github.com/0xPolygon/supernets2-node@v0.0.0-20230711153321-2fe574524eaa/docs/production-setup.md (about)

     1  # Production Setup
     2  
     3  This guide describes how to run a node that:
     4  
     5  - Synchronizes the network
     6  - Expose a JSON RPC interface, acting as an archive node
     7  
     8  Note that sequencing and proving functionalities are not covered in this document **yet**.
     9  
    10  ## Requirements
    11  
    12  - A machine to run the zkEVM node with the following requirements:
    13    - Hardware: 32G RAM, 4 cores, 128G Disk with high IOPS (as the network is super young the current disk requirements are quite low, but they will increase overtime. Also note that this requirement is true if the DBs run on the same machine, but it's recommended to run Postgres on dedicated infra). Currently ARM-based CPUs are not supported
    14    - Software: Ubuntu 22.04, Docker
    15  - A L1 node: we recommend using geth, but what it's actually needed is access to a JSON RPC interface for the L1 network (Goerli for zkEVM testnet, Ethereum mainnet for zkEVM mainnet)
    16  
    17  ## Setup
    18  
    19  This is the most straightforward path to run a zkEVM node, and it's perfectly fine for most use cases, however if you are interested in providing service to many users it's recommended to do some tweaking over the default configuration. Furthermore, this is quite opinionated, feel free to run this software in a different way, for instance it's not needed to use Docker, you could use the Go and C++ binaries directly.
    20  
    21  tl;dr:
    22  
    23  ```bash
    24  # DOWNLOAD ARTIFACTS
    25  ZKEVM_NET=mainnet
    26  ZKEVM_DIR=./path/to/install # CHANGE THIS
    27  ZKEVM_CONFIG_DIR=./path/to/config  # CHANGE THIS
    28  curl -L https://github.com/0xPolygon/supernets2-node/releases/latest/download/$ZKEVM_NET.zip > $ZKEVM_NET.zip && unzip -o $ZKEVM_NET.zip -d $ZKEVM_DIR && rm $ZKEVM_NET.zip
    29  cp $ZKEVM_DIR/$ZKEVM_NET/example.env $ZKEVM_CONFIG_DIR/.env
    30  
    31  # EDIT THIS env file:
    32  nano $ZKEVM_CONFIG_DIR/.env
    33  
    34  # RUN:
    35  docker compose --env-file $ZKEVM_CONFIG_DIR/.env -f $ZKEVM_DIR/$ZKEVM_NET/docker-compose.yml up -d
    36  ```
    37  
    38  Explained step by step:
    39  
    40  1. Define network: `ZKEVM_NET=testnet` or `ZKEVM_NET=mainnet`
    41  2. Define installation path: `ZKEVM_DIR=./path/to/install`
    42  3. Define a config directory: `ZKEVM_CONFIG_DIR=./path/to/config`
    43  4. It's recommended to source this env vars in your `~/.bashrc`, `~/.zshrc` or whatever you're using
    44  5. Download and extract the artifacts: `curl -L https://github.com/0xPolygon/supernets2-node/releases/latest/download/$ZKEVM_NET.zip > $ZKEVM_NET.zip && unzip -o $ZKEVM_NET.zip -d $ZKEVM_DIR && rm $ZKEVM_NET.zip`. Note you may need to install `unzip` for this command to work
    45  6. Copy the file with the env parameters: `cp $ZKEVM_DIR/$ZKEVM_NET/example.env $ZKEVM_CONFIG_DIR/.env`
    46  7. Edit the env file, with your favourite editor. The example will use nano: `nano $ZKEVM_CONFIG_DIR/.env`. This file contains the configuration that anyone should modify. For advanced configuration:
    47     1. Copy the config files into the config directory `cp $ZKEVM_DIR/$ZKEVM_NET/config/environments/public/* $ZKEVM_CONFIG_DIR/`
    48     2. Make sure the modify the `ZKEVM_ADVANCED_CONFIG_DIR` from `$ZKEVM_CONFIG_DIR/.env` with the correct path
    49     3. Edit the different configuration files
    50  8. Run the node: `docker compose --env-file $ZKEVM_CONFIG_DIR/.env -f $ZKEVM_DIR/$ZKEVM_NET/docker-compose.yml up -d`. You may need to run this command using `sudo` depending on your Docker setup.
    51  9. Make sure that all components are running: `docker compose --env-file $ZKEVM_CONFIG_DIR/.env -f $ZKEVM_DIR/$ZKEVM_NET/docker-compose.yml ps`. You should see the following containers:
    52     1. supernets2-rpc
    53     2. supernets2-sync
    54     3. supernets2-state-db
    55     4. supernets2-pool-db
    56     5. supernets2-prover
    57  
    58  If everything has gone as expected you should be able to run queries to the JSON RPC at `http://localhost:8545`. For instance you can run the following query that fetches the latest synchronized L2 block, if you call this every few seconds, you should see the number increasing:
    59  
    60  `curl -H "Content-Type: application/json" -X POST --data '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":83}' http://localhost:8545`
    61  
    62  ## Troubleshooting
    63  
    64  - It's possible that the machine you're using already uses some of the necessary ports. In this case you can change them directly at `$ZKEVM_DIR/$ZKEVM_NET/docker-compose.yml`
    65  - If one or more containers are crashing please check the logs using `docker compose --env-file $ZKEVM_CONFIG_DIR/.env -f $ZKEVM_DIR/$ZKEVM_NET/docker-compose.yml logs <cointainer_name>`
    66  
    67  ## Stop
    68  
    69  ```bash
    70  docker compose --env-file $ZKEVM_CONFIG_DIR/.env -f $ZKEVM_DIR/$ZKEVM_NET/docker-compose.yml down
    71  ```
    72  
    73  ## Updating
    74  
    75  In order to update the software, you have to repeat the steps of the setup, but taking care of not overriding the config that you have modified. Basically, instead of running `cp $ZKEVM_DIR/$ZKEVM_NET/example.env $ZKEVM_CONFIG_DIR/.env`, check if the variables of `$ZKEVM_DIR/$ZKEVM_NET/example.env` have been renamed or there are new ones, and update `$ZKEVM_CONFIG_DIR/.env` accordingly.
    76  
    77  ## Advanced setup
    78  
    79  > DISCLAIMER: right now this part of the documentation attempts to give ideas on how to improve the setup for better performance, but is far from being a detailed guide on how to achieve this. Please open issues requesting more details if you don't understand how to achieve something. We will keep improving this doc for sure!
    80  
    81  There are some fundamental changes that can be done towards the basic setup, in order to get better performance and scale better:
    82  
    83  ### DB
    84  
    85  In the basic setup, there are Postgres being instanciated as Docker containers. For better performance is recommended to:
    86  
    87  - Run dedicated instances for Postgres. To achieve this you will need to:
    88    - Remove the Postgres services (`supernets2-pool-db` and `supernets2-state-db`) from the `docker-compose.yml`
    89    - Instantiate Postgres elsewhere (note that you will have to create credentials and run some queries to make this work, following the config files and docker-compose should give a clear idea of what to do)
    90    - Update the `public.node.config.toml` to use the correct URI for both DBs
    91    - Update `prover.public.config.json` to use the correct URI for the state DB
    92  - Use a setup of Postgres that allows to have separated endpoints for read / write replicas
    93  
    94  ### JSON RPC
    95  
    96  Unlike the synchronizer, that needs to have only one instance running (having more than one synchronizer running at the same time connected to the same DB can be fatal), the JSON RPC can scale horizontally.
    97  
    98  There can be as many instances of it as needed, but in order to not introduce other bottlenecks, it's important to consider the following:
    99  
   100  - Read replicas of the State DB should be used
   101  - Synchronizer should have an exclusive instance of `supernets2-prover`
   102  - JSON RPCs should scale in correlation with instances of `supernets2-prover`. The most obvious way to do so is by having a dedicated `supernets2-prover` for each `supernets2-rpc`. But depending on the payload of your solution it could be worth to have `1 supernets2-rpc : many supernets2-prover` or `many supernets2-rpc : 1 supernets2-prover`, ... For reference, the `supernets2-prover` implements the EVM, and therefore will be heavily used when calling endpoints such as `eth_call`. On the other hand, there are other endpoints that relay on the `supernets2-state-db`