github.com/osdi23p228/fabric@v0.0.0-20221218062954-77808885f5db/docs/source/deployorderer/ordererchecklist.md

# Checklist for a production ordering node

As you prepare to build a production ordering service (or a single ordering node), you need to customize the configuration by editing the [orderer.yaml](https://github.com/osdi23p228/fabric/blob/{BRANCH}/sampleconfig/orderer.yaml) file, which is copied into the `/config` directory when downloading the Fabric binaries, and available within the Fabric ordering node image at `/etc/hyperledger/fabric/orderer.yaml`.

While in a production environment you could override the environment variables in the `orderer.yaml` file in your Docker container or your Kubernetes job, these instructions show how to edit `orderer.yaml` instead. It’s important to understand the parameters in the configuration file and their dependencies on other parameter settings in the file. Blindly overriding one setting using an environment variable could affect the functionality of another setting. Therefore, the recommendation is that before starting the ordering node, you make the modifications to the settings in the configuration file to become familiar with the available settings and how they work. Afterwards, you may choose to override these parameters using environment variables.

This checklist covers key configuration parameters for setting up a production ordering service. Of course, you can always refer to the orderer.yaml file for additional parameters or more information. It also provides guidance on which parameters should be overridden. The list of parameters that you need to understand and that are described in this topic include:

* [General.ListenAddress](#general-listenaddress)
* [General.ListenPort](#general-listenport)
* [General.TLS.*](#general-tls)
* [General.Keepalive.*](#general-keepalive)
* [General.Cluster.*](#general-cluster)
* [General.BoostrapMethod](#general-bootstrapmethod)
* [General.BoostrapFile](#general-bootstrapfile)
* [General.LocalMSPDir](#general-localmspdir)
* [General.LocalMSPID](#general-localmspid)
* [FileLedger.Location](#fileledger-location)
* [Operations.*](#operations)
* [Metrics.*](#metrics)
* [Consensus.*](#consensus)

## General.ListenAddress

```
# Listen address: The IP on which to bind to listen.
ListenAddress: 127.0.0.1
```

* **`ListenAddress`**: (default value should be overridden) This is the location where the orderer will listen, for example, `0.0.0.0`. Note: unlike the peer, the `orderer.yaml` separates the address and the port, hence the [General.ListenPort](#general-listenport) parameter.

## General.ListenPort

```
# Listen port: The port on which to bind to listen.
ListenPort: 7050
```

* **`ListenPort`**: (default value should be overridden) This is the port that the orderer listens on.

## General.TLS

```
Enabled: false
# PrivateKey governs the file location of the private key of the TLS certificate.
PrivateKey: tls/server.key
# Certificate governs the file location of the server TLS certificate.
Certificate: tls/server.crt
RootCAs:
  - tls/ca.crt
ClientAuthRequired: false
ClientRootCAs:
```

* **`Enabled`**: (default value should be overridden) In a production network, you should be using TLS-secured communications. This value should be `true`.
* **`PrivateKey`**: (default value should be overridden). Provide the path to, and filename of, the private key generated by your TLS CA for this node.
* **`Certificate`**: (default value should be overridden) Provide the path to, and filename of, the public certificate (also known as the sign certificate) generated by your TLS CA for this node.
* **`RootCAs`**: (should be commented out) This parameter is typically unset for normal use. It is a list of the paths to additional root certificates used for verifying certificates of other orderer nodes during outbound connections. It can be used to augment the set of TLS CA certificates available from the MSPs of each channel's configuration.
* **`ClientAuthRequired`**: (Mutual TLS only) Setting this value to “true” will enable mutual TLS on your network, and must be done for the entire network, not just one node.
* **`ClientRootCAs`**: (Mutual TLS only) Can be left blank if mutual TLS is disabled. If mutual TLS is enabled, this is a list of the paths to additional root certificates used for verifying certificates of client connections. It can be used to augment the set of TLS CA certificates available from the MSPs of each channel’s configuration.

## General.KeepAlive

The `KeepAlive` values might need to be tuned for compatibility with any networking devices or software (like firewalls or proxies) in between components of your network. Ideally, these settings would be manipulated if needed in a test or pre-prod environment and then set accordingly for your production environment.

```
# ServerMinInterval is the minimum permitted time between client pings.
# If clients send pings more frequently, the server will
# disconnect them.
ServerMinInterval: 60s
# ServerInterval is the time between pings to clients.
ServerInterval: 7200s
# ServerTimeout is the duration the server waits for a response from
# a client before closing the connection.
ServerTimeout: 20s
```

* **`ServerMinInterval`**: (default value should not be overridden, unless determined necessary through testing)
* **`ServerInterval`**: (default value should not be overridden, unless determined necessary through testing)
* **`ServerTimeout`**: (default value should not be overridden, unless determined necessary through testing)

## General.Cluster

```
# SendBufferSize is the maximum number of messages in the egress buffer.
# Consensus messages are dropped if the buffer is full, and transaction
# messages are waiting for space to be freed.
SendBufferSize: 10
# ClientCertificate governs the file location of the client TLS certificate
# If not set, the server General.TLS.Certificate is re-used.
ClientCertificate:
# If not set, the server General.TLS.PrivateKey is re-used.
ClientPrivateKey:
# The below 4 properties should be either set together, or be unset together.
# If they are set, then the orderer node uses a separate listener for intra-cluster
# communication. If they are unset, then the general orderer listener is used.
# This is useful if you want to use a different TLS server certificates on the
# client-facing and the intra-cluster listeners.

# ListenPort defines the port on which the cluster listens to connections.
ListenPort:
# ListenAddress defines the IP on which to listen to intra-cluster communication.
ListenAddress:
# ServerCertificate defines the file location of the server TLS certificate used for intra-cluster
# communication.
ServerCertificate:
# ServerPrivateKey defines the file location of the private key of the TLS certificate.
ServerPrivateKey:
```

If unset, the `ClientCertificate` and `ClientPrivateKey` default to the server `General.TLS.Certificate` and `General.TLS.PrivateKey` when the orderer is not configured to use a separate cluster port.

* **`ClientCertificate`**:  Provide the path to, and filename of, the public certificate (also known as a signed certificate) generated by your TLS CA for this node.
* **`ClientPrivateKey`**: Provide the path to, and filename of, the private key generated by your TLS CA for this node.

In general, these four parameters would only need to be configured if you want to configure a separate listener and TLS certificates for intra-cluster communication (with other Raft orderers), as opposed to using the listener that peer clients and application clients utilize. This is an advanced deployment option. These four parameters should be set together or left unset, and if they are set, note that the `ClientCertificate` and `ClientPrivateKey` must be set as well.

* **`ListenPort`**
* **`ListenAddress`**
* **`ServerCertificate`**
* **`ServerPrivateKey`**

## General.BoostrapMethod

```
# Bootstrap method: The method by which to obtain the bootstrap block
# system channel is specified. The option can be one of:
#   "file" - path to a file containing the genesis block or config block of system channel
#   "none" - allows an orderer to start without a system channel configuration
BootstrapMethod: file
```

* **`BootstrapMethod`**: (default value should not be overridden) Unless you plan to use a file type other than “file”, this value should be left as is.

## General.BoostrapFile

```
# Bootstrap file: The file containing the bootstrap block to use when
# initializing the orderer system channel and BootstrapMethod is set to
# "file".  The bootstrap file can be the genesis block, and it can also be
# a config block for late bootstrap of some consensus methods like Raft.
# Generate a genesis block by updating $FABRIC_CFG_PATH/configtx.yaml and
# using configtxgen command with "-outputBlock" option.
# Defaults to file "genesisblock" (in $FABRIC_CFG_PATH directory) if not specified.
BootstrapFile:
```

* **`BoostrapFile`**: (default value should be overridden) Specify the location and name of the system channel genesis block to use when this node is created.

## General.LocalMSPDir

```
# LocalMSPDir is where to find the private crypto material needed by the
# orderer. It is set relative here as a default for dev environments but
# should be changed to the real location in production.
LocalMSPDir: msp
```

**`LocalMSPDir`**: (default value will often be overriden be overridden) This is the path to the ordering node's local MSP, which must be created before it can be deployed. The path can be absolute or relative to `FABRIC_CFG_PATH` (by default, it is `/etc/hyperledger/fabric` in the orderer image). Unless an absolute path is specified to a folder named something other than "msp", the ordering node defaults to looking for a folder called “msp” at the path (in other words, `FABRIC_CFG_PATH/msp`) and when using the orderer image: `/etc/hyperledger/fabric/msp`. If you are using the recommended folder structure described in the [Registering and enrolling identities with a CA](https://hyperledger-fabric-ca.readthedocs.io/en/release-1.4/deployguide/use_CA.html) topic, it would be relative to the `FABRIC_CFG_PATH` as follows:
`config/organizations/ordererOrganizations/org0.example.com/orderers/orderer0.org0.example.com/msp`. **The best practice is to store this data in persistent storage**. This prevents the MSP from being lost if your orderer containers are destroyed for some reason.

## General.LocalMSPID

```
# LocalMSPID is the identity to register the local MSP material with the MSP
# manager. IMPORTANT: The local MSP ID of an orderer needs to match the MSP
# ID of one of the organizations defined in the orderer system channel's
# /Channel/Orderer configuration. The sample organization defined in the
# sample configuration provided has an MSP ID of "SampleOrg".
LocalMSPID: SampleOrg
```

* **`LocalMSPID`**: (default value should be overridden) The MSP ID must match the orderer organization MSP ID that exists in the configuration of the system channel. This means the MSP ID must have been listed in the `configtx.yaml` used to create the genesis block of the system channel (or have been added later to the list of system channel administrators).

## General.BCCSP.*

```
# Default specifies the preferred blockchain crypto service provider
        # to use. If the preferred provider is not available, the software
        # based provider ("SW") will be used.
        # Valid providers are:
        #  - SW: a software based crypto provider
        #  - PKCS11: a CA hardware security module crypto provider.
        Default: SW

        # SW configures the software based blockchain crypto provider.
        SW:
            # TODO: The default Hash and Security level needs refactoring to be
            # fully configurable. Changing these defaults requires coordination
            # SHA2 is hardcoded in several places, not only BCCSP
            Hash: SHA2
            Security: 256
            # Location of key store. If this is unset, a location will be
            # chosen using: 'LocalMSPDir'/keystore
            FileKeyStore:
                KeyStore:
```

(Optional) This section is used to configure the Blockchain crypto provider.

* **`BCCSP.Default:`** If you plan to use a Hardware Security Module (HSM), then this must be set to `PKCS11`.

```
# Settings for the PKCS#11 crypto provider (i.e. when DEFAULT: PKCS11)
        PKCS11:
            # Location of the PKCS11 module library
            Library:
            # Token Label
            Label:
            # User PIN
            Pin:
            Hash:
            Security:
            FileKeyStore:
                KeyStore:
```

* **`BCCSP.PKCS11.*:`** Provide this set of parameters according to your HSM configuration. Refer to this [example](../hsm.html) of an HSM configuration for more information.

## FileLedger.Location

```
# Location: The directory to store the blocks in.
Location: /var/hyperledger/production/orderer
```

* **`Location`**: (default value should be overridden in the unlikely event where two ordering nodes are running on the same node) Every channel on which the node is a consenter will have its own subdirectory at this location. The user running the orderer needs to own and have write access to this directory. **The best practice is to store this data in persistent storage**. This prevents the ledger from being lost if your orderer containers are destroyed for some reason.

## Operations.*

The operations service is used for monitoring the health of the ordering node and relies on mutual TLS to secure its communication. Therefore, you need to set `operations.tls.clientAuthRequired` to `true`. When this parameter is set to `true`, clients attempting to ascertain the health of the node are required to provide a valid certificate for authentication. If the client does not provide a certificate or the service cannot verify the client’s certificate, the request is rejected. This means that the clients will need to register with the ordering node's TLS CA and provide their TLS signing certificate on the requests. See [The Operations Service](../operations_service.html) to learn more.

If you plan to use Prometheus [metrics](#metrics) to monitor your ordering node, you must configure the operations service here.

In the unlikely case where two ordering nodes are running on the same node on your infrastructure, you need to modify the addresses for the second ordering node to use a different port. Otherwise, when you start the second ordering node, it will fail to start, reporting that the addresses are already in use.

```
# host and port for the operations server
    ListenAddress: 127.0.0.1:8443

    # TLS configuration for the operations endpoint
    TLS:
        # TLS enabled
        Enabled: false

        # Certificate is the location of the PEM encoded TLS certificate
        Certificate:

        # PrivateKey points to the location of the PEM-encoded key
        PrivateKey:

        # Most operations service endpoints require client authentication when TLS
        # is enabled. ClientAuthRequired requires client certificate authentication
        # at the TLS layer to access all resources.
        ClientAuthRequired: false

        # Paths to PEM encoded ca certificates to trust for client authentication
        ClientRootCAs: []
```

* **`ListenAddress`**: (required when using the operations service) Specify the address and port of the operations server.
* **`Enabled`**: (required when using the operations service) Must be `true` if the operations service is being used.
* **`Certificate`**: (required when using the operations service) Can be the same file as the `General.TLS.Certificate`.
* **`PrivateKey`**: (required when using the operations service) Can be the same file as the `General.TLS.PrivateKey`.
* **`ClientAuthRequired`**: (required when using the operations service) Must be set to `true` to enable mutual TLS between the client and the server.
* **`ClientRootCAs`**: (required when using the operations service) Similar to the client root CA cert file in TLS, it contains a list of client root CA certificates that can be used to verify client certificates. If the client enrolled with the orderer organization CA, then this value is the orderer organization root CA cert.

## Metrics.*

By default this is disabled, but if you want to monitor the metrics for the orderer, you need to use `StatsD` or `Prometheus` as your metric provider. `StatsD` uses a "push" model, pushing metrics from the ordering node to a `StatsD` endpoint. Because of this, it does not require configuration of the operations service itself. `Prometheus` metrics, by contrast, are pulled from an ordering node.

For more information about the available `Prometheus` metrics, check out [Prometheus](../metrics_reference.html#prometheus)

For more information about the available `StatsD` metrics, check out [StatsD](../metrics_reference.html#statsd).

Because Prometheus utilizes a "pull" model there is not any configuration required, beyond making the operations service available. Rather, Prometheus will send requests to the operations URL to poll for available metrics.

```
    # The metrics provider is one of statsd, prometheus, or disabled
    Provider: disabled

    # The statsd configuration
    Statsd:
      # network type: tcp or udp
      Network: udp

      # the statsd server address
      Address: 127.0.0.1:8125

      # The interval at which locally cached counters and gauges are pushed
      # to statsd; timings are pushed immediately
      WriteInterval: 30s

      # The prefix is prepended to all emitted statsd metrics
      Prefix:
```

* **`Provider`**: Set this value to `statsd` if using `StatsD` or `prometheus` if using `Prometheus`.
* **`Statsd.Address`**: (required to use `StatsD` metrics for the ordering node) When `StatsD` is enabled, you will need to configure the `hostname` and `port` of the `StatsD` server so that the ordering node can push metric updates.

## Consensus.*

The values of this section vary by consensus plugin. The values below are for the `etcdraft` consensus plugin. If you are using a different consensus plugin, refer to its documentation for allowed keys and recommended values.

```
# The allowed key-value pairs here depend on consensus plugin. For etcd/raft,
# we use following options:

# WALDir specifies the location at which Write Ahead Logs for etcd/raft are
# stored. Each channel will have its own subdir named after channel ID.
WALDir: /var/hyperledger/production/orderer/etcdraft/wal

# SnapDir specifies the location at which snapshots for etcd/raft are
# stored. Each channel will have its own subdir named after channel ID.
SnapDir: /var/hyperledger/production/orderer/etcdraft/snapshot
```

* **`WALDir`**: (default value should be overridden) This is the path to the write ahead logs on the local filesystem of the ordering node. It can be an absolute path or relative to `FABRIC_CFG_PATH`. It defaults to `/var/hyperledger/production/orderer/etcdraft/wal`. Each channel will have its own subdirectory named after the channel ID. The user running the ordering node needs to own and have write access to this directory. **The best practice is to store this data in persistent storage**. This prevents the write ahead log from being lost if your orderer containers are destroyed for some reason.
* **`SnapDir`**: (default value should be overridden) This is the path to the snapshots on the local filesystem of the ordering node. For more information about how snapshots work in a Raft ordering service, check out [Snapshots](../orderer/ordering_service.html#snapshots)/ It can be an absolute path or relative to `FABRIC_CFG_PATH`. It defaults to `/var/hyperledger/production/orderer/etcdraft/wal`. Each channel will have its own subdirectory named after the channel ID. The user running the ordering node needs to own and have write access to this directory. **The best practice is to store this data in persistent storage**. This prevents snapshots from being lost if your orderer containers are destroyed for some reason.

For more information about ordering node configuration, including how to set parameters that are not available in `orderer.yaml`, check out [Configuring and operating a Raft ordering service](../raft_configuration.html).

<!--- Licensed under Creative Commons Attribution 4.0 International License
https://creativecommons.org/licenses/by/4.0/) -->