github.com/outbrain/consul@v1.4.5/website/source/docs/guides/deployment-guide.html.md (about)

     1  ---
     2  layout: "docs"
     3  page_title: "Consul Deployment Guide"
     4  sidebar_current: "docs-guides-deployment-guide"
     5  description: |-
     6    This deployment guide covers the steps required to install and
     7    configure a single HashiCorp Consul cluster as defined in the
     8    Consul Reference Architecture.
     9  ea_version: 1.3
    10  ---
    11  
    12  # Consul Deployment Guide
    13  
    14  This deployment guide covers the steps required to install and configure a single HashiCorp Consul cluster as defined in the [Consul Reference Architecture](/docs/guides/deployment.html).
    15  
    16  These instructions are for installing and configuring Consul on Linux hosts running the systemd system and service manager.
    17  
    18  ## Reference Material
    19  
    20  This deployment guide is designed to work in combination with the [Consul Reference Architecture](/docs/guides/deployment.html). Although not a strict requirement to follow the Consul Reference Architecture, please ensure you are familiar with the overall architecture design; for example installing Consul server agents on multiple physical or virtual (with correct anti-affinity) hosts for high-availability.
    21  
    22  ## Overview
    23  
    24  To provide a highly-available single cluster architecture, we recommend Consul server agents be deployed to more than one host, as shown in the [Consul Reference Architecture](/docs/guides/deployment.html).
    25  
    26  ![Reference Diagram](/assets/images/consul-arch-single.png "Reference Diagram")
    27  
    28  These setup steps should be completed on all Consul hosts.
    29  
    30  - [Download Consul](#download-consul)
    31  - [Install Consul](#install-consul)
    32  - [Configure systemd](#configure-systemd)
    33  - Configure Consul [(server)](#configure-consul-server-) or [(client)](#configure-consul-client-)
    34  - [Start Consul](#start-consul)
    35  
    36  ## Download Consul
    37  
    38  Precompiled Consul binaries are available for download at [https://releases.hashicorp.com/consul/](https://releases.hashicorp.com/consul/) and Consul Enterprise binaries are available for download by following the instructions made available to HashiCorp Consul customers.
    39  
    40  You should perform checksum verification of the zip packages using the SHA256SUMS and SHA256SUMS.sig files available for the specific release version. HashiCorp provides [a guide on checksum verification](https://www.hashicorp.com/security.html) for precompiled binaries.
    41  
    42  ```text
    43  CONSUL_VERSION="x.x.x"
    44  curl --silent --remote-name https://releases.hashicorp.com/consul/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_linux_amd64.zip
    45  curl --silent --remote-name https://releases.hashicorp.com/consul/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_SHA256SUMS
    46  curl --silent --remote-name https://releases.hashicorp.com/consul/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_SHA256SUMS.sig
    47  ```
    48  
    49  ## Install Consul
    50  
    51  Unzip the downloaded package and move the `consul` binary to `/usr/local/bin/`. Check `consul` is available on the system path.
    52  
    53  ```text
    54  unzip consul_${CONSUL_VERSION}_linux_amd64.zip
    55  sudo chown root:root consul
    56  sudo mv consul /usr/local/bin/
    57  consul --version
    58  ```
    59  
    60  The `consul` command features opt-in autocompletion for flags, subcommands, and arguments (where supported). Enable autocompletion.
    61  
    62  ```text
    63  consul -autocomplete-install
    64  complete -C /usr/local/bin/consul consul
    65  ```
    66  
    67  Create a unique, non-privileged system user to run Consul and create its data directory.
    68  
    69  ```text
    70  sudo useradd --system --home /etc/consul.d --shell /bin/false consul
    71  sudo mkdir --parents /opt/consul
    72  sudo chown --recursive consul:consul /opt/consul
    73  ```
    74  
    75  ## Configure systemd
    76  
    77  Systemd uses [documented sane defaults](https://www.freedesktop.org/software/systemd/man/systemd.directives.html) so only non-default values must be set in the configuration file.
    78  
    79  Create a Consul service file at /etc/systemd/system/consul.service.
    80  
    81  ```text
    82  sudo touch /etc/systemd/system/consul.service
    83  ```
    84  
    85  Add this configuration to the Consul service file:
    86  
    87  ```text
    88  [Unit]
    89  Description="HashiCorp Consul - A service mesh solution"
    90  Documentation=https://www.consul.io/
    91  Requires=network-online.target
    92  After=network-online.target
    93  ConditionFileNotEmpty=/etc/consul.d/consul.hcl
    94  
    95  [Service]
    96  User=consul
    97  Group=consul
    98  ExecStart=/usr/local/bin/consul agent -config-dir=/etc/consul.d/
    99  ExecReload=/usr/local/bin/consul reload
   100  KillMode=process
   101  Restart=on-failure
   102  LimitNOFILE=65536
   103  
   104  [Install]
   105  WantedBy=multi-user.target
   106  ```
   107  
   108  The following parameters are set for the `[Unit]` stanza:
   109  
   110  - [`Description`](https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Description=) - Free-form string describing the consul service
   111  - [`Documentation`](https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Documentation=) - Link to the consul documentation
   112  - [`Requires`](https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Requires=) - Configure a requirement dependency on the network service
   113  - [`After`](https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Before=) - Configure an ordering dependency on the network service being started before the consul service
   114  - [`ConditionFileNotEmpty`](https://www.freedesktop.org/software/systemd/man/systemd.unit.html#ConditionArchitecture=) - Check for a non-zero sized configuration file before consul is started
   115  
   116  The following parameters are set for the `[Service]` stanza:
   117  
   118  - [`User`, `Group`](https://www.freedesktop.org/software/systemd/man/systemd.exec.html#User=) - Run consul as the consul user
   119  - [`ExecStart`](https://www.freedesktop.org/software/systemd/man/systemd.service.html#ExecStart=) - Start consul with the `agent` argument and path to the configuration file
   120  - [`ExecReload`](https://www.freedesktop.org/software/systemd/man/systemd.service.html#ExecReload=) - Send consul a reload signal to trigger a configuration reload in consul
   121  - [`KillMode`](https://www.freedesktop.org/software/systemd/man/systemd.kill.html#KillMode=) - Treat consul as a single process
   122  - [`Restart`](https://www.freedesktop.org/software/systemd/man/systemd.service.html#RestartSec=) - Restart consul unless it returned a clean exit code
   123  - [`LimitNOFILE`](https://www.freedesktop.org/software/systemd/man/systemd.exec.html#Process%20Properties) - Set an increased Limit for File Descriptors
   124  
   125  The following parameters are set for the `[Install]` stanza:
   126  
   127  - [`WantedBy`](https://www.freedesktop.org/software/systemd/man/systemd.unit.html#WantedBy=) - Creates a weak dependency on consul being started by the multi-user run level
   128  
   129  ## Configure Consul (server)
   130  
   131  Consul uses [documented sane defaults](/docs/agent/options.html) so only non-default values must be set in the configuration file. Configuration can be read from multiple files and is loaded in lexical order. See the [full description](/docs/agent/options.html) for more information about configuration loading and merge semantics.
   132  
   133  Consul server agents typically require a superset of configuration required by Consul client agents. We will specify common configuration used by all Consul agents in `consul.hcl` and server specific configuration in `server.hcl`.
   134  
   135  ### General configuration
   136  
   137  Create a configuration file at `/etc/consul.d/consul.hcl`:
   138  
   139  ```text
   140  sudo mkdir --parents /etc/consul.d
   141  sudo touch /etc/consul.d/consul.hcl
   142  sudo chown --recursive consul:consul /etc/consul.d
   143  sudo chmod 640 /etc/consul.d/consul.hcl
   144  ```
   145  
   146  Add this configuration to the `consul.hcl` configuration file:
   147  
   148  ~> **NOTE** Replace the `datacenter` parameter value with the identifier you will use for the datacenter this Consul cluster is deployed in. Replace the `encrypt` parameter value with the output from running `consul keygen` on any host with the `consul` binary installed.
   149  
   150  ```hcl
   151  datacenter = "dc1"
   152  data_dir = "/opt/consul"
   153  encrypt = "Luj2FZWwlt8475wD1WtwUQ=="
   154  ```
   155  
   156  - [`datacenter`](/docs/agent/options.html#_datacenter) - The datacenter in which the agent is running.
   157  - [`data_dir`](/docs/agent/options.html#_data_dir) - The data directory for the agent to store state.
   158  - [`encrypt`](/docs/agent/options.html#_encrypt) - Specifies the secret key to use for encryption of Consul network traffic.
   159  
   160  ### ACL configuration
   161  
   162  The [ACL](/docs/guides/acl.html) guide provides instructions on configuring and enabling ACLs.
   163  
   164  ### Cluster auto-join
   165  
   166  The `retry_join` parameter allows you to configure all Consul agents to automatically form a cluster using a common Consul server accessed via DNS address, IP address or using Cloud Auto-join. This removes the need to manually join the Consul cluster nodes together.
   167  
   168  Add the retry_join parameter to the `consul.hcl` configuration file:
   169  
   170  ~> **NOTE** Replace the `retry_join` parameter value with the correct DNS address, IP address or [cloud auto-join configuration](/docs/agent/cloud-auto-join.html) for your environment.
   171  
   172  ```hcl
   173  retry_join = ["172.16.0.11"]
   174  ```
   175  
   176  - [`retry_join`](/docs/agent/options.html#retry-join) - Address of another agent to join upon starting up.
   177  
   178  ### Performance stanza
   179  
   180  The [`performance`](/docs/agent/options.html#performance) stanza allows tuning the performance of different subsystems in Consul.
   181  
   182  Add the performance configuration to the `consul.hcl` configuration file:
   183  
   184  ```hcl
   185  performance {
   186    raft_multiplier = 1
   187  }
   188  ```
   189  
   190  - [`raft_multiplier`](/docs/agent/options.html#raft_multiplier) - An integer multiplier used by Consul servers to scale key Raft timing parameters. Setting this to a value of 1 will configure Raft to its highest-performance mode, equivalent to the default timing of Consul prior to 0.7, and is recommended for production Consul servers.
   191  
   192  For more information on Raft tuning and the `raft_multiplier` setting, see the [server performance](/docs/guides/performance.html) documentation.
   193  
   194  ### Telemetry stanza
   195  
   196  The [`telemetry`](/docs/agent/options.html#telemetry) stanza specifies various configurations for Consul to publish metrics to upstream systems.
   197  
   198  If you decide to configure Consul to publish telemtery data, you should review the [telemetry configuration section](/docs/agent/options.html#telemetry) of our documentation.
   199  
   200  ### TLS configuration
   201  
   202  The [Creating Certificates](/docs/guides/creating-certificates.html) guide provides instructions on configuring and enabling TLS.
   203  
   204  ### Server configuration
   205  
   206  Create a configuration file at `/etc/consul.d/server.hcl`:
   207  
   208  ```text
   209  sudo mkdir --parents /etc/consul.d
   210  sudo touch /etc/consul.d/server.hcl
   211  sudo chown --recursive consul:consul /etc/consul.d
   212  sudo chmod 640 /etc/consul.d/server.hcl
   213  ```
   214  
   215  Add this configuration to the `server.hcl` configuration file:
   216  
   217  ~> **NOTE** Replace the `bootstrap_expect` value with the number of Consul servers you will use; three or five [is recommended](/docs/internals/consensus.html#deployment-table).
   218  
   219  ```hcl
   220  server = true
   221  bootstrap_expect = 3
   222  ```
   223  
   224  - [`server`](/docs/agent/options.html#_server) -  This flag is used to control if an agent is in server or client mode.
   225  - [`bootstrap-expect`](/docs/agent/options.html#_bootstrap_expect) - This flag provides the number of expected servers in the datacenter. Either this value should not be provided or the value must agree with other servers in the cluster.
   226  
   227  ### Consul UI
   228  
   229  Consul features a web-based user interface, allowing you to easily view all services, nodes, intentions and more using a graphical user interface, rather than the CLI or API.
   230  
   231  ~> **NOTE** You should consider running the Consul UI on select Consul hosts rather than all hosts.
   232  
   233  Optionally, add the UI configuration to the `server.hcl` configuration file to enable the Consul UI:
   234  
   235  ```hcl
   236  ui = true
   237  ```
   238  
   239  ## Configure Consul (client)
   240  
   241  Consul client agents typically require a subset of configuration required by Consul server agents. All Consul clients can use the `consul.hcl` file created when [configuring the Consul servers](#general-configuration). If you have added host-specific configuration such as identifiers, you will need to set these individually.
   242  
   243  ## Start Consul
   244  
   245  Enable and start Consul using the systemctl command responsible for controlling systemd managed services. Check the status of the consul service using systemctl.
   246  
   247  ```text
   248  sudo systemctl enable consul
   249  sudo systemctl start consul
   250  sudo systemctl status consul
   251  ```
   252  
   253  ## Backups
   254  
   255  Creating server backups is an important step in production deployments. Backups provide a mechanism for the server to recover from an outage (network loss, operator error, or a corrupted data directory). All agents write to the `-data-dir` before commit. This directory persists the local agent’s state and — in the case of servers — it also holds the Raft information.
   256  
   257  Consul provides the [snapshot](/docs/commands/snapshot.html) command which can be run using the CLI command or the API. The `snapshot` command saves the point-in-time snapshot of the state of the Consul servers which includes KV entries, the service catalog, prepared queries, sessions, and ACL.
   258  
   259  With [Consul Enterprise](/docs/commands/snapshot/agent.html), the `snapshot agent` command runs periodically and writes to local or remote storage (such as Amazon S3).
   260  
   261  By default, all snapshots are taken using `consistent` mode where requests are forwarded to the leader which verifies that it is still in power before taking the snapshot. Snapshots will not be saved if the clusted is degraded or if no leader is available. To reduce the burden on the leader, it is possible to [run the snapshot](/docs/commands/snapshot/save.html) on any non-leader server using `stale` consistency mode:
   262  
   263  ```text
   264  consul snapshot save -stale backup.snap
   265  ```
   266  
   267  This spreads the load across nodes at the possible expense of losing full consistency guarantees. Typically this means that a very small number of recent writes may not be included. The omitted writes are typically limited to data written in the last `100ms` or less from the recovery point. This is usually suitable for disaster recovery. However, the system can’t guarantee how stale this may be if executed against a partitioned server.
   268  
   269  ## Next Steps
   270  
   271  - Read [Monitoring Consul with Telegraf](/docs/guides/monitoring-telegraf.html)
   272    for an example guide to monitoring Consul for improved operational visibility.
   273  
   274  - Read [Outage Recovery](/docs/guides/outage.html) to learn the steps required
   275    for recovery from a Consul outage due to a majority of server nodes in a
   276    datacenter being lost.
   277  
   278  - Read [Server Performance](/docs/guides/performance.html) to learn about
   279    additional configuration that benefits production deployments.