github.com/outbrain/consul@v1.4.5/website/source/docs/guides/deployment-guide.html.md (about) 1 --- 2 layout: "docs" 3 page_title: "Consul Deployment Guide" 4 sidebar_current: "docs-guides-deployment-guide" 5 description: |- 6 This deployment guide covers the steps required to install and 7 configure a single HashiCorp Consul cluster as defined in the 8 Consul Reference Architecture. 9 ea_version: 1.3 10 --- 11 12 # Consul Deployment Guide 13 14 This deployment guide covers the steps required to install and configure a single HashiCorp Consul cluster as defined in the [Consul Reference Architecture](/docs/guides/deployment.html). 15 16 These instructions are for installing and configuring Consul on Linux hosts running the systemd system and service manager. 17 18 ## Reference Material 19 20 This deployment guide is designed to work in combination with the [Consul Reference Architecture](/docs/guides/deployment.html). Although not a strict requirement to follow the Consul Reference Architecture, please ensure you are familiar with the overall architecture design; for example installing Consul server agents on multiple physical or virtual (with correct anti-affinity) hosts for high-availability. 21 22 ## Overview 23 24 To provide a highly-available single cluster architecture, we recommend Consul server agents be deployed to more than one host, as shown in the [Consul Reference Architecture](/docs/guides/deployment.html). 25 26 ![Reference Diagram](/assets/images/consul-arch-single.png "Reference Diagram") 27 28 These setup steps should be completed on all Consul hosts. 29 30 - [Download Consul](#download-consul) 31 - [Install Consul](#install-consul) 32 - [Configure systemd](#configure-systemd) 33 - Configure Consul [(server)](#configure-consul-server-) or [(client)](#configure-consul-client-) 34 - [Start Consul](#start-consul) 35 36 ## Download Consul 37 38 Precompiled Consul binaries are available for download at [https://releases.hashicorp.com/consul/](https://releases.hashicorp.com/consul/) and Consul Enterprise binaries are available for download by following the instructions made available to HashiCorp Consul customers. 39 40 You should perform checksum verification of the zip packages using the SHA256SUMS and SHA256SUMS.sig files available for the specific release version. HashiCorp provides [a guide on checksum verification](https://www.hashicorp.com/security.html) for precompiled binaries. 41 42 ```text 43 CONSUL_VERSION="x.x.x" 44 curl --silent --remote-name https://releases.hashicorp.com/consul/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_linux_amd64.zip 45 curl --silent --remote-name https://releases.hashicorp.com/consul/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_SHA256SUMS 46 curl --silent --remote-name https://releases.hashicorp.com/consul/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_SHA256SUMS.sig 47 ``` 48 49 ## Install Consul 50 51 Unzip the downloaded package and move the `consul` binary to `/usr/local/bin/`. Check `consul` is available on the system path. 52 53 ```text 54 unzip consul_${CONSUL_VERSION}_linux_amd64.zip 55 sudo chown root:root consul 56 sudo mv consul /usr/local/bin/ 57 consul --version 58 ``` 59 60 The `consul` command features opt-in autocompletion for flags, subcommands, and arguments (where supported). Enable autocompletion. 61 62 ```text 63 consul -autocomplete-install 64 complete -C /usr/local/bin/consul consul 65 ``` 66 67 Create a unique, non-privileged system user to run Consul and create its data directory. 68 69 ```text 70 sudo useradd --system --home /etc/consul.d --shell /bin/false consul 71 sudo mkdir --parents /opt/consul 72 sudo chown --recursive consul:consul /opt/consul 73 ``` 74 75 ## Configure systemd 76 77 Systemd uses [documented sane defaults](https://www.freedesktop.org/software/systemd/man/systemd.directives.html) so only non-default values must be set in the configuration file. 78 79 Create a Consul service file at /etc/systemd/system/consul.service. 80 81 ```text 82 sudo touch /etc/systemd/system/consul.service 83 ``` 84 85 Add this configuration to the Consul service file: 86 87 ```text 88 [Unit] 89 Description="HashiCorp Consul - A service mesh solution" 90 Documentation=https://www.consul.io/ 91 Requires=network-online.target 92 After=network-online.target 93 ConditionFileNotEmpty=/etc/consul.d/consul.hcl 94 95 [Service] 96 User=consul 97 Group=consul 98 ExecStart=/usr/local/bin/consul agent -config-dir=/etc/consul.d/ 99 ExecReload=/usr/local/bin/consul reload 100 KillMode=process 101 Restart=on-failure 102 LimitNOFILE=65536 103 104 [Install] 105 WantedBy=multi-user.target 106 ``` 107 108 The following parameters are set for the `[Unit]` stanza: 109 110 - [`Description`](https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Description=) - Free-form string describing the consul service 111 - [`Documentation`](https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Documentation=) - Link to the consul documentation 112 - [`Requires`](https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Requires=) - Configure a requirement dependency on the network service 113 - [`After`](https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Before=) - Configure an ordering dependency on the network service being started before the consul service 114 - [`ConditionFileNotEmpty`](https://www.freedesktop.org/software/systemd/man/systemd.unit.html#ConditionArchitecture=) - Check for a non-zero sized configuration file before consul is started 115 116 The following parameters are set for the `[Service]` stanza: 117 118 - [`User`, `Group`](https://www.freedesktop.org/software/systemd/man/systemd.exec.html#User=) - Run consul as the consul user 119 - [`ExecStart`](https://www.freedesktop.org/software/systemd/man/systemd.service.html#ExecStart=) - Start consul with the `agent` argument and path to the configuration file 120 - [`ExecReload`](https://www.freedesktop.org/software/systemd/man/systemd.service.html#ExecReload=) - Send consul a reload signal to trigger a configuration reload in consul 121 - [`KillMode`](https://www.freedesktop.org/software/systemd/man/systemd.kill.html#KillMode=) - Treat consul as a single process 122 - [`Restart`](https://www.freedesktop.org/software/systemd/man/systemd.service.html#RestartSec=) - Restart consul unless it returned a clean exit code 123 - [`LimitNOFILE`](https://www.freedesktop.org/software/systemd/man/systemd.exec.html#Process%20Properties) - Set an increased Limit for File Descriptors 124 125 The following parameters are set for the `[Install]` stanza: 126 127 - [`WantedBy`](https://www.freedesktop.org/software/systemd/man/systemd.unit.html#WantedBy=) - Creates a weak dependency on consul being started by the multi-user run level 128 129 ## Configure Consul (server) 130 131 Consul uses [documented sane defaults](/docs/agent/options.html) so only non-default values must be set in the configuration file. Configuration can be read from multiple files and is loaded in lexical order. See the [full description](/docs/agent/options.html) for more information about configuration loading and merge semantics. 132 133 Consul server agents typically require a superset of configuration required by Consul client agents. We will specify common configuration used by all Consul agents in `consul.hcl` and server specific configuration in `server.hcl`. 134 135 ### General configuration 136 137 Create a configuration file at `/etc/consul.d/consul.hcl`: 138 139 ```text 140 sudo mkdir --parents /etc/consul.d 141 sudo touch /etc/consul.d/consul.hcl 142 sudo chown --recursive consul:consul /etc/consul.d 143 sudo chmod 640 /etc/consul.d/consul.hcl 144 ``` 145 146 Add this configuration to the `consul.hcl` configuration file: 147 148 ~> **NOTE** Replace the `datacenter` parameter value with the identifier you will use for the datacenter this Consul cluster is deployed in. Replace the `encrypt` parameter value with the output from running `consul keygen` on any host with the `consul` binary installed. 149 150 ```hcl 151 datacenter = "dc1" 152 data_dir = "/opt/consul" 153 encrypt = "Luj2FZWwlt8475wD1WtwUQ==" 154 ``` 155 156 - [`datacenter`](/docs/agent/options.html#_datacenter) - The datacenter in which the agent is running. 157 - [`data_dir`](/docs/agent/options.html#_data_dir) - The data directory for the agent to store state. 158 - [`encrypt`](/docs/agent/options.html#_encrypt) - Specifies the secret key to use for encryption of Consul network traffic. 159 160 ### ACL configuration 161 162 The [ACL](/docs/guides/acl.html) guide provides instructions on configuring and enabling ACLs. 163 164 ### Cluster auto-join 165 166 The `retry_join` parameter allows you to configure all Consul agents to automatically form a cluster using a common Consul server accessed via DNS address, IP address or using Cloud Auto-join. This removes the need to manually join the Consul cluster nodes together. 167 168 Add the retry_join parameter to the `consul.hcl` configuration file: 169 170 ~> **NOTE** Replace the `retry_join` parameter value with the correct DNS address, IP address or [cloud auto-join configuration](/docs/agent/cloud-auto-join.html) for your environment. 171 172 ```hcl 173 retry_join = ["172.16.0.11"] 174 ``` 175 176 - [`retry_join`](/docs/agent/options.html#retry-join) - Address of another agent to join upon starting up. 177 178 ### Performance stanza 179 180 The [`performance`](/docs/agent/options.html#performance) stanza allows tuning the performance of different subsystems in Consul. 181 182 Add the performance configuration to the `consul.hcl` configuration file: 183 184 ```hcl 185 performance { 186 raft_multiplier = 1 187 } 188 ``` 189 190 - [`raft_multiplier`](/docs/agent/options.html#raft_multiplier) - An integer multiplier used by Consul servers to scale key Raft timing parameters. Setting this to a value of 1 will configure Raft to its highest-performance mode, equivalent to the default timing of Consul prior to 0.7, and is recommended for production Consul servers. 191 192 For more information on Raft tuning and the `raft_multiplier` setting, see the [server performance](/docs/guides/performance.html) documentation. 193 194 ### Telemetry stanza 195 196 The [`telemetry`](/docs/agent/options.html#telemetry) stanza specifies various configurations for Consul to publish metrics to upstream systems. 197 198 If you decide to configure Consul to publish telemtery data, you should review the [telemetry configuration section](/docs/agent/options.html#telemetry) of our documentation. 199 200 ### TLS configuration 201 202 The [Creating Certificates](/docs/guides/creating-certificates.html) guide provides instructions on configuring and enabling TLS. 203 204 ### Server configuration 205 206 Create a configuration file at `/etc/consul.d/server.hcl`: 207 208 ```text 209 sudo mkdir --parents /etc/consul.d 210 sudo touch /etc/consul.d/server.hcl 211 sudo chown --recursive consul:consul /etc/consul.d 212 sudo chmod 640 /etc/consul.d/server.hcl 213 ``` 214 215 Add this configuration to the `server.hcl` configuration file: 216 217 ~> **NOTE** Replace the `bootstrap_expect` value with the number of Consul servers you will use; three or five [is recommended](/docs/internals/consensus.html#deployment-table). 218 219 ```hcl 220 server = true 221 bootstrap_expect = 3 222 ``` 223 224 - [`server`](/docs/agent/options.html#_server) - This flag is used to control if an agent is in server or client mode. 225 - [`bootstrap-expect`](/docs/agent/options.html#_bootstrap_expect) - This flag provides the number of expected servers in the datacenter. Either this value should not be provided or the value must agree with other servers in the cluster. 226 227 ### Consul UI 228 229 Consul features a web-based user interface, allowing you to easily view all services, nodes, intentions and more using a graphical user interface, rather than the CLI or API. 230 231 ~> **NOTE** You should consider running the Consul UI on select Consul hosts rather than all hosts. 232 233 Optionally, add the UI configuration to the `server.hcl` configuration file to enable the Consul UI: 234 235 ```hcl 236 ui = true 237 ``` 238 239 ## Configure Consul (client) 240 241 Consul client agents typically require a subset of configuration required by Consul server agents. All Consul clients can use the `consul.hcl` file created when [configuring the Consul servers](#general-configuration). If you have added host-specific configuration such as identifiers, you will need to set these individually. 242 243 ## Start Consul 244 245 Enable and start Consul using the systemctl command responsible for controlling systemd managed services. Check the status of the consul service using systemctl. 246 247 ```text 248 sudo systemctl enable consul 249 sudo systemctl start consul 250 sudo systemctl status consul 251 ``` 252 253 ## Backups 254 255 Creating server backups is an important step in production deployments. Backups provide a mechanism for the server to recover from an outage (network loss, operator error, or a corrupted data directory). All agents write to the `-data-dir` before commit. This directory persists the local agent’s state and — in the case of servers — it also holds the Raft information. 256 257 Consul provides the [snapshot](/docs/commands/snapshot.html) command which can be run using the CLI command or the API. The `snapshot` command saves the point-in-time snapshot of the state of the Consul servers which includes KV entries, the service catalog, prepared queries, sessions, and ACL. 258 259 With [Consul Enterprise](/docs/commands/snapshot/agent.html), the `snapshot agent` command runs periodically and writes to local or remote storage (such as Amazon S3). 260 261 By default, all snapshots are taken using `consistent` mode where requests are forwarded to the leader which verifies that it is still in power before taking the snapshot. Snapshots will not be saved if the clusted is degraded or if no leader is available. To reduce the burden on the leader, it is possible to [run the snapshot](/docs/commands/snapshot/save.html) on any non-leader server using `stale` consistency mode: 262 263 ```text 264 consul snapshot save -stale backup.snap 265 ``` 266 267 This spreads the load across nodes at the possible expense of losing full consistency guarantees. Typically this means that a very small number of recent writes may not be included. The omitted writes are typically limited to data written in the last `100ms` or less from the recovery point. This is usually suitable for disaster recovery. However, the system can’t guarantee how stale this may be if executed against a partitioned server. 268 269 ## Next Steps 270 271 - Read [Monitoring Consul with Telegraf](/docs/guides/monitoring-telegraf.html) 272 for an example guide to monitoring Consul for improved operational visibility. 273 274 - Read [Outage Recovery](/docs/guides/outage.html) to learn the steps required 275 for recovery from a Consul outage due to a majority of server nodes in a 276 datacenter being lost. 277 278 - Read [Server Performance](/docs/guides/performance.html) to learn about 279 additional configuration that benefits production deployments.