github.com/outbrain/consul@v1.4.5/website/source/docs/guides/connect-production.md (about) 1 --- 2 layout: "docs" 3 page_title: "Connect in Production" 4 sidebar_current: "docs-guides-connect-production" 5 description: |- 6 This guide describes best practices for running Consul Connect in production. 7 --- 8 9 # Running Connect in Production 10 11 Consul Connect can secure all inter-service communication via mutual TLS. It's 12 designed to work with [minimal configuration out of the 13 box](https://learn.hashicorp.com/consul/getting-started/connect), but completing the [security 14 checklist](/docs/connect/security.html) and understanding the [Consul security 15 model](/docs/internals/security.html) are prerequisites for production 16 deployments. 17 18 This guide aims to walk through the steps required to ensure the security 19 guarantees hold. 20 21 We assume a cluster is already running with an appropriate number of servers and 22 clients and that other reference material like the 23 [deployment](/docs/guides/deployment.html) and 24 [performance](/docs/guides/performance.html) guides have been followed. 25 26 In practical deployments it may be necessary to incrementally adopt Connect 27 service-by-service. In this case some or all of the advice below may not apply 28 during the transition but should give a good understanding on which security 29 properties have been sacrificed in the interim. The final deployment goal should 30 be to end up compliant with all the advice below. 31 32 The steps we need to get to a secure Connect cluster are: 33 34 1. [Configure ACLs](#configure-acls) 35 1. [Configure Agent Transport Encryption](#configure-agent-transport-encryption) 36 1. [Bootstrap Certificate Authority](#bootstrap-certificate-authority) 37 1. [Setup Host Firewall](#setup-host-firewall) 38 1. [Configure Service Instances](#configure-service-instances) 39 40 ## Configure ACLs 41 42 Consul Connect's security is based on service identity. In practice the identity 43 of the service is only enforcible with sufficiently restrictive ACLs. 44 45 This section will not replace reading the full [ACL 46 guide](/docs/guides/acl.html) but will highlight the specific requirements 47 Connect relies on to ensure it's security properties. 48 49 A service's identity, in the form of an x.509 certificate, will only be issued 50 to an API client that has `service:write` permission for that service. In other 51 words, any client that has permission to _register_ an instance of a service 52 will be able to identify as that service and access all of the resources that that 53 service is allowed to access. 54 55 A secure ACL setup must meet these criteria: 56 57 1. **[ACL default 58 policy](/docs/agent/options.html#acl_default_policy) 59 must be `deny`.** It is technically sufficient to keep the default policy of 60 `allow` but add an explicit ACL denying anonymous `service:write`. Note 61 however that in this case the Connect intention graph will also default to 62 `allow` and explicit `deny` intentions will be needed to restrict service 63 access. Also note that explicit rules to limit who can manage intentions are 64 necessary in this case. It is assumed for the remainder of this guide that 65 ACL policy defaults to `deny`. 66 2. **Each service must have a distinct ACL token** that is restricted to 67 `service:write` only for the named service. Current Consul ACLs only support 68 prefix matching but in a near-future release we will allow exact name 69 matching. It is possible for all instances of the service to share the same 70 token although best practices is for each instance to get a unique token as 71 described below. 72 73 ### Fine Grained Enforcement 74 75 Connect intentions manage access based only on service identity so it is 76 sufficient for ACL tokens to only be unique per _service_ and shared between 77 instances. 78 79 It is much better though if ACL tokens are unique per service _instance_ because 80 it limits the blast radius of a compromise. 81 82 A future release of Connect will support revoking specific certificates that 83 have been issued. For example if a single node in a datacenter has been 84 compromised, it will be possible to find all certificates issued to the agent on 85 that node and revoke them. This will block all access to the intruder without 86 taking instances of the service(s) on other nodes offline too. 87 88 While this will work with service-unique tokens, there is nothing stopping an 89 attacker from obtaining certificates while spoofing the agent ID or other 90 identifier – these certificates will not appear to have been issued to the 91 compromised agent and so will not be revoked. 92 93 If every service instance has a unique token however, it will be possible to 94 revoke all certificates that were requested under that token. Assuming the 95 attacker can only access the tokens present on the compromised host, this 96 guarantees that any certificate they might have access to or requested directly 97 will be revoked. 98 99 In practice, managing per-instance tokens requires automated ACL provisioning, 100 for example using [HashiCorp's 101 Vault](https://www.vaultproject.io/docs/secrets/consul/index.html). 102 103 ## Configure Agent Transport Encryption 104 105 Consul's gossip (UDP) and RPC (TCP) communications need to be encrypted 106 otherwise attackers may be able to see ACL tokens while in flight 107 between the server and client agents (RPC) or between client agent and 108 application (HTTP). Certificate private keys never leave the host they 109 are used on but are delivered to the application or proxy over local 110 HTTP so local agent traffic should be encrypted where potentially 111 untrusted parties might be able to observe localhost agent API traffic. 112 113 Follow the [encryption documentation](/docs/agent/encryption.html) to ensure 114 both gossip encryption and RPC/HTTP TLS are configured securely. 115 116 For now client and server TLS certificates are still managed by manual 117 configuration. In the future we plan to automate more of that with the same 118 mechanisms Connect offers to user applications. 119 120 ## Bootstrap Certificate Authority 121 122 Consul Connect comes with a built in Certificate Authority (CA) that will 123 bootstrap by default when you first enable Connect on your servers. 124 125 To use the built-in CA, enable it in the server's configuration. 126 127 ```text 128 connect { 129 enabled = true 130 } 131 ``` 132 133 This config change requires a restart which you can perform one server at a time 134 to maintain availability in an existing cluster. 135 136 As soon as a server that has Connect enabled becomes the leader, it will 137 bootstrap a new CA and generate it's own private key which is written to the 138 Raft state. 139 140 Alternatively, an external private key can be provided via the [CA 141 configuration](/docs/connect/ca.html#specifying-a-private-key-and-root-certificate). 142 143 ### External CAs 144 145 Connect has been designed with a pluggable CA component so external CAs can be 146 integrated. We will expand the external CA systems that are supported in the 147 future and will allow seamless online migration to a different CA or 148 bootstrapping with an external CA. 149 150 For production workloads we recommend using [Vault or another external 151 CA](/docs/connect/ca.html#external-ca-certificate-authority-providers) once 152 available such that the root key is not stored within Consul state at all. 153 154 ## Setup Host Firewall 155 156 In order to enable inbound connections to connect proxies, you may need to 157 configure host or network firewalls to allow incoming connections to proxy 158 ports. 159 160 In addition to Consul agent's [communication 161 ports](/docs/agent/options.html#ports) any 162 [proxies](/docs/connect/proxies.html) will need to have 163 ports open to accept incoming connections. 164 165 If using [sidecar service 166 registration](/docs/connect/proxies/sidecar-service.html) Consul will by default 167 assign ports from [a configurable 168 range](/docs/agent/options.html#sidecar_min_port) the default range is 21000 - 169 21255. If this feature is used, the agent assumes all ports in that range are 170 both free to use (no other processes listening on them) and are exposed in the 171 firewall to accept connections from other service hosts. 172 173 It is possible to prevent automated port selection by [configuring 174 `sidecar_min_port` and 175 `sidecar_max_port`](/docs/agent/options.html#sidecar_min_port) to both be `0`, 176 forcing any sidecar service registrations to need an explicit port configured. 177 178 It then becomes the same problem as opening ports necessary for any other 179 application and might be managed by configuration management or a scheduler. 180 181 ## Configure Service Instances 182 183 With [necessary ACL tokens](#configure-acls) in place, all service registrations 184 need to have an appropriate ACL token present. 185 186 For on-disk configuration the `token` parameter of the service definition must 187 be set. 188 189 For registration via the API [the token is passed in the request 190 header](/api/index.html#acls) or by using the [Go 191 client configuration](https://godoc.org/github.com/hashicorp/consul/api#Config). 192 193 For examples of proxy service definitions see the [proxy 194 documentation](/docs/connect/proxies.html). 195 196 To avoid the overhead of a proxy, applications may [natively 197 integrate](/docs/connect/native.html) with connect. 198 199 ### Protect Application Listener 200 201 If using any kind of proxy for connect, the application must ensure no untrusted 202 connections can be made to it's unprotected listening port. This is typically 203 done by binding to `localhost` and only allowing loopback traffic, but may also 204 be achieved using firewall rules or network namespacing.