github.com/uchennaokeke444/nomad@v0.11.8/website/pages/docs/integrations/consul-connect.mdx (about) 1 --- 2 layout: docs 3 page_title: Consul Connect 4 sidebar_title: Consul Connect 5 description: >- 6 Learn how to use Nomad with Consul Connect to enable secure service to service 7 communication 8 --- 9 10 # Consul Connect 11 12 ~> **Note:** This guide requires Nomad 0.10.0 or later and Consul 1.6.0 or 13 later. 14 15 ~> **Note:** Nomad's Connect integration requires Linux network namespaces. 16 Nomad Connect will not run on Windows or macOS. 17 18 [Consul Connect](https://www.consul.io/docs/connect) provides 19 service-to-service connection authorization and encryption using mutual 20 Transport Layer Security (TLS). Applications can use sidecar proxies in a 21 service mesh configuration to automatically establish TLS connections for 22 inbound and outbound connections without being aware of Connect at all. 23 24 # Nomad with Consul Connect Integration 25 26 Nomad integrates with Consul to provide secure service-to-service communication 27 between Nomad jobs and task groups. In order to support Consul Connect, Nomad 28 adds a new networking mode for jobs that enables tasks in the same task group to 29 share their networking stack. With a few changes to the job specification, job 30 authors can opt into Connect integration. When Connect is enabled, Nomad will 31 launch a proxy alongside the application in the job file. The proxy (Envoy) 32 provides secure communication with other applications in the cluster. 33 34 Nomad job specification authors can use Nomad's Consul Connect integration to 35 implement [service segmentation](https://www.consul.io/segmentation.html) in a 36 microservice architecture running in public clouds without having to directly 37 manage TLS certificates. This is transparent to job specification authors as 38 security features in Connect continue to work even as the application scales up 39 or down or gets rescheduled by Nomad. 40 41 For using the Consul Connect integration with Consul ACLs enabled, see the 42 [Secure Nomad Jobs with Consul Connect](https://learn.hashicorp.com/nomad/consul-integration/nomad-connect-acl) 43 guide. 44 45 # Nomad Consul Connect Example 46 47 The following section walks through an example to enable secure communication 48 between a web dashboard and a backend counting service. The web dashboard and 49 the counting service are managed by Nomad. Nomad additionally configures Envoy 50 proxies to run along side these applications. The dashboard is configured to 51 connect to the counting service via localhost on port 9001. The proxy is managed 52 by Nomad, and handles mTLS communication to the counting service. 53 54 ## Prerequisites 55 56 ### Consul 57 58 Connect integration with Nomad requires [Consul 1.6 or 59 later.](https://releases.hashicorp.com/consul/1.6.0/) The Consul agent can be 60 run in dev mode with the following command: 61 62 **Note**: Nomad's Connect integration requires Consul in your `$PATH` 63 64 ```shell-session 65 $ consul agent -dev 66 ``` 67 68 To use Connect on a non-dev Consul agent, you will minimally need to enable the 69 GRPC port and set `connect` to enabled by adding some additional information to 70 your Consul client configurations, depending on format. 71 72 For HCL configurations: 73 74 ```hcl 75 # ... 76 77 ports { 78 grpc = 8502 79 } 80 81 connect { 82 enabled = true 83 } 84 ``` 85 86 For JSON configurations: 87 88 ```javascript 89 { 90 // ... 91 "ports": { 92 "grpc": 8502 93 }, 94 "connect": { 95 "enabled": true 96 } 97 } 98 ``` 99 100 ### Nomad 101 102 Nomad must schedule onto a routable interface in order for the proxies to 103 connect to each other. The following steps show how to start a Nomad dev agent 104 configured for Connect. 105 106 ```shell-session 107 $ sudo nomad agent -dev-connect 108 ``` 109 110 ### CNI Plugins 111 112 Nomad uses CNI plugins to configure the network namespace used to secure the 113 Consul Connect sidecar proxy. All Nomad client nodes using network namespaces 114 must have CNI plugins installed. 115 116 The following commands install CNI plugins: 117 118 ```shell-session 119 $ curl -L -o cni-plugins.tgz https://github.com/containernetworking/plugins/releases/download/v0.8.6/cni-plugins-linux-amd64-v0.8.6.tgz 120 $ sudo mkdir -p /opt/cni/bin 121 $ sudo tar -C /opt/cni/bin -xzf cni-plugins.tgz 122 ``` 123 124 Ensure the your Linux operating system distribution has been configured to allow 125 container traffic through the bridge network to be routed via iptables. These 126 tunables can be set as follows: 127 128 ```shell-session 129 $ echo 1 > /proc/sys/net/bridge/bridge-nf-call-arptables 130 $ echo 1 > /proc/sys/net/bridge/bridge-nf-call-ip6tables 131 $ echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables 132 ``` 133 134 To preserve these settings on startup of a client node, add a file including the 135 following to `/etc/sysctl.d/` or remove the file your Linux distribution puts in 136 that directory. 137 138 ``` 139 net.bridge.bridge-nf-call-arptables = 1 140 net.bridge.bridge-nf-call-ip6tables = 1 141 net.bridge.bridge-nf-call-iptables = 1 142 ``` 143 144 ## Run the Connect-enabled Services 145 146 Once Nomad and Consul are running, submit the following Connect-enabled services 147 to Nomad by copying the HCL into a file named `connect.nomad` and running: 148 `nomad run connect.nomad` 149 150 ```hcl 151 job "countdash" { 152 datacenters = ["dc1"] 153 154 group "api" { 155 network { 156 mode = "bridge" 157 } 158 159 service { 160 name = "count-api" 161 port = "9001" 162 163 connect { 164 sidecar_service {} 165 } 166 } 167 168 task "web" { 169 driver = "docker" 170 171 config { 172 image = "hashicorpnomad/counter-api:v1" 173 } 174 } 175 } 176 177 group "dashboard" { 178 network { 179 mode = "bridge" 180 181 port "http" { 182 static = 9002 183 to = 9002 184 } 185 } 186 187 service { 188 name = "count-dashboard" 189 port = "9002" 190 191 connect { 192 sidecar_service { 193 proxy { 194 upstreams { 195 destination_name = "count-api" 196 local_bind_port = 8080 197 } 198 } 199 } 200 } 201 } 202 203 task "dashboard" { 204 driver = "docker" 205 206 env { 207 COUNTING_SERVICE_URL = "http://${NOMAD_UPSTREAM_ADDR_count_api}" 208 } 209 210 config { 211 image = "hashicorpnomad/counter-dashboard:v1" 212 } 213 } 214 } 215 } 216 ``` 217 218 The job contains two task groups: an API service and a web frontend. 219 220 ### API Service 221 222 The API service is defined as a task group with a bridge network: 223 224 ```hcl 225 group "api" { 226 network { 227 mode = "bridge" 228 } 229 230 # ... 231 } 232 ``` 233 234 Since the API service is only accessible via Consul Connect, it does not define 235 any ports in its network. The service stanza enables Connect: 236 237 ```hcl 238 group "api" { 239 240 # ... 241 242 service { 243 name = "count-api" 244 port = "9001" 245 246 connect { 247 sidecar_service {} 248 } 249 } 250 251 # ... 252 253 } 254 ``` 255 256 The `port` in the service stanza is the port the API service listens on. The 257 Envoy proxy will automatically route traffic to that port inside the network 258 namespace. 259 260 ### Web Frontend 261 262 The web frontend is defined as a task group with a bridge network and a static 263 forwarded port: 264 265 ```hcl 266 group "dashboard" { 267 network { 268 mode = "bridge" 269 270 port "http" { 271 static = 9002 272 to = 9002 273 } 274 } 275 276 # ... 277 278 } 279 ``` 280 281 The `static = 9002` parameter requests the Nomad scheduler reserve port 9002 on 282 a host network interface. The `to = 9002` parameter forwards that host port to 283 port 9002 inside the network namespace. 284 285 This allows you to connect to the web frontend in a browser by visiting 286 `http://<host_ip>:9002` as show below: 287 288 [![Count Dashboard][count-dashboard]][count-dashboard] 289 290 The web frontend connects to the API service via Consul Connect: 291 292 ```hcl 293 service { 294 name = "count-dashboard" 295 port = "9002" 296 297 connect { 298 sidecar_service { 299 proxy { 300 upstreams { 301 destination_name = "count-api" 302 local_bind_port = 8080 303 } 304 } 305 } 306 } 307 } 308 ``` 309 310 The `upstreams` stanza defines the remote service to access (`count-api`) and 311 what port to expose that service on inside the network namespace (`8080`). 312 313 The web frontend is configured to communicate with the API service with an 314 environment variable: 315 316 ```hcl 317 env { 318 COUNTING_SERVICE_URL = "http://${NOMAD_UPSTREAM_ADDR_count_api}" 319 } 320 ``` 321 322 The web frontend is configured via the `$COUNTING_SERVICE_URL`, so you must 323 interpolate the upstream's address into that environment variable. Note that 324 dashes (`-`) are converted to underscores (`_`) in environment variables so 325 `count-api` becomes `count_api`. 326 327 ## Limitations 328 329 - The `consul` binary must be present in Nomad's `$PATH` to run the Envoy 330 proxy sidecar on client nodes. 331 - Consul Connect Native is not yet supported ([#6083][gh6083]). 332 - Only the Docker, `exec`, `raw_exec`, and `java` drivers support network namespaces 333 and Connect. 334 - Changes to the `connect` stanza may not properly trigger a job update 335 ([#6459][gh6459]). Changing a `meta` variable is the suggested workaround as 336 this will always cause an update to occur. 337 - Consul Connect and network namespaces are only supported on Linux. 338 339 [count-dashboard]: /img/count-dashboard.png 340 [gh6083]: https://github.com/hashicorp/nomad/issues/6083 341 [gh6120]: https://github.com/hashicorp/nomad/issues/6120 342 [gh6701]: https://github.com/hashicorp/nomad/issues/6701 343 [gh6459]: https://github.com/hashicorp/nomad/issues/6459