github.com/adityamillind98/nomad@v0.11.8/website/pages/docs/integrations/consul-connect.mdx (about)

     1  ---
     2  layout: docs
     3  page_title: Consul Connect
     4  sidebar_title: Consul Connect
     5  description: >-
     6    Learn how to use Nomad with Consul Connect to enable secure service to service
     7    communication
     8  ---
     9  
    10  # Consul Connect
    11  
    12  ~> **Note:** This guide requires Nomad 0.10.0 or later and Consul 1.6.0 or
    13  later.
    14  
    15  ~> **Note:** Nomad's Connect integration requires Linux network namespaces.
    16  Nomad Connect will not run on Windows or macOS.
    17  
    18  [Consul Connect](https://www.consul.io/docs/connect) provides
    19  service-to-service connection authorization and encryption using mutual
    20  Transport Layer Security (TLS). Applications can use sidecar proxies in a
    21  service mesh configuration to automatically establish TLS connections for
    22  inbound and outbound connections without being aware of Connect at all.
    23  
    24  # Nomad with Consul Connect Integration
    25  
    26  Nomad integrates with Consul to provide secure service-to-service communication
    27  between Nomad jobs and task groups. In order to support Consul Connect, Nomad
    28  adds a new networking mode for jobs that enables tasks in the same task group to
    29  share their networking stack. With a few changes to the job specification, job
    30  authors can opt into Connect integration. When Connect is enabled, Nomad will
    31  launch a proxy alongside the application in the job file. The proxy (Envoy)
    32  provides secure communication with other applications in the cluster.
    33  
    34  Nomad job specification authors can use Nomad's Consul Connect integration to
    35  implement [service segmentation](https://www.consul.io/segmentation.html) in a
    36  microservice architecture running in public clouds without having to directly
    37  manage TLS certificates. This is transparent to job specification authors as
    38  security features in Connect continue to work even as the application scales up
    39  or down or gets rescheduled by Nomad.
    40  
    41  For using the Consul Connect integration with Consul ACLs enabled, see the
    42  [Secure Nomad Jobs with Consul Connect](https://learn.hashicorp.com/nomad/consul-integration/nomad-connect-acl)
    43  guide.
    44  
    45  # Nomad Consul Connect Example
    46  
    47  The following section walks through an example to enable secure communication
    48  between a web dashboard and a backend counting service. The web dashboard and
    49  the counting service are managed by Nomad. Nomad additionally configures Envoy
    50  proxies to run along side these applications. The dashboard is configured to
    51  connect to the counting service via localhost on port 9001. The proxy is managed
    52  by Nomad, and handles mTLS communication to the counting service.
    53  
    54  ## Prerequisites
    55  
    56  ### Consul
    57  
    58  Connect integration with Nomad requires [Consul 1.6 or
    59  later.](https://releases.hashicorp.com/consul/1.6.0/) The Consul agent can be
    60  run in dev mode with the following command:
    61  
    62  **Note**: Nomad's Connect integration requires Consul in your `$PATH`
    63  
    64  ```shell-session
    65  $ consul agent -dev
    66  ```
    67  
    68  To use Connect on a non-dev Consul agent, you will minimally need to enable the
    69  GRPC port and set `connect` to enabled by adding some additional information to
    70  your Consul client configurations, depending on format.
    71  
    72  For HCL configurations:
    73  
    74  ```hcl
    75  # ...
    76  
    77  ports {
    78    grpc = 8502
    79  }
    80  
    81  connect {
    82    enabled = true
    83  }
    84  ```
    85  
    86  For JSON configurations:
    87  
    88  ```javascript
    89  {
    90    // ...
    91    "ports": {
    92      "grpc": 8502
    93    },
    94    "connect": {
    95       "enabled": true
    96    }
    97  }
    98  ```
    99  
   100  ### Nomad
   101  
   102  Nomad must schedule onto a routable interface in order for the proxies to
   103  connect to each other. The following steps show how to start a Nomad dev agent
   104  configured for Connect.
   105  
   106  ```shell-session
   107  $ sudo nomad agent -dev-connect
   108  ```
   109  
   110  ### CNI Plugins
   111  
   112  Nomad uses CNI plugins to configure the network namespace used to secure the
   113  Consul Connect sidecar proxy. All Nomad client nodes using network namespaces
   114  must have CNI plugins installed.
   115  
   116  The following commands install CNI plugins:
   117  
   118  ```shell-session
   119  $ curl -L -o cni-plugins.tgz https://github.com/containernetworking/plugins/releases/download/v0.8.6/cni-plugins-linux-amd64-v0.8.6.tgz
   120  $ sudo mkdir -p /opt/cni/bin
   121  $ sudo tar -C /opt/cni/bin -xzf cni-plugins.tgz
   122  ```
   123  
   124  Ensure the your Linux operating system distribution has been configured to allow
   125  container traffic through the bridge network to be routed via iptables. These
   126  tunables can be set as follows:
   127  
   128  ```shell-session
   129  $ echo 1 > /proc/sys/net/bridge/bridge-nf-call-arptables
   130  $ echo 1 > /proc/sys/net/bridge/bridge-nf-call-ip6tables
   131  $ echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
   132  ```
   133  
   134  To preserve these settings on startup of a client node, add a file including the
   135  following to `/etc/sysctl.d/` or remove the file your Linux distribution puts in
   136  that directory.
   137  
   138  ```
   139  net.bridge.bridge-nf-call-arptables = 1
   140  net.bridge.bridge-nf-call-ip6tables = 1
   141  net.bridge.bridge-nf-call-iptables = 1
   142  ```
   143  
   144  ## Run the Connect-enabled Services
   145  
   146  Once Nomad and Consul are running, submit the following Connect-enabled services
   147  to Nomad by copying the HCL into a file named `connect.nomad` and running:
   148  `nomad run connect.nomad`
   149  
   150  ```hcl
   151  job "countdash" {
   152    datacenters = ["dc1"]
   153  
   154    group "api" {
   155      network {
   156        mode = "bridge"
   157      }
   158  
   159      service {
   160        name = "count-api"
   161        port = "9001"
   162  
   163        connect {
   164          sidecar_service {}
   165        }
   166      }
   167  
   168      task "web" {
   169        driver = "docker"
   170  
   171        config {
   172          image = "hashicorpnomad/counter-api:v1"
   173        }
   174      }
   175    }
   176  
   177    group "dashboard" {
   178      network {
   179        mode = "bridge"
   180  
   181        port "http" {
   182          static = 9002
   183          to     = 9002
   184        }
   185      }
   186  
   187      service {
   188        name = "count-dashboard"
   189        port = "9002"
   190  
   191        connect {
   192          sidecar_service {
   193            proxy {
   194              upstreams {
   195                destination_name = "count-api"
   196                local_bind_port  = 8080
   197              }
   198            }
   199          }
   200        }
   201      }
   202  
   203      task "dashboard" {
   204        driver = "docker"
   205  
   206        env {
   207          COUNTING_SERVICE_URL = "http://${NOMAD_UPSTREAM_ADDR_count_api}"
   208        }
   209  
   210        config {
   211          image = "hashicorpnomad/counter-dashboard:v1"
   212        }
   213      }
   214    }
   215  }
   216  ```
   217  
   218  The job contains two task groups: an API service and a web frontend.
   219  
   220  ### API Service
   221  
   222  The API service is defined as a task group with a bridge network:
   223  
   224  ```hcl
   225    group "api" {
   226      network {
   227        mode = "bridge"
   228      }
   229  
   230      # ...
   231    }
   232  ```
   233  
   234  Since the API service is only accessible via Consul Connect, it does not define
   235  any ports in its network. The service stanza enables Connect:
   236  
   237  ```hcl
   238    group "api" {
   239  
   240      # ...
   241  
   242      service {
   243        name = "count-api"
   244        port = "9001"
   245  
   246        connect {
   247          sidecar_service {}
   248        }
   249      }
   250  
   251      # ...
   252  
   253    }
   254  ```
   255  
   256  The `port` in the service stanza is the port the API service listens on. The
   257  Envoy proxy will automatically route traffic to that port inside the network
   258  namespace.
   259  
   260  ### Web Frontend
   261  
   262  The web frontend is defined as a task group with a bridge network and a static
   263  forwarded port:
   264  
   265  ```hcl
   266    group "dashboard" {
   267      network {
   268        mode = "bridge"
   269  
   270        port "http" {
   271          static = 9002
   272          to     = 9002
   273        }
   274      }
   275  
   276      # ...
   277  
   278    }
   279  ```
   280  
   281  The `static = 9002` parameter requests the Nomad scheduler reserve port 9002 on
   282  a host network interface. The `to = 9002` parameter forwards that host port to
   283  port 9002 inside the network namespace.
   284  
   285  This allows you to connect to the web frontend in a browser by visiting
   286  `http://<host_ip>:9002` as show below:
   287  
   288  [![Count Dashboard][count-dashboard]][count-dashboard]
   289  
   290  The web frontend connects to the API service via Consul Connect:
   291  
   292  ```hcl
   293      service {
   294        name = "count-dashboard"
   295        port = "9002"
   296  
   297        connect {
   298          sidecar_service {
   299            proxy {
   300              upstreams {
   301                destination_name = "count-api"
   302                local_bind_port  = 8080
   303              }
   304            }
   305          }
   306        }
   307      }
   308  ```
   309  
   310  The `upstreams` stanza defines the remote service to access (`count-api`) and
   311  what port to expose that service on inside the network namespace (`8080`).
   312  
   313  The web frontend is configured to communicate with the API service with an
   314  environment variable:
   315  
   316  ```hcl
   317        env {
   318          COUNTING_SERVICE_URL = "http://${NOMAD_UPSTREAM_ADDR_count_api}"
   319        }
   320  ```
   321  
   322  The web frontend is configured via the `$COUNTING_SERVICE_URL`, so you must
   323  interpolate the upstream's address into that environment variable. Note that
   324  dashes (`-`) are converted to underscores (`_`) in environment variables so
   325  `count-api` becomes `count_api`.
   326  
   327  ## Limitations
   328  
   329  - The `consul` binary must be present in Nomad's `$PATH` to run the Envoy
   330    proxy sidecar on client nodes.
   331  - Consul Connect Native is not yet supported ([#6083][gh6083]).
   332  - Only the Docker, `exec`, `raw_exec`, and `java` drivers support network namespaces
   333    and Connect.
   334  - Changes to the `connect` stanza may not properly trigger a job update
   335    ([#6459][gh6459]). Changing a `meta` variable is the suggested workaround as
   336    this will always cause an update to occur.
   337  - Consul Connect and network namespaces are only supported on Linux.
   338  
   339  [count-dashboard]: /img/count-dashboard.png
   340  [gh6083]: https://github.com/hashicorp/nomad/issues/6083
   341  [gh6120]: https://github.com/hashicorp/nomad/issues/6120
   342  [gh6701]: https://github.com/hashicorp/nomad/issues/6701
   343  [gh6459]: https://github.com/hashicorp/nomad/issues/6459