github.com/Ilhicas/nomad@v1.0.4-0.20210304152020-e86851182bc3/website/content/docs/configuration/index.mdx (about) 1 --- 2 layout: docs 3 page_title: Agent Configuration 4 sidebar_title: Configuration 5 description: Learn about the configuration options available for the Nomad agent. 6 --- 7 8 # Nomad Configuration 9 10 Nomad agents have a variety of parameters that can be specified via 11 configuration files or command-line flags. Configuration files are written in 12 [HCL][hcl]. Nomad can read and combine parameters from multiple configuration 13 files or directories to configure the Nomad agent. 14 15 ## Load Order and Merging 16 17 The Nomad agent supports multiple configuration files, which can be provided 18 using the `-config` CLI flag. The flag can accept either a file or folder. In 19 the case of a folder, any `.hcl` and `.json` files in the folder will be loaded 20 and merged in lexicographical order. Directories are not loaded recursively. 21 22 For example: 23 24 ```shell-session 25 $ nomad agent -config=server.conf -config=/etc/nomad -config=extra.json 26 ``` 27 28 This will load configuration from `server.conf`, from `.hcl` and `.json` files 29 under `/etc/nomad`, and finally from `extra.json`. 30 31 As each file is processed, its contents are merged into the existing 32 configuration. When merging, any non-empty values from the latest config file 33 will append or replace parameters in the current configuration. An empty value 34 means `""` for strings, `0` for integer or float values, and `false` for 35 booleans. Since empty values are ignored you cannot disable a parameter like 36 `server` mode once you've enabled it. 37 38 Please note that `plugin` blocks for a given plugin always replace and 39 override each other, rather than being merged. 40 41 Here is an example Nomad agent configuration that runs in both client and server 42 mode. 43 44 ```hcl 45 data_dir = "/var/lib/nomad" 46 47 bind_addr = "0.0.0.0" # the default 48 49 advertise { 50 # Defaults to the first private IP address. 51 http = "1.2.3.4" 52 rpc = "1.2.3.4" 53 serf = "1.2.3.4:5648" # non-default ports may be specified 54 } 55 56 server { 57 enabled = true 58 bootstrap_expect = 3 59 } 60 61 client { 62 enabled = true 63 } 64 65 plugin "raw_exec" { 66 config { 67 enabled = true 68 } 69 } 70 71 consul { 72 address = "1.2.3.4:8500" 73 } 74 75 ``` 76 77 ~> Note that it is strongly recommended **not** to operate a node as both 78 `client` and `server`, although this is supported to simplify development and 79 testing. 80 81 ## General Parameters 82 83 - `acl` `(`[`ACL`]`: nil)` - Specifies configuration which is specific to ACLs. 84 85 - `addresses` `(Addresses: see below)` - Specifies the bind address for 86 individual network services. Any values configured in this stanza take 87 precedence over the default [bind_addr](#bind_addr). These values should be 88 specified in IP format without a port (ex. `"0.0.0.0"`). To set the port, 89 see the [`ports`](#ports) field. The values support [go-sockaddr/template 90 format][go-sockaddr/template]. 91 92 - `http` - The address the HTTP server is bound to. This is the most common 93 bind address to change. 94 95 - `rpc` - The address to bind the internal RPC interfaces to. Should be 96 exposed only to other cluster members if possible. 97 98 - `serf` - The address used to bind the gossip layer to. Both a TCP and UDP 99 listener will be exposed on this address. Should be exposed only to other 100 cluster members if possible. 101 102 - `advertise` `(Advertise: see below)` - Specifies the advertise address for 103 individual network services. This can be used to advertise a different address 104 to the peers of a server or a client node to support more complex network 105 configurations such as NAT. This configuration is optional, and defaults to 106 the bind address of the specific network service if it is not provided. Any 107 values configured in this stanza take precedence over the default 108 [bind_addr](#bind_addr). 109 110 If the bind address is `0.0.0.0` then the IP address of the default private 111 network interface advertised. The `advertise` values may include an 112 alternate port, but otherwise default to the port used by the bind address. 113 The values support [go-sockaddr/template format][go-sockaddr/template]. 114 115 - `http` - The address to advertise for the HTTP interface. This should be 116 reachable by all the nodes from which end users are going to use the Nomad 117 CLI tools. 118 119 - `rpc` - The address advertised to Nomad client nodes. This allows 120 advertising a different RPC address than is used by Nomad Servers such that 121 the clients can connect to the Nomad servers if they are behind a NAT. 122 123 - `serf` - The address advertised for the gossip layer. This address must be 124 reachable from all server nodes. It is not required that clients can reach 125 this address. Nomad servers will communicate to each other over RPC using 126 the advertised Serf IP and advertised RPC Port. 127 128 - `audit` `(`[`Audit`]`: nil)` - Enterprise-only. Specifies audit logging 129 configuration. 130 131 - `bind_addr` `(string: "0.0.0.0")` - Specifies which address the Nomad 132 agent should bind to for network services, including the HTTP interface as 133 well as the internal gossip protocol and RPC mechanism. This should be 134 specified in IP format, and can be used to easily bind all network services to 135 the same address. It is also possible to bind the individual services to 136 different addresses using the [addresses](#addresses) configuration option. 137 Dev mode (`-dev`) defaults to localhost. 138 The value supports [go-sockaddr/template format][go-sockaddr/template]. 139 140 - `client` `(`[`Client`]`: nil)` - Specifies configuration which is specific 141 to the Nomad client. 142 143 - `consul` `(`[`Consul`]`: nil)` - Specifies configuration for 144 connecting to Consul. 145 146 - `datacenter` `(string: "dc1")` - Specifies the data center of the local agent. 147 All members of a datacenter should share a local LAN connection. 148 149 - `data_dir` `(string: required)` - Specifies a local directory used to store 150 agent state. Client nodes use this directory by default to store temporary 151 allocation data as well as cluster information. Server nodes use this 152 directory to store cluster state, including the replicated log and snapshot 153 data. This must be specified as an absolute path. 154 155 ~> **WARNING**: This directory **must not** be set to a directory that is 156 [included in the chroot](/docs/drivers/exec#chroot) if you use the 157 [`exec`](/docs/drivers/exec) driver. 158 159 - `disable_anonymous_signature` `(bool: false)` - Specifies if Nomad should 160 provide an anonymous signature for de-duplication with the update check. 161 162 - `disable_update_check` `(bool: false)` - Specifies if Nomad should not check 163 for updates and security bulletins. _This defaults to `true` in Nomad Enterprise._ 164 165 - `enable_debug` `(bool: false)` - Specifies if the debugging HTTP endpoints 166 should be enabled. These endpoints can be used with profiling tools to dump 167 diagnostic information about Nomad's internals. 168 169 - `enable_syslog` `(bool: false)` - Specifies if the agent should log to syslog. 170 This option only works on Unix based systems. 171 172 - `http_api_response_headers` `(map<string|string>: nil)` - Specifies 173 user-defined headers to add to the HTTP API responses. 174 175 - `leave_on_interrupt` `(bool: false)` - Specifies if the agent should 176 gracefully leave when receiving the interrupt signal. By default, the agent 177 will exit forcefully on any signal. This value should only be set to true on 178 server agents if it is expected that a terminated server instance will never 179 join the cluster again. 180 181 - `leave_on_terminate` `(bool: false)` - Specifies if the agent should 182 gracefully leave when receiving the terminate signal. By default, the agent 183 will exit forcefully on any signal. This value should only be set to true on 184 server agents if it is expected that a terminated server instance will never 185 join the cluster again. 186 187 - `limits` - Available in Nomad 0.10.3 and later, this is a nested object that 188 configures limits that are enforced by the agent. The following parameters 189 are available: 190 191 - `https_handshake_timeout` `(string: "5s")` - Configures the limit for how 192 long the HTTPS server in both client and server agents will wait for a 193 client to complete a TLS handshake. This should be kept conservative as it 194 limits how many connections an unauthenticated attacker can open if 195 [`tls.http = true`][tls] is being used (strongly recommended in 196 production). Default value is `5s`. `0` disables HTTP handshake timeouts. 197 198 - `http_max_conns_per_client` `(int: 100)` - Configures a limit of how many 199 concurrent TCP connections a single client IP address is allowed to open to 200 the agent's HTTP server. This affects the HTTP servers in both client and 201 server agents. Default value is `100`. `0` disables HTTP connection limits. 202 203 - `rpc_handshake_timeout` `(string: "5s")` - Configures the limit for how 204 long servers will wait after a client TCP connection is established before 205 they complete the connection handshake. When TLS is used, the same timeout 206 applies to the TLS handshake separately from the initial protocol 207 negotiation. All Nomad clients should perform this immediately on 208 establishing a new connection. This should be kept conservative as it 209 limits how many connections an unauthenticated attacker can open if 210 TLS is being using to authenticate clients (strongly recommended in 211 production). When `tls.rpc` is true on servers, this limits how long the 212 connection and associated goroutines will be held open before the client 213 successfully authenticates. Default value is `5s`. `0` disables RPC handshake 214 timeouts. 215 216 - `rpc_max_conns_per_client` `(int: 100)` - Configures a limit of how 217 many concurrent TCP connections a single source IP address is allowed 218 to open to a single server. Client agents do not accept RPC TCP connections 219 directly and therefore are not affected. It affects both clients connections 220 and other server connections. Nomad clients multiplex many RPC calls over a 221 single TCP connection, except for streaming endpoints such as [log 222 streaming][log-api] which require their own connection when routed through 223 servers. A server needs at least 2 TCP connections (1 Raft, 1 RPC) per peer 224 server locally and in any federated region. Servers also need a TCP connection 225 per routed streaming endpoint concurrently in use. Only operators use streaming 226 endpoints; as of 0.10.3 Nomad client code does not. A reasonably low limit 227 significantly reduces the ability of an unauthenticated attacker to consume 228 unbounded resources by holding open many connections. You may need to 229 increase this if WAN federated servers connect via proxies or NAT gateways 230 or similar causing many legitimate connections from a single source IP. 231 Default value is `100` which is designed to support the majority of users. 232 `0` disables RPC connection limits. `26` is the minimum as `20` connections 233 are always reserved for non-streaming connections (Raft and RPC) to ensure 234 streaming RPCs do not prevent normal server operation. This minimum may be 235 lowered in the future when streaming RPCs no longer require their own TCP 236 connection. 237 238 - `log_level` `(string: "INFO")` - Specifies the verbosity of logs the Nomad 239 agent will output. Valid log levels include `WARN`, `INFO`, or `DEBUG` in 240 increasing order of verbosity. 241 242 - `log_json` `(bool: false)` - Output logs in a JSON format. 243 244 - `log_file` `(string: "")` - Specifies the path for logging. If the path 245 does not includes a filename, the filename defaults to "nomad-{timestamp}.log". 246 This setting can be combined with `log_rotate_bytes` and `log_rotate_duration` 247 for a fine-grained log rotation control. 248 249 - `log_rotate_bytes` `(int: 0)` - Specifies the number of bytes that should be 250 written to a log before it needs to be rotated. Unless specified, there is no 251 limit to the number of bytes that can be written to a log file. 252 253 - `log_rotate_duration` `(duration: "24h")` - Specifies the maximum duration a 254 log should be written to before it needs to be rotated. Must be a duration 255 value such as 30s. 256 257 - `log_rotate_max_files` `(int: 0)` - Specifies the maximum number of older 258 log file archives to keep, not including the log file currently being 259 written. If set to 0 no files are ever deleted. Note that the total number 260 of log files, for each of `stderr` and `stdout`, will be 1 greater than the 261 `log_rotate_max_files` value. 262 263 - `name` `(string: [hostname])` - Specifies the name of the local node. This 264 value is used to identify individual agents. When specified on a server, the 265 name must be unique within the region. 266 267 - `plugin_dir` `(string: "[data_dir]/plugins")` - Specifies the directory to 268 use for looking up plugins. By default, this is the top-level 269 [data_dir](#data_dir) suffixed with "plugins", like `"/opt/nomad/plugins"`. 270 This must be an absolute path. 271 272 - `plugin` `(`[`Plugin`]`: nil)` - Specifies configuration for a 273 specific plugin. The plugin stanza may be repeated, once for each plugin being 274 configured. The key of the stanza is the plugin's executable name relative to 275 the [plugin_dir](#plugin_dir). 276 277 - `ports` `(Port: see below)` - Specifies the network ports used for different 278 services required by the Nomad agent. 279 280 - `http` - The port used to run the HTTP server. 281 282 - `rpc` - The port used for internal RPC communication between 283 agents and servers, and for inter-server traffic for the consensus algorithm 284 (raft). 285 286 - `serf` - The port used for the gossip protocol for cluster 287 membership. Both TCP and UDP should be routable between the server nodes on 288 this port. 289 290 The default values are: 291 292 ```hcl 293 ports { 294 http = 4646 295 rpc = 4647 296 serf = 4648 297 } 298 ``` 299 300 - `region` `(string: "global")` - Specifies the region the Nomad agent is a 301 member of. A region typically maps to a geographic region, for example `us`, 302 with potentially multiple zones, which map to [datacenters](#datacenter) such 303 as `us-west` and `us-east`. 304 305 - `sentinel` `(`[`Sentinel`]`: nil)` - Specifies configuration for Sentinel 306 policies. 307 308 - `server` `(`[`Server`]`: nil)` - Specifies configuration which is specific 309 to the Nomad server. 310 311 - `syslog_facility` `(string: "LOCAL0")` - Specifies the syslog facility to 312 write to. This has no effect unless `enable_syslog` is true. 313 314 - `tls` `(`[`TLS`][tls]`: nil)` - Specifies configuration for TLS. 315 316 - `vault` `(`[`Vault`]`: nil)` - Specifies configuration for 317 connecting to Vault. 318 319 ## Configuration Reload 320 321 You can send the Nomad process a `SIGHUP` signal to reload a limited subset of 322 its configuration. The fields that currently support reloading are: 323 324 - [`log_level`](#log_level): the log level is reloaded but not any other 325 logging configuration value. 326 - [`tls`][tls-reload]: note this only reloads the TLS configuration between 327 Nomad agents (servers and clients), and not the TLS configuration for 328 communication with Consul or Vault. 329 - [`vault`][vault-reload]: note this only reloads the TLS configuration 330 between Nomad and Vault, but not other configuration values. 331 332 In order to reload any other configuration values, you must restart the Nomad 333 agent. 334 335 If the Nomad agent receives a `SIGHUP` during initialization, it may crash 336 (see [GH-3885]). Ensure that the Nomad agent is able to receive RPC traffic 337 before attempting to reload its configuration. 338 339 ## Examples 340 341 ### Custom Region and Datacenter 342 343 This example shows configuring a custom region and data center for the Nomad 344 agent: 345 346 ```hcl 347 region = "europe" 348 datacenter = "ams" 349 ``` 350 351 ### Enable CORS 352 353 This example shows how to enable CORS on the HTTP API endpoints: 354 355 ```hcl 356 http_api_response_headers { 357 "Access-Control-Allow-Origin" = "*" 358 } 359 ``` 360 361 [`acl`]: /docs/configuration/acl 'Nomad Agent ACL Configuration' 362 [`audit`]: /docs/configuration/audit 'Nomad Agent Audit Logging Configuration' 363 [`client`]: /docs/configuration/client 'Nomad Agent client Configuration' 364 [`consul`]: /docs/configuration/consul 'Nomad Agent consul Configuration' 365 [`plugin`]: /docs/configuration/plugin 'Nomad Agent Plugin Configuration' 366 [`sentinel`]: /docs/configuration/sentinel 'Nomad Agent sentinel Configuration' 367 [`server`]: /docs/configuration/server 'Nomad Agent server Configuration' 368 [tls]: /docs/configuration/tls 'Nomad Agent tls Configuration' 369 [`vault`]: /docs/configuration/vault 'Nomad Agent vault Configuration' 370 [go-sockaddr/template]: https://godoc.org/github.com/hashicorp/go-sockaddr/template 371 [log-api]: /api-docs/client#stream-logs 372 [hcl]: https://github.com/hashicorp/hcl 'HashiCorp Configuration Language' 373 [tls-reload]: /docs/configuration/tls#tls-configuration-reloads 374 [vault-reload]: /docs/configuration/vault#vault-configuration-reloads 375 [GH-3885]: https://github.com/hashicorp/nomad/issues/3885