github.com/dkerwin/nomad@v0.3.3-0.20160525181927-74554135514b/website/source/docs/agent/config.html.md (about) 1 --- 2 layout: "docs" 3 page_title: "Configuration" 4 sidebar_current: "docs-agent-config" 5 description: |- 6 Learn about the configuration options available for the Nomad agent. 7 --- 8 9 # Configuration 10 11 Nomad agents have a variety of options that can be specified via configuration 12 files or command-line flags. Config files must be written in 13 [HCL](https://github.com/hashicorp/hcl) or JSON syntax. Nomad can read and 14 combine options from multiple configuration files or directories to configure 15 the Nomad agent. 16 17 ## Loading Configuration Files 18 19 When specifying multiple config file options on the command-line, the files are 20 loaded in the order they are specified. For example: 21 22 nomad agent -config server.conf -config /etc/nomad -config extra.json 23 24 Will load configuration from `server.conf`, from `.hcl` and `.json` files under 25 `/etc/nomad`, and finally from `extra.json`. 26 27 Configuration files in directories are loaded alphabetically. With the 28 directory option, only files ending with the `.hcl` or `.json` extensions are 29 used. Directories are not loaded recursively. 30 31 As each file is processed, its contents are merged into the existing 32 configuration. When merging, any non-empty values from the latest config file 33 will append or replace options in the current configuration. An empty value 34 means `""` for strings, `0` for integer or float values, and `false` for 35 booleans. Since empty values are ignored you cannot disable an option like 36 server mode once you've enabled it. 37 38 Complex data types like arrays or maps are usually merged. [Some configuration 39 options](#cli) can also be specified using the command-line interface. Please 40 refer to the sections below for the details of each option. 41 42 ## Configuration Syntax 43 44 The preferred configuration syntax is HCL, which supports comments, but you can 45 also use JSON. Below is an example configuration file in HCL syntax. 46 47 ``` 48 bind_addr = "0.0.0.0" 49 data_dir = "/var/lib/nomad" 50 51 advertise { 52 # We need to specify our host's IP because we can't 53 # advertise 0.0.0.0 to other nodes in our cluster. 54 rpc = "1.2.3.4:4647" 55 } 56 57 server { 58 enabled = true 59 bootstrap_expect = 3 60 } 61 62 client { 63 enabled = true 64 network_speed = 10 65 options { 66 "driver.raw_exec.enable" = "1" 67 } 68 } 69 70 atlas { 71 infrastructure = "hashicorp/mars" 72 token = "atlas.v1.AFE84330943" 73 } 74 ``` 75 76 Note that it is strongly recommended _not_ to operate a node as both `client` 77 and `server`, although this is supported to simplify development and testing. 78 79 ## General Options 80 81 The following configuration options are available to both client and server 82 nodes, unless otherwise specified: 83 84 * <a id="region">`region`</a>: Specifies the region the Nomad agent is a 85 member of. A region typically maps to a geographic region, for example `us`, 86 with potentially multiple zones, which map to [datacenters](#datacenter) such 87 as `us-west` and `us-east`. Defaults to `global`. 88 89 * `datacenter`: Datacenter of the local agent. All members of a datacenter 90 should share a local LAN connection. Defaults to `dc1`. 91 92 * <a id="name">`name`</a>: The name of the local node. This value is used to 93 identify individual nodes in a given datacenter and must be unique 94 per-datacenter. By default this is set to the local host's name. 95 96 * `data_dir`: A local directory used to store agent state. Client nodes use this 97 directory by default to store temporary allocation data as well as cluster 98 information. Server nodes use this directory to store cluster state, including 99 the replicated log and snapshot data. This option is required to start the 100 Nomad agent and must be specified as an absolute path. 101 102 * `log_level`: Controls the verbosity of logs the Nomad agent will output. Valid 103 log levels include `WARN`, `INFO`, or `DEBUG` in increasing order of 104 verbosity. Defaults to `INFO`. 105 106 * <a id="bind_addr">`bind_addr`</a>: Used to indicate which address the Nomad 107 agent should bind to for network services, including the HTTP interface as 108 well as the internal gossip protocol and RPC mechanism. This should be 109 specified in IP format, and can be used to easily bind all network services to 110 the same address. It is also possible to bind the individual services to 111 different addresses using the [addresses](#addresses) configuration option. 112 Defaults to the local loopback address `127.0.0.1`. 113 114 * `enable_debug`: Enables the debugging HTTP endpoints. These endpoints can be 115 used with profiling tools to dump diagnostic information about Nomad's 116 internals. It is not recommended to leave this enabled in production 117 environments. Defaults to `false`. 118 119 * `ports`: Controls the network ports used for different services required by 120 the Nomad agent. The value is a key/value mapping of port numbers, and accepts 121 the following keys: 122 <br> 123 * `http`: The port used to run the HTTP server. Applies to both client and 124 server nodes. Defaults to `4646`. 125 * `rpc`: The port used for internal RPC communication between agents and 126 servers, and for inter-server traffic for the consensus algorithm (raft). 127 Defaults to `4647`. Only used on server nodes. 128 * `serf`: The port used for the gossip protocol for cluster membership. Both 129 TCP and UDP should be routable between the server nodes on this port. 130 Defaults to `4648`. Only used on server nodes. 131 132 * <a id="addresses">`addresses`</a>: Controls the bind address for individual 133 network services. Any values configured in this block take precedence over the 134 default [bind_addr](#bind_addr). The value is a map of IP addresses and 135 supports the following keys: 136 <br> 137 * `http`: The address the HTTP server is bound to. This is the most common 138 bind address to change. Applies to both clients and servers. 139 * `rpc`: The address to bind the internal RPC interfaces to. Should be exposed 140 only to other cluster members if possible. Used only on server nodes, but 141 must be accessible from all agents. 142 * `serf`: The address used to bind the gossip layer to. Both a TCP and UDP 143 listener will be exposed on this address. Should be restricted to only 144 server nodes from the same datacenter if possible. Used only on server 145 nodes. 146 147 * `advertise`: Controls the advertise address for individual network services. 148 This can be used to advertise a different address to the peers of a server or 149 a client node to support more complex network configurations such as NAT. This 150 configuration is optional, and defaults to the bind address of the specific 151 network service if it is not provided. The value is a map of IP addresses and 152 ports and supports the following keys: 153 <br> 154 * `http`: The address to advertise for the HTTP interface. This should be 155 reachable by all the nodes from which end users are going to use the Nomad 156 CLI tools. 157 ``` 158 advertise { 159 http = "1.2.3.4:4646" 160 } 161 ``` 162 * `rpc`: The address to advertise for the RPC interface. This address should 163 be reachable by all of the agents in the cluster. For example: 164 ``` 165 advertise { 166 rpc = "1.2.3.4:4647" 167 } 168 ``` 169 * `serf`: The address advertised for the gossip layer. This address must be 170 reachable from all server nodes. It is not required that clients can reach 171 this address. 172 173 * `telemetry`: Used to control how the Nomad agent exposes telemetry data to 174 external metrics collection servers. This is a key/value mapping and supports 175 the following keys: 176 <br> 177 * `statsite_address`: Address of a 178 [statsite](https://github.com/armon/statsite) server to forward metrics data 179 to. 180 * `statsd_address`: Address of a [statsd](https://github.com/etsy/statsd) 181 server to forward metrics to. 182 * `disable_hostname`: A boolean indicating if gauge values should not be 183 prefixed with the local hostname. 184 185 * `leave_on_interrupt`: Enables gracefully leaving when receiving the 186 interrupt signal. By default, the agent will exit forcefully on any signal. 187 188 * `leave_on_terminate`: Enables gracefully leaving when receiving the 189 terminate signal. By default, the agent will exit forcefully on any signal. 190 191 * `enable_syslog`: Enables logging to syslog. This option only works on 192 Unix based systems. 193 194 * `syslog_facility`: Controls the syslog facility that is used. By default, 195 `LOCAL0` will be used. This should be used with `enable_syslog`. 196 197 * `disable_update_check`: Disables automatic checking for security bulletins 198 and new version releases. 199 200 * `disable_anonymous_signature`: Disables providing an anonymous signature 201 for de-duplication with the update check. See `disable_update_check`. 202 203 * `http_api_response_headers`: This object allows adding headers to the 204 HTTP API responses. For example, the following config can be used to enable 205 CORS on the HTTP API endpoints: 206 ``` 207 http_api_response_headers { 208 Access-Control-Allow-Origin = "*" 209 } 210 ``` 211 212 ## Server-specific Options 213 214 The following options are applicable to server agents only and need not be 215 configured on client nodes. 216 217 * `server`: This is the top-level key used to define the Nomad server 218 configuration. It is a key/value mapping which supports the following keys: 219 <br> 220 * `enabled`: A boolean indicating if server mode should be enabled for the 221 local agent. All other server options depend on this value being set. 222 Defaults to `false`. 223 * <a id="bootstrap_expect">`bootstrap_expect`</a>: This is an integer 224 representing the number of server nodes to wait for before bootstrapping. It 225 is most common to use the odd-numbered integers `3` or `5` for this value, 226 depending on the cluster size. A value of `1` does not provide any fault 227 tolerance and is not recommended for production use cases. 228 * `data_dir`: This is the data directory used for server-specific data, 229 including the replicated log. By default, this directory lives inside of the 230 [data_dir](#data_dir) in the "server" sub-path. 231 * `protocol_version`: The Nomad protocol version spoken when communicating 232 with other Nomad servers. This value is typically not required as the agent 233 internally knows the latest version, but may be useful in some upgrade 234 scenarios. 235 * `num_schedulers`: The number of parallel scheduler threads to run. This 236 can be as many as one per core, or `0` to disallow this server from making 237 any scheduling decisions. This defaults to the number of CPU cores. 238 * `enabled_schedulers`: This is an array of strings indicating which 239 sub-schedulers this server will handle. This can be used to restrict the 240 evaluations that worker threads will dequeue for processing. This 241 defaults to all available schedulers. 242 * `node_gc_threshold` This is a string with a unit suffix, such as "300ms", 243 "1.5h" or "25m". Valid time units are "ns", "us" (or "µs"), "ms", "s", 244 "m", "h". Controls how long a node must be in a terminal state before it is 245 garbage collected and purged from the system. 246 * <a id="rejoin_after_leave">`rejoin_after_leave`</a> When provided, Nomad will ignore a previous leave and 247 attempt to rejoin the cluster when starting. By default, Nomad treats leave 248 as a permanent intent and does not attempt to join the cluster again when 249 starting. This flag allows the previous state to be used to rejoin the 250 cluster. 251 * <a id="retry_join">`retry_join`</a> Similar to [`start_join`](#start_join) but allows retrying a join 252 if the first attempt fails. This is useful for cases where we know the 253 address will become available eventually. Use `retry_join` with an array as a replacement for 254 `start_join`, do not use both options. 255 * <a id="retry_interval">`retry_interval`</a> The time to wait between join attempts. Defaults to 30s. 256 * <a id="retry_max">`retry_max`</a> The maximum number of join attempts to be made before exiting 257 with a return code of 1. By default, this is set to 0 which is interpreted 258 as infinite retries. 259 * <a id="start_join">`start_join`</a> An array of strings specifying addresses 260 of nodes to join upon startup. If Nomad is unable to join with any of the 261 specified addresses, agent startup will fail. By default, the agent won't 262 join any nodes when it starts up. Addresses can be given as an IP, a domain 263 name, or an IP:Port pair. If the port isn't specified the default Serf port, 264 4648, is used. DNS names may also be used. 265 266 ## Client-specific Options 267 268 The following options are applicable to client agents only and need not be 269 configured on server nodes. 270 271 * `client`: This is the top-level key used to define the Nomad client 272 configuration. Like the server configuration, it is a key/value mapping which 273 supports the following keys: 274 <br> 275 * `enabled`: A boolean indicating if client mode is enabled. All other client 276 configuration options depend on this value. Defaults to `false`. 277 * <a id="state_dir">`state_dir`</a>: This is the state dir used to store 278 client state. By default, it lives inside of the [data_dir](#data_dir), in 279 the "client" sub-path. It must be specified as an absolute path. 280 * <a id="alloc_dir">`alloc_dir`</a>: A directory used to store allocation data. 281 Depending on the workload, the size of this directory can grow arbitrarily 282 large as it is used to store downloaded artifacts for drivers (QEMU images, 283 JAR files, etc.). It is therefore important to ensure this directory is 284 placed some place on the filesystem with adequate storage capacity. By 285 default, this directory lives under the [data_dir](#data_dir) at the 286 "alloc" sub-path. It must be specified as an absolute path. 287 * <a id="servers">`servers`</a>: An array of server addresses. This list is 288 used to register the client with the server nodes and advertise the 289 available resources so that the agent can receive work. If a port is not specified 290 in the array of server addresses, the default port `4647` will be used. 291 * <a id="node_class">`node_class`</a>: A string used to logically group client 292 nodes by class. This can be used during job placement as a filter. This 293 option is not required and has no default. 294 * <a id="meta">`meta`</a>: This is a key/value mapping of metadata pairs. This 295 is a free-form map and can contain any string values. 296 * <a id="options">`options`</a>: This is a key/value mapping of internal 297 configuration for clients, such as for driver configuration. Please see 298 [here](#options_map) for a description of available options. 299 * <a id="network_interface">`network_interface`</a>: This is a string to force 300 network fingerprinting to use a specific network interface 301 * <a id="network_speed">`network_speed`</a>: This is an int that sets the 302 default link speed of network interfaces, in megabits, if their speed can 303 not be determined dynamically. 304 * `max_kill_timeout`: `max_kill_timeout` is a time duration that can be 305 specified using the `s`, `m`, and `h` suffixes, such as `30s`. If a job's 306 task specifies a `kill_timeout` greater than `max_kill_timeout`, 307 `max_kill_timeout` is used. This is to prevent a user being able to set an 308 unreasonable timeout. If unset, a default is used. 309 * `reserved`: `reserved` is used to reserve a portion of the nodes resources 310 from being used by Nomad when placing tasks. It can be used to target 311 a certain capacity usage for the node. For example, 20% of the nodes CPU 312 could be reserved to target a CPU utilization of 80%. The block has the 313 following format: 314 315 ``` 316 reserved { 317 cpu = 500 318 memory = 512 319 disk = 1024 320 reserved_ports = "22,80,8500-8600" 321 } 322 ``` 323 324 * `cpu`: `cpu` is given as MHz to reserve. 325 * `memory`: `memory` is given as MB to reserve. 326 * `disk`: `disk` is given as MB to reserve. 327 * `reserved_ports`: `reserved_ports` is a comma separated list of ports 328 to reserve on all fingerprinted network devices. Ranges can be 329 specified by using a hyphen separated the two inclusive ends. 330 331 ### Client Options Map <a id="options_map"></a> 332 333 The following is not an exhaustive list of options that can be passed to the 334 Client, but rather the set of options that configure the Client and not the 335 drivers. To find the options supported by an individual driver, see the drivers 336 documentation [here](/docs/drivers/index.html) 337 338 * `consul.address`: The address to the local Consul agent given in the format of 339 `host:port`. The default is the same as the Consul default address, 340 `127.0.0.1:8500`. 341 342 * `consul.token`: Token is used to provide a per-request ACL token.This options 343 overrides the agent's default token 344 345 * `consul.auth`: The auth information to use for http access to the Consul 346 Agent. 347 348 * `consul.ssl`: This boolean option sets the transport scheme to talk to the Consul 349 Agent as `https`. This option is unset by default and so the default transport 350 scheme for the consul api client is `http`. 351 352 * `consul.verifyssl`: This option enables SSL verification when the transport 353 scheme for the Consul API client is `https`. This is set to true by default. 354 355 * `driver.whitelist`: A comma separated list of whitelisted drivers (e.g. 356 "docker,qemu"). If specified, drivers not in the whitelist will be disabled. 357 If the whitelist is empty, all drivers are fingerprinted and enabled where 358 applicable. 359 360 * `env.blacklist`: Nomad passes the host environment variables to `exec`, 361 `raw_exec` and `java` tasks. `env.blacklist` is a comma separated list of 362 environment variable keys not to pass to these tasks. If specified, the 363 defaults are overridden. The following are the default: 364 365 * `CONSUL_TOKEN` 366 * `VAULT_TOKEN` 367 * `ATLAS_TOKEN` 368 * `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN` 369 * `GOOGLE_APPLICATION_CREDENTIALS` 370 371 * `user.blacklist`: An operator specifiable blacklist of users which a task is 372 not allowed to run as when using a driver in `user.checked_drivers`. 373 Defaults to: 374 375 * `root` 376 * `Administrator` 377 378 * `user.checked_drivers`: An operator specifiable list of drivers to enforce 379 the the `user.blacklist`. For drivers using containers, this enforcement often 380 doesn't make sense and as such the default is set to: 381 382 * `exec` 383 * `qemu` 384 * `java` 385 386 * `fingerprint.whitelist`: A comma separated list of whitelisted fingerprinters. 387 If specified, fingerprinters not in the whitelist will be disabled. If the 388 whitelist is empty, all fingerprinters are used. 389 390 ## Atlas Options 391 392 **NOTE**: Nomad integration with Atlas is awaiting release of Atlas features 393 for Nomad support. Nomad currently only validates configuration options for 394 Atlas but does not use them. 395 See [#183](https://github.com/hashicorp/nomad/issues/183) for more details. 396 397 The following options are used to configure [Atlas](https://atlas.hashicorp.com) 398 integration and are entirely optional. 399 400 * `atlas`: The top-level config key used to contain all Atlas-related 401 configuration options. The value is a key/value map which supports the 402 following keys: 403 <br> 404 * <a id="infrastructure">`infrastructure`</a>: The Atlas infrastructure name to 405 connect this agent to. This value should be of the form 406 `<org>/<infrastructure>`, and requires a valid [token](#token) authorized on 407 the infrastructure. 408 * <a id="token">`token`</a>: The Atlas token to use for authentication. This 409 token should have access to the provided [infrastructure](#infrastructure). 410 * <a id="join">`join`</a>: A boolean indicating if the auto-join feature of 411 Atlas should be enabled. Defaults to `false`. 412 * `endpoint`: The address of the Atlas instance to connect to. Defaults to the 413 public Atlas endpoint and is only used if both 414 [infrastructure](#infrastructure) and [token](#token) are provided. 415 416 ## Command-line Options <a id="cli"></a> 417 418 A subset of the available Nomad agent configuration can optionally be passed in 419 via CLI arguments. The `agent` command accepts the following arguments: 420 421 * `alloc-dir=<path>`: Equivalent to the Client [alloc_dir](#alloc_dir) config 422 option. 423 * `-atlas=<infrastructure>`: Equivalent to the Atlas 424 [infrastructure](#infrastructure) config option. 425 * `-atlas-join`: Equivalent to the Atlas [join](#join) config option. 426 * `-atlas-token=<token>`: Equivalent to the Atlas [token](#token) config option. 427 * `-bind=<address>`: Equivalent to the [bind_addr](#bind_addr) config option. 428 * `-bootstrap-expect=<num>`: Equivalent to the 429 [bootstrap_expect](#bootstrap_expect) config option. 430 * `-client`: Enable client mode on the local agent. 431 * `-config=<path>`: Specifies the path to a configuration file or a directory of 432 configuration files to load. Can be specified multiple times. 433 * `-data-dir=<path>`: Equivalent to the [data_dir](#data_dir) config option. 434 * `-dc=<datacenter>`: Equivalent to the [datacenter](#datacenter) config option. 435 * `-dev`: Start the agent in development mode. This enables a pre-configured 436 dual-role agent (client + server) which is useful for developing or testing 437 Nomad. No other configuration is required to start the agent in this mode. 438 * `-join=<address>`: Address of another agent to join upon starting up. This can 439 be specified multiple times to specify multiple agents to join. 440 * `-log-level=<level>`: Equivalent to the [log_level](#log_level) config option. 441 * `-meta=<key=value>`: Equivalent to the Client [meta](#meta) config option. 442 * `-network-interface<interface>`: Equivalent to the Client 443 [network_interface](#network_interface) config option. 444 * `-network-speed<MBits>`: Equivalent to the Client 445 [network_speed](#network_speed) config option. 446 * `-node=<name>`: Equivalent to the [name](#name) config option. 447 * `-node-class=<class>`: Equivalent to the Client [node_class](#node_class) 448 config option. 449 * `-region=<region>`: Equivalent to the [region](#region) config option. 450 * `-rejoin`: Equivalent to the [rejoin_after_leave](#rejoin_after_leave) config option. 451 * `-retry-interval`: Equivalent to the [retry_interval](#retry_interval) config option. 452 * `-retry-join`: Similar to `-join` but allows retrying a join if the first attempt fails. 453 * `-retry-max`: Similar to the [retry_max](#retry_max) config option. 454 * `-server`: Enable server mode on the local agent. 455 * `-servers=<host:port>`: Equivalent to the Client [servers](#servers) config 456 option. 457 * `-state-dir=<path>`: Equivalent to the Client [state_dir](#state_dir) config 458 option.