github.com/maier/nomad@v0.4.1-0.20161110003312-a9e3d0b8549d/website/source/docs/http/json-jobs.html.md (about)

     1  ---
     2  layout: "http"
     3  page_title: "HTTP API: JSON Job Specification"
     4  sidebar_current: "docs-http-json-jobs"
     5  description: |-
     6    Jobs can also be specified via the HTTP API using a JSON format. This guide
     7    discusses the job specification in JSON format.
     8  ---
     9  
    10  # Job Specification
    11  
    12  This guide covers the JSON syntax for submitting jobs to Nomad. A useful command
    13  for generating valid JSON versions of HCL jobs is
    14  `nomad run -output <job.nomad>` which will emit a JSON version of the job.
    15  
    16  ## JSON Syntax
    17  
    18  Below is an example of a JSON object that submits a `periodic` job to Nomad:
    19  
    20  ```json
    21  {
    22    "Job":{
    23      "Region":"global",
    24      "ID":"example",
    25      "Name":"example",
    26      "Type":"batch",
    27      "Priority":50,
    28      "AllAtOnce":false,
    29      "Datacenters":[
    30        "dc1"
    31      ],
    32      "Constraints":[
    33        {
    34          "LTarget":"${attr.kernel.name}",
    35          "RTarget":"linux",
    36          "Operand":"="
    37        }
    38      ],
    39      "TaskGroups":[
    40        {
    41          "Name":"cache",
    42          "Count":1,
    43          "Constraints":null,
    44          "Tasks":[
    45            {
    46              "Name":"redis",
    47              "Driver":"docker",
    48              "User":"foo-user",
    49              "Config":{
    50                "image":"redis:latest",
    51                "port_map":[
    52                  {
    53                    "db":6379
    54                  }
    55                ]
    56              },
    57              "Constraints":null,
    58              "Env":{
    59                "foo":"bar",
    60                "baz":"pipe"
    61              },
    62              "Services":[
    63                {
    64                  "Name":"cache-redis",
    65                  "Tags":[
    66                    "global",
    67                    "cache"
    68                  ],
    69                  "PortLabel":"db",
    70                  "Checks":[
    71                    {
    72                      "Id":"",
    73                      "Name":"alive",
    74                      "Type":"tcp",
    75                      "Command":"",
    76                      "Args":null,
    77                      "Path":"",
    78                      "Protocol":"",
    79                      "Interval":10000000000,
    80                      "Timeout":2000000000
    81                    }
    82                  ]
    83                }
    84              ],
    85              "Resources":{
    86                "CPU":500,
    87                "MemoryMB":256,
    88                "DiskMB":300,
    89                "IOPS":0,
    90                "Networks":[
    91                  {
    92                    "ReservedPorts":[
    93                      {
    94                        "Label":"rpc",
    95                        "Value":25566
    96                      }
    97                    ],
    98                    "DynamicPorts":[
    99                      {
   100                        "Label":"db"
   101                      }
   102                    ],
   103                    "MBits":10
   104                  }
   105                ]
   106              },
   107              "Meta":{
   108                "foo":"bar",
   109                "baz":"pipe"
   110              },
   111              "KillTimeout":5000000000,
   112              "LogConfig":{
   113                "MaxFiles":10,
   114                "MaxFileSizeMB":10
   115              },
   116              "Artifacts":[
   117                {
   118                  "GetterSource":"http://foo.com/artifact.tar.gz",
   119                  "GetterOptions":{
   120                    "checksum":"md5:c4aa853ad2215426eb7d70a21922e794"
   121                  },
   122                  "RelativeDest":"local/"
   123                }
   124              ]
   125            }
   126          ],
   127          "RestartPolicy":{
   128            "Interval":300000000000,
   129            "Attempts":10,
   130            "Delay":25000000000,
   131            "Mode":"delay"
   132          },
   133          "Meta":{
   134            "foo":"bar",
   135            "baz":"pipe"
   136          }
   137        }
   138      ],
   139      "Update":{
   140        "Stagger":10000000000,
   141        "MaxParallel":1
   142      },
   143      "Periodic":{
   144        "Enabled":true,
   145        "Spec":"* * * * *",
   146        "SpecType":"cron",
   147        "ProhibitOverlap":true
   148      },
   149      "Meta":{
   150        "foo":"bar",
   151        "baz":"pipe"
   152      }
   153    }
   154  }
   155  ```
   156  
   157  ## Syntax Reference
   158  
   159  Following is a syntax reference for the possible keys that are supported
   160  and their default values if any for each type of object.
   161  
   162  ### Job
   163  
   164  The `Job` object supports the following keys:
   165  
   166  * `AllAtOnce` - Controls if the entire set of tasks in the job must
   167    be placed atomically or if they can be scheduled incrementally.
   168    This should only be used for special circumstances. Defaults to `false`.
   169  
   170  * `Constraints` - A list to define additional constraints where a job can be
   171    run. See the constraint reference for more details.
   172  
   173  * `Datacenters` - A list of datacenters in the region which are eligible
   174    for task placement. This must be provided, and does not have a default.
   175  
   176  * `TaskGroups` - A list to define additional task groups. See the task group
   177    reference for more details.
   178  
   179  * `Meta` - Annotates the job with opaque metadata.
   180  
   181  * `Priority` - Specifies the job priority which is used to prioritize
   182    scheduling and access to resources. Must be between 1 and 100 inclusively,
   183    and defaults to 50.
   184  
   185  * `Region` - The region to run the job in, defaults to "global".
   186  
   187  * `Type` - Specifies the job type and switches which scheduler
   188    is used. Nomad provides the `service`, `system` and `batch` schedulers,
   189    and defaults to `service`. To learn more about each scheduler type visit
   190    [here](/docs/runtime/schedulers.html)
   191  
   192  * `Update` - Specifies the task's update strategy. When omitted, rolling
   193    updates are disabled. The `Update` object supports the following attributes:
   194  
   195    * `MaxParallel` - `MaxParallel` is given as an integer value and specifies
   196    the number of tasks that can be updated at the same time.
   197  
   198    * `Stagger` - `Stagger` introduces a delay between sets of task updates and
   199    is given in nanoseconds.
   200  
   201      An example `Update` block:
   202  
   203      ```json
   204      {
   205        "Update": {
   206          "MaxParallel" : 3,
   207          "Stagger" : 10000000000
   208        }
   209      }
   210      ```
   211  
   212  *   `Periodic` - `Periodic` allows the job to be scheduled at fixed times, dates
   213      or intervals. The periodic expression is always evaluated in the UTC
   214      timezone to ensure consistent evaluation when Nomad Servers span multiple
   215      time zones. The `Periodic` object is optional and supports the following attributes:
   216  
   217      * `Enabled` - `Enabled` determines whether the periodic job will spawn child
   218      jobs.
   219  
   220      * `SpecType` - `SpecType` determines how Nomad is going to interpret the
   221        periodic expression. `cron` is the only supported `SpecType` currently.
   222  
   223      * `Spec` - A cron expression configuring the interval the job is launched
   224      at. Supports predefined expressions such as "@daily" and "@weekly" See
   225      [here](https://github.com/gorhill/cronexpr#implementation) for full
   226      documentation of supported cron specs and the predefined expressions.
   227  
   228      * <a id="prohibit_overlap">`ProhibitOverlap`</a> - `ProhibitOverlap` can
   229        be set to true to enforce that the periodic job doesn't spawn a new
   230        instance of the job if any of the previous jobs are still running. It is
   231        defaulted to false.
   232  
   233      An example `periodic` block:
   234  
   235      ```json
   236      {
   237        "Periodic": {
   238            "Spec": "*/15 * * * * *"
   239            "SpecType": "cron",
   240            "Enabled": true,
   241            "ProhibitOverlap": true
   242        }
   243      }
   244      ```
   245  
   246  ### Task Group
   247  
   248  `TaskGroups` is a list of `TaskGroup` objects, each supports the following
   249  attributes:
   250  
   251  * `Constraints` - This is a list of `Constraint` objects. See the constraint
   252    reference for more details.
   253  
   254  * `Count` - Specifies the number of the task groups that should
   255    be running. Must be non-negative, defaults to one.
   256  
   257  * `Meta` - A key-value map that annotates the task group with opaque metadata.
   258  
   259  * `Name` - The name of the task group. Must be specified.
   260  
   261  * `RestartPolicy` - Specifies the restart policy to be applied to tasks in this group.
   262    If omitted, a default policy for batch and non-batch jobs is used based on the
   263    job type. See the [restart policy reference](#restart_policy) for more details.
   264  
   265  * `Tasks` - A list of `Task` object that are part of the task group.
   266  
   267  ### Task
   268  
   269  The `Task` object supports the following keys:
   270  
   271  * `Artifacts` - `Artifacts` is a list of `Artifact` objects which define
   272    artifacts to be downloaded before the task is run. See the artifacts
   273    reference for more details.
   274  
   275  * `Config` - A map of key-value configuration passed into the driver
   276    to start the task. The details of configurations are specific to
   277    each driver.
   278  
   279  * `Constraints` - This is a list of `Constraint` objects. See the constraint
   280    reference for more details.
   281  
   282  * `Driver` - Specifies the task driver that should be used to run the
   283    task. See the [driver documentation](/docs/drivers/index.html) for what
   284    is available. Examples include `docker`, `qemu`, `java`, and `exec`.
   285  
   286  *   `Env` - A map of key-value representing environment variables that
   287      will be passed along to the running process. Nomad variables are
   288      interpreted when set in the environment variable values. See the table of
   289      interpreted variables [here](/docs/runtime/interpolation.html).
   290  
   291      For example the below environment map will be reinterpreted:
   292  
   293      ```json
   294      {
   295        "Env": {
   296          "NODE_CLASS" : "${nomad.class}"
   297        }
   298      }
   299      ```
   300  
   301  * `KillTimeout` - `KillTimeout` is a time duration in nanoseconds. It can be
   302    used to configure the time between signaling a task it will be killed and
   303    actually killing it. Drivers first sends a task the `SIGINT` signal and then
   304    sends `SIGTERM` if the task doesn't die after the `KillTimeout` duration has
   305    elapsed. The default `KillTimeout` is 5 seconds.
   306  
   307  * `LogConfig` - This allows configuring log rotation for the `stdout` and `stderr`
   308    buffers of a Task. See the log rotation reference below for more details.
   309  
   310  * `Meta` - Annotates the task group with opaque metadata.
   311  
   312  * `Name` - The name of the task. This field is required.
   313  
   314  * `Resources` - Provides the resource requirements of the task.
   315    See the resources reference for more details.
   316  
   317  * `Services` - `Services` is a list of `Service` objects. Nomad integrates with
   318    Consul for service discovery. A `Service` object represents a routable and
   319    discoverable service on the network. Nomad automatically registers when a task
   320    is started and de-registers it when the task transitions to the dead state.
   321    [Click here](/docs/service-discovery/index.html) to learn more about
   322    services. Below is the fields in the `Service` object:
   323  
   324       * `Name`: An explicit name for the Service. Nomad will replace `${JOB}`,
   325         `${TASKGROUP}` and `${TASK}` by the name of the job, task group or task,
   326         respectively. `${BASE}` expands to the equivalent of
   327         `${JOB}-${TASKGROUP}-${TASK}`, and is the default name for a Service.
   328         Each service defined for a given task must have a distinct name, so if
   329         a task has multiple services only one of them can use the default name
   330         and the others must be explicitly named. Names must adhere to
   331         [RFC-1123 ยง2.1](https://tools.ietf.org/html/rfc1123#section-2) and are
   332         limited to alphanumeric and hyphen characters (i.e. `[a-z0-9\-]`), and be
   333         less than 64 characters in length.
   334  
   335       * `Tags`: A list of string tags associated with this Service. String
   336         interpolation is supported in tags.
   337  
   338       * `PortLabel`: `PortLabel` is an optional string and is used to associate
   339         a port with the service.  If specified, the port label must match one
   340         defined in the resources block.  This could be a label of either a
   341         dynamic or a static port.
   342  
   343       * `Checks`: `Checks` is an array of check objects. A check object defines a
   344         health check associated with the service. Nomad supports the `script`,
   345         `http` and `tcp` Consul Checks. Script checks are not supported for the
   346         qemu driver since the Nomad client doesn't have access to the file system
   347         of a task using the Qemu driver.
   348  
   349           * `Type`:  This indicates the check types supported by Nomad. Valid
   350             options are currently `script`, `http` and `tcp`.
   351  
   352           * `Name`: The name of the health check.
   353  
   354           * `Interval`: This indicates the frequency of the health checks that
   355             Consul will perform.
   356  
   357           * `Timeout`: This indicates how long Consul will wait for a health
   358             check query to succeed.
   359  
   360           * `Path`:The path of the http endpoint which Consul will query to query
   361             the health of a service if the type of the check is `http`. Nomad
   362             will add the IP of the service and the port, users are only required
   363             to add the relative URL of the health check endpoint.
   364  
   365           * `Protocol`: This indicates the protocol for the http checks. Valid
   366             options are `http` and `https`. We default it to `http`.
   367  
   368           * `Command`: This is the command that the Nomad client runs for doing
   369             script based health check.
   370  
   371           * `Args`: Additional arguments to the `command` for script based health
   372             checks.
   373  
   374  
   375  * `User` - Set the user that will run the task. It defaults to the same user
   376    the Nomad client is being run as. This can only be set on Linux platforms.
   377  
   378  ### Resources
   379  
   380  The `Resources` object supports the following keys:
   381  
   382  * `CPU` - The CPU required in MHz.
   383  
   384  * `DiskMB` - The disk required in MB.
   385  
   386  * `IOPS` - The number of IOPS required given as a weight between 10-1000.
   387  
   388  * `MemoryMB` - The memory required in MB.
   389  
   390  * `Networks` - A list of network objects.
   391  
   392  The Network object supports the following keys:
   393  
   394  * `MBits` - The number of MBits in bandwidth required.
   395  
   396  Nomad can allocate two types of ports to a task - Dynamic and Static/Reserved
   397  ports. A network object allows the user to specify a list of `DynamicPorts` and
   398  `ReservedPorts`. Each object supports the following attributes:
   399  
   400  * `Value` - The port number for static ports. If the port is dynamic, then this
   401    attribute is ignored.
   402  * `Label` - The label to annotate a port so that it can be referred in the
   403    service discovery block or environment variables.
   404  
   405  <a id="restart_policy"></a>
   406  
   407  ### Restart Policy
   408  
   409  The `RestartPolicy` object supports the following keys:
   410  
   411  * `Attempts` - `Attempts` is the number of restarts allowed in an `Interval`.
   412  
   413  * `Interval` - `Interval` is a time duration that is specified in nanoseconds.
   414    The `Interval` begins when the first task starts and ensures that only
   415    `Attempts` number of restarts happens within it. If more than `Attempts`
   416    number of failures happen, behavior is controlled by `Mode`.
   417  
   418  * `Delay` - A duration to wait before restarting a task. It is specified in
   419    nanoseconds. A random jitter of up to 25% is added to the delay.
   420  
   421  *   `Mode` - `Mode` is given as a string and controls the behavior when the task
   422      fails more than `Attempts` times in an `Interval`. Possible values are listed
   423      below:
   424  
   425      * `delay` - `delay` will delay the next restart until the next `Interval` is
   426        reached.
   427  
   428      * `fail` - `fail` will not restart the task again.
   429  
   430  ### Constraint
   431  
   432  The `Constraint` object supports the following keys:
   433  
   434  * `LTarget` - Specifies the attribute to examine for the
   435    constraint. See the table of attributes [here](/docs/runtime/interpolation.html#interpreted_node_vars).
   436  
   437  * `RTarget` - Specifies the value to compare the attribute against.
   438    This can be a literal value, another attribute or a regular expression if
   439    the `Operator` is in "regexp" mode.
   440  
   441  * `Operand` - Specifies the test to be performed on the two targets. It takes on the
   442    following values:
   443  
   444    * `regexp` - Allows the `RTarget` to be a regular expression to be matched.
   445  
   446    * `set_contains` - Allows the `RTarget` to be a comma separated list of values
   447      that should be contained in the LTarget's value.
   448  
   449    * `distinct_host` - If set, the scheduler will not co-locate any task groups on the same
   450          machine. This can be specified as a job constraint which applies the
   451          constraint to all task groups in the job, or as a task group constraint which
   452          scopes the effect to just that group.
   453  
   454          Placing the constraint at both the job level and at the task group level is
   455          redundant since when placed at the job level, the constraint will be applied
   456          to all task groups. When specified, `LTarget` and `RTarget` should be
   457          omitted.
   458  
   459    * Comparison Operators - `=`, `==`, `is`, `!=`, `not`, `>`, `>=`, `<`, `<=`. The
   460      ordering is compared lexically.
   461  
   462  ### Log Rotation
   463  
   464  The `LogConfig` object configures the log rotation policy for a task's `stdout` and
   465  `stderr`. The `LogConfig` object supports the following attributes:
   466  
   467  * `MaxFiles` - The maximum number of rotated files Nomad will retain for
   468    `stdout` and `stderr`, each tracked individually.
   469  
   470  * `MaxFileSizeMB` - The size of each rotated file. The size is specified in
   471    `MB`.
   472  
   473  If the amount of disk resource requested for the task is less than the total
   474  amount of disk space needed to retain the rotated set of files, Nomad will return
   475  a validation error when a job is submitted.
   476  
   477  ```json
   478  {
   479    "LogConfig": {
   480      "MaxFiles": 3,
   481      "MaxFileSizeMB": 10
   482    }
   483  }
   484  ```
   485  
   486  In the above example we have asked Nomad to retain 3 rotated files for both
   487  `stderr` and `stdout` and size of each file is 10MB. The minimum disk space that
   488  would be required for the task would be 60MB.
   489  
   490  ### Artifact
   491  
   492  Nomad downloads artifacts using
   493  [`go-getter`](https://github.com/hashicorp/go-getter). The `go-getter` library
   494  allows downloading of artifacts from various sources using a URL as the input
   495  source. The key-value pairs given in the `options` block map directly to
   496  parameters appended to the supplied `source` URL. These are then used by
   497  `go-getter` to appropriately download the artifact. `go-getter` also has a CLI
   498  tool to validate its URL and can be used to check if the Nomad `artifact` is
   499  valid.
   500  
   501  Nomad allows downloading `http`, `https`, and `S3` artifacts. If these artifacts
   502  are archives (zip, tar.gz, bz2, etc.), these will be unarchived before the task
   503  is started.
   504  
   505  The `Artifact` object supports the following keys:
   506  
   507  * `GetterSource` - The path to the artifact to download.
   508  
   509  * `RelativeDest` - An optional path to download the artifact into relative to the
   510    root of the task's directory. If omitted, it will default to `local/`.
   511  
   512  * `GetterOptions` - A `map[string]string` block of options for `go-getter`.
   513    Full documentation of supported options are available
   514    [here](https://github.com/hashicorp/go-getter/tree/ef5edd3d8f6f482b775199be2f3734fd20e04d4a#protocol-specific-options-1).
   515    An example is given below:
   516  
   517  ```json
   518  {
   519    "GetterOptions": {
   520      "checksum": "md5:c4aa853ad2215426eb7d70a21922e794",
   521  
   522      "aws_access_key_id": "<id>",
   523      "aws_access_key_secret": "<secret>",
   524      "aws_access_token": "<token>"
   525    }
   526  }
   527  ```
   528  
   529  An example of downloading and unzipping an archive is as simple as:
   530  
   531  ```json
   532  {
   533    "Artifacts": [
   534      {
   535        "GetterSource": "https://example.com/my.zip",
   536        "GetterOptions": {
   537          "checksum": "md5:7f4b3e3b4dd5150d4e5aaaa5efada4c3"
   538        }
   539      }
   540    ]
   541  }
   542  ```
   543  
   544  #### S3 examples
   545  
   546  S3 has several different types of addressing and more detail can be found
   547  [here](http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#access-bucket-intro)
   548  
   549  S3 region specific endpoints can be found
   550  [here](http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region)
   551  
   552  Path based style:
   553  
   554  ```json
   555  {
   556    "Artifacts": [
   557      {
   558        "GetterSource": "https://s3-us-west-2.amazonaws.com/my-bucket-example/my_app.tar.gz",
   559      }
   560    ]
   561  }
   562  ```
   563  
   564  or to override automatic detection in the URL, use the S3-specific syntax
   565  
   566  ```json
   567  {
   568    "Artifacts": [
   569      {
   570        "GetterSource": "s3::https://s3-eu-west-1.amazonaws.com/my-bucket-example/my_app.tar.gz",
   571      }
   572    ]
   573  }
   574  ```
   575  
   576  Virtual hosted based style
   577  
   578  ```json
   579  {
   580    "Artifacts": [
   581      {
   582        "GetterSource": "my-bucket-example.s3-eu-west-1.amazonaws.com/my_app.tar.gz",
   583      }
   584    ]
   585  }
   586  ```