github.com/anth0d/nomad@v0.0.0-20221214183521-ae3a0a2cad06/website/content/docs/job-specification/artifact.mdx (about)

     1  ---
     2  layout: docs
     3  page_title: artifact Stanza - Job Specification
     4  description: |-
     5    The "artifact" stanza instructs Nomad to fetch and unpack a remote resource,
     6    such as a file, tarball, or binary, and permits downloading artifacts from a
     7    variety of locations using a URL as the input source.
     8  ---
     9  
    10  # `artifact` Stanza
    11  
    12  <Placement groups={['job', 'group', 'task', 'artifact']} />
    13  
    14  The `artifact` stanza instructs Nomad to fetch and unpack a remote resource,
    15  such as a file, tarball, or binary. Nomad downloads artifacts using the popular
    16  [`go-getter`][go-getter] library, which permits downloading artifacts from a
    17  variety of locations using a URL as the input source.
    18  
    19  ```hcl
    20  job "docs" {
    21    group "example" {
    22      task "server" {
    23        artifact {
    24          source      = "https://example.com/file.tar.gz"
    25          destination = "local/some-directory"
    26          options {
    27            checksum = "md5:df6a4178aec9fbdc1d6d7e3634d1bc33"
    28          }
    29        }
    30      }
    31    }
    32  }
    33  ```
    34  
    35  Nomad supports downloading `http`, `https`, `git`, `hg` and `S3` artifacts. If
    36  these artifacts are archived (`zip`, `tgz`, `bz2`, `xz`), they are
    37  automatically unarchived before the starting the task.
    38  
    39  ## `artifact` Parameters
    40  
    41  - `destination` `(string: "local/")` - Specifies the directory path to
    42    download the artifact, relative to the root of the [task's working
    43    directory]. If omitted, the default value is to place the artifact in
    44    `local/`. The destination is treated as a directory unless `mode` is set to
    45    `file`. Source files will be downloaded into that directory path. For more
    46    details on how the `destination` interacts with task drivers, see the
    47    [Filesystem internals] documentation.
    48  
    49  - `mode` `(string: "any")` - One of `any`, `file`, or `dir`. If set to `file`
    50    the `destination` must be a file, not a directory. By default the
    51    `destination` will be `local/<filename>`.
    52  
    53  - `options` `(map<string|string>: nil)` - Specifies configuration parameters to
    54    fetch the artifact. The key-value pairs map directly to parameters appended to
    55    the supplied `source` URL. Please see the [`go-getter`
    56    documentation][go-getter] for a complete list of options and examples.
    57  
    58  - `headers` `(map<string|string>: nil)` - Specifies HTTP headers to set when
    59    fetching the artifact using `http` or `https` protocol. Please see the
    60    [`go-getter` headers documentation][go-getter-headers] for more information.
    61  
    62  - `source` `(string: <required>)` - Specifies the URL of the artifact to download.
    63    See [`go-getter`][go-getter] for details.
    64  
    65  ## Operation Limits
    66  
    67  The client [`artifact`][client_artifact] configuration can set limits to
    68  specific artifact operations to prevent excessive data download or operation
    69  time.
    70  
    71  If a task's `artifact` retrieval exceeds one of those limits, the task will be
    72  interrupted and fail to start. Refer to the task events for more information.
    73  
    74  ## `artifact` Examples
    75  
    76  The following examples only show the `artifact` stanzas. Remember that the
    77  `artifact` stanza is only valid in the placements listed above.
    78  
    79  ### Download File
    80  
    81  This example downloads the artifact from the provided URL and places it in
    82  `local/file.txt`. The `local/` path is relative to the [task's working
    83  directory].
    84  
    85  ```hcl
    86  artifact {
    87    source = "https://example.com/file.txt"
    88  }
    89  ```
    90  
    91  To set HTTP headers in the request for the source the optional `headers` field
    92  can be configured.
    93  
    94  ```hcl
    95  artifact {
    96    source = "https://example.com/file.txt"
    97  
    98    headers {
    99      User-Agent    = "nomad-[${NOMAD_JOB_ID}]-[${NOMAD_GROUP_NAME}]-[${NOMAD_TASK_NAME}]"
   100      X-Nomad-Alloc = "${NOMAD_ALLOC_ID}"
   101    }
   102  }
   103  ```
   104  
   105  To use HTTP basic authentication, preprend the username and password to the
   106  hostname in the URL. All special characters, including the username and
   107  password, must be URL encoded. For example, for a username `exampleUser` and
   108  the password `pass/word!`:
   109  
   110  ```hcl
   111  artifact {
   112    source = "https://exampleUser:pass%2Fword%21@example.com/file.txt"
   113  }
   114  ```
   115  
   116  ### Download using git
   117  
   118  This example downloads the artifact from the provided GitHub URL and places it at
   119  `local/repo`, as specified by the optional `destination` parameter.
   120  
   121  ```hcl
   122  artifact {
   123    source      = "git::https://github.com/hashicorp/nomad-guides"
   124    destination = "local/repo"
   125  }
   126  ```
   127  
   128  To download from a private repo, sshkey needs to be set. The key must be
   129  base64-encoded string. On Linux, you can run `base64 -w0 <file>` to encode the
   130  file. Or use [HCL2](https://www.nomadproject.io/docs/job-specification/hcl2)
   131  expressions to read and encode the key from a file on your machine:
   132  
   133  ```hcl
   134  artifact {
   135    # The git:: prefix forces go-getter's protocol detection to use the git ssh
   136    # protocol. It can also automatically detect the protocol from the domain of
   137    # some git hosting providers (such as GitHub) without the prefix.
   138    source      = "git::git@bitbucket.org:example/nomad-examples"
   139    destination = "local/repo"
   140    options {
   141      # Make sure that the system known hosts file is populated:
   142      # ssh-keyscan github.com | sudo tee -a /etc/ssh/ssh_known_hosts
   143      # https://github.com/hashicorp/go-getter/issues/55
   144      sshkey = "${base64encode(file("/etc/ssh/ssh_known_hosts"))}"
   145    }
   146  }
   147  ```
   148  
   149  To clone specific refs or at a specific depth, use the `ref` and `depth`
   150  options:
   151  
   152  ```hcl
   153  artifact {
   154    source      = "git::https://github.com/hashicorp/nomad-guides"
   155    destination = "local/repo"
   156    options {
   157      ref = "main"
   158      depth = 1
   159    }
   160  }
   161  ```
   162  
   163  ### Download and Unarchive
   164  
   165  This example downloads and unarchives the result in `local/file`. Because the
   166  source URL is an archive extension, Nomad will automatically decompress it:
   167  
   168  ```hcl
   169  artifact {
   170    source = "https://example.com/file.tar.gz"
   171  }
   172  ```
   173  
   174  To disable automatic unarchiving, set the `archive` option to false:
   175  
   176  ```hcl
   177  artifact {
   178    source = "https://example.com/file.tar.gz"
   179    options {
   180      archive = false
   181    }
   182  }
   183  ```
   184  
   185  ### Download and Verify Checksums
   186  
   187  This example downloads an artifact and verifies the resulting artifact's
   188  checksum before proceeding. If the checksum is invalid, an error will be
   189  returned.
   190  
   191  ```hcl
   192  artifact {
   193    source = "https://example.com/file.zip"
   194  
   195    options {
   196      checksum = "md5:df6a4178aec9fbdc1d6d7e3634d1bc33"
   197    }
   198  }
   199  ```
   200  
   201  ### Download from an S3-compatible Bucket
   202  
   203  These examples download artifacts from Amazon S3. There are several different
   204  types of [S3 bucket addressing][s3-bucket-addr] and [S3 region-specific
   205  endpoints][s3-region-endpoints]. As of Nomad 0.6 non-Amazon S3-compatible
   206  endpoints like [Minio] are supported, but you must explicitly set the "s3::"
   207  prefix.
   208  
   209  This example uses path-based notation on a publicly-accessible bucket:
   210  
   211  ```hcl
   212  artifact {
   213    source = "s3://my-bucket-example.s3-us-west-2.amazonaws.com/my_app.tar.gz"
   214  }
   215  ```
   216  
   217  If a bucket requires authentication, you can avoid the use of credentials by
   218  using [EC2 IAM instance profiles][iam-instance-profiles]. If this is not possible,
   219  credentials may be supplied via the `options` parameter:
   220  
   221  ```hcl
   222  artifact {
   223    options {
   224      aws_access_key_id     = "<id>"
   225      aws_access_key_secret = "<secret>"
   226      aws_access_token      = "<token>"
   227    }
   228  }
   229  ```
   230  
   231  To force the S3-specific syntax, use the `s3::` prefix:
   232  
   233  ```hcl
   234  artifact {
   235    source = "s3::https://my-bucket-example.s3-eu-west-1.amazonaws.com/my_app.tar.gz"
   236  }
   237  ```
   238  
   239  Alternatively you can use virtual hosted style:
   240  
   241  ```hcl
   242  artifact {
   243    source = "s3://my-bucket-example.s3-eu-west-1.amazonaws.com/my_app.tar.gz"
   244  }
   245  ```
   246  
   247  [client_artifact]: /docs/configuration/client#artifact-parameters
   248  [go-getter]: https://github.com/hashicorp/go-getter 'HashiCorp go-getter Library'
   249  [go-getter-headers]: https://github.com/hashicorp/go-getter#headers 'HashiCorp go-getter Headers'
   250  [minio]: https://www.minio.io/
   251  [s3-bucket-addr]: http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#access-bucket-intro 'Amazon S3 Bucket Addressing'
   252  [s3-region-endpoints]: http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region 'Amazon S3 Region Endpoints'
   253  [iam-instance-profiles]: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html 'EC2 IAM instance profiles'
   254  [task's working directory]: /docs/runtime/environment#task-directories 'Task Directories'
   255  [filesystem internals]: /docs/concepts/filesystem#templates-artifacts-and-dispatch-payloads