github.com/anth0d/nomad@v0.0.0-20221214183521-ae3a0a2cad06/website/content/docs/job-specification/artifact.mdx (about) 1 --- 2 layout: docs 3 page_title: artifact Stanza - Job Specification 4 description: |- 5 The "artifact" stanza instructs Nomad to fetch and unpack a remote resource, 6 such as a file, tarball, or binary, and permits downloading artifacts from a 7 variety of locations using a URL as the input source. 8 --- 9 10 # `artifact` Stanza 11 12 <Placement groups={['job', 'group', 'task', 'artifact']} /> 13 14 The `artifact` stanza instructs Nomad to fetch and unpack a remote resource, 15 such as a file, tarball, or binary. Nomad downloads artifacts using the popular 16 [`go-getter`][go-getter] library, which permits downloading artifacts from a 17 variety of locations using a URL as the input source. 18 19 ```hcl 20 job "docs" { 21 group "example" { 22 task "server" { 23 artifact { 24 source = "https://example.com/file.tar.gz" 25 destination = "local/some-directory" 26 options { 27 checksum = "md5:df6a4178aec9fbdc1d6d7e3634d1bc33" 28 } 29 } 30 } 31 } 32 } 33 ``` 34 35 Nomad supports downloading `http`, `https`, `git`, `hg` and `S3` artifacts. If 36 these artifacts are archived (`zip`, `tgz`, `bz2`, `xz`), they are 37 automatically unarchived before the starting the task. 38 39 ## `artifact` Parameters 40 41 - `destination` `(string: "local/")` - Specifies the directory path to 42 download the artifact, relative to the root of the [task's working 43 directory]. If omitted, the default value is to place the artifact in 44 `local/`. The destination is treated as a directory unless `mode` is set to 45 `file`. Source files will be downloaded into that directory path. For more 46 details on how the `destination` interacts with task drivers, see the 47 [Filesystem internals] documentation. 48 49 - `mode` `(string: "any")` - One of `any`, `file`, or `dir`. If set to `file` 50 the `destination` must be a file, not a directory. By default the 51 `destination` will be `local/<filename>`. 52 53 - `options` `(map<string|string>: nil)` - Specifies configuration parameters to 54 fetch the artifact. The key-value pairs map directly to parameters appended to 55 the supplied `source` URL. Please see the [`go-getter` 56 documentation][go-getter] for a complete list of options and examples. 57 58 - `headers` `(map<string|string>: nil)` - Specifies HTTP headers to set when 59 fetching the artifact using `http` or `https` protocol. Please see the 60 [`go-getter` headers documentation][go-getter-headers] for more information. 61 62 - `source` `(string: <required>)` - Specifies the URL of the artifact to download. 63 See [`go-getter`][go-getter] for details. 64 65 ## Operation Limits 66 67 The client [`artifact`][client_artifact] configuration can set limits to 68 specific artifact operations to prevent excessive data download or operation 69 time. 70 71 If a task's `artifact` retrieval exceeds one of those limits, the task will be 72 interrupted and fail to start. Refer to the task events for more information. 73 74 ## `artifact` Examples 75 76 The following examples only show the `artifact` stanzas. Remember that the 77 `artifact` stanza is only valid in the placements listed above. 78 79 ### Download File 80 81 This example downloads the artifact from the provided URL and places it in 82 `local/file.txt`. The `local/` path is relative to the [task's working 83 directory]. 84 85 ```hcl 86 artifact { 87 source = "https://example.com/file.txt" 88 } 89 ``` 90 91 To set HTTP headers in the request for the source the optional `headers` field 92 can be configured. 93 94 ```hcl 95 artifact { 96 source = "https://example.com/file.txt" 97 98 headers { 99 User-Agent = "nomad-[${NOMAD_JOB_ID}]-[${NOMAD_GROUP_NAME}]-[${NOMAD_TASK_NAME}]" 100 X-Nomad-Alloc = "${NOMAD_ALLOC_ID}" 101 } 102 } 103 ``` 104 105 To use HTTP basic authentication, preprend the username and password to the 106 hostname in the URL. All special characters, including the username and 107 password, must be URL encoded. For example, for a username `exampleUser` and 108 the password `pass/word!`: 109 110 ```hcl 111 artifact { 112 source = "https://exampleUser:pass%2Fword%21@example.com/file.txt" 113 } 114 ``` 115 116 ### Download using git 117 118 This example downloads the artifact from the provided GitHub URL and places it at 119 `local/repo`, as specified by the optional `destination` parameter. 120 121 ```hcl 122 artifact { 123 source = "git::https://github.com/hashicorp/nomad-guides" 124 destination = "local/repo" 125 } 126 ``` 127 128 To download from a private repo, sshkey needs to be set. The key must be 129 base64-encoded string. On Linux, you can run `base64 -w0 <file>` to encode the 130 file. Or use [HCL2](https://www.nomadproject.io/docs/job-specification/hcl2) 131 expressions to read and encode the key from a file on your machine: 132 133 ```hcl 134 artifact { 135 # The git:: prefix forces go-getter's protocol detection to use the git ssh 136 # protocol. It can also automatically detect the protocol from the domain of 137 # some git hosting providers (such as GitHub) without the prefix. 138 source = "git::git@bitbucket.org:example/nomad-examples" 139 destination = "local/repo" 140 options { 141 # Make sure that the system known hosts file is populated: 142 # ssh-keyscan github.com | sudo tee -a /etc/ssh/ssh_known_hosts 143 # https://github.com/hashicorp/go-getter/issues/55 144 sshkey = "${base64encode(file("/etc/ssh/ssh_known_hosts"))}" 145 } 146 } 147 ``` 148 149 To clone specific refs or at a specific depth, use the `ref` and `depth` 150 options: 151 152 ```hcl 153 artifact { 154 source = "git::https://github.com/hashicorp/nomad-guides" 155 destination = "local/repo" 156 options { 157 ref = "main" 158 depth = 1 159 } 160 } 161 ``` 162 163 ### Download and Unarchive 164 165 This example downloads and unarchives the result in `local/file`. Because the 166 source URL is an archive extension, Nomad will automatically decompress it: 167 168 ```hcl 169 artifact { 170 source = "https://example.com/file.tar.gz" 171 } 172 ``` 173 174 To disable automatic unarchiving, set the `archive` option to false: 175 176 ```hcl 177 artifact { 178 source = "https://example.com/file.tar.gz" 179 options { 180 archive = false 181 } 182 } 183 ``` 184 185 ### Download and Verify Checksums 186 187 This example downloads an artifact and verifies the resulting artifact's 188 checksum before proceeding. If the checksum is invalid, an error will be 189 returned. 190 191 ```hcl 192 artifact { 193 source = "https://example.com/file.zip" 194 195 options { 196 checksum = "md5:df6a4178aec9fbdc1d6d7e3634d1bc33" 197 } 198 } 199 ``` 200 201 ### Download from an S3-compatible Bucket 202 203 These examples download artifacts from Amazon S3. There are several different 204 types of [S3 bucket addressing][s3-bucket-addr] and [S3 region-specific 205 endpoints][s3-region-endpoints]. As of Nomad 0.6 non-Amazon S3-compatible 206 endpoints like [Minio] are supported, but you must explicitly set the "s3::" 207 prefix. 208 209 This example uses path-based notation on a publicly-accessible bucket: 210 211 ```hcl 212 artifact { 213 source = "s3://my-bucket-example.s3-us-west-2.amazonaws.com/my_app.tar.gz" 214 } 215 ``` 216 217 If a bucket requires authentication, you can avoid the use of credentials by 218 using [EC2 IAM instance profiles][iam-instance-profiles]. If this is not possible, 219 credentials may be supplied via the `options` parameter: 220 221 ```hcl 222 artifact { 223 options { 224 aws_access_key_id = "<id>" 225 aws_access_key_secret = "<secret>" 226 aws_access_token = "<token>" 227 } 228 } 229 ``` 230 231 To force the S3-specific syntax, use the `s3::` prefix: 232 233 ```hcl 234 artifact { 235 source = "s3::https://my-bucket-example.s3-eu-west-1.amazonaws.com/my_app.tar.gz" 236 } 237 ``` 238 239 Alternatively you can use virtual hosted style: 240 241 ```hcl 242 artifact { 243 source = "s3://my-bucket-example.s3-eu-west-1.amazonaws.com/my_app.tar.gz" 244 } 245 ``` 246 247 [client_artifact]: /docs/configuration/client#artifact-parameters 248 [go-getter]: https://github.com/hashicorp/go-getter 'HashiCorp go-getter Library' 249 [go-getter-headers]: https://github.com/hashicorp/go-getter#headers 'HashiCorp go-getter Headers' 250 [minio]: https://www.minio.io/ 251 [s3-bucket-addr]: http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#access-bucket-intro 'Amazon S3 Bucket Addressing' 252 [s3-region-endpoints]: http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region 'Amazon S3 Region Endpoints' 253 [iam-instance-profiles]: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html 'EC2 IAM instance profiles' 254 [task's working directory]: /docs/runtime/environment#task-directories 'Task Directories' 255 [filesystem internals]: /docs/concepts/filesystem#templates-artifacts-and-dispatch-payloads