HashiConf 2024 Now streaming live from Boston! Attend for free
Nomad
Nomad Home

artifact Block

Placementjob -> group -> task -> artifact

The artifact block instructs Nomad to fetch and unpack a remote resource, such as a file, tarball, or binary. Nomad downloads artifacts using the popular go-getter library, which permits downloading artifacts from a variety of locations using a URL as the input source.

job "docs" {
  group "example" {
    task "server" {
      artifact {
        source      = "https://example.com/file.tar.gz"
        destination = "local/some-directory"
        options {
          checksum = "md5:df6a4178aec9fbdc1d6d7e3634d1bc33"
        }
      }
    }
  }
}

Nomad supports downloading http, https, git, hg and S3 artifacts. If these artifacts are archived (zip, tgz, bz2, xz), they are automatically unarchived before the starting the task.

artifact Parameters

  • destination (string: "local/") - Specifies the directory path to download the artifact, relative to the root of the task's working directory. If omitted, the default value is to place the artifact in local/. The destination is treated as a directory unless mode is set to file. Source files will be downloaded into that directory path. For more details on how the destination interacts with task drivers, see the Filesystem internals documentation.

  • mode (string: "any") - One of any, file, or dir. If set to file the destination must be a file, not a directory. By default the destination will be local/<filename>.

  • options (map<string|string>: nil) - Specifies configuration parameters to fetch the artifact. The key-value pairs map directly to parameters appended to the supplied source URL. Please see the go-getter documentation for a complete list of options and examples.

  • headers (map<string|string>: nil) - Specifies HTTP headers to set when fetching the artifact using http or https protocol. Please see the go-getter headers documentation for more information.

  • source (string: <required>) - Specifies the URL of the artifact to download. See go-getter for details.

Operation Limits

The client artifact configuration can set limits to specific artifact operations to prevent excessive data download or operation time.

If a task's artifact retrieval exceeds one of those limits, the task will be interrupted and fail to start. Refer to the task events for more information.

artifact Examples

The following examples only show the artifact blocks. Remember that the artifact block is only valid in the placements listed above.

Download File

This example downloads the artifact from the provided URL and places it in local/file.txt. The local/ path is relative to the task's working directory.

artifact {
  source = "https://example.com/file.txt"
}

To set HTTP headers in the request for the source the optional headers field can be configured.

artifact {
  source = "https://example.com/file.txt"

  headers {
    User-Agent    = "nomad-[${NOMAD_JOB_ID}]-[${NOMAD_GROUP_NAME}]-[${NOMAD_TASK_NAME}]"
    X-Nomad-Alloc = "${NOMAD_ALLOC_ID}"
  }
}

To use HTTP basic authentication, preprend the username and password to the hostname in the URL. All special characters, including the username and password, must be URL encoded. For example, for a username exampleUser and the password pass/word!:

artifact {
  source = "https://exampleUser:pass%2Fword%21@example.com/file.txt"
}

Download using git

This example downloads the artifact from the provided GitHub URL and places it at local/repo, as specified by the optional destination parameter.

artifact {
  source      = "git::https://github.com/hashicorp/nomad-guides"
  destination = "local/repo"
}

To download from a private repo, sshkey needs to be set. The key must be base64-encoded string. On Linux, you can run base64 -w0 <file> to encode the file. Or use HCL2 expressions to read and encode the key from a file on your machine:

artifact {
  # The git:: prefix forces go-getter's protocol detection to use the git ssh
  # protocol. It can also automatically detect the protocol from the domain of
  # some git hosting providers (such as GitHub) without the prefix.
  source      = "git::git@bitbucket.org:example/nomad-examples"
  destination = "local/repo"
  options {
    # Make sure that the system known hosts file is populated:
    # ssh-keyscan github.com | sudo tee -a /etc/ssh/ssh_known_hosts
    # https://github.com/hashicorp/go-getter/issues/55
    sshkey = "${base64encode(file("/path/to/private-key"))}"
  }
}

To clone specific refs or at a specific depth, use the ref and depth options:

artifact {
  source      = "git::https://github.com/hashicorp/nomad-guides"
  destination = "local/repo"
  options {
    ref = "main"
    depth = 1
  }
}

Download and Unarchive

This example downloads and unarchives the result in local/file. Because the source URL is an archive extension, Nomad will automatically decompress it:

artifact {
  source = "https://example.com/file.tar.gz"
}

To disable automatic unarchiving, set the archive option to false:

artifact {
  source = "https://example.com/file.tar.gz"
  options {
    archive = false
  }
}

Download and Verify Checksums

This example downloads an artifact and verifies the resulting artifact's checksum before proceeding. If the checksum is invalid, an error will be returned.

artifact {
  source = "https://example.com/file.zip"

  options {
    checksum = "md5:df6a4178aec9fbdc1d6d7e3634d1bc33"
  }
}

Download from an S3-compatible Bucket

These examples download artifacts from Amazon S3. There are several different types of S3 bucket addressing and S3 region-specific endpoints. Non-Amazon S3-compatible endpoints like Minio are supported, but you must explicitly set the "s3::" prefix.

This example uses path-based notation on a publicly-accessible bucket:

artifact {
  source = "s3://my-bucket-example.s3-us-west-2.amazonaws.com/my_app.tar.gz"
}

If a bucket requires authentication, you can avoid the use of credentials by using EC2 IAM instance profiles. If this is not possible, credentials may be supplied via the options parameter:

artifact {
  options {
    aws_access_key_id     = "<id>"
    aws_access_key_secret = "<secret>"
    aws_access_token      = "<token>"
  }
}

To force the S3-specific syntax, use the s3:: prefix:

artifact {
  source = "s3::https://my-bucket-example.s3-eu-west-1.amazonaws.com/my_app.tar.gz"
}

Alternatively you can use virtual hosted style:

artifact {
  source = "s3://my-bucket-example.s3-eu-west-1.amazonaws.com/my_app.tar.gz"
}

DigitalOcean Spaces provide S3-compatible object storage and can be used in conjunction with the artifact block. Given a bucket named my-bucket-example located in the sfo3 region and a file named my_app.tar.gz, the following artifact block will work, provided the bucket and file are public. This uses the origin endpoint as detailed directly from Digital Ocean.

artifact {
  source = "https://my-bucket-example.sfo3.digitaloceanspaces.com/my_app.tar.gz"
}

If the bucket or file are private, the artifact source string needs to be modified to add the s3:: prefix. The bucket name is also moved to form part of the URL path component.

artifact {
  source = "s3::https://sfo3.digitaloceanspaces.com/my-bucket-example/my_app.tar.gz"
}

Environment

The artifact downloader by default does not have access to the environment variables set for the Nomad client. Inheritance of environment variables can be managed through the artifact.set_environment_variables client configuration.

Edit this page on GitHub