Nomad
Runtime Environment
Some settings you specify in your job specification are passed to tasks when they start. Other settings are dynamically allocated when your job is scheduled. Both types of values are made available to your job through environment variables.
Summary
Variable | Description |
---|---|
NOMAD_ALLOC_DIR | The path to the shared alloc/ directory. Seehere for more information. |
NOMAD_TASK_DIR | The path to the task local/ directory. Seehere for more information. |
NOMAD_SECRETS_DIR | Path to the task's secrets directory. Seehere for more information. |
NOMAD_MEMORY_LIMIT | Memory limit in MB for the task |
NOMAD_CPU_LIMIT | CPU limit in MHz for the task |
NOMAD_ALLOC_ID | Allocation ID of the task |
NOMAD_ALLOC_NAME | Allocation name of the task |
NOMAD_ALLOC_INDEX | Allocation index; useful to distinguish instances of task groups. From 0 to (count - 1). The index is unique within a given version of a job, but canaries or failed tasks in a deployment may reuse the index. |
NOMAD_TASK_NAME | Task's name |
NOMAD_GROUP_NAME | Group's name |
NOMAD_JOB_ID | Job's ID, which is equal to the Job name when submitted through CLI but can be different when using the API |
NOMAD_JOB_NAME | Job's name |
NOMAD_JOB_PARENT_ID | ID of the Job's parent if it has one |
NOMAD_DC | Datacenter in which the allocation is running |
NOMAD_NAMESPACE | Namespace in which the allocation is running |
NOMAD_REGION | Region in which the allocation is running |
NOMAD_META_<key> | The metadata value given by key on the task's metadata. Note that this is different from${meta.<key>} which are keys in the node's metadata. |
VAULT_TOKEN | The task's Vault token. SeeVault Integrationfor more details |
Network-related Variables | |
NOMAD_IP_<label> | Host IP for the given port label . Seehere for more information. |
NOMAD_PORT_<label> | Port for the given port label . Driver-specified port when a port map is used, otherwise the host's static or dynamic port allocation. Services should bind to this port. Seehere for more information. |
NOMAD_ADDR_<label> | Host IP:Port pair for the given port label . |
NOMAD_HOST_PORT_<label> | Port on the host for the port label . Seehere for more information. |
NOMAD_IP_<task>_<label> | Host IP for the given port label and task for tasks in the same task group. |
NOMAD_PORT_<task>_<label> | Port for the given port label and task for tasks in the same task group. Driver-specified port when a port map is used, otherwise the host's static or dynamic port allocation. Services should bind to this port. |
NOMAD_ADDR_<task>_<label> | Host IP:Port pair for the given port label andtask for tasks in the same task group. |
NOMAD_HOST_PORT_<task>_<label> | Port on the host for the port label and task for tasks in the same task group. |
NOMAD_UPSTREAM_IP_<service> | IP for the given service when defined as a Consul Connectupstream. |
NOMAD_UPSTREAM_PORT_<service> | Port for the given service when defined as a Consul Connectupstream. |
NOMAD_UPSTREAM_ADDR_<service> | Host IP:Port for the given service when defined as a Consul Connectupstream. |
NOMAD_ENVOY_ADMIN_ADDR_<service> | Local address localhost:Port for the admin port of the envoy sidecar for the given service when defined as a Consul Connect enabled service. |
Port labels and task names will have any non-alphanumeric or underscore
characters in their names replaced by underscores _
when they're used in
environment variable names such as NOMAD_ADDR_<task>_<label>
.
Task Identifiers
Nomad will pass both the allocation ID and name, the deployment ID that created
the allocation, the job ID and name, the parent job ID as well as
the task and group's names. These are given as NOMAD_ALLOC_ID
, NOMAD_ALLOC_NAME
,
NOMAD_ALLOC_INDEX
, NOMAD_JOB_NAME
, NOMAD_JOB_ID
,
NOMAD_JOB_PARENT_ID
, NOMAD_GROUP_NAME
and NOMAD_TASK_NAME
. The allocation ID
and index can be useful when the task being run needs a unique identifier or to
know its instance count.
Resources
When you request resources for a job, Nomad creates a resource offer. The final resources for your job are not determined until it is scheduled. Nomad will tell you which resources have been allocated after evaluation and placement.
CPU and Memory
Nomad will pass CPU and memory limits to your job as NOMAD_CPU_LIMIT
and
NOMAD_MEMORY_LIMIT
. Your task should use these values to adapt its behavior to
fit inside the resource allocation that nomad provides. For example, you can use
the memory limit to inform how large your in-process cache should be, or to
decide when to flush buffers to disk.
Both CPU and memory are presented as integers. The unit for CPU limit is
1024 = 1GHz
. The unit for memory is 1 = 1 megabyte
.
Writing your applications to adjust to these values at runtime provides greater scheduling flexibility since you can adjust the resource allocations in your job specification without needing to change your code. You can also schedule workloads that accept dynamic resource allocations so they can scale down/up as your cluster gets more or less busy.
Networking
Nomad assigns IPs and ports to your jobs and exposes them via environment variables. See the Networking page for more details.
Task Directories
Nomad creates a working directory for each allocation on a client. The allocation working directory contains a task working directory for each task in the allocation.
Nomad makes the following directories available to tasks, relative to the task working directory:
alloc/
: This directory is shared across all tasks in a task group and can be used to store data that needs to be used by multiple tasks, such as a log shipper.local/
: This directory is private to each task. It can be used to store arbitrary data that should not be shared by tasks in the task group.secrets/
: This directory is private to each task, not accessible via thenomad alloc fs
command or filesystem APIs and where possible backed by an in-memory filesystem. It can be used to store secret data that should not be visible outside the task.
These directories are persisted until the allocation is removed, which occurs hours after all the tasks in the task group enter terminal states. This gives time to view the data produced by tasks.
Depending on the driver and operating system being targeted, the directories
are made available in various ways. For example, on docker
the directories
are bound to the container, while on exec
on Linux the chroot is built in
the task working directory, and the directories are mounted into that
chroot. Regardless of how the directories are made available, the path to the
directories can be read through the NOMAD_ALLOC_DIR
, NOMAD_TASK_DIR
, and
NOMAD_SECRETS_DIR
environment variables.
For more details on the task directories, see the Filesystem internals.
Meta
The job specification also allows you to specify a meta
block to supply arbitrary
configuration to a task. This allows you to easily provide job-specific
configuration even if you use the same executable unit in multiple jobs. These
key-value pairs are passed through to the job as NOMAD_META_<key>=<value>
environment variables. Prior to Nomad 0.5.5 the key was uppercased and since
then both the original case and an uppercased version are injected. The
uppercased version will be deprecated in a future release.
Currently there is no enforcement that the meta keys be lowercase, but using multiple keys with the same uppercased representation will lead to undefined behavior.