Placement

When the Nomad scheduler receives a job registration request, it needs to determine which clients will run allocations for the job.

This process is called allocation placement and can be important to understand it to help achieve important goals for your applications, such as high availability and resilience.

By default, all nodes are considered for placements but this process can be adjusted via agent and job configuration.

There are several options that can be used depending on the desired outcome.

Affinities and Constraints

Affinities and constraints allow users to define soft or hard requirements for their jobs. The affinity block specifies a soft requirement on certain node properties, meaning allocations for the job have a preference for some nodes, but may be placed elsewhere if the rules can't be matched, while the constraint block creates hard requirements and allocations can only be placed in nodes that match these rules. Job placement fails if a constraint cannot be satisfied.

These rules can reference intrinsic node characteristics, which are called node attributes and are automatically detected by Nomad, static values defined in the agent configuration file by cluster administrators, or dynamic values defined after the agent starts.

One restriction of using affinities and constraints is that they only express relationships from jobs to nodes, so it is not possible to use them to restrict a node to only receive allocations for specific jobs.

Use affinities and constraints when some jobs have certain node preferences or requirements but it is acceptable to have other jobs sharing the same nodes.

The sections below describe the node values that can be configured and used in job affinity and constraint rules.

Node Class

Node class is an arbitrary value that can be used to group nodes based on some characteristics, like the instance size or the presence of fast hard drives, and is specified in the client configuration file using the node_class parameter.

Dynamic and Static Node Metadata

Node metadata are arbitrary key-value mappings specified either in the client configuration file using the meta parameter or dynamically via the nomad node meta command and the /v1/client/metadata API endpoint.

There are no preconceived use cases for metadata values, and each team may choose to use them in different ways. Some examples of static metadata include resource ownership, such as owner = "team-qa", or fine-grained locality, rack = "3". Dynamic metadata may be used to track runtime information, such as jobs running in a given client.

Datacenter

Datacenters represent a geographical location in a region that can be used for fault tolerance and infrastructure isolation.

It is defined in the agent configuration file using the datacenter parameter and, unlike affinities and constraints, datacenters are opt-in at the job level, meaning that a job only places allocations in the datacenters it uses, and, more importantly, only jobs in a given datacenter are allowed to place allocations in those nodes.

Given the strong connotation of a geographical location, use datacenters to represent where a node resides rather than its intended use. The spread block can help achieve fault tolerance across datacenters.

Node Pool

Node pools allow grouping nodes that can be targeted by jobs to achieve workload isolation.

Similarly to datacenters, node pools are configured in an agent configuration file using the node_pool attribute, and are opt-in on jobs, allowing restricted use of certain nodes by specific jobs without extra configuration.

But unlike datacenters, node pools don't have a preconceived notion and can be used for several use cases, such as segmenting infrastructure per environment (development, staging, production), by department (engineering, finance, support), or by functionality (databases, ingress proxy, applications).

Node pools are also a first-class concept and can hold additional metadata and configuration.

Use node pools when there is a need to restrict and reserve certain nodes for specific workloads, or when you need to adjust specific scheduler configuration values.

Nomad Enterprise also allows associating a node pool to a namespace to facilitate managing the relationships between jobs, namespaces, and node pools.

Refer to the Node Pools concept page for more information.