QEMU Driver

Name: qemu

The qemu driver provides a generic virtual machine runner. QEMU can utilize the KVM kernel module to utilize hardware virtualization features and provide great performance. Currently the qemu driver can map a set of ports from the host machine to the guest virtual machine, and provides configuration for resource allocation.

The qemu driver can execute any regular qemu image (e.g. qcow, img, iso), and is currently invoked with qemu-system-x86_64.

The driver requires the image to be accessible from the Nomad client via the artifact downloader.

Task Configuration

task "webservice" {
  driver = "qemu"

  config {
    image_path        = "/path/to/my/linux.img"
    accelerator       = "kvm"
    graceful_shutdown = true
    args              = ["-nodefaults", "-nodefconfig"]
  }
}

The qemu driver supports the following configuration in the job spec:

image_path - The path to the downloaded image. In most cases this will just be the name of the image. However, if the supplied artifact is an archive that contains the image in a subfolder, the path will need to be the relative path (subdir/from_archive/my.img).
drive_interface - (Optional) This option defines on which type of interface the drive is connected. Available types are: ide, scsi, sd, mtd, floppy, pflash, virtio and none. Default is ide.
accelerator - (Optional) The type of accelerator to use in the invocation. If the host machine has qemu installed with KVM support, users can specify kvm for the accelerator. Default is tcg.
graceful_shutdown (bool: false) - Using the qemu monitor, send an ACPI shutdown signal to virtual machines rather than simply terminating them. This emulates a physical power button press, and gives instances a chance to shut down cleanly. If the VM is still running after kill_timeout, it will be forcefully terminated. This feature uses a Unix socket that is placed within the task directory and operating systems may impose a limit on how long these paths can be. This feature is currently not supported on Windows.
guest_agent (bool: false) - Enable support for the QEMU Guest Agent for this virtual machine. This will add the necessary virtual hardware and create a qa.sock file in the task's working directory for interacting with the agent. The QEMU Guest Agent must be running in the guest VM. This feature is currently not supported on Windows.

port_map - (Optional) A key-value map of port labels.

config {
  # Forward the host port with the label "db" to the guest VM's port 6539.
  port_map {
    db = 6539
  }
}

args - (Optional) A list of strings that is passed to QEMU as command line options.

Examples

A simple config block to run a qemu image:

task "virtual" {
  driver = "qemu"
 
  config {
    image_path  = "local/linux.img"
    accelerator = "kvm"
    args        = ["-nodefaults", "-nodefconfig"]
  }
 
  # Specifying an artifact is required with the "qemu"
  # driver. This is the # mechanism to ship the image to be run.
  artifact {
    source = "https://internal.file.server/linux.img"
 
    options {
      checksum = "md5:123445555555555"
    }
  }

Capabilities

The qemu driver implements the following capabilities.

Feature	Implementation
`nomad alloc signal`	false
`nomad alloc exec`	false
filesystem isolation	image
network isolation	none
volume mounting	none

Client Requirements

The qemu driver requires QEMU to be installed and in your system's $PATH. The task must also specify at least one artifact to download, as this is the only way to retrieve the image being run.

Client Attributes

The qemu driver will set the following client attributes:

driver.qemu - Set to 1 if QEMU is found on the host node. Nomad determines this by executing qemu-system-x86_64 -version on the host and parsing the output
driver.qemu.version - Version of qemu-system-x86_64, ex: 2.4.0

Here is an example of using these properties in a job file:

job "docs" {
  # Only run this job where the qemu version is higher than 1.2.3.
  constraint {
    attribute = "${driver.qemu.version}"
    operator  = ">"
    value     = "1.2.3"
  }
}

Plugin Options

plugin "qemu" {
  config {
    image_paths    = ["/mnt/image/paths"]
    args_allowlist = ["-drive", "-usbdevice"]
  }
}

image_paths ([]string: []) - Specifies the host paths the QEMU driver is allowed to load images from.
args_allowlist ([]string: []) - Specifies the command line flags that the args option is permitted to pass to QEMU. If unset, a job submitter can pass any command line flag into QEMU, including flags that provide the VM with access to host devices such as USB drives. Refer to the QEMU documentation for the available flags.

Resource Isolation

Nomad uses QEMU to provide full software virtualization for virtual machine workloads. Nomad can use QEMU KVM's hardware-assisted virtualization to deliver better performance.

Virtualization provides the highest level of isolation for workloads that require additional security, and resource use is constrained by the QEMU hypervisor rather than the host kernel. VM network traffic still flows through the host's interface(s).

Note that the strong isolation provided by virtualization only applies to the workload once the VM is started. Operators should use the args_allowlist option to prevent job submitters from accessing devices and resources they are not allowed to access.