Parallel builds

4min
|
Packer

In the previous tutorials, you used Packer to automatically build an image and provision it from a template. Then, you parameterized the template allowing you to build different images using the same template. While this is already quite powerful, Packer can create multiple images in parallel, all configured from a single template.

Parellel build is a very useful and important feature of Packer. For example, Packer can build an Amazon AMI and a VMware virtual machine in parallel provisioned with the same scripts, resulting in near-identical images. The AMI can be used for production, the VMware machine can be used for development. Or, another example, if you're using Packer to build software appliances, then you can build the appliance for every supported platform all in parallel and configured from a single template.

In this tutorial, you will build a second Docker image in parallel.While the second image is a Docker image for the sake of this tutorial, the second source could just as easily come from another hypervisor or a cloud platform.

Prerequisites

This tutorial assumes that you are continuing from the previous tutorials. If not, follow the steps below before continuing.

Install Packer

Create a directory named packer_tutorial and paste the following configuration into a file named aws-ubuntu.pkr.hcl.

packer {
  required_plugins {
    amazon = {
      version = ">= 1.2.8"
      source  = "github.com/hashicorp/amazon"
    }
  }
}

variable "ami_prefix" {
  type    = string
  default = "learn-packer-linux-aws-redis"
}

locals {
  timestamp = regex_replace(timestamp(), "[- TZ:]", "")
}

source "amazon-ebs" "ubuntu" {
  ami_name      = "${var.ami_prefix}-${local.timestamp}"
  instance_type = "t2.micro"
  region        = "us-west-2"
  source_ami_filter {
    filters = {
      name                = "ubuntu/images/*ubuntu-jammy-22.04-amd64-server-*"
      root-device-type    = "ebs"
      virtualization-type = "hvm"
    }
    most_recent = true
    owners      = ["099720109477"]
  }
  ssh_username = "ubuntu"
}

build {
  name = "learn-packer"
  sources = [
    "source.amazon-ebs.ubuntu"
  ]

  provisioner "shell" {
    environment_vars = [
      "FOO=hello world",
    ]
    inline = [
      "echo Installing Redis",
      "sleep 30",
      "sudo apt-get update",
      "sudo apt-get install -y redis-server",
      "echo \"FOO is $FOO\" > example.txt",
    ]
  }

  provisioner "shell" {
    inline = ["echo This provisioner runs last"]
  }
}

Initialize the Packer template.
```
$ packer init .
```

Once you have successfully initialized the template, you can continue with the rest of this tutorial.

Add parallel builds to template

To use parallel builds, create a source then add the source to the sources array in the build block. Your sources do not need to be the same type. This tells Packer to build multiple images when that build is run.

Add the following source block to your aws-ubuntu.pkr.hcl file. This is similar to the existing source block but has -focal in the AMI name and uses ubuntu-focal-20.04 as the base AMI.

source "amazon-ebs" "ubuntu-focal" {
  ami_name      = "${var.ami_prefix}-focal-${local.timestamp}"
  instance_type = "t2.micro"
  region        = "us-west-2"
  source_ami_filter {
    filters = {
      name                = "ubuntu/images/*ubuntu-focal-20.04-amd64-server-*"
      root-device-type    = "ebs"
      virtualization-type = "hvm"
    }
    most_recent = true
    owners      = ["099720109477"]
  }
  ssh_username = "ubuntu"
}

Then update your build block to use the new source.

build {
  name    = "learn-packer"
  sources = [
-    "source.amazon-ebs.ubuntu"
+    "source.amazon-ebs.ubuntu",
+    "source.amazon-ebs.ubuntu-focal"
  ]
  ## ...
}

Build images

Build the images. Packer will display color-coded output for both builds. You can tell which build source an output line is associated with using either the line color or prefix. In this example, there are two builds: amazon-ebs.ubuntu and amazon-ebs.ubuntu-focal.

$ packer build .
mazon-ebs.ubuntu: output will be in this color.
learn-packer.amazon-ebs.ubuntu-focal: output will be in this color.

==> learn-packer.amazon-ebs.ubuntu-focal: Prevalidating any provided VPC information
==> learn-packer.amazon-ebs.ubuntu-focal: Prevalidating AMI Name: learn-packer-linux-aws-redis-focal-20210429173057
==> learn-packer.amazon-ebs.ubuntu: Prevalidating any provided VPC information
==> learn-packer.amazon-ebs.ubuntu: Prevalidating AMI Name: learn-packer-linux-aws-redis-20210429173057
    learn-packer.amazon-ebs.ubuntu-focal: Found Image ID: ami-0554443e7f55d3eeb
### ...

==> Wait completed after 8 minutes 47 seconds

==> Builds finished. The artifacts of successful builds are:
--> learn-packer.amazon-ebs.ubuntu: AMIs were created:
us-west-2: ami-0e3795bd3628d430b

--> learn-packer.amazon-ebs.ubuntu-focal: AMIs were created:
us-west-2: ami-014a9d431eddddc2d

Next steps

In this tutorial, you updated your template to build multiple images in parallel. Continue to the next tutorial to add a post-processing step to your Packer template.

Refer to the following resources for additional details on the concepts covered in this tutorial:

Learn how to restrict provisioners to certain build sources with the only and except attributes.

Variables

Post-processors - Vagrant