Automate Packer with GitHub Actions

Packer templates let you define artifacts as code, so you can establish a version control repository as the source of truth for artifact contents. This lets you adopt code review for changes and use automated builds to keep your artifacts up-to-date. HCP Packer further simplifies artifact creation and deployment by providing a central artifact registry for operations and development teams. By integrating Packer and HCP Packer into continuous integration (CI) pipelines, you can automatically trigger artifact builds on changes to your version control repository.

In this tutorial, you will set up a complete GitHub Actions workflow to automatically build and manage different versions of an artifact. Whenever you push changes to designated branches (main, development, staging) or create a new Git tag, the workflow will trigger a Packer build, send the build metadata to HCP Packer, and set the corresponding HCP Packer channel to the build version. In the process, you will learn how to automate and scale Packer builds across your organization.

Prerequisites

This tutorial assumes that you are familiar with the standard Packer and HCP Packer workflows. If you are new to Packer, complete the Get Started tutorials first. If you are new to HCP Packer, complete the Get Started HCP Packer tutorials first.

For this tutorial, you will need:

• A GitHub account
• An HCP account
• An HCP Packer Registry and HCP service principal
• An AWS account and AWS Access Credentials

Create example repository

Navigate to the template repository for this tutorial. Click the Use this template button and select Create a new repository. Choose a GitHub account to create the repository in and name the new repository learn-packer-github-actions. Leave the rest of the settings at their default values.

Configure GitHub action

In the repository you created, go to Settings, Secrets and variables, then Actions. Create the following repository secrets and set them to their respective values. The GitHub Action workflow in this repository will use these credentials to provision your resources and push metadata to HCP Packer.

Once complete, the Actions secrets page displays the six secrets.

Navigate to the Actions tab and enable the pre-configured workflow defined in the repository's .github/workflows directory by clicking I understand my workflows, go ahead and enable them.

Then, clone your repository to your local machine. If using the CLI, replace USERNAME with your GitHub username.

$ git clone https://github.com/USERNAME/learn-packer-github-actions

Change to the repository directory.

$ cd learn-packer-github-actions

Review configuration

In this section, you will review the Packer template file and the GitHub Actions workflow to understand how GitHub Actions will automate your artifact builds.

Review Packer template

Open build.pkr.hcl. This file contains Packer HCL code for generating a HashiCups machine image. HashiCups is a demo application that lets you view and order customized HashiCorp branded coffee. The HashiCups application consists of a frontend React application and multiple backend services.

The amazon-ebs.ubuntu-lts source block retrieves an Ubuntu 22.04 image from the us-west-1 region to use as the base image.

source "amazon-ebs" "ubuntu-lts" {
  region = "us-west-1"
  
  source_ami_filter {
    filters = {
      virtualization-type = "hvm"
      name                = "ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"
      root-device-type    = "ebs"
    }
    owners      = ["099720109477"]
    most_recent = true
  }
  
  instance_type  = "t2.small"
  ssh_username   = "ubuntu"
  ssh_agent_auth = false

  ami_name    = "hashicups_{{timestamp}}"
  ami_regions = ["us-west-1"]
}

Within the build block, the hcp_packer_registry block configures packer to send artifact metadata to the HCP Packer registry.

build {
  hcp_packer_registry {
    bucket_name = "learn-packer-github-actions"
    description = <<EOT
This is an image for HashiCups.
    EOT

    bucket_labels = {
      "hashicorp-learn" = "learn-packer-github-actions",
    }
  }
  ## ...
}

The amazon-ebs.ubuntu-lts block retrieves the build source. The file provisioner copies a systemd unit file for the HashiCups service. The shell provisioner runs the setup-deps-hashicups.sh script to install and configure the service.

build {
  ## ...
  sources = [
    "source.amazon-ebs.ubuntu-lts",
  ]

  # systemd unit for HashiCups service
  provisioner "file" {
    source      = "hashicups.service"
    destination = "/tmp/hashicups.service"
  }

  # Set up HashiCups
  provisioner "shell" {
    scripts = [
      "setup-deps-hashicups.sh"
    ]
  }
  ## ...
}

After building the artifact, Packer stores the version fingerprint in a file named packer_manifest.json. The GitHub Action retrieves the version fingerprint from this file and updates the respective channel to reference it. You will review how the GitHub action uses this value in the next section.

build {
  ## ...
  post-processor "manifest" {
    output     = "packer_manifest.json"
    strip_path = true
    custom_data = {
      version_fingerprint = packer.versionFingerprint
    }
  }
}

The image built by packer is now fully configured to run the HashiCups application. When a server using this image launches, systemd will start HashiCups.

Review Actions workflow

The .github directory contains the GitHub Actions workflow and a helper script.

Open .github/workflows/build_and_deploy.yaml to review the GitHub Actions workflow. The first line sets a name for the workflow.

name: Build and Deploy

Next, the on block restricts this workflow to new tags that comply with semantic versioning or pushes to the development, staging, or main branches.

on:
  push:
    tags: ["v[0-9].[0-9]+.[0-9]+"]
    branches:
      - "development"
      - "staging"
      - "main"

The env section defines environment variables, which all jobs and steps in the workflow will have access to. The workflow uses HCP_CLIENT_ID and HCP_CLIENT_SECRET to authenticate and send build metadata to HCP Packer. The helper script uses the remaining environment variables to create and update HCP Packer channels using the HCP Packer API.

env:
  HCP_CLIENT_ID: ${{ secrets.HCP_CLIENT_ID }}
  HCP_CLIENT_SECRET: ${{ secrets.HCP_CLIENT_SECRET }}
  HCP_PROJECT_ID: ${{ secrets.HCP_PROJECT_ID }}
  HCP_ORGANIZATION_ID: ${{ secrets.HCP_ORGANIZATION_ID }}
  HCP_BUCKET_NAME: "learn-packer-github-actions"

Finally, the configuration defines two jobs. The build-artifact job uses Packer to build the artifact, and the update-hcp-packer-channel job updates the HCP Packer channel to point to the newly built artifact.

Both jobs run on the ubuntu-latest image, which contains the Packer binary.

jobs:
  build-artifact:
    name: Build
    runs-on: ubuntu-latest
    ##...
  update-hcp-packer-channel:
    name: Update HCP Packer channel
    runs-on: ubuntu-latest
    ##...

The build-artifact job has the following steps:

1. Checkout clones the GitHub repository to retrieve the current configuration. The uses argument defines the Docker image to run for the step and uses GitHub's actions/checkout@v3 action.

   .github/workflows/build_and_deploy.yml

   - name: Checkout
     uses: actions/checkout@v3
   

2. Configure AWS Credentials creates environment variables using values retrieved from GitHub secrets and sets the AWS region to us-west-1.

   .github/workflows/build_and_deploy.yml

   - name: Configure AWS Credentials
         uses: aws-actions/configure-aws-credentials@v1
         with:
           aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
           aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
           aws-region: us-west-1
   

3. Packer Init initializes the Packer template by installing all plugins referenced in the template.

   .github/workflows/build_and_deploy.yml

   - name: Packer Init
         run: packer init .
   

4. Packer Build - Branches builds the artifacts defined in the root directory. The github.ref contains the branch or tag name that triggered the workflow, however, this step only runs when the workflow is triggered by a push to a branch.

   .github/workflows/build_and_deploy.yml

   - name: Packer Build - Branches
         if: "startsWith(github.ref, 'refs/heads/')"
         run: packer build .
   

5. Packer Build - Tags builds the artifacts defined in the root directory for new semantically-versioned tags. Since HCP Packer build fingerprints must be unique, this job sets the build fingerprint to the execution's timestamp. This ensures that builds from tagged commits do not share the same fingerprint as builds in the main, development, and staging branches.

   .github/workflows/build_and_deploy.yml

   - name: Packer Build - Tags
         if: startsWith(github.ref, 'refs/tags/v')
         run: HCP_PACKER_BUILD_FINGERPRINT=$(date +'%m%d%YT%H%M%S') packer build .
   

6. Get HCP Packer version fingerprint from Packer Manifest retrieves the version fingerprint from the packer_manifest.json file generated by the Packer build post-processor. This step exports the version fingerprint so the update-hcp-packer-channel job can use it to update the HCP Packer channel.

   Tip

   GitHub generates a new commit SHA when you create a Pull Request (PR). If your version control system does not create new commit SHAs for PRs, you must create a unique fingerprint for the Packer build when merging PRs.

   .github/workflows/build_and_deploy.yml

   - name: Get HCP Packer version fingerprint from Packer Manifest
         id: hcp
         run: |
           last_run_uuid=$(jq -r '.last_run_uuid' "./packer_manifest.json")
           build=$(jq -r '.builds[] | select(.packer_run_uuid == "'"$last_run_uuid"'")' "./packer_manifest.json")
           version_fingerprint=$(echo "$build" | jq -r '.custom_data.version_fingerprint')
           echo "::set-output name=version_fingerprint::$version_fingerprint"
   

Once the build-artifact job completes, the Actions workflow triggers the update-hcp-packer-channel job. This job has two steps:

1. Checkout clones the current configuration.

   .github/workflows/build_and_deploy.yml

   - name: Checkout
     uses: actions/checkout@v3
   

2. Create and set channel updates the HCP Packer channel corresponding to the branch or tag to serve the new version. This step determines the channel name using the github.ref_name and replaces all dots in tags to dashes to create a valid channel name (for example: v1.0.0 becomes v1-0-0). Then, it runs the create_channel_version.sh script with the channel name and the version fingerprint from the build-artifact job.

   .github/workflows/build_and_deploy.yml

   - name: Create and set channel
         working-directory: .github/scripts
         run: |
           channel_name=$( echo ${{github.ref_name}} | sed 's/\./-/g')
           ./create_channel_version.sh $HCP_BUCKET_NAME $channel_name "${{ needs.build-artifact.outputs.version_fingerprint }}"
   

Open the .github/scripts/create_channel_version.sh file. This script authenticates to HCP, creates the HCP Packer channel if it does not exist, and updates the channel to point to the given version. The main branch is mapped to the HCP Packer release channel to show how you can decouple branch names from channel names.

# If on main branch, set channel to release
if [ "$channel_name" == "main" ]; then
  channel_name="release"
Fi

Create development build

Create a new development branch in your repository.

$ git checkout -b 'development'

Change the HASHICUPS_VERSION variable in setup-deps-hashicups.sh to v1.0.0.

# Get HashiCups config
git clone https://github.com/hashicorp-demoapp/hashicups-setups
cd hashicups-setups/docker-compose-deployment
git checkout server

HASHICUPS_VERSION="v1.0.0"
sed -i 's/HashiCups/HashiCups - ${HASHICUPS_VERSION}/g' docker-compose.yaml

# Use `compose create` to fetch container data without running the container
sudo docker compose create

Stage your changes.

$ git add setup-deps-hashicups.sh

Commit the changes with a message.

$ git commit -m 'Update HashiCups footer message with version'
[development aba0bd5] Update HashiCups footer message with version
 1 file changed, 1 insertion(+), 1 deletion(-)

Push the changes.

$ git push --set-upstream origin development

Verify development build

Click on your repository's Actions tab, then click on the running workflow.

Click on the Build job. Notice that it builds the artifact using the Packer Build - Branches step.

Once the workflow completes, go to your HCP Packer registry. Click on the learn-packer-github-actions bucket, then click on Versions. Packer displays the completed build, and the build fingerprint matches the Git commit's SHA.

Click on the v1 version to view an overview of the build. Notice that Packer includes metadata about the build, including information about Packer, plugins, and build options. Since you triggered this build through a GitHub action, Packer also included information about the repository and action. Refer to rich CI/CD pipeline metadata for more information.

Click on Channels. Notice the Update HCP Packer channel workflow step created a development channel and set it to the version triggered by the development branch.

Create another development build

Change the HASHICUPS_VERSION variable in setup-deps-hashicups.sh to v1.1.0.

# Get HashiCups config
git clone https://github.com/hashicorp-demoapp/hashicups-setups
cd hashicups-setups/docker-compose-deployment
git checkout server

HASHICUPS_VERSION="v1.1.0"
sed -i 's/HashiCups/HashiCups - ${HASHICUPS_VERSION}/g' docker-compose.yaml

# Use `compose create` to fetch container data without running the container
sudo docker compose create

Stage your changes.

$ git add setup-deps-hashicups.sh

Commit the changes with a message.

$ git commit -m 'Update HashiCups footer message with new version'
[development a9724f9] Update HashiCups footer message with new version
 1 file changed, 1 insertion(+), 1 deletion(-)

Push the changes.

$ git push

The push triggered a new workflow in your repository Actions.

Once the workflow completes, go to the learn-packer-github-actions bucket's Channels page. Notice that the workflow updated the development channel to point to the second version, the latest build in the development branch. This workflow makes the development GitHub branch the source of truth for the development channel.

Create and merge a pull request

In your repository within GitHub, create a pull request from the development branch, then click Create pull request.

Merge the pull request. Do not delete the development branch.

The Actions tab shows that the merged pull request triggered a workflow for the main branch.

Once the workflow completes, go to the learn-packer-github-actions bucket's Channels page. Notice that the workflow created a channel named release and set it to the build triggered by the merge to the main branch. This makes the main GitHub branch the source of truth for the release channel.

Create tagged build

Git tags let you version your changes. By tagging specific commits, you can support multiple artifact build versions. This lets downstream artifact consumers query specific build versions outside of the development, staging, and production channels.

Checkout the main branch.

$ git checkout main
Switched to branch 'main'
Your branch is up to date with 'origin/main'.

Pull the latest changes from main.

$ git pull

Create a new Git tag.

$ git tag v1.1.0

Push the tag.

$ git push --tags

Verify tagged build

Go to your repository's Actions tab to find the new workflow run. Notice the commit SHA for the main branch and the v1.1.0 tag are the same. To ensure HCP Packer build fingerprints are unique, the Build Artifact - Tags step set the fingerprint to the current timestamp instead of the commit SHA.

Once the workflow completes, go to the learn-packer-github-actions bucket's Channels page in HCP Packer. Notice that the workflow created a channel named v1-1-0 and set it to the build triggered by the v1.1.0 tag. This makes the v1.1.0 Git tag the source of truth for the v1-1-0 channel.

Destroy resources

Before moving on, destroy the AMIs created during this tutorial and their snapshots to avoid incurring additional costs. In the AMI console for us-west-1 select the AMIs, then click on the Actions button and the Deregister option. Delete the snapshots by selecting the snapshots, then click on the Actions button and the Delete option.

Next steps

In this tutorial, you set up a GitHub Actions workflow to automatically build and manage different versions of an artifact. Then, you triggered builds by pushing commits to the development and main branches, and by creating a new Git tag. You can adapt this Actions workflow to automate and scale Packer builds across your organization.

For more information on topics covered in this tutorial, review the following resources.

• Refer to the Build Artifacts in CI/CD Packer documentation for more guidance on automating Packer builds.
• Complete the Schedule Artifact Version Revocation for Compliance tutorial to learn how to revoke artifact versions to ensure your artifacts are compliant and secure.
• Read the Build a Golden Image Pipeline with HCP Packer tutorial to learn how create a build pipeline for base artifact and application artifacts.
• Watch the Automating Artifact Pipelines with HCP Packer HashiTalks presentation to learn how HashiCorp leverages HCP Packer and GitHub Actions in production to automate our artifact build, test, and deployment pipeline.