Custom Builders

Packer builders are responsible for creating a virtual machine, setting the virtual machine up for provisioning, and then turning that provisioned virtual machine into a machine image. We officially maintain and distribute several builders, including builders to create images on Amazon EC2, VMware, Google Compute Engine, and many more. Refer to the Builders documentation for details.

This page explains how to use the Packer plugin interface to write custom builders. If you want your builder to support HashiCorp Cloud Platform (HCP) Packer, you should also review the HCP Packer Support documentation.

Before You Begin

We recommend reviewing the following resources before you begin development:

• Developing Plugins - Overview
• The Go language. You must write custom plugins in Go, so this guide assumes you are familiar with the language.

The Interface

To create your own builder, you must create a struct that implements the packer.Builder interface. It is reproduced below for reference.

type Builder interface {
  ConfigSpec() hcldec.ObjectSpec
  Prepare(...interface{}) ([]string, []string, error)
  Run(context.Context, ui Ui, hook Hook) (Artifact, error)
}

The "ConfigSpec" Method

This method returns a hcldec.ObjectSpec, which is a spec necessary for using HCL2 templates with Packer. For information on how to use and implement this function, check our object spec docs

The "Prepare" Method

The Prepare method for each builder will be called by the Packer core at the beginning of the build. Its purpose is to parse and validate the configuration template provided to Packer with packer build your_packer_template.json, but not to execute API calls or begin creating any resources or artifacts.

The configuration from your Packer template is passed into the Prepare() method as an array of interface{} types, but is generally map[string]interface{}. The Prepare method is responsible for translating this configuration into an internal structure, validating it, and returning any errors.

If multiple parameters are passed into Prepare(), they should be merged together into the final configuration, with later parameters overwriting any previous configuration. The exact semantics of the merge are left to the builder author.

We recommend that you use the mapstructure library to decode the interface{} into a meaningful structure. Mapstructure will take an interface{} and decode it into an arbitrarily complex struct. If there are any errors, it generates very human friendly errors that can be returned directly from the prepare method. You can find many usage examples of this library within the Prepare() methods of HashiCorp-maintained Packer plugins.

While Packer does not actively enforce this, no side effects should occur from running the Prepare method. Specifically: don't create files, don't launch virtual machines, etc. Prepare's purpose is solely to load the configuration from the template into a format usable by your builder, to validate that configuration, and to apply necessary defaults to that configuration.

In addition to the configuration provided in the Packer template, Packer will also supply a common.PackerConfig containing meta-information such as the build name, builder type, core version, etc, and coded into a map[string]interface{}. One important piece of meta information in this map is the packer.DebugConfigKey set to boolean true if debug mode is enabled for the build. If this is set to true, then the builder should enable a debug mode which assists builder developers and advanced users to introspect what is going on during a build. During debug builds, parallelism is strictly disabled, so it is safe to request input from stdin and so on.

Prepare() returns an array of strings and an error. The array of strings is a special list of keys for variables created at runtime which your builder will make accessible to provisioners using the generatedData mechanism (see below for more details) An example could be an instance ID for the cloud instance created by your builder. If you do not plan to make use of the generatedData feature, just return an empty list. The error should be used if there is something wrong with the user-provided configuration and the build should not proceed.

The "Run" Method

Run is executed, often in parallel for multiple builders, to actually build the machine, provision it, and create the resulting machine image, which is returned as an implementation of the packer.Artifact interface.

The Run method takes three parameters. The context.Context used to cancel the build. The packer.Ui object is used to send output to the console. packer.Hook is used to execute hooks, which are covered in more detail in the Provisioning section below.

Because builder runs are typically a complex set of many steps, the packer-plugin-sdk contains a multistep module. Multistep allows you to separate your build logic into multiple distinct "steps" with separate run and cleanup phases, and run them in order. It supports cancellation mid-step, pausing between steps when debugging, the CLI's on-error flag, and more. All of the HashiCorp maintained builders make use of this module, and while it is not required for builder implementation, it will help you create your builder in a way that matches user and Packer Core assumptions. The SDK also provides a number of "helper" generic steps that may prevent you from having to re-implement work that has already been done by the HashiCorp maintainers. Examples include sending boot commands, connecting to SSH, and creating virtual CDs to mount on your VM. Take a look at the communicator and multistep/commonsteps modules in the SDK to see what tools are available to you.

Finally, Run should return an implementation of packer.Artifact. More details on creating a packer.Artifact are covered in the artifact section below. If something goes wrong during the build that prevents an artifact from being correctly created, Run should return an error and a nil artifact. Note that your builder is allowed to produce no artifact and no error, although this is a rare use case.

Cancellation

With the "Cancel" Method ( for plugins for Packer < v1.3 )

The Cancel method can be called at any time and requests cancellation of any builder run in progress. This method should block until the run actually stops. Note that the Cancel method will not be called by Packer versions >= 1.4.0.

Context cancellation ( from Packer v1.4 )

The <-ctx.Done() can unblock at any time and signifies request for cancellation of any builder run in progress.

Cancels are most commonly triggered by external interrupts, such as the user pressing Ctrl-C. Packer will only exit once all the builders clean up, so it is important that you architect your builder in a way that it is quick to respond to these cancellations and clean up after itself. If your builder makes a long-running call, you should consider the possibility that a user may cancel the build during that call, and make sure that such a cancellation is not blocked.

Creating an Artifact

The Run method is expected to return an implementation of the packer.Artifact interface. Each builder must create its own implementation of this interface.

Most of the pieces of an artifact should be fairly self-explanatory by reading the packer.Artifact interface documentation.

However one part of an artifact that may be confusing is the BuilderId method. This method must return an absolutely unique ID for the builder. In general, a reasonable ID would be the github username or organization that created the builder, followed by the platform it is building for. For example, the builder ID of the VMware builder is "hashicorp.vmware".

Post-processors use the builder ID value in order to make some assumptions about the artifact results and to determine whether they are even able to run against a given artifact, so it is important that this ID never changes once the builder is published.

The builder ID for each builder is included on the associated documentation page.

Provisioning

Packer has built-in support for provisioning using the Provisioner plugins. But builders themselves, rather than the Packer core, must determine when to invoke the provisioners since only the builder knows when the machine is running and ready for communication.

When the machine is ready to be provisioned, run the packer.HookProvision hook, making sure the communicator is not nil, since this is required for provisioners. An example of calling the hook is shown below:

hook.Run(context.Context, packer.HookProvision, ui, comm, nil)

At this point, Packer will run the provisioners and no additional work is necessary.

If you are using the multistep tooling, the Packer plugin SDK contains a generic StepProvision which handles execution the provision hook for you and automatically supplies any custom builder generatedData you would like to provide to provisioners (see below for more details on generatedData.)

Template Engine

Build variables

Packer JSON makes it possible to provide custom template engine variables to be shared with provisioners and post-processors using the build function. JSON template build docs are here and HCL template build docs are here.

As of Packer v1.5.0, builder Prepare() methods return a list of custom variables which we call generated data. We use that list of variables to generate a custom placeholder map per builder that combines custom variables with the placeholder map of default build variables created by Packer. Here's an example snippet telling packer what will be made available by the builder:

func (b *Builder) Prepare(raws ...interface{}) ([]string, []string, error) {
    // ...

    generatedData := []string{"SourceImageName"}
    return generatedData, warns, nil
}

When a user provides a Packer template that uses a build function, Packer validates that the key in the build function exists for a given builder using this generatedData array. If it does not exist, then Packer validation will fail.

Once the placeholder is set, it's necessary to pass the variables' real values when calling the provisioner. This can be done as the example below:

func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (packer.Artifact, error) {
    // ...

    // Create map of custom variable
    generatedData := map[string]interface{}{"SourceImageName": "the source image name value"}
    // Pass map to provisioner
    hook.Run(context.Context, packer.HookProvision, ui, comm, generatedData)

    // ...
}

In order to make these same variables and the Packer default ones also available to post-processors, your builder will need to add them to its Artifact. This can be done by adding an attribute of type map[string]interface{} to the Artifact and putting the generated data in it. The post-processor will access this data later via the Artifact's State method.

The Artifact code should be implemented similar to the below:

type Artifact struct {
    // ...

    // StateData should store data such as GeneratedData
    // to be shared with post-processors
    StateData map[string]interface{}
}

// ...

func (a *Artifact) State(name string) interface{} {
    return a.StateData[name]
}

// ...

The builder should return the above Artifact containing the generated data and the code should be similar to the example snippet below:

func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (packer.Artifact, error) {
    // ...

    return &Artifact{
                // ...
                StateData: map[string]interface{}{"generated_data": state.Get("generated_data")},
            }, nil
}

The code above assigns the generated_data state to the StateData map with the key generated_data.

Here some example of how this data will be used by post-processors:

func (p *PostProcessor) PostProcess(ctx context.Context, ui packer.Ui, source packer.Artifact) (packer.Artifact, bool, bool, error) {
    generatedData := source.State("generated_data")

    // generatedData will then be used for interpolation

    // ...
}

Putting it all together

This page has focused up until now on the implementation details for the Builder interface. You will need to create a server and store the builder in a binary in order to make it available to the Packer core as a plugin. We have created a scaffolding repo to give you an idea of the relationship between the builder implementation and the server implementation within a repository, and then read basics of how Plugins work, which breaks down all the server details.