Resources - State Migration

Resources define the data types and API interactions required to create, update, and destroy infrastructure with a cloud vendor while the Terraform state stores mapping and metadata information for those remote objects. There are several reasons why a resource implementation needs to change: backend APIs Terraform interacts with will change overtime, or the current implementation might be incorrect or unmaintainable. Some of these changes may not be backward compatible and a migration is needed for resources provisioned in the wild with old schema configurations.

The mechanism that is used for state migrations changed between v0.11 and v0.12 of the SDK bundled with Terraform core. Be sure to choose the method that matches your Terraform dependency.

Terraform v0.12 SDK State Migrations

Note: This method of state migration does not work if the provider has a dependency on the Terraform v0.11 SDK. See the Terraform v0.11 SDK State Migrations section for details on using MigrateState instead.

For this task provider developers should use a resource's SchemaVersion and StateUpgraders fields. Resources typically do not have these fields configured unless state migrations have been performed in the past.

When Terraform encounters a newer resource SchemaVersion during planning, it will automatically migrate the state through each StateUpgrader function until it matches the current SchemaVersion.

State migrations performed with StateUpgraders are compatible with the Terraform 0.11 runtime, if the provider still supports the Terraform 0.11 protocol. Additional MigrateState implementation is not necessary and any existing MigrateState implementations do not need to be converted to StateUpgraders.

The general overview of this process is:

Create a new function that copies the existing schema.Resource, but only includes the Schema field. Terraform needs the type information of each attribute in the previous schema version to successfully migrate the state.
Change the existing resource Schema as necessary.
If the SchemaVersion field for the resource is already defined, increase its value by one. If SchemaVersion is not defined for the resource, add SchemaVersion: 1 to the resource (resources default to SchemaVersion: 0 if undefined).
Implement the StateUpgraders field for the resource, which is a list of StateUpgrade. The new StateUpgrade should be configured with the following:
- Type set to CoreConfigSchema().ImpliedType() of the saved schema.Resource function above.
- Upgrade set to a function that modifies the attribute(s) appropriately for the migration.
- Version set to the version of the schema before this migration. If no previous state migrations were performed, this should be set to 0.

For example, with a resource without previous state migrations:

package example

import "github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"

func resourceExampleInstance() *schema.Resource {
    return &schema.Resource{
        Create: resourceExampleInstanceCreate,
        Read:   resourceExampleInstanceRead,
        Update: resourceExampleInstanceUpdate,
        Delete: resourceExampleInstanceDelete,

        Schema: map[string]*schema.Schema{
            "name": {
                Type:     schema.TypeString,
                Required: true,
            },
        },
    }
}

Say the instance resource API now requires the name attribute to end with a period "."

package example

import (
    "fmt"
    "strings"

    "github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
)

func resourceExampleInstance() *schema.Resource {
    return &schema.Resource{
        Create: resourceExampleInstanceCreate,
        Read:   resourceExampleInstanceRead,
        Update: resourceExampleInstanceUpdate,
        Delete: resourceExampleInstanceDelete,

        Schema: map[string]*schema.Schema{
            "name": {
                Type:     schema.TypeString,
                Required: true,
                ValidateFunc: func(v any, k string) (warns []string, errs []error) {
                    if !strings.HasSuffix(v.(string), ".") {
                        errs = append(errs, fmt.Errorf("%q must end with a period '.'", k))
                    }
                    return
                },
            },
        },
        SchemaVersion: 1,
        StateUpgraders: []schema.StateUpgrader{
            {
                Type:    resourceExampleInstanceResourceV0().CoreConfigSchema().ImpliedType(),
                Upgrade: resourceExampleInstanceStateUpgradeV0,
                Version: 0,
            },
        },
    }
}

func resourceExampleInstanceResourceV0() *schema.Resource {
    return &schema.Resource{
        Schema: map[string]*schema.Schema{
            "name": {
                Type:     schema.TypeString,
                Required: true,
            },
        },
    }
}

func resourceExampleInstanceStateUpgradeV0(ctx context.Context, rawState map[string]any, meta any) (map[string]any, error) {
    rawState["name"] = rawState["name"].(string) + "."

    return rawState, nil
}

To unit test this migration, the following can be written:

func testResourceExampleInstanceStateDataV0() map[string]any {
    return map[string]any{
        "name": "test",
    }
}

func testResourceExampleInstanceStateDataV1() map[string]any {
    v0 := testResourceExampleInstanceStateDataV0()
    return map[string]any{
        "name": v0["name"] + ".",
    }
}

func TestResourceExampleInstanceStateUpgradeV0(t *testing.T) {
    expected := testResourceExampleInstanceStateDataV1()
    actual, err := resourceExampleInstanceStateUpgradeV0(testResourceExampleInstanceStateDataV0(), nil)
    if err != nil {
        t.Fatalf("error migrating state: %s", err)
    }

    if !reflect.DeepEqual(expected, actual) {
        t.Fatalf("\n\nexpected:\n\n%#v\n\ngot:\n\n%#v\n\n", expected, actual)
    }
}

Terraform v0.11 SDK State Migrations

NOTE: This method of state migration does not work if the provider has a dependency on the Terraform v0.12 SDK. See the Terraform v0.12 SDK State Migrations section for details on using StateUpgraders instead.

For this task provider developers should use a resource's SchemaVersion and MigrateState function. Resources do not have these options set on first implementation, the SchemaVersion defaults to 0.

package example

import "github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"

func resourceExampleInstance() *schema.Resource {
    return &schema.Resource{
        Create: resourceExampleInstanceCreate,
        Read:   resourceExampleInstanceRead,
        Update: resourceExampleInstanceUpdate,
        Delete: resourceExampleInstanceDelete,
      
        Schema: map[string]*schema.Schema{
            "name": {
                Type:     schema.TypeString,
                Required: true,
            },
        },
    }
}

Say the instance resource API now requires the name attribute to end with a period "."

package example

import (
    "fmt"
    "strings"

    "github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
)

func resourceExampleInstance() *schema.Resource {
    return &schema.Resource{
        Create: resourceExampleInstanceCreate,
        Read:   resourceExampleInstanceRead,
        Update: resourceExampleInstanceUpdate,
        Delete: resourceExampleInstanceDelete,
      
        Schema: map[string]*schema.Schema{
            "name": {
                Type:     schema.TypeString,
                Required: true,
                ValidateFunc: func(v any, k string) (warns []string, errs []error) {
                    if !strings.HasSuffix(v.(string), ".") {
                        errs = append(errs, fmt.Errorf("%q must end with a period '.'", k))
                    }
                    return
                },
            },
        },
        SchemaVersion: 1,
        MigrateState: resourceExampleInstanceMigrateState,
    }
}

To trigger the migration we set the SchemaVersion to 1. When Terraform saves state it also sets the SchemaVersion at the time, that way when differences are calculated, if the saved SchemaVersion is less than what the Resource is currently set to, the state is run through the MigrateState function.

func resourceExampleInstanceMigrateState(v int, inst *terraform.InstanceState, meta any) (*terraform.InstanceState, error) {
    switch v {
    case 0:
        log.Println("[INFO] Found Example Instance State v0; migrating to v1")
        return migrateExampleInstanceStateV0toV1(inst)
    default:
        return inst, fmt.Errorf("Unexpected schema version: %d", v)
    }
}

func migrateExampleInstanceStateV0toV1(inst *terraform.InstanceState) (*terraform.InstanceState, error) {
    if inst.Empty() {
        log.Println("[DEBUG] Empty InstanceState; nothing to migrate.")
        return inst, nil
    }

    if !strings.HasSuffix(inst.Attributes["name"], ".") {
        log.Printf("[DEBUG] Attributes before migration: %#v", inst.Attributes)
        inst.Attributes["name"] = inst.Attributes["name"] + "."
        log.Printf("[DEBUG] Attributes after migration: %#v", inst.Attributes)
    }

    return inst, nil
}

Although not required, it's a good idea to break the migration function up into version jumps. As the provider developer you will have to account for migrations that are larger than one version upgrade, using the switch/case pattern above will allow you to create codepaths for states coming from all the versions of state in the wild. Please be careful to allow all legacy versions to migrate to the latest schema. Consider the code now where the name attribute has moved to an attribute called fqdn.

func resourceExampleInstanceMigrateState(v int, inst *terraform.InstanceState, meta any) (*terraform.InstanceState, error) {
    var err error
    switch v {
    case 0:
        log.Println("[INFO] Found Example Instance State v0; migrating to v1")
        inst, err = migrateExampleInstanceV0toV1(inst)
        if err != nil {
            return inst, err
        }
        fallthrough
    case 1:
        log.Println("[INFO] Found Example Instance State v1; migrating to v2")
        return migrateExampleInstanceStateV1toV2(inst)
    default:
        return inst, fmt.Errorf("Unexpected schema version: %d", v)
    }
}

func migrateExampleInstanceStateV1toV2(inst *terraform.InstanceState) (*terraform.InstanceState, error) {
    if inst.Empty() {
        log.Println("[DEBUG] Empty InstanceState; nothing to migrate.")
        return inst, nil
    }

    if inst.Attributes["name"] != "" {
        inst.Attributes["fqdn"] = inst.Attributes["name"]
        delete(inst.Attributes, "name")
    }
    return inst, nil
}

The fallthrough allows a very old state to move from 0 to 1 and now to 2. Sometimes state migrations are more complicated, and requires making API calls, to allow this the configured meta any is also passed to the MigrateState function.