Muxing

Muxing enables multiple underlying provider implementations to exist within the same logical provider server via the terraform-plugin-mux Go module. Each underlying provider implementation serves different managed resources and data sources. Refer to the Combining and Translating documentation for full details about muxing configuration.

Use Cases

Use muxing when:

You have an existing terraform-plugin-sdk based provider.
The provider includes more than a few managed resources and data sources.
You want to iteratively develop or release a version of your provider with only some of the managed resources and data sources migrated to the Framework.

Otherwise for simplicity, it is recommended to migrate directly to the framework without temporarily introducing muxing.

Requirements

Ensure github.com/hashicorp/terraform-plugin-sdk/v2 is upgraded to the latest version. For example, running the go get github.com/hashicorp/terraform-plugin-sdk/v2@latest command.
Ensure existing acceptance testing is passing. Acceptance testing can be used to verify the muxing implementation before release.

Implementation

Introduce a Go type implementing the Framework's provider.Provider interface. Refer to the provider definition section of the migration guide for additional details.
Implement the provider.Provider type Schema and Configure methods so it is compatible for muxing. The schema and configuration handling must exactly match between all underlying providers of the mux server. Refer to the provider schema section of the migration guide for additional details.
Introduce a mux server using terraform-plugin-mux functionality. This code eventually must be referenced by the codebase's main() function, which is responsible for starting the provider server. Refer to the Mux Server Examples section for additional details.
Introduce an acceptance test for the mux server implementation. Refer to the Testing Examples section for additional details.
Ensure github.com/hashicorp/terraform-plugin-mux is added to the provider Go module dependencies. For example, running the go get github.com/hashicorp/terraform-plugin-mux@latest command.

Mux Server Examples

Terraform 0.12 Compatibility Example

The following main.go example shows how to set up muxing for a provider that uses Protocol Version 5 to maintain compatibility with Terraform 0.12 and later. The example also shows how to use the debug flag to optionally run the provider in debug mode.

import (
    "context"
    "flag"
    "log"

    "github.com/hashicorp/terraform-plugin-framework/providerserver"
    "github.com/hashicorp/terraform-plugin-go/tfprotov5"
    "github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server"
    "github.com/hashicorp/terraform-plugin-mux/tf5muxserver"

    "example.com/terraform-provider-examplecloud/internal/provider"
)

func main() {
    ctx := context.Background()

    var debug bool

    flag.BoolVar(&debug, "debug", false, "set to true to run the provider with support for debuggers like delve")
    flag.Parse()

    providers := []func() tfprotov5.ProviderServer{
        providerserver.NewProtocol5(provider.New()), // Example terraform-plugin-framework provider
        provider.Provider().GRPCProvider, // Example terraform-plugin-sdk provider
    }

    muxServer, err := tf5muxserver.NewMuxServer(ctx, providers...)

    if err != nil {
        log.Fatal(err)
    }

    var serveOpts []tf5server.ServeOpt

    if debug {
        serveOpts = append(serveOpts, tf5server.WithManagedDebug())
    }

    err = tf5server.Serve(
        "registry.terraform.io/<namespace>/<provider_name>",
        muxServer.ProviderServer,
        serveOpts...,
    )

    if err != nil {
        log.Fatal(err)
    }
}

Terraform 1.X Compatibility Example

The mux server can be setup to break compatibility with Terraform 0.12 through 1.1.6, but enable Protocol Version 6 capabilities in the Framework provider, such as nested attributes.

The following main.go example shows how to set up muxing for a provider that upgrades the terraform-plugin-sdk based provider to Protocol Version 6 to support those new features in the Framework provider. The example also shows how to use the debug flag to optionally run the provider in debug mode.

import (
    "context"
    "flag"
    "log"

    "github.com/hashicorp/terraform-plugin-framework/providerserver"
    "github.com/hashicorp/terraform-plugin-go/tfprotov6"
    "github.com/hashicorp/terraform-plugin-go/tfprotov6/tf6server"
    "github.com/hashicorp/terraform-plugin-mux/tf5to6server"
    "github.com/hashicorp/terraform-plugin-mux/tf6muxserver"

    "example.com/terraform-provider-examplecloud/internal/provider"
)

func main() {
    ctx := context.Background()

    var debug bool

    flag.BoolVar(&debug, "debug", false, "set to true to run the provider with support for debuggers like delve")
    flag.Parse()

    upgradedSdkServer, err := tf5to6server.UpgradeServer(
        ctx,
        provider.Provider().GRPCProvider, // Example terraform-plugin-sdk provider
    )

    if err != nil {
        log.Fatal(err)
    }

    providers := []func() tfprotov6.ProviderServer{
        providerserver.NewProtocol6(provider.New()()), // Example terraform-plugin-framework provider
        func() tfprotov6.ProviderServer {
            return upgradedSdkServer,
        },
    }

    muxServer, err := tf6muxserver.NewMuxServer(ctx, providers...)

    if err != nil {
        log.Fatal(err)
    }

    var serveOpts []tf6server.ServeOpt

    if debug {
        serveOpts = append(serveOpts, tf6server.WithManagedDebug())
    }

    err = tf6server.Serve(
        "registry.terraform.io/<namespace>/<provider_name>",
        muxServer.ProviderServer,
        serveOpts...,
    )

    if err != nil {
        log.Fatal(err)
    }
}

Testing Examples

Protocol Version 5

The following acceptance test example would be included in the same Go package that defines the provider code to verify the muxing setup:

import (
    "context"
    "testing"

    "github.com/hashicorp/terraform-plugin-go/tfprotov5"
    "github.com/hashicorp/terraform-plugin-mux/tf5muxserver"
    "github.com/hashicorp/terraform-plugin-testing/helper/resource"
)

func TestMuxServer(t *testing.T) {
    resource.Test(t, resource.TestCase{
        ProtoV5ProviderFactories: map[string]func() (tfprotov5.ProviderServer, error) {
            "examplecloud": func() (tfprotov5.ProviderServer, error) {
                ctx := context.Background()
                providers := []func() tfprotov5.ProviderServer{
                    providerserver.NewProtocol5(New()), // Example terraform-plugin-framework provider
                    Provider().GRPCProvider, // Example terraform-plugin-sdk provider
                }

                muxServer, err := tf5muxserver.NewMuxServer(ctx, providers...)

                if err != nil {
                    return nil, err
                }

                return muxServer.ProviderServer(), nil
            },
        },
        Steps: []resource.TestStep{
            {
                Config: "... configuration including simplest data source or managed resource",
            },
        },
    })
}

Protocol Version 6

The following acceptance test example would be included in the same Go package that defines the provider code to verify the muxing setup:

import (
    "context"
    "testing"

    "github.com/hashicorp/terraform-plugin-go/tfprotov6"
    "github.com/hashicorp/terraform-plugin-mux/tf5to6server"
    "github.com/hashicorp/terraform-plugin-mux/tf6muxserver"
    "github.com/hashicorp/terraform-plugin-testing/helper/resource"
)

func TestMuxServer(t *testing.T) {
    resource.Test(t, resource.TestCase{
        ProtoV6ProviderFactories: map[string]func() (tfprotov6.ProviderServer, error) {
            "examplecloud": func() (tfprotov6.ProviderServer, error) {
                ctx := context.Background()

                upgradedSdkServer, err := tf5to6server.UpgradeServer(
                    ctx,
                    Provider().GRPCProvider, // Example terraform-plugin-sdk provider
                )

                if err != nil {
                    return nil, err
                }

                providers := []func() tfprotov6.ProviderServer{
                    providerserver.NewProtocol6(New()), // Example terraform-plugin-framework provider
                    func() tfprotov6.ProviderServer {
                        return upgradedSdkServer
                    },
                }

                muxServer, err := tf6muxserver.NewMuxServer(ctx, providers...)

                if err != nil {
                    return nil, err
                }

                return muxServer.ProviderServer(), nil
            },
        },
        Steps: []resource.TestStep{
            {
                Config: "... configuration including simplest data source or managed resource",
            },
        },
    })
}

Tips

Only acceptance tests for migrated managed resources and data sources require testing code updates as noted in the testing page of the migration guide.

Troubleshooting

PreparedConfig response from multiple servers

Muxed providers may receive a new error, such as:

Error: Plugin error

  with provider["registry.terraform.io/example/examplecloud"],
  on <empty> line 0:
  (source code not available)

The plugin returned an unexpected error from
plugin.(*GRPCProvider).ValidateProviderConfig: rpc error: code = Unknown desc
= got different PrepareProviderConfig PreparedConfig response from multiple
servers, not sure which to use

If the terraform-plugin-sdk based provider was using Default or DefaultFunc, you must remove the usage of Default and DefaultFunc in that provider implementation. Transfer the logic into the provider ConfigureFunc or ConfigureContextFunc, similar to how it must be implemented in a terraform-plugin-framework based provider.