Vault
Add a containerized secrets plugin to Vault
Run your external secrets plugins in containers to increases the isolation between the plugin and Vault.
Before you start
- Your Vault instance must be running on Linux.
- Your Vault instance must have local access to the Docker Engine API. Vault uses the Docker SDK to manage containerized plugins.
- You must have gVisor
installed. Vault uses
runsc
as the entrypoint to your container runtime. - If you are using a container runtime other than gVisor, you must have a
runsc
-compatible container runtime installed.
Step 1: Install your container engine
Install one of the supported container engines:
Step 2: Configure your container runtime
Update your container engine to use runsc
for Unix sockets between the host
and plugin binary.
Add
runsc
to your Docker daemon configuration:$ sudo tee PATH_TO_DOCKER_DAEMON_CONFIG_FILE <<EOF { "runtimes": { "runsc": { "path": "PATH_TO_RUNSC_INSTALLATION", "runtimeArgs": [ "--host-uds=all" ] } } } EOF
Restart Docker:
$ sudo systemctl reload docker
Step 3: Update the HashiCorp go-plugin
library
You must build your plugin locally with v1.5.0+ of the HashiCorp
go-plugin
library to ensure the
finished binary is compatible with containerization.
Use go install
to pull the latest version of the plugin library from the
hashicorp/go-plugin
repo on GitHub:
$ go install github.com/hashicorp/go-plugin@latest
The Vault SDK includes go-plugin
If you build with the Vault SDK, you can update go-plugin
with go install
by pulling the latest SDK version from the hashicorp/vault
repo:
go install github.com/hashicorp/vault/sdk@latest
Step 4: Build the plugin container
Containerized plugins must run as a binary in the finished container and behave the same whether run in a container or as a standalone application:
- Build your plugin binary so it runs on Linux.
- Create a container file for your plugin with the compiled binary as the entry-point.
- Build the image with a unique tag.
For example, to build a containerized version of the built-in key-value (KV) secrets plugin for Docker:
- Clone the latest version of the KV secrets plugin from
hashicorp/vault-plugin-secrets-kv
.$ git clone https://github.com/hashicorp/vault-plugin-secrets-kv.git
- Build the Go binary for Linux.
$ cd vault-plugin-secrets-kv ; CGO_ENABLED=0 GOOS=linux \ go build -o kv cmd/vault-plugin-secrets-kv/main.go
- Create an empty Dockerfile.
$ touch Dockerfile
- Update the empty
Dockerfile
with your infrastructure build details and the compiled binary as the entry-point.FROM gcr.io/distroless/static-debian12 COPY kv /bin/kv ENTRYPOINT [ "/bin/kv" ]
- Build the container image and assign an identifiable tag.
$ docker build -t hashicorp/vault-plugin-secrets-kv:mycontainer .
Step 5: Register the plugin
Registering a containerized plugin with Vault is similar to registering any other external plugin that is available locally to Vault.
Store the SHA256 of the plugin image:
$ export SHA256=$(docker images \ --no-trunc \ --format="{{ .ID }}" \ YOUR_PLUGIN_IMAGE_TAG | cut -d: -f2)
For example:
$ export SHA256=$(docker images \ --no-trunc \ --format="{{ .ID }}" \ hashicorp/vault-plugin-secrets-kv:mycontainer | cut -d: -f2)
Register the plugin with
vault plugin register
and specify your plugin image with theoci_image
flag:$ vault plugin register \ -sha256="${SHA256}" \ -oci_image=YOUR_PLUGIN_IMAGE_TAG \ NEW_PLUGIN_TYPE NEW_PLUGIN_ID
For example:
$ vault plugin register \ -sha256="${SHA256}" \ -oci_image=hashicorp/vault-plugin-secrets-kv:mycontainer \ secret my-kv-container
Enable the new plugin for your Vault instance with
vault secrets enable
and the new plugin ID:$ vault secrets enable NEW_PLUGIN_ID
For example:
$ vault secrets enable my-kv-container
Customize container behavior with registration flags
You can provide additional information about the image entrypoint, command,
and environment with the -command
, -args
, and -env
flags for
vault plugin register
.
Step 6: Test your plugin
Now that the container is registered with Vault, you should be able to interact with it like any other plugin. Try writing then fetching a new secret with your new plugin.
Use
vault write
to store a secret with your containerized plugin:$ vault write NEW_PLUGIN_ID/SECRET_PATH SECRET_KEY=SECRET_VALUE
For example:
$ vault write my-kv-container/testing subject=containers Success! Data written to: my-kv-container/testing
Fetch the secret you just wrote:
$ vault read NEW_PLUGIN_ID/SECRET_PATH
For example:
$ vault read my-kv-container/testing ===== Data ===== Key Value --- ----- subject containers
Use alternative runtimes
You can force Vault to use alternative runtimes provided the runtime is installed locally.
To use an alternative runtime:
Register and name the runtime with
vault plugin runtime register
. For example, to register the default Docker runtime (runc
) asdocker-rt
:$ vault plugin runtime register \ -oci_runtime=runc \ -type=container docker-rt
Use the
--runtime
flag during plugin registration to tell Vault what runtime to use:$ vault plugin register \ -runtime=RUNTIME_NAME \ -sha256="${SHA256}" \ -oci_image=YOUR_PLUGIN_IMAGE_TAG \ NEW_PLUGIN_TYPE NEW_PLUGIN_ID
For example:
$ vault plugin register \ -runtime=docker-rt \ -sha256="${SHA256}" \ -oci_image=hashicorp/vault-plugin-secrets-kv:mycontainer \ secret my-kv-container
Troubleshooting
Invalid backend version error
If you run into the following error while registering your plugin:
invalid backend version error: 2 errors occurred:
* error creating container: Error response from daemon: error while looking up the specified runtime path: exec: " /usr/bin/runsc": stat /usr/bin/runsc: no such file or directory
* error creating container: Error response from daemon: error while looking up the specified runtime path: exec: " /usr/bin/runsc": stat /usr/bin/runsc: no such file or directory
it means that Vault cannot find the executable for runsc
. Confirm the
following is true before trying again:
- You have gVisor installed locally to Vault.
- The path to
runsc
is correct in you your Docker configuration. - Vault has permission to run the
runsc
executable.
If you still get errors when registering a plugin, the recommended workaround is
to use the default Docker runtime (runc
) as an
alternative runtime.