Secret data transformation

Utilizing advanced templating and data filters, the Vault Secrets Operator for Kubernetes (VSO) can transform source secret data, secret metadata, resource labels and annotations into a format that is compatible with your application. All secret data sources are supported. Secret transformations can be specified directly within a secret custom resource (CR), or by references to one or more SecretTransformation custom resource instances, or both.

Templating

VSO utilizes the data-driven templates for Golang to generate secret data output. The template data input holds the secret data, secret metadata, resource labels and annotations.

Templates are configured in a secret custom resource's spec.Destination.Transformation.Templates, or in a SecretTransformation resource's spec.templates.

VSO provides access to a large library of template functions, some of which are documented below.

Secret data input

Secret data is accessed through the .Secrets input member. It contains a map of secret key-value pairs, which are assumed to be sensitive information fetched from a secret source.

For example, to include a password in your application's secret, you might specify a template like:

{{- printf "password=%s" (get .Secrets "password") -}}

Secret metadata input

Secret metadata is accessed through the .Metadata input member. It contains a map of metadata key to its value. The data should not contain any confidential information.

For example, to include a secret metadata value in your application's secret, you might specify a template like:

{{- printf "secretGroup=%s" (get .Metadata "secretGroup") -}}

Resource annotations input

Resource annotations are accessed through the .Annotations input member. The annotations consist of all metadata.annotations configured on the secret custom resource.

For example, to include a value from the resource's annotations in your application's secret, you might specify a template like:

{{- printf "host=%s" (get .Annotations "myapp.config/host") -}}

Resource labels input

Resource labels are accessed through the .Labels input member. The labels consist of all metadata.labels configured on the secret custom resource.

For example, to include a value from the resource's labels in your application's secret, you might specify a template like:

{{- printf "appType=%s" (get .Labels "appType") -}}

Filters

Filters are used to control which source secret data fields are included in the destination secret's data. They are specified as a set of exclude/include RE2 accepted regular expressions.

Filters are configured in the excludes and includes fields of a secret custom resource's spec.Destination.Transformation, or in a SecretTransformation resource's spec.

All exclude patterns take precedence over any include patterns, and are never applied to templated keys.

Examples

Local transformation

A VaultDynamicSecret configured to sync Postgres database credentials from Vault to the Kubernetes secret named example-vds.

---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultDynamicSecret
metadata:
  namespace: example-ns
  name: example-vds
  annotations:
    myapp.config/postgres-host: postgres-postgresql.postgres.svc.cluster.local:5432
spec:
  destination:
    create: true
    name: app-secret
    transformation:
      excludes:
       - .*
      templates:
        url:
          text: |
            {{- $host := get .Annotations "myapp.config/postgres-host" -}}
            {{- printf "postgresql://%s:%s@%s/postgres?sslmode=disable" (get .Secrets "username") (get .Secrets "password") $host -}}
  path: creds/dev-postgres

The resulting Kubernetes secret includes a single key named url, with a valid Postgres connection URL as its value.

url: postgresql://v-postgres-user:XUpah-password@postgres-postgresql.postgres.svc.cluster.local:5432/postgres?sslmode=disable

Shared transformation

The following manifest contains shared transformation templates and filters. All templates it provides will be included in the destination k8s secret. It also provides sourceTemplates that can be included in any template text configured in a secret CR or within the same resource instance.

---
apiVersion: secrets.hashicorp.com/v1beta1
kind: SecretTransformation
metadata:
  name: vso-templates
  namespace: example-vds
spec:
  excludes:
    - password|username
  templates:
    url:
      text: '{{- template "dbUrl" . -}}'
  sourceTemplates:
    - name: helpers
      text: |
        {{/*
          create a Java props from SecretInput for this app
        */}}
        {{- define "appProps" -}}
        {{- $host := get .Annotations "myapp.config/postgres-host" -}}
        {{- printf "db.host=%s\n" $host -}}
        {{- range $k, $v := .Secrets -}}
        {{- printf "db.%s=%s\n" $k $v -}}
        {{- end -}}
        {{- end -}}
        {{/*
          create a JSON config from SecretInput for this app
        */}}
        {{- define "appJson" -}}
        {{- $host := get .Annotations "myapp.config/postgres-host" -}}
        {{- $copy := .Secrets | mustDeepCopy -}}
        {{- $_ := set $copy "host" $host -}}
        {{- mustToPrettyJson $copy -}}
        {{- end -}}
        {{/*
          compose a Postgres URL from SecretInput for this app
        */}}
        {{- define "dbUrl" -}}
        {{- $host := get .Annotations "myapp.config/postgres-host" -}}
        {{- printf "postgresql://%s:%s@%s/postgres?sslmode=disable" (get .Secrets "username") (get .Secrets "password") $host -}}
        {{- end -}}
        {{/*
          get the app name from the VSO resource's label
        */}}
        {{- define "appName" -}}
        {{- get .Labels "myapp/name" -}}
        {{- end -}}

The following VaultDynamicSecret manifest references the SecretTransformation above. All templates and filters from the reference object will be applied to the destination secret data.

---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultDynamicSecret
metadata:
  namespace: example-ns
  name: example-vds
  annotations:
    myapp.config/postgres-host: postgres-postgresql.postgres.svc.cluster.local:5432
spec:
  destination:
    create: true
    name: app-secret
    transformation:
      transformationRefs:
        - name: vso-templates
  path: creds/dev-postgres

The resulting Kubernetes secret includes a single key named url, with a valid Postgres connection URL as its value

url: postgresql://v-postgres-user:XUpah-password@postgres-postgresql.postgres.svc.cluster.local:5432/postgres?sslmode=disable

Template functions

All template functions are provided by the sprig library. Some common functions are mentioned below. For the complete list of functions see allowedSprigFuncs

String functions

trim removes any leading or trailing whitespaces from the input:

trim " host " -> `host`

Encoding functions

b64enc base64 encodes an input value

b64enc "host" -> `aG9zdAo=`

b64dec base64 decodes an input value

b64dec "aG9zdAo=" -> `host`

Map functions

get retrieves a value from a map input:

get .Secrets "baz" -> `qux`

Given a nested map input:

{
  "foo": {
    "bar": "baz",
    "quz": "quux"
  }
}

get can retrieve a specific value:

get (get .Secrets "foo") "bar" -> `baz`

dig can also retrieve a specific value, or return a default if any of the keys are not found:

dig "foo" "quz" "<not found>" .Secrets -> `quux`
 
dig "foo" "nux" "<not found>" .Secrets -> `<not found>`