Couchbase secrets engine

Vault provides powerful dynamic credential lifecycle management for a wide range of database solutions.

This tutorial demonstrates the use of the database secrets engine to dynamically generate credentials for Couchbase Server database users.

Challenge

Credential management is critical for secrets hygiene, but managing the lifecycle of credentials across several heterogeneous platforms such as database solutions can be cumbersome and time consuming.

An application requires credentials to access a specific database platform, but you should never hard code credentials into the application or allow credentials to persist past their useful lifetime.

Solution

Vault provides a databases secrets engine with support for credential lifecycle management across a range of database solutions.

Administrators can define credential attributes, such as attached policies and time to live values, such that the credential provides least privileged access. The credential is valid for just the allowed time-frame, and Vault revokes it when no longer needed.

Personas

The end-to-end scenario described in this tutorial involves 3 personas:

• couchbase admin manages Couchbase Server and by Vault for managing Couchbase Server user credentials.
• vault admin has privileged capabilities to configure Vault secrets engines.
• user needs credentials from Vault that allow access to Couchbase Server documents.

Prerequisites

This lab was tested on macOS using an x86_64 based processor. If you are running macOS on an Apple silicon-based processor, use a x86_64 based Linux virtual machine in your preferred cloud provider.

You need the following to perform all tasks in the example scenario.

• Docker

Docker version:

$ docker --version
Docker version 20.10.2, build 2291f61

This tutorial alternates between using a web UI to configure Couchbase, and a terminal session with the vault CLI. You should be comfortable working with a terminal session to complete the example scenario.

Deploy Couchbase

(Persona: couchbase admin)

This scenario uses a single Couchbase Server container for simplicity.

1. Use the Docker image to deploy a Couchbase server that detaches and runs in the background while publishing the necessary ports for access.

   $ docker run \
       --name couchbase-server \
       --publish 8091-8094:8091-8094 \
       --detach \
       --rm \
       couchbase
   

   Example output:

   ...snip...
   Digest: sha256:8a0bc6ad73b5eff52b094e0f59a6f0b8adb35e7a527b2248d950c4238a1f669c
   Status: Downloaded newer image for couchbase:latest
   e5bad8056fe651ec56ff5e5a5273b0cdb5a46231761a8189716af2ca92212767
   

2. Verify that the Couchbase container is running.

   $ docker ps -f name=couchbase-server --format "table {{.Names}}\t{{.Status}}"
   NAMES              STATUS
   couchbase-server   Up 15 seconds
   

   The example output shows that the container has been up for 15 seconds.

3. Visit http://localhost:8091/ in a browser to confirm UI access and complete configuration of the server.

   

4. Click Setup New Cluster to begin the setup process.

5. Enter a name into Cluster Name (for example, couchbase-learn).

6. Enter an initial password value into Create Password and Confirm Password. For this scenario, the example password value is dZLDNkkuGyk=.

7. Click Next: Accept Terms.

8. Review the license terms and if you agree, click I accept the terms & conditions.

9. Click Finish With Defaults.

   After you complete the configuration, you arrive at the dashboard.

   

   While you are here, you can add an example bucket and document, which you can access as part of the example scenario in later steps.

10. Select Buckets from the side menu, and click ADD BUCKET.

    

11. Enter art-inventory in the Name field.

12. Click Advanced bucket settings.

13. Under Replicas, remove the check from the Enable checkbox. You are using a development environment for this example scenario and just one Couchbase Server, so you do not need data replication.

    

14. Click Add Bucket.

15. You should observe the bucket created with a green status bar.

    The example document represents a simplified record from an imaginary art catalog. The inventory records contain sensitive information, so applications and users require valid Couchbase credentials to access them.

    You will access this document as the user persona using dynamic credentials retrieved from Vault in later steps.

16. Select Documents from the side menu.

17. With art-inventory selected in the Bucket, click ADD DOCUMENT.

18. Enter a name for the document; for this example, it is important that you use the value art-launch-hearts.

19. Click Save.

20. Add the example JSON, replacing the entire contents of the text area as shown in the screenshot.

    [
      {
        "name": "Launch Hearts",
        "dateCreated": "2014",
        "artform": "painting",
        "artMedium": "acrylic",
        "image": "https://art.example.com/painting/art-creator/launch-hearts.png",
        "description": "Launch depicts the advent of space travel...",
        "creator": [
          {
            "name": "Art Creator"
          }
        ],
        "owner": [
          {
            "name": "Art Foundation",
            "price": 9001.0
          }
        ]
      }
    ]
    

    

21. Click Save.

    In the next step, add a global index to the art-inventory bucket so that you can search in it later.

22. Click Query in the side navigation.

23. In the Query Editor on line 1, enter the following:

    CREATE PRIMARY INDEX `art-inventory-primary-index` ON `art-inventory` USING GSI;
    

    

24. Click Execute.

    The results should show success.

You completed the initial Couchbase Server cluster deployment, and used the Couchbase web UI to create a document and add an index on the art-inventory bucket.

You can now proceed to deploying the Vault server.

Deploy Vault

(Persona: vault admin)

Use a terminal session and the Vault Docker image to install Vault.

$ docker run \
    --name=vault-dev \
    --cap-add=IPC_LOCK \
    --env 'VAULT_DEV_ROOT_TOKEN_ID=root' \
    --publish 8200:8200 \
    --detach \
    --rm \
    vault

Example output:

Unable to find image 'vault:latest' locally
latest: Pulling from library/vault
Digest: sha256:6281c2d63ac61769dd8cdd553ec8a92547ca6ed58252dac9a0a7a664ffb4289a
Status: Downloaded newer image for vault:latest
fda42dca322a99578bf61b162c7cbea2d1c827e06d5acf195109061d05f9e517

You deployed a single Vault server in dev mode for this tutorial. This is a special mode of Vault server operation where all data persist to in-memory storage, and the server is automatically unsealed when started.

You also set an initial root token value for use in later steps.

Before accessing the Vault container, you must export two environment variables to specify the server address and token value.

Export the dev mode server's container address as the value of VAULT_ADDR.

$ export VAULT_ADDR="http://$(docker inspect -f '{{.NetworkSettings.IPAddress}}' vault-dev):8200"

Export the dev mode initial root token value as the value of VAULT_TOKEN.

$ export VAULT_TOKEN=root

Finally, to avoid the need for a separate Vault binary installation on the host computer, you can set an alias to run the vault CLI commands inside the Vault container itself.

$ alias vault="docker run --rm --cap-add=IPC_LOCK --env VAULT_ADDR --env VAULT_TOKEN vault"

Check the Vault status.

$ vault status
Key             Value
---             -----
Seal Type       shamir
Initialized     true
Sealed          false
Total Shares    1
Threshold       1
Version         1.6.2
Storage Type    inmem
Cluster Name    vault-cluster-dadc5c6e
Cluster ID      927f9779-63d6-662f-657c-a731d36be176
HA Enabled      false

Scenario Introduction

In this example scenario, you use vault commands in a terminal session to enable the database secrets engine. Then, you configure an instance of the Couchbase secrets engine, define a dynamic role, and read a credential from the role.

After configuring and reading Couchbase credentials from Vault, you can use the credentials to access a document from a bucket in the Couchbase cluster.

Step 1: Enable the database secrets engine

(Persona: vault admin)

You use the databases secrets engine to define an instance of a supported database secrets engine. You must enable the database secrets engine to use any of the supported databases.

Enable the database secrets engine.

$ vault secrets enable database
Success! Enabled the database secrets engine at: database/

Step 2: Configure Couchbase Server secrets engine

(Persona: vault admin)

In this step you create an example Couchbase secrets engine configuration. In this example, you use the initial Couchbase Administrator credentials and do not enable TLS.

Write the Couchbase configuration to the path database/config/couchbase-database; successful configuration results in no output.

$ vault write database/config/couchbase-database \
    plugin_name="couchbase-database-plugin" \
    hosts="couchbase://$(docker inspect -f '{{.NetworkSettings.IPAddress}}' couchbase-server)" \
    tls=false \
    username="Administrator" \
    password="dZLDNkkuGyk=" \
    allowed_roles="art-inventory"

Details for the example command follow.

• plugin_name: This is the databases secrets engine plugin name to use.

• hosts: connection string to the Couchbase cluster hosts; in this example, you are filling in the couchbase-server container IP address by querying it dynamically with docker inspect.

• tls: In production you should use tls by setting this to true, but in this example you set it to false for simplicity.

• username: The Couchbase Server administrator user. You should use a dedicated administrator account in production. In this example you specify the initial Administrator username from the Couchbase cluster startup output.

• password: The Couchbase Server administrator account password.

• allowed_roles: The allowed Vault roles allowed by with this Couchbase Server configuration. Here you specify the art-inventory role which you'll define in the following steps.

Step 3: Define a dynamic role

(Persona: vault admin)

The Couchbase secrets engine provides two types of roles:

• Static Roles: represent a 1 to 1 mapping of Vault roles to Couchbase usernames. Vault manages the Couchbase user's password, and automatically rotates it in the period of time that you configure.

• Dynamic roles: represent a mapping of Vault roles to Couchbase RBAC roles. When a client requests a credential, Vault dynamically generates a unique username and password pair in Couchbase while automatically associating it with the corresponding RBAC role, and returns the credentials to the client. Once the credential lease TTL expires, Vault revokes the credential and deletes the user from Couchbase.

To keep this scenario brief, you write a dynamic role named art-inventory that maps to the Couchbase Query Select role. This grants the user persona the ability to later query documents in the art-inventory bucket. The credentials are time-limited, to a 5 minute TTL with a maximum TTL of 1 hour, and will become expired and revoked after that time.

$ vault write database/roles/art-inventory \
    db_name="couchbase-database" \
    creation_statements='{"Roles": [{"role":"query_select[art-inventory]"}]}' \
    default_ttl="5m" \
    max_ttl="1h"

Output:

Success! Data written to: database/roles/art-inventory

The role is successfully written.

Step 4: Generate a credential

(Persona: user)

Verify that you can generate a new credential by reading from the /database/creds/art-inventory endpoint.

$ vault read database/creds/art-inventory
Key                Value
---                -----
lease_id           database/creds/art-inventory/UbFlbnL2mcN3OprtPkRinadt
lease_duration     5m
lease_renewable    true
password           Y4OxBfBwG56DHjwmy-CP
username           V_TOKEN_ART-INVENTORY_CYSLACEQINAQD6M6CXIW_1613070685

Step 5: Access bucket item

(Persona: user)

Now you are ready to try accessing the example art inventory item art-launch-hearts from the Couchbase art-inventory bucket. You can do so in a terminal session with curl and the Couchbase HTTP N1QL query language HTTP API or with the Couchbase Server web UI.

$ curl --silent \
    http://localhost:8093/query/service \
    --data 'statement=SELECT * FROM `art-inventory`;'

{
  "requestID": "b04540e8-4c77-413f-bd85-5d56a8bf3e7e",
  "signature": { "$1": "number" },
  "results": [],
  "errors": [
    {
      "code": 13014,
      "msg": "User does not have credentials to run SELECT queries on the art-inventory bucket. Add role query_select on art-inventory to allow the query to run."
    }
  ],
  "status": "fatal",
  "metrics": {
    "elapsedTime": "6.8009ms",
    "executionTime": "6.6883ms",
    "resultCount": 0,
    "resultSize": 0,
    "errorCount": 1
  }
}

$ CB_CREDENTIALS="$(vault read database/creds/art-inventory)"

$ CB_USER="$(echo "$CB_CREDENTIALS" | awk '/username/{print $2}')"

$ CB_PASS="$(echo "$CB_CREDENTIALS" | awk '/password/{print $2}')"

$ curl --silent \
    http://localhost:8093/query/service \
    --data 'statement=SELECT * FROM `art-inventory`;' \
    --user $CB_USER:$CB_PASS

{
  "requestID": "3ba68855-090d-41d9-84b3-f3aea6026024",
  "signature": { "*": "*" },
  "results": [
    {
      "art-inventory": [
        {
          "name": "Launch Hearts",
          "dateCreated": "2014",
          "artform": "painting",
          "artMedium": "acrylic",
          "image": "https://art.example.com/painting/art-creator/launch-hearts.png",
          "description": "Launch depicts the advent of space travel...",
          "creator": [
            {
              "name": "Art Creator"
            }
          ],
          "owner": [
            {
              "name": "Art Foundation",
              "price": 9001.0
            }
          ]
        }
      ]
    }
  ],
  "status": "success",
  "metrics": {
    "elapsedTime": "9.8035ms",
    "executionTime": "9.218ms",
    "resultCount": 1,
    "resultSize": 452
  }
}

Example ACL policy requirements

In this tutorial, you were able to accomplish all steps with the Vault server in dev mode using an initial root token.

In a production scenario, each scenario persona requires a different set of capabilities, expressed as Vault ACL policies. If you are not familiar with ACL policies, complete the policies tutorial.

Use a non-root token with a policy attached having the following capabilities for the vault admin persona.

# Mount secrets engines
path "sys/mounts/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Configure the database secrets engine and create roles
path "database/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Manage leases
path "sys/leases/+/database/creds/art-inventory/*" {
  capabilities = [ "create", "read", "update", "delete", "list", "sudo" ]
}

path "sys/leases/+/database/creds/art-inventory" {
  capabilities = [ "create", "read", "update", "delete", "list", "sudo" ]
}

# Write ACL policies
path "sys/policies/acl/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Manage tokens for verification
path "auth/token/create" {
  capabilities = [ "create", "read", "update", "delete", "list", "sudo" ]
}

The user persona tasks require the following capabilities.

# Get credentials from the database secrets engine 'art-inventory' role.
path "database/creds/art-inventory" {
  capabilities = [ "read" ]
}

Cleanup

When you have completed the example scenario, you can clean up the containers used if you no longer wish to keep them.

Stop and remove the Docker containers.

$ docker stop couchbase-server vault-dev

Remove the vault alias.

$ unalias vault

Next steps

You have learned how to use the Couchbase secrets engine by enabling it, configuring its connection to Vault, and creating a dynamic role mapping to a Couchbase RBAC role.

You then demonstrated that the example document you created could not be accessed through a Couchbase N1QL query when attempted without authentication. You retrieved Couchbase credentials from Vault and were able to use them to access the example document.

You can learn more about database secrets engines in general, including how to specify a password policy on dynamic secrets in the Dynamic Secrets: Database Secrets Engine tutorial. If you are ready to update your system to retrieve Couchbase credentials from Vault, read the Vault Agent templates tutorial.

Resources

• Announcing the Couchbase Secrets Engine

• Vault dev mode server

• Couchbase Database Secrets Engine

• Couchbase Database Plugin HTTP API

• RBAC for Administrators