Define credentials for the secrets engine

In the Implement secrets for the secrets engine tutorial, you defined a new secret for your secrets engine.

In this tutorial, you will create a credentials schema for your secrets engine to renew and revoke secrets. You define this at the creds/* path of the secrets engine.

To do this, you will:

1. Set up your development environment.
   You will clone the HashiCups secrets engine repository. This contains many of the interfaces and objects you need to create a secrets engine.
2. Define the fields for the secrets engine's credentials.
   You will set the schema for the role name in the credentials path.
3. Implement read for the secrets engine's credentials.
   You will add code to generate new tokens for the HashiCups API and store them in the credentials path.
4. Add the credentials path to the backend.
   You will update the backend to add a new API path for credentials.
5. Explore acceptance tests that verify the credentials path.
   You will examine acceptance tests that check that Vault can create new HashiCups API tokens on-demand.
6. Test the credentials path.
   You will set up and run acceptance tests to create new HashiCups API tokens for a local version of HashiCups.

Prerequisites

• Golang 1.16+ installed and configured.
• Vault 1.8+ CLI installed locally.
• Docker and Docker Compose to run an instance of HashiCups locally.

Set up your development environment

Clone the learn-vault-plugin-secrets-hashicups repository.

$ git clone https://github.com/hashicorp-education/learn-vault-plugin-secrets-hashicups

Change into the repository directory.

$ cd vault-plugin-secrets-hashicups

Define the fields for the secrets engine's credentials

Open path_credentials.go. The file contains all of the objects and methods related to setting up the creds path for the secrets engine.

The pathCredentials method returns an object with empty fields. These attributes defined framework.Path extend the Vault API for the secrets engine's creds path.

Replace the pathCredentials method in the scaffold with a schema defining the name of the role for Fields.

func pathCredentials(b *hashiCupsBackend) *framework.Path {
    return &framework.Path{
        Pattern: "creds/" + framework.GenericNameRegex("name"),
        Fields: map[string]*framework.FieldSchema{
            "name": {
                Type:        framework.TypeLowerCaseString,
                Description: "Name of the role",
                Required:    true,
            },
        },
        Callbacks:       map[logical.Operation]framework.OperationFunc{},
        HelpSynopsis:    pathCredentialsHelpSyn,
        HelpDescription: pathCredentialsHelpDesc,
    }
}

The path includes attributes for help text, such as HelpSynopsis and HelpDescription. The help text describes the attributes someone needs to configure the secrets engine.

func pathCredentials(b *hashiCupsBackend) *framework.Path {
    return &framework.Path{
        Pattern: "creds/" + framework.GenericNameRegex("name"),
        Fields: map[string]*framework.FieldSchema{
            "name": {
                Type:        framework.TypeLowerCaseString,
                Description: "Name of the role",
                Required:    true,
            },
        },
        Callbacks:       map[logical.Operation]framework.OperationFunc{},
        HelpSynopsis:    pathCredentialsHelpSyn,
        HelpDescription: pathCredentialsHelpDesc,
    }
}

const pathCredentialsHelpSyn = `
Generate a HashiCups API token from a specific Vault role.
`

const pathCredentialsHelpDesc = `
This path generates a HashiCups API user tokens
based on a particular role. A role can only represent a user token,
since HashiCups doesn't have other types of tokens.
`

Implement read for the secrets engine's credentials

Open path_credentials.go.

The Operations field in the pathCredentials method starts empty. You need to add methods to the Operations field to tell Vault how to handle creating, reading, updating, and deleting information at the creds path.

Add a method named createToken that extends the hashiCupsBackend. It creates the token based on the username in the role entry passed to the creds endpoint.

func (b *hashiCupsBackend) createToken(ctx context.Context, s logical.Storage, roleEntry *hashiCupsRoleEntry) (*hashiCupsToken, error) {
    client, err := b.getClient(ctx, s)
    if err != nil {
        return nil, err
    }

    var token *hashiCupsToken

    token, err = createToken(ctx, client, roleEntry.Username)
    if err != nil {
        return nil, fmt.Errorf("error creating HashiCups token: %w", err)
    }

    if token == nil {
        return nil, errors.New("error creating HashiCups token")
    }

    return token, nil
}

You need to parse the hashiCupsToken and map the fields to the response from the /creds endpoint.

Add a new method named createUserCreds in path_credentials.go. The method creates the HashiCups token and maps it to a response for the secrets engine backend to return. It also sets the time to live (TTL) and maximum TTL for the secret.

func (b *hashiCupsBackend) createUserCreds(ctx context.Context, req *logical.Request, role *hashiCupsRoleEntry) (*logical.Response, error) {
    token, err := b.createToken(ctx, req.Storage, role)
    if err != nil {
        return nil, err
    }

    resp := b.Secret(hashiCupsTokenType).Response(map[string]interface{}{
        "token":    token.Token,
        "token_id": token.TokenID,
        "user_id":  token.UserID,
        "username": token.Username,
    }, map[string]interface{}{
        "token": token.Token,
        "role":  role.Name,
    })

    if role.TTL > 0 {
        resp.Secret.TTL = role.TTL
    }

    if role.MaxTTL > 0 {
        resp.Secret.MaxTTL = role.MaxTTL
    }

    return resp, nil
}

Add a method named pathCredentialsRead. The method verifies the role exists for the secrets engine and creates a new HashiCups token based on the role entry.

func (b *hashiCupsBackend) pathCredentialsRead(ctx context.Context, req *logical.Request, d *framework.FieldData) (*logical.Response, error) {
    roleName := d.Get("name").(string)

    roleEntry, err := b.getRole(ctx, req.Storage, roleName)
    if err != nil {
        return nil, fmt.Errorf("error retrieving role: %w", err)
    }

    if roleEntry == nil {
        return nil, errors.New("error retrieving role: role is nil")
    }

    return b.createUserCreds(ctx, req, roleEntry)
}

Replace pathCredentials with an update to the list of Callbacks for operations on the /creds endpoint. Add logical.ReadOperation and logical.UpdateOperation and set the callback method to b.pathCredentialsRead. Both methods use the same callback to generate a new token each time you call the creds endpoint.

func pathCredentials(b *hashiCupsBackend) *framework.Path {
    return &framework.Path{
        Pattern: "creds/" + framework.GenericNameRegex("name"),
        Fields: map[string]*framework.FieldSchema{
            "name": {
                Type:        framework.TypeLowerCaseString,
                Description: "Name of the role",
                Required:    true,
            },
        },
        Callbacks: map[logical.Operation]framework.OperationFunc{
            logical.ReadOperation:   b.pathCredentialsRead,
            logical.UpdateOperation: b.pathCredentialsRead,
        },
        HelpSynopsis:    pathCredentialsHelpSyn,
        HelpDescription: pathCredentialsHelpDesc,
    }
}

Add the credentials path to the backend

For each API path you extend on the secrets engine, you must add it to the secrets engine backend.

Open backend.go and replace backend to add pathCredentials to the list of valid paths for the backend.

func backend() *hashiCupsBackend {
    var b = hashiCupsBackend{}

    b.Backend = &framework.Backend{
        Help: strings.TrimSpace(backendHelp),
        PathsSpecial: &logical.Paths{
            LocalStorage: []string{},
            SealWrapStorage: []string{
                "config",
                "role/*",
            },
        },
        Paths: framework.PathAppend(
            pathRole(&b),
            []*framework.Path{
                pathConfig(&b),
                pathCredentials(&b),
            },
        ),
        Secrets: []*framework.Secret{
            b.hashiCupsToken(),
        },
        BackendType: logical.TypeLogical,
        Invalidate:  b.invalidate,
    }
    return &b
}

Explore acceptance tests that verify the credentials path

The Vault Plugin SDK includes a testing framework for unit and acceptance tests.

• Unit tests: Use mocks to verify the functionality of the secrets engine
• Acceptance tests: Require a Vault instance, an active target API endpoint, and binary for the secrets engine.

You can write a set of acceptance tests to verify the secrets engine creates and updates the new HashiCups token.

Open backend_test.go. The file defines parameters related to managing and setting input values to tests.

Examine the constants for the names of environment variables. The tests retrieve environment variables for a few testing parameters, including:

• VAULT_ACC: Set to 1 to run acceptance tests. Acceptance tests have external dependencies, so you may not want to run them all the time.
• TEST_HASHICUPS_USERNAME: Username for HashiCups API. Make a request to /signup at the HashiCups API.
• TEST_HASHICUPS_PASSWORD: Password for HashiCups API. Make a request to /signup at the HashiCups API.
• TEST_HASHICUPS_URL: Testing endpoint for HashiCups API. Make a request to /signup at the HashiCups API.

const (
    envVarRunAccTests       = "VAULT_ACC"
    envVarHashiCupsUsername = "TEST_HASHICUPS_USERNAME"
    envVarHashiCupsPassword = "TEST_HASHICUPS_PASSWORD"
    envVarHashiCupsURL      = "TEST_HASHICUPS_URL"
)

Open path_credentials_test.go.

Examine the method newAcceptanceTestEnv. It creates an acceptance testing environment that retrieves username and password information from environment variables to access HashiCups.

func newAcceptanceTestEnv() (*testEnv, error) {
    ctx := context.Background()

    maxLease, _ := time.ParseDuration("60s")
    defaultLease, _ := time.ParseDuration("30s")
    conf := &logical.BackendConfig{
        System: &logical.StaticSystemView{
            DefaultLeaseTTLVal: defaultLease,
            MaxLeaseTTLVal:     maxLease,
        },
        Logger: logging.NewVaultLogger(log.Debug),
    }
    b, err := Factory(ctx, conf)
    if err != nil {
        return nil, err
    }
    return &testEnv{
        Username: os.Getenv(envVarHashiCupsUsername),
        Password: os.Getenv(envVarHashiCupsPassword),
        URL:      os.Getenv(envVarHashiCupsURL),
        Backend:  b,
        Context:  ctx,
        Storage:  &logical.InmemStorage{},
    }, nil
}

Examine the testing method, TestAcceptanceUserToken. The test checks if you want the test suite to include acceptance tests, initializes a new acceptance testing environment, and runs a series of commands to test that you generated a new HashiCups API token.

func TestAcceptanceUserToken(t *testing.T) {
    if !runAcceptanceTests {
        t.SkipNow()
    }

    acceptanceTestEnv, err := newAcceptanceTestEnv()
    if err != nil {
        t.Fatal(err)
    }

    t.Run("add config", acceptanceTestEnv.AddConfig)
    t.Run("add user token role", acceptanceTestEnv.AddUserTokenRole)
    t.Run("read user token cred", acceptanceTestEnv.ReadUserToken)
    t.Run("read user token cred", acceptanceTestEnv.ReadUserToken)
    t.Run("cleanup user tokens", acceptanceTestEnv.CleanupUserTokens)
}

TestAcceptanceUserToken runs the tests sequentially and uses a secrets engine backend. You need to set up the secrets engine configuration and create a role before you can generate new credentials. You should write your test sequence as follows:

1. Add a configuration for your secrets engine to /config.
2. Add a role for your secrets engine to /role/test.
3. Read credentials from your secrets engine at /creds/test.
4. Read another set of credentials from your secrets engine at /creds/test. They should not match the first set of credentials.
5. Clean up the tokens by accessing the target API with its client and invalidating the credentials.

You will find the helper methods for the testing sequence in backend_test.go.

Examine ReadUserToken as an example. It calls the secrets engine backend with a ReadOperation at the path, creds/test-user-token. It checks that the response includes the JSON Web Token (JWT) from HashiCups. The method stores the tokens into a list so the test suite cleans them up after it finishes.

func (e *testEnv) ReadUserToken(t *testing.T) {
    req := &logical.Request{
        Operation: logical.ReadOperation,
        Path:      "creds/test-user-token",
        Storage:   e.Storage,
    }
    resp, err := e.Backend.HandleRequest(e.Context, req)
    require.Nil(t, err)
    require.NotNil(t, resp)

    if t, ok := resp.Data["token"]; ok {
        e.Tokens = append(e.Tokens, t.(string))
    }
    require.NotEmpty(t, resp.Data["token"])

    if e.SecretToken != "" {
        require.NotEqual(t, e.SecretToken, resp.Data["token"])
    }

    require.NotNil(t, resp.Secret)
    if t, ok := resp.Secret.InternalData["token"]; ok {
        e.SecretToken = t.(string)
    }
}

Test the credentials path

Open a terminal and make sure your working directory uses the plugins/vault-plugin-secrets-hashicups.

$ pwd

${HOME}/hashicorp/vault-guides/plugins/vault-plugin-secrets-hashicups

The secrets engine directory includes files to create a local instance of HashiCups for acceptance tests.

Use the terminal to create the database and API for HashiCups.

$ cd docker_compose && docker-compose up -d

Creating network "docker_compose_default" with the default driver
Creating docker_compose_db_1 ... done
Creating docker_compose_api_1 ... done

Verify with the terminal that Docker started HashiCups. Send a request to its health check endpoint.

$ cd .. && curl localhost:19090/health

ok

Set the username you will use to test the HashiCups API as the TEST_HASHICUPS_USERNAME environment variable.

$ export TEST_HASHICUPS_USERNAME='vault-plugin-testing'

Set the password you will use to test the HashiCups API as the TEST_HASHICUPS_PASSWORD environment variable.

$ export TEST_HASHICUPS_PASSWORD='Testing!123'

Set the URL you will use to test the HashiCups API as the TEST_HASHICUPS_URL environment variable. You should use localhost:19090 to access the HashiCups Docker containers.

$ export TEST_HASHICUPS_URL='http://localhost:19090'

Sign up for the HashiCups API with a username and password by calling the /signup API endpoint.

$ curl -X POST -H 'Content-Type:application/json' \
  ${TEST_HASHICUPS_URL}/signup \
  -d '{"username": "'${TEST_HASHICUPS_USERNAME}'", "password": "'${TEST_HASHICUPS_PASSWORD}'"}'

Output:

{"UserID":1,"Username":"vault-plugin-testing","token":"${TEST_HASHICUPS_JSON_WEB_TOKEN}"}

Run the acceptance tests for the credentials path in your terminal. You need to set the environment variable VAULT_ACC=1 to run them. The tests should pass.

$ VAULT_ACC=1 go test -v -run TestAcceptanceUserToken

Output:

=== RUN   TestAcceptanceUserToken
=== RUN   TestAcceptanceUserToken/add_config
=== RUN   TestAcceptanceUserToken/add_user_token_role
=== RUN   TestAcceptanceUserToken/read_user_token_cred
=== RUN   TestAcceptanceUserToken/read_user_token_cred#01
=== RUN   TestAcceptanceUserToken/cleanup_user_tokens
--- PASS: TestAcceptanceUserToken (0.03s)
    --- PASS: TestAcceptanceUserToken/add_config (0.00s)
    --- PASS: TestAcceptanceUserToken/add_user_token_role (0.00s)
    --- PASS: TestAcceptanceUserToken/read_user_token_cred (0.02s)
    --- PASS: TestAcceptanceUserToken/read_user_token_cred#01 (0.01s)
    --- PASS: TestAcceptanceUserToken/cleanup_user_tokens (0.01s)
PASS
ok      github.com/hashicorp/vault-guides/plugins/vault-plugin-secrets-hashicups    1.021s

Clean up

Use the terminal to remove the database and API for HashiCups.

$ cd docker_compose && docker-compose down

Next steps

Congratulations! You defined the /creds endpoint for your secrets engine.

If you are stuck in this tutorial, refer to the plugins/vault-plugin-secrets-hashicups/solution directory.

• To learn more about Vault plugins, refer to the Vault Plugin System Documentation.
• Build the secrets engine in the next tutorial.