Using the HTTP APIs with authentication

All of Vault's capabilities are accessible via the HTTP API in addition to the CLI. In fact, most calls from the CLI actually invoke the HTTP API. In some cases, Vault features are not available via the CLI and can only be accessed via the HTTP API.

Accessing Secrets via the REST APIs

Machines that need access to information stored in Vault will most likely access Vault via its REST API. For example, if a machine were using AppRole for authentication, the application would first authenticate to Vault which would return a Vault API token. The application would use that token for future communication with Vault.

For the purpose of this tutorial, you will use the following configuration which disables TLS and uses a file-based backend. TLS is disabled here only for example purposes; it should never be disabled in production.

Create a server configuration file, config.hcl.

$ tee config.hcl <<EOF
storage "file" {
  path = "vault-data"
}

listener "tcp" {
  tls_disable = "true"
}
EOF

Start a new Vault instance using the newly created configuration.

$ vault server -config=config.hcl

At this point, you can use Vault's HTTP API for all your interactions.

Launch a new terminal session, and use curl to initialize Vault with the API.

$ curl \
    --request POST \
    --data '{"secret_shares": 1, "secret_threshold": 1}' \
    http://127.0.0.1:8200/v1/sys/init | jq

The response should be JSON and looks something like this:

{
  "keys": [
    "ff27b63de46b77faabba1f4fa6ef44c948e4d6f2ea21f960d6aab0eb0f4e1391"
  ],
  "keys_base64": [
    "/ye2PeRrd/qruh9Ppu9EyUjk1vLqIflg1qqw6w9OE5E="
  ],
  "root_token": "s.Ga5jyNq6kNfRMVQk2LY1j9iu"
}

This response contains your initial root token. It also includes the unseal key. You can use the unseal key to unseal the Vault and use the root token perform other requests in Vault that require authentication.

To make this tutorial easy to copy-and-paste, you will be using the environment variable $VAULT_TOKEN to store the root token.

Example:

$ export VAULT_TOKEN="s.Ga5jyNq6kNfRMVQk2LY1j9iu"

Using the unseal key (not the root token) from above, you can unseal the Vault via the HTTP API.

Example:

$ curl \
    --request POST \
    --data '{"key": "/ye2PeRrd/qruh9Ppu9EyUjk1vLqIflg1qqw6w9OE5E="}' \
    http://127.0.0.1:8200/v1/sys/unseal | jq

Note that you should replace /ye2PeRrd/qru... with the generated key from your output. This will return a JSON response:

{
  "type": "shamir",
  "initialized": true,
  "sealed": false,
  "t": 1,
  "n": 1,
  "progress": 0,
  "nonce": "",
  "version": "1.7.0",
  "migration": false,
  "cluster_name": "vault-cluster-1b34e68e",
  "cluster_id": "2cccf342-091a-b060-900b-04c29bb71ed4",
  "recovery_seal": false,
  "storage_type": "file"
}

You can invoke the Vault API to validate the initialization status.

$ curl http://127.0.0.1:8200/v1/sys/init

This returns a JSON response.

{ "initialized": true }

Now any of the available auth methods can be enabled and configured. For the purposes of this tutorial lets enable AppRole authentication.

The Authentication tutorial showed how to enable the GitHub auth method using Vault CLI.

$ vault auth enable <auth_method_type>

To see the cURL equivalent of the CLI command to enable AppRole auth method, use the -output-curl-string flag.

$ vault auth enable -output-curl-string approle

Enable the AppRole auth method by invoking the Vault API.

$ curl \
    --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data '{"type": "approle"}' \
    http://127.0.0.1:8200/v1/sys/auth/approle

Notice that the request to enable the AppRole endpoint needed an authentication token. In this case you are passing the root token generated when you started the Vault server. You could also generate tokens using any other authentication mechanisms, but you will use the root token for simplicity.

Now create an AppRole with desired set of ACL policies.

The Policies tutorial used CLI to create my-policy. In this tutorial, use the /sys/policies/acl endpoint to create the same policy via Vault API.

$ curl \
    --header "X-Vault-Token: $VAULT_TOKEN" \
    --request PUT \
    --data '{"policy":"# Dev servers have version 2 of KV secrets engine mounted by default, so will\n# need these paths to grant permissions:\npath \"secret/data/*\" {\n  capabilities = [\"create\", \"update\"]\n}\n\npath \"secret/data/foo\" {\n  capabilities = [\"read\"]\n}\n"}' \
    http://127.0.0.1:8200/v1/sys/policies/acl/my-policy

Since my-policy expects secret/data path to exist, enable KV v2 secrets engine at secret/ using API.

$ curl \
    --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data '{ "type":"kv-v2" }' \
    http://127.0.0.1:8200/v1/sys/mounts/secret

The following command specifies that the tokens issued under the AppRole my-role should be associated with my-policy.

$ curl \
    --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data '{"policies": ["my-policy"]}' \
    http://127.0.0.1:8200/v1/auth/approle/role/my-role

The AppRole auth method expects a RoleID and a SecretID as its input. The RoleID is similar to a username and the SecretID can be thought as the RoleID's password.

The following command fetches the RoleID of the role named my-role.

$ curl \
    --header "X-Vault-Token: $VAULT_TOKEN" \
     http://127.0.0.1:8200/v1/auth/approle/role/my-role/role-id | jq -r ".data"

The response will include the role_id:

{
  "role_id": "3c301960-8a02-d776-f025-c3443d513a18"
}

This command creates a new SecretID under the my-role.

$ curl \
    --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    http://127.0.0.1:8200/v1/auth/approle/role/my-role/secret-id | jq -r ".data"

The response will include the secret_id:

{
  "secret_id": "22d1e0d6-a70b-f91f-f918-a0ee8902666b",
  "secret_id_accessor": "726ab786-70d0-8cc4-e775-c0a75070e5e5",
  "secret_id_ttl": 0
}

These two credentials can be supplied to the login endpoint to fetch a new Vault token.

$ curl --request POST \
       --data '{"role_id": "3c301960-8a02-d776-f025-c3443d513a18", "secret_id": "22d1e0d6-a70b-f91f-f918-a0ee8902666b"}' \
       http://127.0.0.1:8200/v1/auth/approle/login | jq -r ".auth"

The response will be JSON, under the key auth:

{
  "client_token": "s.p5NB4dTlsPiUU94RA5IfbzXv",
  "accessor": "EQTlZwOD4yIFYWIg5YY6Xr29",
  "policies": [
    "default",
    "my-policy"
  ],
  "token_policies": [
    "default",
    "my-policy"
  ],
  "metadata": {
    "role_name": "my-role"
  },
  "lease_duration": 2764800,
  "renewable": true,
  "entity_id": "4526701d-b8fd-3c39-da93-9e17506ec894",
  "token_type": "service",
  "orphan": true
}

The returned client token (s.p5NB4dTlsPiUU94RA5IfbzXv) can be used to authenticate with Vault. This token will be authorized with specific capabilities on all the resources encompassed by the default and my-policy policies. (As it was mentioned in the Policies tutorial, the default policy is attached to all tokens by default. )

The newly acquired token can be exported as the VAULT_TOKEN environment variable value and used to authenticate subsequent Vault requests.

$ export VAULT_TOKEN="s.p5NB4dTlsPiUU94RA5IfbzXv"

Create a version 1 of secret named creds with a key password and its value set to my-long-password.

$ curl \
    --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data '{ "data": {"password": "my-long-password"} }' \
    http://127.0.0.1:8200/v1/secret/data/creds | jq -r ".data"

Example output:

{
  "created_time": "2020-02-05T16:51:34.0887877Z",
  "deletion_time": "",
  "destroyed": false,
  "version": 1
}

Clean up

You can stop the server and unset the VAULT_TOKEN environment variable.

$ unset VAULT_TOKEN

Avoid reusing settings from this lab and delete the vault-data folder.

rm -r vault-data

Help and reference

You can see the documentation on the HTTP APIs for more details on other available endpoints.

Congratulations! You now know all the basics needed to get started with Vault.