Serve custom TLS certificates from an external service

This is an advanced topic that describes how to configure ingress gateways to serve TLS certificates sourced from an external service to inbound traffic using secret discovery service (SDS). SDS is a low-level feature designed for developers building integrations with custom TLS management solutions. For instructions on more common ingress gateway implementations, refer to Implement an ingress gateway.

Overview

The following process describes the general procedure for configuring ingress gateways to serve TLS certificates sourced from external services:

Configure static SDS clusters in the ingress gateway service definition.
Register the service definition.
Configure TLS client authentication
Start Envoy.
Configure SDS settings in an ingress gateway configuration entry.
Register the ingress gateway configuration entry with Consul.

Requirements

The external service must implement Envoy's gRPC secret discovery service (SDS) API.
You should have some familiarity with Envoy configuration and the SDS protocol.
The connect.enabled parameter must be set to true for all server agents in the Consul datacenter.
The ports.grpc parameter must be configured for all server agents in the Consul datacenter.

ACL requirements

If ACLs are enabled, you must present a token when registering ingress gateways that grant the following permissions:

service:write for the ingress gateway's service name
service:read for all services in the ingress gateway's configuration entry
node:read for all nodes of the services in the ingress gateway's configuration entry.

These privileges authorize the token to route communications to other services in the mesh. If the Consul client agent on the gateway's node is not configured to use the default gRPC port, 8502, then the gateway's token must also provide agent:read for its node's name in order to discover the agent's gRPC port. gRPC is used to expose Envoy's xDS API to Envoy proxies.

Configure static SDS clusters

You must define one or more additional static clusters in the ingress gateway service definition for each Envoy proxy associated with the gateway. The additional clusters define how Envoy should connect to the required SDS services.

Configure the static clusters in the Proxy.Config.envoy_envoy_extra_static_clusters_json parameter in the service definition.

The clusters must provide connection information and any necessary authentication information, such as mTLS credentials.

You must manually register the ingress gateway with Consul proxy to define extra clusters in Envoy's bootstrap configuration. You can not use the -register flag with consul connect envoy -gateway=ingress to automatically register the proxy to define static clusters.

In the following example, the public-ingress gateway includes a static cluster named sds-cluster that specifies paths to the SDS certificate and SDS certification validation files:

Services {
  Name = "public-ingress"
  Kind = "ingress-gateway"

  Proxy {
    Config {
      envoy_extra_static_clusters_json = <<EOF
{
  "name": "sds-cluster",
  "connect_timeout": "5s",
  "typed_extension_protocol_options": {
    "envoy.extensions.upstreams.http.v3.HttpProtocolOptions": {
      "@type": "type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions",
      "explicit_http_config": {
        "http2_protocol_options": {}
      }
    }
  },
  "type": "LOGICAL_DNS",
  "transport_socket": {
    "name":"tls",
    "typed_config": {
      "@type":"type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext",
      "common_tls_context":{
        "tls_certificate_sds_secret_configs": [
          {
            "name":"tls_sds",
            "sds_config":{
              "path":"/certs/sds-auth-cert.json"
            }
          }
        ],
        "validation_context_sds_secret_config": {
          "name":"validation_context_sds",
          "sds_config":{
              "path":"/certs/sds-validation.json"
          }
        }
      }
    }
  },
  "load_assignment": {
    "cluster_name": "sds-cluster",
    "endpoints": [
      {
        "lb_endpoints": [
          {
            "endpoint": {
              "address": {
                "socket_address": {
                  "address": "sds-server.svc.cluster.local",
                  "port_value": 8080,
                }
              }
            }
          }
        ]
      }
    ]
  }
}
EOF
    }
  }
}

Refer to the Envoy documentation for details about configuration parameters for SDS clusters.

Register the ingress gateway service definition

Issue the consul services register command on the Consul agent on the Envoy proxy's node to register the service. The following example command registers an ingress gateway proxy from a public-ingress.hcl file:

$ consul services register public-ingress.hcl

Refer to Register services and health checks for additional information about registering services in Consul.

Configure TLS client authentication

Store TLS client authentication files, certificate files, and keys on disk where the Envoy proxy runs and ensure that they are available to Consul. Refer to the Envoy documentation for details on configuring authentication files.

The following example specifies certificate chain:

{
  "resources": [
    {
      "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret",
      "name": "tls_sds",
      "tls_certificate": {
        "certificate_chain": {
          "filename": "/certs/sds-client-auth.crt"
        },
        "private_key": {
          "filename": "/certs/sds-client-auth.key"
        }
      }
    }
  ]
}

The following example specifies the validation context:

{
  "resources": [
    {
      "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret",
      "name": "validation_context_sds",
      "validation_context": {
        "trusted_ca": {
          "filename": "/certs/sds-ca.crt"
        }
      }
    }
  ]
}

Start Envoy

Issue the consul connect envoy command to bootstrap Envoy. The following example starts Envoy and registers it as a service called public-ingress:

$ consul connect envoy -gateway=ingress -service public-ingress

Refer to Consul Connect Envoy for additional information about using the consul connect envoy command.

Define an ingress gateway configuration entry

Create an ingress gateway configuration entry that enables the gateway to use certificates from SDS. The configuration entry also maps downstream ingress listeners to upstream services. Configure the following fields:

Kind: Set the value to ingress-gateway.
Name: Consul applies the configuration entry settings to ingress gateway proxies with names that match the Name field.
TLS: The main TLS parameter for the configuration entry holds the SDS configuration. You can also specify TLS configurations per listener and per service.
- TLS.SDS: The SDS map includes the following configuration settings:
  - ClusterName: Specifies the name of the cluster you specified when configuring the SDS cluster.
  - CertResource: Specifies the name of the certificate resource to load.
Listeners: Specify one or more listeners.
- Listeners.Port: Specify a port for the listener. Each listener is uniquely identified by its port number.
- Listeners.Protocol: The default protocol is tcp, but you must specify the protocol used by the services you want to allow traffic from.
- Listeners.Services: The Services field contains the services that you want to expose to upstream services. The field contains several options and sub-configurations that enable granular control over ingress traffic, such as health check and TLS configurations.

For Consul Enterprise service meshes, you may also need to configure the Partition and Namespace fields for the gateway and for each exposed service.

Refer to Ingress gateway configuration entry reference for details about the supported parameters.

The following example directs Consul to retrieve example.com-public-cert certificates from an SDS cluster named sds-cluster and serve them to all listeners:

Kind = "ingress-gateway"
Name = "public-ingress"

TLS {
  SDS {
    ClusterName = "sds-cluster"
    CertResource = "example.com-public-cert"
  }
}

Listeners = [
  {
    Port = 8443
    Protocol = "http"
    Services = ["*"]
  }
]

Register the ingress gateway configuration entry

You can register the configuration entry using the consul config command or by calling the /config API endpoint. Refer to How to Use Configuration Entries for details about applying configuration entries.

The following example registers an ingress gateway configuration entry named public-ingress-cfg.hcl that is stored on the local system:

$ consul config write public-ingress-cfg.hcl

The Envoy instance starts a listener on the port specified in the configuration entry and fetches the TLS certificate named from the SDS server.