Mesh Gateways

Mesh gateways enable service mesh traffic to be routed between different Consul datacenters. Datacenters can reside in different clouds or runtime environments where general interconnectivity between all services in all datacenters isn't feasible.

Prerequisites

Mesh gateways can be used with any of the following Consul configurations for managing separate datacenters or partitions.

WAN Federation
- Mesh gateways can be used to route service-to-service traffic between datacenters
- Mesh gateways can be used to route all WAN traffic, including from Consul servers
Cluster Peering
- Mesh gateways can be used to route service-to-service traffic between datacenters
- Mesh gateways can be used to route control-plane traffic from Consul servers
Admin Partitions
- Mesh gateways can be used to route service-to-service traffic between admin partitions in the same Consul datacenter

Consul

Review the specific guide for your use case to determine the required version of Consul.

Network

General network connectivity to all services within its local Consul datacenter.
General network connectivity to all mesh gateways within remote Consul datacenters.

Proxy

Envoy is the only proxy with mesh gateway capabilities in Consul.

Mesh gateway proxies receive their configuration through Consul, which automatically generates it based on the proxy's registration. Consul can only translate mesh gateway registration information into Envoy configuration.

Sidecar proxies that send traffic to an upstream service through a gateway need to know the location of that gateway. They discover the gateway based on their sidecar proxy registrations. Consul can only translate the gateway registration information into Envoy configuration.

Sidecar proxies that do not send upstream traffic through a gateway are not affected when you deploy gateways. If you are using Consul's built-in proxy as a Connect sidecar it will continue to work for intra-datacenter traffic and will receive incoming traffic even if that traffic has passed through a gateway.

Configuration

Configure the following settings to register the mesh gateway as a service in Consul.

Specify mesh-gateway in the kind field to register the gateway with Consul.
Configure the proxy.upstreams parameters to route traffic to the correct service, namespace, and datacenter. Refer to the upstreams documentation for details. The service proxy.upstreams.destination_name is always required. The proxy.upstreams.datacenter must be configured to enable cross-datacenter traffic. The proxy.upstreams.destination_namespace configuration is only necessary if the destination service is in a different namespace.
Define the Proxy.Config settings using opaque parameters compatible with your proxy (i.e., Envoy). For Envoy, refer to the Gateway Options and Escape-hatch Overrides documentation for additional configuration information.
If ACLs are enabled, a token granting service:write for the gateway's service name and service:read for all services in the datacenter or partition must be added to the gateway's service definition. These permissions authorize the token to route communications for other Consul service mesh services, but does not allow decrypting any of their communications.

Modes

Each upstream associated with a service mesh proxy can be configured so that it is routed through a mesh gateway. Depending on your network, the proxy's connection to the gateway can operate in one of the following modes:

none - No gateway is used and a service mesh sidecar proxy makes its outbound connections directly to the destination services. This is the default for WAN federation. This setting is invalid for peered clusters and will be treated as remote instead.
local - The service mesh sidecar proxy makes an outbound connection to a gateway running in the same datacenter. That gateway is responsible for ensuring that the data is forwarded to gateways in the destination datacenter.
remote - The service mesh sidecar proxy makes an outbound connection to a gateway running in the destination datacenter. The gateway forwards the data to the final destination service. This is the default for peered clusters.

Service Mesh Proxy Configuration

Set the proxy to the preferred mode to configure the service mesh proxy. You can specify the mode globally or within child configurations to control proxy behaviors at a lower level. Consul recognizes the following order of precedence if the gateway mode is configured in multiple locations the order of precedence:

Upstream definition (highest priority)
Service instance definition
Centralized service-defaults configuration entry
Centralized proxy-defaults configuration entry

Example Configurations

Use the following example configurations to help you understand some of the common scenarios.

Enabling Gateways Globally

The following proxy-defaults configuration will enable gateways for all mesh services in the local mode.

Example: Enabling gateways globally

Kind = "proxy-defaults"
Name = "global"
MeshGateway {
  Mode = "local"
}

Enabling Gateways Per Service

The following service-defaults configuration will enable gateways for all mesh services with the name web.

Example: Enabling gateways per service.

Kind = "service-defaults"
Name = "web"
MeshGateway {
  Mode = "local"
}

Enabling Gateways for a Service Instance

The following proxy service configuration enables gateways for the service instance in the remote mode.

Example: Enabling gateways for a service instance.

service {
  name = "web-sidecar-proxy"
  kind = "connect-proxy"
  port = 8181
  proxy {
    destination_service_name = "web"
    mesh_gateway {
      mode = "remote"
    }
    upstreams = [
      {
        destination_name = "api"
        datacenter = "secondary"
        local_bind_port = 10000
      }
    ]
  }
}

# Or alternatively inline with the service definition:

service {
  name = "web"
  port = 8181
  connect {
    sidecar_service {
      proxy {
        mesh_gateway {
          mode = "remote"
        }
        upstreams = [
          {
            destination_name = "api"
            datacenter = "secondary"
            local_bind_port = 10000
          }
        ]
      }
    }
  }
}

{
  "service": {
    "kind": "connect-proxy",
    "name": "web-sidecar-proxy",
    "port": 8181,
    "proxy": {
      "destination_service_name": "web",
      "mesh_gateway": {
        "mode": "remote"
      },
      "upstreams": [
        {
          "destination_name": "api",
          "datacenter": "secondary",
          "local_bind_port": 10000
        }
      ]
    }
  }
}

Enabling Gateways for a Proxy Upstream

The following service definition will enable gateways in the local mode for one upstream, the remote mode for a second upstream and will disable gateways for a third upstream.

Example: Enabling gateways for a proxy upstream.

service {
  name = "web-sidecar-proxy"
  kind = "connect-proxy"
  port = 8181
  proxy {
    destination_service_name = "web"
    upstreams = [
      {
        destination_name = "api"
        destination_peer = "cluster-01"
        local_bind_port = 10000
        mesh_gateway {
          mode = "remote"
        }
      },
      {
        destination_name = "db"
        datacenter = "secondary"
        local_bind_port = 10001
        mesh_gateway {
          mode = "local"
        }
      },
      {
        destination_name = "logging"
        datacenter = "secondary"
        local_bind_port = 10002
        mesh_gateway {
          mode = "none"
        }
      },
    ]
  }
}

{
  "service": {
    "kind": "connect-proxy",
    "name": "web-sidecar-proxy",
    "port": 8181,
    "proxy": {
      "destination_service_name": "web",
      "upstreams": [
        {
          "destination_name": "api",
          "local_bind_port": 10000,
          "mesh_gateway": {
            "mode": "remote"
          }
        },
        {
          "destination_name": "db",
          "local_bind_port": 10001,
          "mesh_gateway": {
            "mode": "local"
          }
        },
        {
          "destination_name": "logging",
          "local_bind_port": 10002,
          "mesh_gateway": {
            "mode": "none"
          }
        }
      ]
    }
  }
}