Automate Lambda function registration

This topic describes how to automate Lambda function registration using Consul's Lambda registrator module for Terraform.

Introduction

You can deploy the Lambda registrator to your environment to automatically register and deregister Lambda functions with Consul based on the function's tags. Refer to the AWS Lambda tags documentation to learn about tags.

The registrator also stores and periodically updates information required to make mTLS requests to upstream services in the AWS parameter store. This enables Lambda functions to invoke mesh services. Refer to Invoke Services from Lambda Functions for additional information.

The registrator runs as a Lambda function that is invoked by AWS EventBridge. Refer to the AWS EventBridge documentation for additional information.

EventBridge invokes the registrator using either AWS CloudTrail to synchronize with Consul in real-time or in scheduled intervals.

CloudTrail events typically synchronize updates, registration, and deregistration within one minute, but events may occasionally be delayed.

Scheduled events fully synchronize functions between Lambda and Consul to prevent entropy. By default, EventBridge triggers a full sync every five minutes.

The following diagram shows the flow of events from EventBridge into Consul:

1. EventBridge invokes the Lambda registrator based on CloudTrail Lambda events or a schedule.
2. Lambda registrator determines how to reconcile Lambda's control plane state with Consul state and ensures they are in sync by registering, updating, and deregistering Lambda services.

Requirements

Verify that your environment meets the requirements specified in Lambda Function Registration Requirements.

Configuration

The Lambda registrator stores data in the AWS Parameter Store. You can configure the type of data stored and how to store it.

Optional: Store the CA certificate in Parameter Store

When Lambda registrator makes a request to Consul's HTTP API over HTTPS and the Consul API is signed by a custom CA, Lambda registrator uses the CA certificate stored in AWS Parameter Store to verify the authenticity of the Consul API. Refer to the Parameter Store documentation for additional information.

You can apply the following Terraform configuration to store Consul's server CA in Parameter Store:

resource "aws_ssm_parameter" "ca-cert" {
  name        = "/lambda-registrator/ca-cert"
  type        = "SecureString"
  value       = <VALUE>
}

Optional: Store the ACL token in Parameter Store

If Consul access control lists (ACLs) are enabled, Lambda registrator must present an ACL token stored in Parameter Store to access resources. You can use the Consul CLI, API, or the Terraform provider to facilitate the ACL workflow. The following procedure describes how to create and store a token from the command line:

1. Create an ACL policy that includes the following rule:

   rules.hcl

    service_prefix "" {
        policy = "write"
    }
   

2. Run consul acl policy create to create the policy. The following example creates a policy called lambda-registrator-policy containing permissions specified in rules.hcl:

    $ consul acl policy create -name "lambda-registrator-policy" -rules @rules.hcl
   

3. Issue the consul acl token create command to create the token. The following example creates a token linked to the lambda-registrator-policy policy:

    $ consul acl token create -policy-name "lambda-registrator-policy"
   

4. Store the token in Parameter Store by applying the following Terraform:

   resource "aws_ssm_parameter" "acl-token" {
       name        = "/lambda-registrator/acl-token"
       type        = "SecureString"
       value       = <VALUE>
    }
   

Optional: Store extension data in Parameter Store

If you want to enable Lambda functions to invoke services in the mesh, then you must specify a non-empty string in the consul_extension_data_prefix configuration. The string represents a path in the AWS Parameter Store so that the registrator can store data necessary for making mTLS requests to upstream services and update the data periodically. If the path does not exist it will be created.

Lambda registrator encrypts and stores all data for Lambda functions in the AWS Parameter Store according to the Lambda registrator configuration options. The data is stored in the following directory as a SecureString type:

${consul_extension_data_prefix}/${<partition>}/${<namespace>}/${<service_name>}

The registrator also requires the following IAM permissions to access the parameter store:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": ["ssm:PutParameter","ssm:DeleteParameter"],
            "Resource": "arn:aws:ssm:us-east-2:123456789012:parameter/${var.consul_extension_data_prefix}/*"
        }
    ]
}

Lambda Registrator configuration options

Deploy the Lambda registrator

1. Create a Terraform configuration and specify the lambda-registrator module. In the following example, the Lambda registrator is deployed to https://consul.example.com:8501. Refer to the Lambda registrator module documentation for additional usage information:

   module "lambda-registrator" {
      source  = "hashicorp/consul-lambda/consul-lambda-registrator"
      version = "x.y.z"
      name = "consul-lambda-registrator"
      consul_http_addr = "https://aecfe39d629774e348a9844439f5e3c1-1471365273.us-east-1.elb.amazonaws.com:8501"
      ca_cert_path = aws_ssm_parameter.ca-cert.name
      http_token_path = aws_ssm_parameter.acl-token.name
      consul_extension_data_prefix = "/lambda_extension_data"
   }
   

2. Deploy Lambda registrator with terraform apply.

Register Lambda functions

Lambda registrator registers Lambda functions into Consul, regardless of how the functions are deployed. The following procedure describes how to register Lambda functions with the Lambda registrator using Terraform, but you can also deploy a Lambda function with CloudFormation, the AWS user interface, or Cloud Development Kit (CDK):

1. Add the aws_lambda_function resource to your Terraform configuration and specify the name of the Lambda function.
2. Add a tags block to the resource and specify the tags you want to use to register the function (refer to Supported tags).

In the following example, the example Lambda function is registered using the enabled, payload-passthrough, and invocation-mode tags:

resource "aws_lambda_function" "example" {
  …
  function_name = "lambda"
  tags = {
    "serverless.consul.hashicorp.com/v1alpha1/lambda/enabled" = "true"
    "serverless.consul.hashicorp.com/v1alpha1/lambda/payload-passthrough" = "true"
    "serverless.consul.hashicorp.com/v1alpha1/lambda/invocation-mode" = "ASYNCHRONOUS"
  }
}

Supported tags

The following tags are supported. The path prefix for all tags is serverless.consul.hashicorp.com/v1alpha1/lambda. For example, specify the following tag to enable the Lambda registrator to sync the Lambda with Consul:

serverless.consul.hashicorp.com/v1alpha1/lambda/enabled.