Filtering

Filter expressions refine data queries for some API listing endpoints, as notated in the individual API documentation.

To create a filter expression, you will write one or more expressions. Each expression has a matching operators composed with selectors and values.

curl --get <path> --data-urlencode 'filter="<filter expression>"'

Filtering is executed on the Consul server, before data is returned, reducing the network load. To pass a filter expression to Consul, use the filter query parameter when sending requests to HTTP API endpoints that support it.

Creating Expressions

A single expression is a matching operator with a selector and value. They are written in plain text format, boolean logic and parenthesization are supported. In general whitespace is ignored, except within literal strings.

Matching Operators

All matching operators use a selector or value to choose what data should be matched. Each endpoint that supports filtering accepts a potentially different list of selectors and is detailed in the API documentation for those endpoints.

// Equality & Inequality checks
<Selector> == "<Value>"
<Selector> != "<Value>"

// Emptiness checks
<Selector> is empty
<Selector> is not empty

// Contains checks or Substring Matching
"<Value>" in <Selector>
"<Value>" not in <Selector>
<Selector> contains "<Value>"
<Selector> not contains "<Value>"

// Regular Expression Matching
<Selector> matches "<Value>"
<Selector> not matches "<Value>"

Selectors

Selectors are used by matching operators to create an expression. They are defined by a . separated list of names. Each name must start with a an ASCII letter and can contain ASCII letters, numbers, and underscores. When part of the selector references a map value it may be expressed using the form ["<map key name>"] instead of .<map key name>. This allows the possibility of using map keys that are not valid selectors in and of themselves.

// selects the foo key within the ServiceMeta mapping for the
// /catalog/service/:service endpoint
ServiceMeta.foo

// Also selects the foo key for the same endpoint
ServiceMeta["foo"]

Values

Values are used by matching operators to create an expression. Values can be any valid selector, a number, or a string. It is best practice to quote values.

Numbers can be base 10 integers or floating point numbers.

When quoting strings, they may either be enclosed in double quotes or backticks. When enclosed in backticks they are treated as raw strings and escape sequences such as \n will not be expanded.

Connecting Expressions

There are several methods for connecting expressions, including

logical or
logical and
logical not
grouping with parenthesis
matching expressions

// Logical Or - evaluates to true if either sub-expression does
<Expression 1> or <Expression 2>

// Logical And - evaluates to true if both sub-expressions do
<Expression 1 > and <Expression 2>

// Logical Not - evaluates to true if the sub-expression does not
not <Expression 1>

// Grouping - Overrides normal precedence rules
( <Expression 1> )

// Inspects data to check for a match
<Matching Expression 1>

Standard operator precedence can be expected for the various forms. For example, the following two expressions would be equivalent.

<Expression 1> and not <Expression 2> or <Expression 3>

( <Expression 1> and (not <Expression 2> )) or <Expression 3>

Filter Utilization

Generally, only the main object is filtered. When filtering for an item within an array that is not at the top level, the entire array that contains the item will be returned. This is usually the outermost object of a response, but in some cases such the /catalog/node/:node endpoint the filtering is performed on a object embedded within the results.

Performance

Filters are executed on the servers and therefore will consume some amount of CPU time on the server. For non-stale queries this means that the filter is executed on the leader.

Filtering Examples

Agent API

Command - Unfiltered

$ curl --request GET localhost:8500/v1/agent/services

Response - Unfiltered

{
  "redis1": {
    "ID": "redis1",
    "Service": "redis",
    "Tags": ["primary", "production"],
    "Meta": {
      "env": "production",
      "foo": "bar"
    },
    "Port": 1234,
    "Address": "",
    "Weights": {
      "Passing": 1,
      "Warning": 1
    },
    "EnableTagOverride": false
  },
  "redis2": {
    "ID": "redis2",
    "Service": "redis",
    "Tags": ["secondary", "production"],
    "Meta": {
      "env": "production",
      "foo": "bar"
    },
    "Port": 1235,
    "Address": "",
    "Weights": {
      "Passing": 1,
      "Warning": 1
    },
    "EnableTagOverride": false
  },
  "redis3": {
    "ID": "redis3",
    "Service": "redis",
    "Tags": ["primary", "qa"],
    "Meta": {
      "env": "qa"
    },
    "Port": 1234,
    "Address": "",
    "Weights": {
      "Passing": 1,
      "Warning": 1
    },
    "EnableTagOverride": false
  }
}

Command - Filtered

curl --get localhost:8500/v1/agent/services --data-urlencode 'filter=Meta.env == "qa"'

Response - Filtered

{
  "redis3": {
    "ID": "redis3",
    "Service": "redis",
    "Tags": ["primary", "qa"],
    "Meta": {
      "env": "qa"
    },
    "Port": 1234,
    "Address": "",
    "Weights": {
      "Passing": 1,
      "Warning": 1
    },
    "EnableTagOverride": false
  }
}

Catalog API

Command - Unfiltered

$ curl --request GET localhost:8500/v1/catalog/service/api-internal

Response - Unfiltered

[
  {
    "ID": "b4f64e8c-5c7d-11e9-bf68-8c8590bd0966",
    "Node": "node-1",
    "Address": "198.18.0.1",
    "Datacenter": "dc1",
    "TaggedAddresses": null,
    "NodeMeta": {
      "agent": "true",
      "arch": "i386",
      "os": "darwin"
    },
    "ServiceKind": "",
    "ServiceID": "api-internal",
    "ServiceName": "api-internal",
    "ServiceTags": ["tag"],
    "ServiceAddress": "",
    "ServiceWeights": {
      "Passing": 1,
      "Warning": 1
    },
    "ServiceMeta": {
      "environment": "qa"
    },
    "ServicePort": 9090,
    "ServiceEnableTagOverride": false,
    "ServiceProxy": {},
    "ServiceConnect": {},
    "CreateIndex": 30,
    "ModifyIndex": 30
  },
  {
    "ID": "b4faf93a-5c7d-11e9-840d-8c8590bd0966",
    "Node": "node-2",
    "Address": "198.18.0.2",
    "Datacenter": "dc1",
    "TaggedAddresses": null,
    "NodeMeta": {
      "arch": "arm",
      "os": "linux"
    },
    "ServiceKind": "",
    "ServiceID": "api-internal",
    "ServiceName": "api-internal",
    "ServiceTags": ["test", "tag"],
    "ServiceAddress": "",
    "ServiceWeights": {
      "Passing": 1,
      "Warning": 1
    },
    "ServiceMeta": {
      "environment": "production"
    },
    "ServicePort": 9090,
    "ServiceEnableTagOverride": false,
    "ServiceProxy": {},
    "ServiceConnect": {},
    "CreateIndex": 29,
    "ModifyIndex": 29
  },
  {
    "ID": "b4fbe7f4-5c7d-11e9-ac82-8c8590bd0966",
    "Node": "node-4",
    "Address": "198.18.0.4",
    "Datacenter": "dc1",
    "TaggedAddresses": null,
    "NodeMeta": {
      "arch": "i386",
      "os": "freebsd"
    },
    "ServiceKind": "",
    "ServiceID": "api-internal",
    "ServiceName": "api-internal",
    "ServiceTags": [],
    "ServiceAddress": "",
    "ServiceWeights": {
      "Passing": 1,
      "Warning": 1
    },
    "ServiceMeta": {
      "environment": "qa"
    },
    "ServicePort": 9090,
    "ServiceEnableTagOverride": false,
    "ServiceProxy": {},
    "ServiceConnect": {},
    "CreateIndex": 28,
    "ModifyIndex": 28
  }
]

Command - Filtered

curl --get localhost:8500/v1/catalog/service/api-internal --data-urlencode 'filter=NodeMeta.os == "linux"'

Response - Filtered

[
  {
    "ID": "b4faf93a-5c7d-11e9-840d-8c8590bd0966",
    "Node": "node-2",
    "Address": "198.18.0.2",
    "Datacenter": "dc1",
    "TaggedAddresses": null,
    "NodeMeta": {
      "arch": "arm",
      "os": "linux"
    },
    "ServiceKind": "",
    "ServiceID": "api-internal",
    "ServiceName": "api-internal",
    "ServiceTags": ["test", "tag"],
    "ServiceAddress": "",
    "ServiceWeights": {
      "Passing": 1,
      "Warning": 1
    },
    "ServiceMeta": {
      "environment": "production"
    },
    "ServicePort": 9090,
    "ServiceEnableTagOverride": false,
    "ServiceProxy": {},
    "ServiceConnect": {},
    "CreateIndex": 29,
    "ModifyIndex": 29
  }
]

Health API

Command - Unfiltered

$ curl --request GET localhost:8500/v1/health/node/node-1

Response - Unfiltered

[
  {
    "Node": "node-1",
    "CheckID": "node-health",
    "Name": "Node level check",
    "Status": "critical",
    "Notes": "",
    "Output": "",
    "ServiceID": "",
    "ServiceName": "",
    "ServiceTags": [],
    "Definition": {},
    "CreateIndex": 13,
    "ModifyIndex": 13
  },
  {
    "Node": "node-1",
    "CheckID": "svc-web-health",
    "Name": "Service level check - web",
    "Status": "warning",
    "Notes": "",
    "Output": "",
    "ServiceID": "",
    "ServiceName": "web",
    "ServiceTags": [],
    "Definition": {},
    "CreateIndex": 18,
    "ModifyIndex": 18
  }
]

Command - Filtered

$ curl --get localhost:8500/v1/health/node/node-1 --data-urlencode 'filter=ServiceName != ""'

Response - Filtered

[
  {
    "Node": "node-1",
    "CheckID": "svc-web-health",
    "Name": "Service level check - web",
    "Status": "warning",
    "Notes": "",
    "Output": "",
    "ServiceID": "",
    "ServiceName": "web",
    "ServiceTags": [],
    "Definition": {},
    "CreateIndex": 18,
    "ModifyIndex": 18
  }
]