CloudWatch Filter Syntax

Overview

CloudWatch Logs uses filter patterns to search, filter, and process log events in real-time. These patterns define terms, numeric values, or regular expressions (regex) to match log entries and are used across metric filters, subscription filters, log event searches, and Live Tail for monitoring logs.

Filter Pattern Uses:

Metric Filters: Convert matched log events into metrics for monitoring and alarms.
Subscription Filters: Route matching log events to AWS services (e.g., Kinesis, Lambda).
Log Events: Search logs interactively to identify key events or patterns.
Live Tail: View logs in real-time as they are ingested.

Supported Filter Pattern Types:

Regex Patterns:
- Surround regex expressions with % (e.g., %ERROR% matches "ERROR").
- Supported regex operators include:
  - ^: Anchors match to the beginning.
  - $: Anchors match to the end.
  - |: OR operator (e.g., %gray|grey%).
  - \: Escape character (e.g., %10\.10\.0\.1% to match an IP address).
  - []: Character class (e.g., %[a-z]% matches any lowercase letter).
  - * / +: Match zero or more, or one or more instances.
Example: %colou?r% matches both "color" and "colour."
Unstructured Log Patterns:
- Match specific terms or phrases.
  - Example: ERROR matches logs containing the word "ERROR."
  - Use double quotes for exact phrases with special characters (e.g., "ERROR 401").
JSON Log Patterns:
- Use {} to define conditions for JSON logs. Example: { $.eventType = "UpdateTrail" }
- Use property selectors like $. to target nested fields or arrays.
- Use AND (&&) and OR (||) for compound conditions.
Example: { ($.user.id = 1) && ($.users[0].email = "John.Doe@example.com") } This matches logs where the user.id is 1 and the first user's email is "John.Doe@example.com".
Space-Delimited Patterns:
- Match terms in logs with space-separated fields. Example:
  [w1=ERROR, w2]
  This matches logs where the first word is "ERROR."

Usage and Limitations:

Filter Testing: Test patterns in the CloudWatch console to verify results.
Case Sensitivity: Filters are case-sensitive.
Regex Limitations:
- Supports a subset of regex symbols and up to 5 regex-based filters per log group.
- Multi-byte characters are not supported.
Quota: A filter pattern can contain up to 2 regex expressions for JSON or space-delimited patterns.

PreviousAlarms for Anomaly Detections NextCloudWatch Service Quota

Last updated 8 months ago

Overview

Filter Pattern Uses:

Metric Filters: Convert matched log events into metrics for monitoring and alarms.

Subscription Filters: Route matching log events to AWS services (e.g., Kinesis, Lambda).

Log Events: Search logs interactively to identify key events or patterns.

Live Tail: View logs in real-time as they are ingested.

Supported Filter Pattern Types:

Regex Patterns:

Surround regex expressions with % (e.g., %ERROR% matches "ERROR").
Supported regex operators include:
- ^: Anchors match to the beginning.
- $: Anchors match to the end.
- |: OR operator (e.g., %gray|grey%).
- \: Escape character (e.g., %10\.10\.0\.1% to match an IP address).
- []: Character class (e.g., %[a-z]% matches any lowercase letter).
- * / +: Match zero or more, or one or more instances.

Example: %colou?r% matches both "color" and "colour."

Unstructured Log Patterns:

Match specific terms or phrases.
- Example: ERROR matches logs containing the word "ERROR."
- Use double quotes for exact phrases with special characters (e.g., "ERROR 401").

JSON Log Patterns:

Use {} to define conditions for JSON logs. Example: { $.eventType = "UpdateTrail" }
Use property selectors like $. to target nested fields or arrays.
Use AND (&&) and OR (||) for compound conditions.

Example: { ($.user.id = 1) && ($.users[0].email = "John.Doe@example.com") } This matches logs where the user.id is 1 and the first user's email is "John.Doe@example.com".

Space-Delimited Patterns:

Match terms in logs with space-separated fields. Example:
```
[w1=ERROR, w2]
```
This matches logs where the first word is "ERROR."

Usage and Limitations:

Filter Testing: Test patterns in the CloudWatch console to verify results.

Case Sensitivity: Filters are case-sensitive.

Regex Limitations:

Supports a subset of regex symbols and up to 5 regex-based filters per log group.
Multi-byte characters are not supported.

Quota: A filter pattern can contain up to 2 regex expressions for JSON or space-delimited patterns.