☁️
CTHFM: AWS
  • Welcome
  • Getting Started
    • Account Setup
  • AWS CLI
    • AWS CLI Overview
    • Installation
  • AWS Fundamentals
    • AWS Documentation
    • AWS Shared Responsibility Model
    • Organizational Hierarchy
    • AWS Principals
    • IAM Fundamentals
      • IAM Policy Components
      • IAM Documentation References
    • AWS Security Services Overview
    • AWS Core Services
    • AWS Frameworks
    • Regions and Availability Zones
  • SQL
    • SQL Refresher for Threat Hunting
  • Logging Reference
    • Cloudtrail
      • What is Cloudtrail?
      • Setting Up Cloudtrail
      • Cloudtrail Events Structure
      • Filtering and Searching CloudTrail Logs
      • IAM ID Prefixes
      • Additional Resources
      • API References
    • VPCFlow Logs
    • GuardDuty
      • Multi-Account Setup
      • GuardDuty Concepts
      • GuardDuty Finding References
      • S3 Protection
      • Malware Protection
        • EC2 Malware Protection
          • EC2 Protection Resources
          • Monitoring Scans
          • EC2 Malware Protection Events: CloudWatch
        • S3 Malware Protection
          • Enabling S3 Malware Protection
          • After Enabling S3 Malware Protection
          • S3 Malware Resource Plan Status
          • S3 Malware Protection Quotas
      • RDS Protection Enablement
      • Lambda Protection Enablement
      • Trusted IP Lists and Threat Lists in Amazon GuardDuty
      • Remediation Recommendations
      • GuardDuty API Reference
      • GuardDuty Quotas
    • Access Analyzer
      • Setup
      • External Access and Unused Access Analyzer Findings
      • Review Findings
      • Access Analyzer Resources
      • Access Analyzer API Reference
    • AWS Network Firewall
      • Permissions
      • Firewall Log Contents
      • Logging Destinations
      • CloudWatch Firewall Metrics
    • AWS Config
      • Resource Management in AWS Config
      • AWS Config Integrations
      • AWS Config Resources
      • Configuration Item
      • Config Rules
        • Evaluation Modes
  • CloudWatch
    • Amazon CloudWatch
      • CloudWatch Concepts
      • CloudWatch Metrics
        • Filter Pattern Syntax
      • CloudWatch Alarms
        • Alarm Recommendations
      • Subscriptions
      • CloudWatch Agent
      • CloudWatch Insights
        • Supported Logs and Discovered Fields
        • CloudWatch Insights Query Syntax
      • Anomaly Detection
        • Create Anomaly Detector
        • Alarms for Anomaly Detections
      • CloudWatch Filter Syntax
      • CloudWatch Service Quota
  • Athena For Threat Hunting
    • Introduction to Athena
    • Setting Up Athena
    • SQL For Threat Hunters
    • Automated Response
    • Query Best Practices
  • AWS Security Research and Resources
    • AWS Security Blog
    • AWS Goat
    • Cloud Goat
    • Pacu
    • Prowler
    • Scout Suite
  • Threat Hunting in AWS
    • Threat Hunting in AWS
    • Threat Hunting Introduction
    • Threat Hunting Process
      • Hypothesis Generation
      • Investigation
      • Identification
      • Resolution & Follow Up
    • Pyramid of Pain
    • MITRE Att&ck
      • MITRE Att&ck Concepts
      • MITRE Att&CK Data Sources
      • MITRE Att&CK Mitigations
    • MITRE Att&ck: AWS
      • MITRE Att&CK Matrix
      • Amazon Web Services Security Control Mappings
    • AWS Threat Hunting Ideas
      • AWS Threat Hunting Ideas: EC2
      • AWS Threat Hunting Ideas: Lambda
      • AWS Threat Hunting Ideas: SQS
      • AWS Threat Hunting Ideas: SNS
      • AWS Threat Hunting Ideas: RDS
Powered by GitBook
On this page
  • Overview
  • How Subscriptions Work
  • Key Components of Subscription Filters:
  • Additional Concepts:
  • Monitoring and Limitations:
  1. CloudWatch
  2. Amazon CloudWatch

Subscriptions

Overview

Subscriptions in CloudWatch Logs enable real-time delivery of log events to other AWS services such as Kinesis Data Streams, Kinesis Firehose, or AWS Lambda for further processing, analysis, or storage. These log events are base64 encoded and compressed using gzip before delivery.

How Subscriptions Work

  1. Subscription Filter: Defines the filter pattern to select which log events to send to a specified destination service.

  2. Supported Destinations:

    • Kinesis Data Streams: For continuous log processing.

    • Kinesis Firehose: For loading data into storage solutions.

    • AWS Lambda: For custom analysis or real-time responses.

    • Amazon OpenSearch Service: For streaming log data into search clusters.

  3. Subscription Levels:

    • Log Group-Level: Each log group can have up to two subscription filters.

    • Account-Level: One account-wide filter can apply to multiple log groups with selection criteria.

  4. Retries and Errors:

    • CloudWatch retries delivery for up to 24 hours for retryable errors (e.g., throttling).

    • For non-retryable errors (e.g., AccessDeniedException), the filter is temporarily disabled for 10 minutes.

Key Components of Subscription Filters:

  • Filter Pattern: Specifies how log data is interpreted and what events are sent to the destination.

  • Destination ARN: The Amazon Resource Name of the destination service (e.g., Kinesis or Lambda).

  • Role ARN: An IAM role with permissions to deliver logs to the destination. This is not required for Lambda.

  • Distribution: For Kinesis Data Streams, logs can be grouped by stream or distributed randomly.

Additional Concepts:

  • Cross-Account & Cross-Region Subscriptions: You can send log events to destinations in other accounts or regions.

  • Log Recursion Prevention: Selection criteria ensure that log loops are avoided by filtering logs appropriately.

  • Batching: CloudWatch may batch log events to optimize transmission, though batching is not guaranteed.

Monitoring and Limitations:

  • Subscriptions work only with Standard log class log groups.

  • Metrics are available to monitor the performance of log forwarding.

  • Account-Level Selection Criteria: If no criteria are specified, the subscription applies to all log groups in the account.

PreviousAlarm RecommendationsNextCloudWatch Agent

Last updated 8 months ago