VPCFlow Logs

What are VPC Flow Logs?

VPC Flow Logs are a feature in AWS that captures metadata about the IP traffic flowing to and from network interfaces within a Virtual Private Cloud (VPC). These logs are essential for network traffic analysis, troubleshooting, compliance auditing, and threat detection.

Each flow log entry provides information such as source IP, destination IP, ports, protocols, and whether the traffic was accepted or rejected. For threat hunters, VPC Flow Logs offer insight into potential attacks, data exfiltration, lateral movement, and policy violations.

How VPC Flow Logs Work

1. Data Collection Scope VPC Flow Logs can be created at three levels:

   • VPC level: Captures all traffic within the VPC.

   • Subnet level: Captures traffic specific to a subnet.

   • Network Interface level: Captures traffic specific to an Elastic Network Interface (ENI).

2. Log Destination VPC Flow Logs are stored in one of the following:

   • CloudWatch Logs: Use CloudWatch Insights to query logs.

   • S3 Bucket: Query logs with Amazon Athena.

   • Kinesis Data Firehose: Stream logs to external systems like SIEMs.

Log Record Structure of VPC Flow Logs

Each log entry contains several fields that describe the flow of network traffic. Below is the structure of a typical VPC Flow Log entry:

Field	Description
version	Version of the flow log format
account-id	AWS account ID of the owner
interface-id	Network interface through which traffic passed
srcaddr	Source IP address
dstaddr	Destination IP address
srcport	Source port number
dstport	Destination port number
protocol	Protocol number (e.g., 6 for TCP, 17 for UDP)
packets	Number of packets transferred during the flow
bytes	Total bytes transferred
start	Start time of the flow in epoch seconds
end	End time of the flow in epoch seconds
action	ACCEPT or REJECT (indicates if traffic was allowed or denied)
log-status	OK, NODATA, or SKIPDATA (log status)

---

Use Cases for VPC Flow Logs in Threat Hunting

1. Identifying Suspicious Network Traffic

Look for blocked traffic or inbound traffic from unfamiliar IPs that could indicate a reconnaissance attempt or attack. Example Query: Find blocked traffic from suspicious IP ranges.

SELECT srcAddr, dstAddr, action, bytes, start, end
FROM vpc_flow_logs
WHERE action = 'REJECT'
AND srcAddr LIKE '203.%';

2. Detecting Data Exfiltration Attempts

Monitor outbound traffic to unexpected regions or large volumes of data leaving the network. Example Query: Identify flows with more than 100MB transferred.

SELECT srcAddr, dstAddr, SUM(bytes) AS total_bytes
FROM vpc_flow_logs
WHERE action = 'ACCEPT'
GROUP BY srcAddr, dstAddr
HAVING total_bytes > 100000000;

3. Tracking Lateral Movement within a VPC

Use VPC Flow Logs to detect unusual internal connections between instances, which could indicate lateral movement by an attacker. Example Query: Identify traffic between EC2 instances within the same VPC.

SELECT srcAddr, dstAddr, protocol, bytes, start, end
FROM vpc_flow_logs
WHERE srcAddr LIKE '10.0.%' AND dstAddr LIKE '10.0.%';

4. Troubleshooting Network Issues

Identify failed connections or blocked attempts to understand misconfigurations or potential denial-of-service (DoS) activity. Example Query: Identify repeated failed connection attempts.

SELECT srcAddr, dstAddr, COUNT(*) AS failed_attempts
FROM vpc_flow_logs
WHERE action = 'REJECT'
GROUP BY srcAddr, dstAddr
HAVING failed_attempts > 10;

---

Challenges and Best Practices in Using VPC Flow Logs

1. Handling Large Data Volumes VPC Flow Logs can generate significant volumes of data. Use partitioned tables in Athena or CloudWatch Insights filters to reduce query time and cost.

2. Monitoring for Silent Traffic Some attacks (e.g., malware) use low-bandwidth traffic to avoid detection. Regularly inspect all traffic types, including low-volume flows.

3. Correlating Logs across Services Combine VPC Flow Logs with CloudTrail and GuardDuty findings to get a complete picture of network and API activities.

4. Using Security Groups and NACLs Tune your Security Groups and Network ACLs to minimize unnecessary traffic. Use VPC Flow Logs to audit these rules and detect policy violations.