Analyze AWS Traffic Using Flow Logs | Fit-DevOps

  • Publish flow logs to Cloudwatch Log group
  • Publish flow logs to S3 buckets

What is VPC Flow Logs?

  • VPC flow logs is a feature which is used to capture the information about the IP traffic going to and from network interfaces in the VPC.
  • We can configure configure flow logs to capture those information and sent it to either Cloudwatch Log groups or S3 bucket.
  • Once the logging is sent to one of the destination , We can then use those data for further analysis.
  • Monitoring the traffic that is coming to the AWS resources such as EC2 Instances
  • Determining the direction of the traffic to and from the network interfaces.
  • Diagnosing overly restrictive security group rules.

Understanding Flow Logs

  • Flow logs can be created for VPC , Network Interfaces or Subnets.
  • If the flow logs is configured for a VPC , All the subnets and the network intefaces within that VPC will be monitored.
  • If the flow logs is enabled on subnet level , Then the network interfaces within that subnet will be monitored.
  • The flow logs data which are collected from network intefaces , subnets or VPC is referred as Flow Log records.

Flow Log records

  • Accepted and rejected traffic
  • Traffic through NAT Gateway
  • Traffic through Transit Gateway
  • TCP flag sequence
  • Security group and Network access control List Rules
  • IPv6 Traffic
  • No data and skipped records
  • Cloudwatch Log group
  • S3 Bucket

Publishing Flow Logs to Cloudwatch Log group

  • All the flow logs data can be directly published to Cloudwatch Log group.
  • The Cloudwatch log streams will be created for each network interfaces.And the log streams will have flow log records.
  • We can create multiple flow logs and based on the traffic event (Eg: Accepted traffic), We can sent them to Cloudwatch log group.
  • Before creating flow logs , We need to grant permission for the flow logs to publish logs to Cloudwatch Log group.
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents",
"logs:DescribeLogGroups",
"logs:DescribeLogStreams"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "vpc-flow-logs.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}

Publishing Flow Logs to S3 Bucket

  • If you wish to send flow logs data to S3 bucket instead of Cloudwatch log group. We can configure it.
  • Flow logs can publish flow logs data to S3 bucket.
  • We should have an existing S3 bucket for the flow logs to send the data to S3 bucket.
  • The traffic information collected by flow logs will be sent to S3 bucket and stored as log file objects.

Understanding Flow log files

Creating S3 Bucket Polices for Flow Logs

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AWSLogDeliveryWrite",
"Effect": "Allow",
"Principal": {"Service": "delivery.logs.amazonaws.com"},
"Action": "s3:PutObject",
"Resource":
"arn:aws:s3:::bucket_name/optional_folder/AWSLogs/account_id/*",
"Condition": {"StringEquals": {"s3:x-amz-acl": "bucket-owner-full-control"}}
},
{
"Sid": "AWSLogDeliveryAclCheck",
"Effect": "Allow",
"Principal": {"Service": "delivery.logs.amazonaws.com"},
"Action": "s3:GetBucketAcl",
"Resource": "arn:aws:s3:::bucket_name"
}
]
}

Conclusion

--

--

--

Pro-Active Devops Engineer with 5+ years of experience in Linux , Amazon Web Services, Azure , GCP , Devops tools. Blogs here : https://fitdevops.in

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

How To Make Learn.co More “Stylish”

How unit testing leads to better quality applications with fewer bugs

Essential Programming Mantras for Beginners

Android — Role of “UI Automator” in UI Testing of Native apps involving WebViews using ‘Espresso’

To check occurence of each word in sentence

MIDI CHRISTMAS LIGHT-SHOW(2018)

Encoding MIPS Instructions with C++17

Shows the bit-field layouts of MIPS I-type, J-type- and R-type instructions.

Insight into Types of Hashing

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Rahul K

Rahul K

Pro-Active Devops Engineer with 5+ years of experience in Linux , Amazon Web Services, Azure , GCP , Devops tools. Blogs here : https://fitdevops.in

More from Medium

Why I think cloud computing is the future!

Creating Tags For Resources in Single Account and Multiple Accounts Using Cloud-Custodian

Multiple Cloud Full-Stack delivery

Navigating the Cyber Security Landscape: The Fight Against Ransomware in 2022