Cost optimization is crucial for maintaining efficient and affordable cloud infrastructure in the ever-evolving cloud landscape. AWS provides numerous services and tools to help manage and optimize costs. One effective cost-saving measure is identifying and deleting stale Amazon Elastic Block Store (EBS) snapshots. EBS snapshots are point-in-time backups of EBS volumes, and over time, they can accumulate and incur significant storage costs if not managed properly.
In this blog, we'll walk through a practical project to create an AWS Lambda function that identifies EBS snapshots no longer associated with any active EC2 instance and deletes them to save on storage costs. This Lambda function will utilize AWS SDK for Python (Boto3) to interact with AWS services.
Prerequisites
Before we dive into the implementation, ensure you have the following:
An AWS account.
Basic understanding of AWS EC2, EBS, and Lambda.
AWS CLI configured on your local machine (optional for testing locally).
IAM role with appropriate permissions to manage EC2 instances, EBS volumes, and snapshots.
Project Overview
Step 1: Create an EC2 Instance and EBS Volume
First, create an EC2 instance (free-tier specs) with an EBS volume. This volume will be used to create a snapshot for demonstration purposes.
The instance is up and running, now check the EBS volume attached to this instance.
Step 2: Create a Snapshot for the EBS Volume
In the EC2 dashboard, navigate to the Snapshots section and create a snapshot for the newly created EBS volume.
Step 3: Create a Lambda Function
Next, create a Lambda function to identify and delete stale EBS snapshots. Navigate to AWS Lambda: Go to Services -> Lambda -> Create Function.
Configure the Function:
Create the Function with default permissions (we'll update permissions later).
The function is successfully created.
Step 4: Add Python Code to the Lambda Function
Replace the default code with the following Python script, which will identify and delete stale EBS snapshots:
After pasting the code, click on Deploy. Since we will manually trigger the function, click on Test, set the event name as “test” and then click on Save.
Increase the Lambda function's execution time to 10 seconds to ensure it completes its task. Navigate to Configuration -> General Configuration -> Edit -> Save.
Step 6: Update IAM Permissions
Update the Lambda function's IAM role to allow it to describe and delete snapshots, as well as list EC2 instances and volumes:
Go to Configuration -> Permissions.
Click on the role name to open it in a new tab.
Add the following inline policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:DescribeInstances",
"ec2:DescribeSnapshots",
"ec2:DeleteSnapshot",
"ec2:DescribeVolumes"
],
"Resource": "*"
}
]
}
Step 7: Test the Lambda Function
Manually trigger the Lambda function to test it. Ensure that the snapshot created earlier is not deleted because it is associated with an active EC2 instance and EBS volume.
The function has been executed successfully. If we go to the EC2 dashboard and see the Snapshot, it would still be there since the EC2 instances are active and as per our Python code, only those Snapshots will be deleted, which are not associated with an active volume/instance.
Step 8: Delete the EC2 Instance and Associated Volume
Delete the EC2 instance and verify that its associated volume is also deleted. Trigger the Lambda function again to confirm that the snapshot is now identified as stale and deleted.
After the instance and its associated volume have been deleted, come back to the Lambda dashboard and click on Test again.
The function has been successfully executed and now let’s see if the Snapshot has been deleted.
The Snapshot has been successfully deleted.
Conclusion
Congratulations! You have successfully created an AWS Lambda function to optimize storage costs by identifying and deleting stale EBS snapshots. This automated approach helps maintain efficient cloud infrastructure and reduce unnecessary storage expenses.
Additional Tips
Automation: Schedule the Lambda function to run periodically using AWS CloudWatch Events.
Cost-Effective Storage: Consider moving snapshots to Amazon S3 Glacier for long-term, low-cost storage.
Monitoring and Alerts: Set up monitoring and alerts to track storage usage and optimization results.
By following these steps, we can implement an effective cost optimization strategy for our AWS environment, ensuring you only pay for the resources you actively use.
