Lambda to Start EC2 Instances on schedule and send instant public IP over email to multiple recipients using SNS

Following article will describe how to start EC2 Instances on schedule using Lambda and send the public IP  instantly to multiple recipients over email using SNS. This activity will help to schedule start EC2 instances on a particular time during weekdays and send the public IP of the instances to the development team who use those instances for some development purpose. Using this method you don’t have to allocate Elastic IP to the EC2 instances and also do not need any human intervention to send the public IP of the instances to users.

Step 1: Create following role for the EC2 Start Stop action:-

1

{

“Version”: “2012-10-17”,

“Statement”: [

{

“Effect”: “Allow”,

“Action”: [

“logs:CreateLogGroup”,

“logs:CreateLogStream”,

“logs:PutLogEvents”

],

“Resource”: “arn:aws:logs:*:*:*”

},

{

“Action”: “ec2:*”,

“Effect”: “Allow”,

“Resource”: “*”

}

]

}

Step 2: Create a lambda function with following Python 2.7 code:-

2.jpg

import boto3

region = ‘eu-central-1’

instances = [‘i- xxxxxxxxxxxx1′,’i- xxxxxxxxxxxx2′,’i- xxxxxxxxxxxx3’]

def lambda_handler(event, context):

ec2 = boto3.client(‘ec2’, region_name=region)

ec2.start_instances(InstanceIds=instances)

print ‘started Bastion and Store1 & Store2 servers’ + str(instances)

Step 3: Schedule the Lambda function from Cloudwatch -> rule -> schedule to run at 11AM IST from Monday to Friday

3

Step 4: Create another role in IAM with Full privilege on EC2 and on SNS (Not recommended for production. Be specific about granting privilages):-

4.jpg

Policy for EC2

{

“Version”: “2012-10-17”,

“Statement”: [

{

“Action”: “ec2:*”,

“Effect”: “Allow”,

“Resource”: “*”

}

]

}

Policy for SNS

{

“Version”: “2012-10-17”,

“Statement”: [

{

“Effect”: “Allow”,

“Action”: [

“sns:*”

],

“Resource”: “arn:aws:sns:eu-central-1:xxxxxxxxxx:fra_tcs_store_ips”

}

]

}

Step 5: Create new Lambda function with the above policy to fetch public IP of instances and send SNS notifications:-

5.jpg

from __future__ import print_function

import json

import boto3

ec2 = boto3.resource(‘ec2′, region_name=’eu-central-1’)

client = boto3.client(

“sns”,

aws_access_key_id=”**********************”,

aws_secret_access_key=”***********************”,

region_name=”eu-central-1″

)

 

topic = client.create_topic(Name=”storeipnotifications”)

topic_arn = topic[‘TopicArn’]

 

client.subscribe(TopicArn=topic_arn,Protocol=’email’,Endpoint=’email.1@xxxxx.com’)

client.subscribe(TopicArn=topic_arn,Protocol=’email’,Endpoint=’email.2@xxxxx.com’)

client.subscribe(TopicArn=topic_arn,Protocol=’email’,Endpoint=’email.3@xxxxxx.com’)

client.subscribe(TopicArn=topic_arn,Protocol=’email’,Endpoint=’email.4@xxxxxx.com’)

 

instances1 = ec2.instances.filter(Filters=[{‘Name’: ‘instance-id’, ‘Values’: [‘i-xxxxxxxxxxxx1’]}])

instances2 = ec2.instances.filter(Filters=[{‘Name’: ‘instance-id’, ‘Values’: [‘i- xxxxxxxxxxxx2’]}])

 

for instance in instances1:

store1IP = instance.public_ip_address

print(“Store1 public IP – “, instance.public_ip_address)

for instance in instances2:

store2IP = instance.public_ip_address

print(“Store2 public IP – “, instance.public_ip_address)

 

print(‘Loading function’)

 

def lambda_handler(event, context):

message = ‘Store1 IP:’+store1IP+’ , Store2 IP:’+store2IP

print(message)

client.publish(Message=message, TopicArn=topic_arn)

return message

Step 6: Now create the schedule to run this Lambda code at 11:05AM IST from Monday to Friday

6.jpg

So once the email subscription is confirmed by the recipients, the recipients will start receiving the IP address of the 2 Stores at 11:05AM IST from Monday to Friday.

Schedule Start & Stop RDS Instances and Save Money

Introduction

If RDS Instances is AWS keep running without 24×7 usage, it can cost you very high in AWS billing.

Take following example how idle RDS instances can cost you:-

Lets take a medium sized RDS instance db.m4.xlarge (4CPU and 16GB RAM) in Sydney region for MySQL engine costs $0.492 per hour.

Now if we use the calculator, usage running 24×7, costs on average $360.15 per month.

Now if we only had that running business hours (9 hrs a day MON-FRI) , talking on average $92.99.

Thats a cost saving of $267.16. That is just for one instance per month/ OR $3,205.92 per annum.

What if you were working in a large enterprise with 10s or 100s of RDS instances.

Hence it is very important to stop the RDS instance when Idle to save significant cost in AWS Billing.

Let’s use aws-sdk, nodeJs Lambda function and CloudWatch to start/stop RDS instances in schedule interval.

Following are step by step guide with screenshot.

Download

Download the source code in zip format from following link and keep the zip file in local machine for further upload in Lambda function.

https://s3.ap-south-1.amazonaws.com/rdsstopstartcode/rds-lambda-stop-start.zip

Step 1: Create IAM Policy RDSManagement

First, let’s create policy, which is in AWS IAM console.

Go to Services -> Policies -> Create Policy -> Create Your Own Policy.
Let’s call it “RDSManagement”. Put the code given below to the ‘Policy Document’ field:

{
    "Version": "2012-10-17",
    "Statement": [        
        {
            "Action": [
                "rds:StopDBInstance",
                "rds:StartDBInstance"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

Press ‘Create policy’, Create Your Own Policy put name, description and JSON to the appropriate fields and press save. Eventually, you have to get something like on the screenshot below:

Screen1:-

1

Screen2:-

2

Screen3:-

3

Screen4:-

4

Step 2: Create Role LambdaRDSManagement

The second step is to create a role which will be associated with lambda function and allow it to manage RDS instances.

Navigate to Services -> Roles -> Create new Role.

Select ‘AWS Lambda’ in ‘AWS Service Role’ section. Search for the policy we created previously, select it, and press ‘Next’.

Screen5:-

5.png

Put “LambdaRDSManagement” as a Role Name, set some description and press ‘Create Role’.

Screen6:-

6.png

Screen7:-

7.png

Screen8:-

8.png

Step 3: Create Lambda Function ManageRDSInstances

Now we are ready to go and create lambda function which will manage our instances. Navigate to Services -> Lambda -> Create a Lambda function -> Blank function. Let’s call it ‘ManageRDSInstances’, select latest Node js 6.x as a runtime. Ignore lambda function code for now, and select ‘Choose an existing role’ in ‘Role’ field. You have to be able to find the previously created role in ‘Existing role’ field. Press ‘Next’ -> ‘Create function’.

Screen9:-

9.png

Now let’s upload our archive zip file previously downloaded in local machine to newly created lambda function.

Services -> Lambda -> ManageRDSInstances, and change Code entry type to ‘Upload a .ZIP file’. Press ‘Upload’, select your zip file and press ‘Save’.

Screen10:-

10

Now we need to configure test event: Actions -> Configure test event.

Screen11:-

11.png

Where “tomcatdb01” and “mysqlbiz” are testing RDS instances. After pressing Press ‘Save’ and ‘Test’, you will see that your RDS instances changed state to ‘Stopping’ and soon to ‘Stopped’.

After they are stopped, you can run the same test with action ‘start’, which will run change state of RDS instances to running.

Step 4: Create CloudWatch rule to schedule the event

The last thing is to set up CloudWatch rules to trigger these function on schedule.
Services -> CloudWatch -> Rules -> Create Rule.

Screen12:-

12.png

Select Schedule instead of default Event Pattern. Now you need to set up cron time. Keep in mind that time must be set in GMT timezone. For instance, to start instances every day at 8 am in cron time will look like this: ‘* 8 * * *’.
After you set cron time for waking up your instances, select Lambda function as a Target and pick your newly created lambda function. Then in Configure Input section put your JSON to Constant(JSON text) field:

{ "instances": ["some-test-instance-1",
"some-test-instance-2"], "action":"start" }

Screen13:-

13.png

Screen14:-

14.png

Now your instances will be woken on every morning at 8AM. Create a similar rule with correct cron time for stopping them, do not forget to change action from start to stop in the json:

{ "instances": ["some-test-instance-1",
"some-test-instance-2"], "action":"stop" }

So now we have schedule the Lambda functions to trigger at schedule interval to start and stop RDS instances and save significant cost in AWS billing.