Deploying a Lambda function to periodically delete EC2 instances

My name is Teraoka, and I am an infrastructure engineer.
Today I would like to summarize my experience deploying a Lambda function internally.

In addition to the AWS accounts on which our customers' servers are running, we also
manage accounts that our in-house engineers use to conduct technical verification.

This account has security settings for login (IAM user/role/MFA authentication) that have been set up with defined rules, but
any authorized member can freely operate within the account without any restrictions. If there are
too many rules for a test account, it will become difficult to use.

In our case, the most frequently created resource during the verification process was the EC2 instance, but
while it could be created freely, there were quite a few instances where it was forgotten to stop or delete them.

Naturally, if this happens frequently, unnecessary costs will be incurred.
Although it is assumed that you will stop or delete
the EC2 instance when you are finished using it, we have introduced a mechanism to automatically delete EC2 instances periodically using Lambda as a precaution in case you forget.

So below is a summary of what I did

Creating a Lambda function

I wrote it in Python.
Personally, I'm more familiar with Golang, so I should have written it in that.

import json import boto3 import os import requests def terminate(event, context): EXCLUDE_TERMINATE_TAG = os.environ['EXCLUDE_TERMINATE_TAG'] CHATWORK_API_KEY = os.environ['CHATWORK_API_KEY'] CHATWORK_ROOM_ID = os.environ['CHATWORK_ROOM_ID'] CHATWORK_ENDPOINT = os.environ['CHATWORK_ENDPOINT'] ec2 = boto3.client('ec2') without_tag = ec2.describe_instances() with_tag = ec2.describe_instances( Filters=[{ 'Name': 'tag:' + EXCLUDE_TERMINATE_TAG, 'Values': ['true'] }] ) without_tag_set = set(ec2['InstanceId'] for resId in without_tag['Reservations'] for ec2 in resId['Instances']) with_tag_set = set(ec2['InstanceId'] for resId in with_tag['Reservations'] for ec2 in resId['Instances']) target_instances = without_tag_set - with_tag_set list_target_instances = list(target_instances) if len(list_target_instances) != 0: terminateInstances = ec2.terminate_instances( InstanceIds=list_target_instances ) notify_instances = '' if len(with_tag['Reservations']) != 0: for reservation in with_tag['Reservations']: for instance in reservation['Instances']: if len(instance['Tags']) != 0: instance_name = '' for tag in instance['Tags']: if tag['Key'] == 'Name': instance_name = tag['Value'] instance_state_enc = json.dumps(instance['State']) instance_state_dec = json.loads(instance_state_enc) if instance_name != '': notify_instances += instance['InstanceId'] + ' -> ' + instance_name + '(' + instance_state_dec['Name'] + ')'+ '\n' else: notify_instances += instance['InstanceId'] + ' -> ' + 'None' + '(' + instance_state_dec['Name'] + ')'+ '\n' message = '[To:350118][To:1786285][info][title][Beyond POC] There are retired instances (devil)[/title]' + notify_instances + '[/info]' PostChatwork(CHATWORK_ENDPOINT,CHATWORK_API_KEY,CHATWORK_ROOM_ID,message) response = { "TerminateInstances": list_target_instances } print (response) return response def PostChatwork(ENDPOINT,API_KEY,ROOM_ID,MESSAGE): post_message_url = '{}/rooms/{}/messages'.format(ENDPOINT, ROOM_ID) headers = { 'X-ChatWorkToken': API_KEY } params = { 'body': MESSAGE } resp = requests.post(post_message_url, headers=headers, params=params)

I think you can understand if you look at the code

• Delete instances that do not have an exclusion tag
• Notify Chatwork of instances that have been excluded from deletion

can
be a problem if the post is deleted without any explanation, so

we have made it a rule within the company to add the exclusion tag "EXCLUDE_TERMINATE: true" in that case

Serverless Framework configuration

This talk will explain how to deploy the code you write to Lambda

We use a tool called Serverless Framework.
https://serverless.com

a CLI tool that deploys functions
to serverless services such as Lambda , and in the case of Lambda, it works in conjunction with CloudFormation to perform the deployment.

Prepare a configuration file named serverless.yml

# Welcome to Serverless! # # This file is the main config file for your service. # It's very minimal at this point and uses default values. # You can always add more config options for more control. # We've included some commented out config examples here. # Just uncomment any of them to get that config option. # # For full config options, check the docs: # docs.serverless.com # # Happy Coding! service: beyond-poc plugins: - serverless-prune-plugin provider: name: aws runtime: python3.8 profile: ${opt:profile, ''} region: ap-northeast-1 role: [IAM Role Name] environment: EXCLUDE_TERMINATE_TAG: EXCLUDE_TERMINATE CHATWORK_API_KEY: [ChatWark API KEY] CHATWORK_ROOM_ID: [ChatWark ROOM ID] CHATWORK_ENDPOINT: https://api.chatwork.com/v2 timeout: 10 # my custom env custom: prune: automatic: true number: 5 # you can overwrite defaults here # stage: dev # region: us-east-1 # you can add packaging information here #package: # include: # - include-me.py # - include-me-dir/** # exclude: # - exclude-me.py # - exclude-me-dir/** functions: terminate-instance: handler: handler.terminate events: - schedule: cron(0 15 ? * * *) timeout: 120 memorySize: 128 reservedConcurrency: 1

In the environment, we set the exclusion tags and ChatWork authentication information as environment variables.
In the functions section, we literally set the function.
In the events section, we set the schedule
This setting will link with CloudWatchEvents and execute the event regularly every day at 15:00 (UTC),
which is midnight JST.

Save the code to be deployed in the directory where this configuration file is located

$ sis deploy --profile [AWS Profile Name]

you can execute and deploy the CloudFormation stack automatically created by the Serverless Framework from the command line

lastly

I've been using it as a topic for my blog, but
the truth is, I'm the one in the company who forgets to delete things the most, so I'm sorry...