A story about deploying a Lambda function that periodically deletes EC2 instances

My name is Teraoka and I am an infrastructure engineer.
This time, I would like to summarize the story of deploying a Lambda function for internal use.

In addition to the AWS accounts in which our customers' servers are running,
we also manage accounts for our in-house engineers to conduct technical verification.

The security settings around login for this account (IAM user role/MFA authentication) are set according to rules, but any
operations within the account can be performed freely without any restrictions by members with authority. I am.
Even though it is for verification, if there are too many rules, it will become difficult to handle.

In our case, the most frequently created resource during the verification process is an EC2 instance, and
while we can create it freely, we often forget to stop or delete it.

Naturally, if this occurs frequently, it will result in unnecessary costs.
There is a premise that you must stop or delete EC2 instances when you have finished using them, but
as a preventive measure in case you forget, we have introduced a mechanism to automatically delete EC2 instances periodically using Lambda.

So I'll summarize what I did below.

Creating a Lambda function

I wrote it in Python.
Personally, I know more about Golang, so I feel like I should have written it there.

import json import boto3 import os import requests def terminate(event, context): EXCLUDE_TERMINATE_TAG = os.environ['EXCLUDE_TERMINATE_TAG'] CHATWORK_API_KEY = os.environ['CHATWORK_API_KEY'] CHATWORK_ROOM_ID = os.environ['CHATWORK_ROOM_ID'] CHATWORK_ENDPOINT = os. environ['CHATWORK_ENDPOINT'] ec2 = boto3.client('ec2') without_tag = ec2.describe_instances() with_tag = ec2.describe_instances( Filters=[{ 'Name': 'tag:' + EXCLUDE_TERMINATE_TAG, 'Values': [' true'] }] ) without_tag_set = set(ec2['InstanceId'] for resId in without_tag['Reservations'] for ec2 in resId['Instances']) with_tag_set = set(ec2['InstanceId'] for resId in with_tag[ 'Reservations'] for ec2 in resId['Instances']) target_instances = without_tag_set - with_tag_set list_target_instances = list(target_instances) if len(list_target_instances) != 0: terminateInstances = ec2.terminate_instances( InstanceIds=list_target_instances ) notify_instances = '' if len (with_tag['Reservations']) != 0: for reservation in with_tag['Reservations']: for instance in reservation['Instances']: if len(instance['Tags']) != 0: instance_name = '' for tag in instance['Tags']: if tag['Key'] == 'Name': instance_name = tag['Value'] instance_state_enc = json.dumps(instance['State']) instance_state_dec = json.loads( instance_state_enc) if instance_name != '': notify_instances += instance['InstanceId'] + ' -> ' + instance_name + '(' + instance_state_dec['Name'] + ')'+ '\n' else: notify_instances += instance['InstanceId'] + ' -> ' + 'None' + '(' + instance_state_dec['Name'] + ')'+ '\n' message = '[To:350118][To:1786285][info ][title][Beyond POC] There are instances that have been terminated. list_target_instances } print (response) return response def PostChatwork(ENDPOINT,API_KEY,ROOM_ID,MESSAGE): post_message_url = '{}/rooms/{}/messages'.format(ENDPOINT, ROOM_ID) headers = { 'X-ChatWorkToken': API_KEY } params = { 'body': MESSAGE } resp = requests.post(post_message_url, headers=headers, params=params)

I think you can understand it if you look at the code.

• Delete instances without exclusion tags
• Notify Chatwork of instances excluded from deletion

It will do this process.
There are cases where it would be a problem if the information is deleted without any questions asked, so

we have made it a rule within the company that you should add the exclusion tag "EXCLUDE_TERMINATE: true"

Serverless Framework settings

A story about how to deploy the code you wrote to Lambda.

We are using a tool called Serverless Framework.
https://serverless.com

is a CLI tool that deploys functions
to serverless services such as Lambda , and in the case of Lambda, it executes the deployment in conjunction with CloudFormation.

Prepare the configuration file with the name serverless.yml.

# Welcome to Serverless! # # This file is the main config file for your service. # It's very minimal at this point and uses default values. # You can always add more config options for more control. # We've included some commented out config examples here. # Just uncomment any of them to get that config option. # # For full config options, check the docs: # docs.serverless.com # # Happy Coding! service: beyond-poc plugins: - serverless-prune- plugin provider: name: aws runtime: python3.8 profile: ${opt:profile, ''} region: ap-northeast-1 role: [IAM Role Name] environment: EXCLUDE_TERMINATE_TAG: EXCLUDE_TERMINATE CHATWORK_API_KEY: [ChatWark API KEY] CHATWORK_ROOM_ID: [ChatWark ROOM ID] CHATWORK_ENDPOINT: https://api.chatwork.com/v2 timeout: 10 # my custom env custom: prune: automatic: true number: 5 # you can overwrite defaults here # stage: dev # region: us- east-1 # you can add packaging information here #package: # include: # - include-me.py # - include-me-dir/** # exclude: # - exclude-me.py # - exclude-me-dir /** functions: terminate-instance: handler: handler.terminate events: - schedule: cron(0 15 ? * * *) timeout: 120 memorySize: 128 reservedConcurrency: 1

The exclusion tag and ChatWork authentication information are set as environment variables in environment.
The functions section literally sets the functions.
The schedule is set in the events section, and
with this setting, it will be executed regularly at 15:00 (UTC) every day in conjunction with CloudWatchEvents.
If it is JST, it will be midnight.

Save the code to be deployed in the directory where this configuration file is located.

$ sis deploy --profile [AWS Profile Name]

you can run and deploy the CloudFormation stack automatically created by Serverless Framework from the command line

lastly

I'm making a blog about it, but
I'm actually the one who forgets to delete it the most in my company. I'm sorry. . .