Build a Fargate deployment flow using Terraform using Github + CodeBuild + CodePipeline

My name is Teraoka and I am an infrastructure engineer.
This time we will talk about deploying an application to Fargate.
Although they are sometimes referred to as Code brothers,
AWS has several services that are useful when deploying to various services.

, I created a pipeline using Terraform to deploy to Fargate using CodeBuild and CodePipeline, so

The Terraform version is "v0.12.24".
Please be careful when using it as a reference.

What we built this time

It looks like below.

The VPC has a three-layer structure of Public, DMZ, and Private, with
ALB and NatGateway in the Public subnet, and
Fargate tasks are started in the DMZ subnet and linked to the ALB target group.
Let's take a closer look at CodeBuild and CodePipeline, which are the core of the deployment pipeline.
A simple diagram is shown below.

The flow is very simple

• Collaborate with Github using WebHook to detect push events to Git and automatically execute CodePipeline
• Get the source code from Github, build a Docker image with CodeBuild, and push it to ECR
• Pull the image pushed to ECR from Fargate and launch a new task
• Register the launched task to the ALB target group
• Exclude old tasks that were originally linked from the target group
• Delete old tasks

It becomes.
When the deployment is executed, the old and new Fargate tasks are swapped.
Now let's take a look at the Terraform code.

Directory structure

$ tree . ├── alb.tf ├── buildspec.yml ├── codebuild.tf ├── codepipeline.tf ├── docker │ ├── nginx │ │ ├── Dockerfile │ │ └── conf │ │ ├── default.conf │ │ └── nginx.conf │ └── sites │ └── index.html ├── docker-compose.yml ├── fargate.tf ├── github.tf ├── iam .f _assume_role.json │ └─ ─ fargate_task_execution_policy.json ├── s3.tf ├── secrets │ └── github_personal_access_token ├── securitygroup.tf ├── ssm.tf ├── tasks │ └── container_definitions.json ├── terraform.tfstate ├─ ─ terraform.tfstate.backup ├── variables.tf └── vpc.tf 7 directories, 28 files

Provider and Variable

Let's write Terraform's Provider and Variable.

provider.tf

provider "aws" { access_key = var.access_key secret_key = var.secret_key region = var.region assume_role { role_arn = var.role_arn } } provider "github" { token = aws_ssm_parameter.github_personal_access_token.value organization = "Teraoka-Org" }

variables.tf

#################### # Provider #################### variable "access_key" { description = " AWS Access Key" } variable "secret_key" { description = "AWS Secret Key" } variable "role_arn" { description = "AWS Role Arn" } variable "region" { default = "ap-northeast-1" }

VPC

Create a VPC.

vpc.tf

#################### # VPC #################### resource "aws_vpc" "vpc" { cidr_block = "10.0.0.0/16" enable_dns_support = true enable_dns_hostnames = true tags = { Name = "vpc-fargate-deploy" } } #################### # Subnet #################### resource "aws_subnet" "public_1a" { vpc_id = aws_vpc.vpc.id availability_zone = "${var.region}a" cidr_block = " 10.0.10.0/24" map_public_ip_on_launch = true tags = { Name = "subnet-fargate-deploy-public-1a" } } resource "aws_subnet" "public_1c" { vpc_id = aws_vpc.vpc.id availability_zone = "${var.region }c" cidr_block = "10.0.11.0/24" map_public_ip_on_launch = true tags = { Name = "subnet-fargate-deploy-public-1c" } } resource "aws_subnet" "dmz_1a" { vpc_id = aws_vpc.vpc.id availability_zone = "${var.region}a" cidr_block = "10.0.20.0/24" map_public_ip_on_launch = true tags = { Name = "subnet-fargate-deploy-dmz-1a" } } resource "aws_subnet" "dmz_1c" { vpc_id = aws_vpc .vpc.id availability_zone = "${var.region}c" cidr_block = "10.0.21.0/24" map_public_ip_on_launch = true tags = { Name = "subnet-fargate-deploy-dmz-1c" } } resource "aws_subnet" " private_1a" { vpc_id = aws_vpc.vpc.id availability_zone = "${var.region}a" cidr_block = "10.0.30.0/24" map_public_ip_on_launch = true tags = { Name = "subnet-fargate-deploy-private-1a" } } resource "aws_subnet" "private_1c" { vpc_id = aws_vpc.vpc.id availability_zone = "${var.region}c" cidr_block = "10.0.31.0/24" map_public_ip_on_launch = true tags = { Name = "subnet-fargate-deploy -private-1c" } } ################### # Route Table #################### resource "aws_route_table" "public" { vpc_id = aws_vpc.vpc.id tags = { Name = "route-fargate-deploy-public" } } resource "aws_route_table" "dmz" { vpc_id = aws_vpc.vpc.id tags = { Name = "route-fargate-deploy-dmz" } } resource "aws_route_table" "private" { vpc_id = aws_vpc.vpc.id tags = { Name = "route-fargate-deploy-private" } } ####### ############# # IGW ################### resource "aws_internet_gateway" "igw" { vpc_id = aws_vpc.vpc.id tags = { Name = "igw-fargate-deploy" } } #################### # NATGW ############## ###### resource "aws_eip" "natgw" { vpc = true tags = { Name = "natgw-fargate-deploy" } } resource "aws_nat_gateway" "natgw" { allocation_id = aws_eip.natgw.id subnet_id = aws_subnet. public_1a.id tags = { Name = "natgw-fargate-deploy" } depends_on = [aws_internet_gateway.igw] } #################### # Route #### ################ resource "aws_route" "public" { route_table_id = aws_route_table.public.id destination_cidr_block = "0.0.0.0/0" gateway_id = aws_internet_gateway.igw.id depends_on = [ aws_route_table.public] } resource "aws_route" "dmz" { route_table_id = aws_route_table.dmz.id destination_cidr_block = "0.0.0.0/0" nat_gateway_id = aws_nat_gateway.natgw.id depends_on = [aws_route_table.dmz] } ###### ############## # Route Association ################### resource "aws_route_table_association" "public_1a" { subnet_id = aws_subnet.public_1a .id route_table_id = aws_route_table.public.id } resource "aws_route_table_association" "public_1c" { subnet_id = aws_subnet.public_1c.id route_table_id = aws_route_table.public.id } resource "aws_route_table_association" "dmz_1a" { subnet_id = aws_subnet.dmz_1a .id route_table_id = aws_route_table.dmz.id } resource "aws_route_table_association" "dmz_1c" { subnet_id = aws_subnet.dmz_1c.id route_table_id = aws_route_table.dmz.id } resource "aws_route_table_association" "private_1a" { subnet_id = aws_subnet.private_1a. id route_table_id = aws_route_table.private. id } resource "aws_route_table_association" "private_1c" { subnet_id = aws_subnet.private_1c.id route_table_id = aws_route_table.private.id }

Security Group

Create a security group.

securitygroup.tf

#################### # Security Group #################### resource "aws_security_group" "alb" { name = "alb-sg" description = "for ALB" vpc_id = aws_vpc.vpc.id } resource "aws_security_group" "fargate" { name = "fargate-sg" description = "for Fargate" vpc_id = aws_vpc.vpc.id } ##################### # Security Group Rule ##################### resource "aws_security_group_rule " "allow_http_for_alb" { security_group_id = aws_security_group.alb.id type = "ingress" protocol = "tcp" from_port = 80 to_port = 80 cidr_blocks = ["0.0.0.0/0"] description = "allow_http_for_alb" } resource "aws_security_group_rule" " From_ALB_TO_FARGATE "{Security_Group_id = Aws_Security_group.fargate.id Type =" Ingress "Protocol =" TCP "from_PORT = 80 t = 80 source_security_Group_id = aws_security_Group.alb.id description = "from_ALB_TO_FARGATE"} RESOURCE_RUP_RU Le "" Egress_alb "{Security_Group_id = Aws_Security_Group .alb.id type = "egress" protocol = "-1" from_port = 0 to_port = 0 cidr_blocks = ["0.0.0.0/0"] description = "Outbound ALL" } resource "aws_security_group_rule" "egress_fargate" { security_group_id = aws_security_group .fargate.id type = "egress" protocol = "-1" from_port = 0 to_port = 0 cidr_blocks = ["0.0.0.0/0"] description = "Outbound ALL" }

ALB

Create an ALB.

alb.tf

#################### # ALB #################### resource "aws_lb" "alb" { name = "alb-fargate-deploy" internal = false load_balancer_type = "application" security_groups = [ aws_security_group.alb.id ] subnets = [ aws_subnet.public_1a.id, aws_subnet.public_1c.id ] } ######## ############ # Target Group ################### resource "aws_lb_target_group" "alb" { name = "fargate-deploy- tg" port = "80" protocol = "HTTP" target_type = "ip" vpc_id = aws_vpc.vpc.id deregistration_delay = "60" health_check { interval = "10" path = "/" port = "traffic-port" protocol = "HTTP" timeout = "4" healthy_threshold = "2" unhealthy_threshold = "10" matcher = "200-302" } } #################### # Listener # ################### resource "aws_lb_listener" "alb" { load_balancer_arn = aws_lb.alb.arn port = "80" protocol = "HTTP" default_action { type = "forward " target_group_arn = aws_lb_target_group.alb.arn } }

Preparing the Docker container

Before starting to build Fargate, create a Dockerfile for the container to be deployed.
Let's manually push the task to ECR in advance to confirm that it has started.

Dockerfile

FROM nginx:alpine COPY ./docker/nginx/conf/default.conf /etc/nginx/conf.d/ ADD ./docker/nginx/conf/nginx.conf /etc/nginx/ COPY ./docker/sites/index .html /var/www/html/ EXPOSE 80

default.conf

server { listen 80 default_server; server_name localhost; index index.php index.html index.htm; location / { root /var/www/html; } }

nginx.conf

user nginx; worker_processes auto; pid /run/nginx.pid; error_log /dev/stdout warn; events { worker_connections 1024; } http { log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $ body_bytes_sent "$http_referer" ' '"$http_user_agent"'; server_tokens off; sendfile on; tcp_nopush on; tcp_nodelay on; include /etc/nginx/mime.types; default_type application/octet-stream; access_log /dev/stdout main; include /etc/nginx/conf.d/*.conf; open_file_cache off; charset UTF-8; }

index.html

<html><body><p>terraform-fargate-deploy</p></body></html>

docker-compose.yml

version: "3" services: nginx: build: context: . dockerfile: ./docker/nginx/Dockerfile image: fargate-deploy-nginx ports: - "80:80"

IAM

Prepare an IAM role to be used with Code-based services on Fargate.

iam.tf

#################### # IAM Role #################### resource "aws_iam_role" "fargate_task_execution" { name = "role-fargate-task-execution" assume_role_policy = file("./roles/fargate_task_assume_role.json") } resource "aws_iam_role" "codebuild_service_role" { name = "role-codebuild-service-role" assume_role_policy = file( "./roles/codebuild_assume_role.json") } resource "aws_iam_role" "codepipeline_service_role" { name = "role-codepipeline-service-role" assume_role_policy = file("./roles/codepipeline_assume_role.json") } ##### ############### # IAM Role Policy #################### resource "aws_iam_role_policy" "fargate_task_execution" { name = " execution-policy" role = aws_iam_role.fargate_task_execution.name policy = file("./roles/fargate_task_execution_policy.json") } resource "aws_iam_role_policy" "codebuild_service_role" { name = "build-policy" role = aws_iam_role.codebuild_service_role.name policy = file("./roles/codebuild_build_policy.json") } resource "aws_iam_role_policy" "codepipeline_service_role" { name = "pipeline-policy" role = aws_iam_role.codepipeline_service_role.name policy = file("./roles/codepipeline_pipeline_policy.json") }

codebuild_assume_role.json

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "codebuild.amazonaws.com" }, "Action": " sts:AssumeRole" } ] }

codebuild_build_policy.json

{ "Version": "2012-10-17", "Statement": [ { "Action": [ "ecr:BatchCheckLayerAvailability", "ecr:CompleteLayerUpload", "ecr:GetAuthorizationToken", "ecr:InitiateLayerUpload", "ecr :PutImage", "ecr:UploadLayerPart", "ecr:GetDownloadUrlForLayer", "ecr:BatchGetImage" ], "Resource": "*", "Effect": "Allow" }, { "Effect": "Allow", " Resource": [ "*" ], "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents" ] }, { "Effect": "Allow", "Resource": [ "* " ], "Action": [ "s3:PutObject", "s3:GetObject", "s3:GetObjectVersion" ] }, { "Effect": "Allow", "Action": "ssm:GetParameters", "Resource" : "*" }, { "Effect": "Allow", "Action": [ "ec2:CreateNetworkInterface", "ec2:DescribeDhcpOptions", "ec2:DescribeNetworkInterfaces", "ec2:DeleteNetworkInterface", "ec2:DescribeSubnets", "ec2:DescribeSecurityGroups", "ec2:DescribeVpcs", "ec2:CreateNetworkInterfacePermission" ], "Resource": "*" } ] }

codepipeline_assume_role.json

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "codepipeline.amazonaws.com" }, "Action": " sts:AssumeRole" } ] }

codepipeline_pipeline_policy.json

{ "Statement": [ { "Action": [ "iam:PassRole" ], "Resource": "*", "Effect": "Allow", "Condition": { "StringEqualsIfExists": { "iam:PassedToService" : [ "cloudformation.amazonaws.com", "elasticbeanstalk.amazonaws.com", "ec2.amazonaws.com", "ecs-tasks.amazonaws.com" ] } } }, { "Action": [ "codecommit:CancelUploadArchive ", "codecommit:GetBranch", "codecommit:GetCommit", "codecommit:GetUploadArchiveStatus", "codecommit:UploadArchive" ], "Resource": "*", "Effect": "Allow" }, { "Action": [ "codedeploy:CreateDeployment", "codedeploy:GetApplication", "codedeploy:GetApplicationRevision", "codedeploy:GetDeployment", "codedeploy:GetDeploymentConfig", "codedeploy:RegisterApplicationRevision" ], "Resource": "*", "Effect": " Allow" }, { "Action": [ "elasticbeanstalk:*", "ec2:*", "elasticloadbalancing:*", "autoscaling:*", "cloudwatch:*", "s3:*", "sns:* ", "cloudformation:*", "rds:*", "sqs:*", "ecs:*" ], "Resource": "*", "Effect": "Allow" }, { "Action": [ "lambda:InvokeFunction", "lambda:ListFunctions" ], "Resource": "*", "Effect": "Allow" }, { "Action": [ "opsworks:CreateDeployment", "opsworks:DescribeApps", "opsworks :DescribeCommands", "opsworks:DescribeDeployments", "opsworks:DescribeInstances", "opsworks:DescribeStacks", "opsworks:UpdateApp", "opsworks:UpdateStack" ], "Resource": "*", "Effect": "Allow" }, { "Action": [ "cloudformation:CreateStack", "cloudformation:DeleteStack", "cloudformation:DescribeStacks", "cloudformation:UpdateStack", "cloudformation:CreateChangeSet", "cloudformation:DeleteChangeSet", "cloudformation:DescribeChangeSet", "cloudformation:ExecuteChangeSet", "cloudformation:SetStackPolicy", "cloudformation:ValidateTemplate" ], "Resource": "*", "Effect": "Allow" }, { "Action": [ "codebuild:BatchGetBuilds", "codebuild :StartBuild" ], "Resource": "*", "Effect": "Allow" }, { "Effect": "Allow", "Action": [ "devicefarm:ListProjects", "devicefarm:ListDevicePools", "devicefarm :GetRun", "devicefarm:GetUpload", "devicefarm:CreateUpload", "devicefarm:ScheduleRun" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "servicecatalog:ListProvisioningArtifacts ", "servicecatalog:CreateProvisioningArtifact", "servicecatalog:DescribeProvisioningArtifact", "servicecatalog:DeleteProvisioningArtifact", "servicecatalog:UpdateProduct" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "cloudformation:ValidateTemplate" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "ecr:DescribeImages" ], "Resource": "*" } ], "Version" : "2012-10-17" }

fargate_task_assume_role.json

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "ecs-tasks.amazonaws.com" }, "Action" : "sts:AssumeRole" } ] }

fargate_task_execution_policy.json

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "ecr:GetAuthorizationToken", "ecr:BatchCheckLayerAvailability", "ecr:GetDownloadUrlForLayer", "ecr:BatchGetImage", "logs:CreateLogStream", "logs:PutLogEvents" ], "Resource": "*" } ] }

Fargate

Build Fargate.

fargate.tf

#################### # ECR #################### resource "aws_ecr_repository" "nginx" { name = "fargate-deploy-nginx" } #################### # Cluster ################## ## resource "aws_ecs_cluster" "cluster" { name = "cluster-fargate-deploy" setting { name = "containerInsights" value = "disabled" } } ################# ### # Task Definition ################### resource "aws_ecs_task_definition" "task" { family = "task-fargate-nginx" container_definitions = file("tasks/ container_definitions.json") cpu = "256" memory = "512" network_mode = "awsvpc" execution_role_arn = aws_iam_role.fargate_task_execution.arn requires_compatibilities = [ "FARGATE" ] } ############### ##### # Service ################### resource "aws_ecs_service" "service" { name = "service-fargate-deploy" cluster = aws_ecs_cluster.cluster. arn task_definition = aws_ecs_task_definition.task.arn desired_count = 1 launch_type = "FARGATE" load_balancer { target_group_arn = aws_lb_target_group.alb.arn container_name = "nginx" container_port = "80" } network_configuration { subnets = [ aws_subnet.dmz_1a.id, aws_subnet.d mz_1c .id ] security_groups = [ aws_security_group.fargate.id ] assign_public_ip = false } }

container_definitions.json

[ { "name": "nginx", "image": "485076298277.dkr.ecr.ap-northeast-1.amazonaws.com/fargate-deploy-nginx:latest", "essential": true, "portMappings": [ { "containerPort": 80, "hostPort": 80 } ] } ]

CodeBuild

Build CodeBuild.

codebuild.tf

resource "aws_codebuild_project" "project" { name = "project-fargate-deploy" description = "project-fargate-deploy" service_role = aws_iam_role.codebuild_service_role.arn artifacts { type = "NO_ARTIFACTS" } environment { compute_type = "BUILD_GENERAL1_SMALL" image = "aws/codebuild/standard:2.0" type = "LINUX_CONTAINER" image_pull_credentials_type = "CODEBUILD" privileged_mode = true environment_variable { name = "AWS_DEFAULT_REGION" value = "ap-northeast-1" } environment_variable { name = "AWS_ACCOUNT_ID" value = "485076298277 " } environment_variable { name = "IMAGE_REPO_NAME_NGINX" value = "fargate-deploy-nginx" } environment_variable { name = "IMAGE_TAG" value = "latest" } } source { type = "GITHUB" location = "https://github.com /beyond-teraoka/fargate-deploy-test.git" git_clone_depth = 1 buildspec = "buildspec.yml" } vpc_config { vpc_id = aws_vpc.vpc.id subnets = [ aws_subnet.dmz_1a.id, aws_subnet.dmz_1c.id ] security_group_ids = [ aws_security_group.fargate.id, ] } }

S3

Create an S3 bucket to store CodePipeline artifacts.

s3.tf

resource "aws_s3_bucket" "pipeline" { bucket = "s3-fargate-deploy" acl = "private" }

SSM

uses Github's personal access token
when retrieving source code from a GitHub repository Because it is confidential information, it is managed in the SSM parameter store.

ssm.tf

#################### # Parameter #################### resource "aws_ssm_parameter" "github_personal_access_token" { name = "github-personal-access-token" description = "github-personal-access-token" type = "String" value = file("./secrets/github_personal_access_token") }

The value part reads the contents of the file under ./secrets.

github_personal_access_token

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

CodePipeline

Build a CodePipeline.

codepipeline.tf

resource "aws_codepipeline" "pipeline" { name = "pipeline-fargate-deploy" role_arn = aws_iam_role.codepipeline_service_role.arn artifact_store { location = aws_s3_bucket.pipeline.bucket type = "S3" } stage { name = "Source" action { name = "Source" category = "Source" owner = "ThirdParty" provider = "GitHub" version = "1" output_artifacts = ["source_output"] configuration = { Owner = "Teraoka-Org" Repo = "fargate-deploy-test" Branch = "master" OAuthToken = aws_ssm_parameter.github_personal_access_token.value PollForSourceChanges = "false" } } } stage { name = "Build" action { name = "Build" category = "Build" owner = "AWS" provider = "CodeBuild" input_artifacts = ["source_output"] output_artifacts = ["build_output"] version = "1" configuration = { ProjectName = aws_codebuild_project.project.name } } } stage { name = "Deploy" action { name = "Deploy" category = "Deploy" owner = "AWS" provider = "ECS" input_artifacts = ["build_output"] version = "1" configuration = { ClusterName = aws_ecs_cluster.cluster.arn ServiceName = aws_ecs_service.service.name FileName = "imagedef.json" } } } } resource "aws_codepipeline_webhook" "webhook" { name = "webhook-fargate-deploy" authentication = "GITHUB_HMAC" target_action = "Source" target_pipeline = aws_codepipeline.pipeline.name authentication_configuration { secret_token = aws_ssm_parameter.github_personal_access_token.value } filter { json_path = "$. ref" match_equals = "refs/heads/{Branch}" } }

Github

Add WebHook settings to Github.

github.tf

resource "github_repository_webhook" "webhook" { repository = "fargate-deploy-test" configuration { url = aws_codepipeline_webhook.webhook.url content_type = "json" insecure_ssl = true secret = aws_ssm_parameter.github_personal_access_token.value } events = ["push"] }

buildspec

That's all the files you need.
The type of build processing to be executed with CodeBuild is written in buildspec.yml.
This time, I will write a description to build a Docker image and push it to ECR.
Additionally, this file must be saved in the root directory of your Git repository.

buildspec.yml

--- version: 0.2 phases: pre_build: commands: - IMAGE_URI_NGINX=$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME_NGINX - IMAGE_URI_PHPFPM=$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws .com/$IMAGE_REPO_NAME_PHPFPM - $(aws ecr get-login --no-include-email --region $AWS_DEFAULT_REGION) build: commands: - docker-compose build - docker tag fargate-deploy-nginx:$IMAGE_TAG $IMAGE_URI_NGINX post_build: commands: - docker push $IMAGE_URI_NGINX:$IMAGE_TAG - echo '[{"name":"nginx","imageUri":"__URI_NGINX__"}]' > imagedef.json - sed -ie "s@__URI_NGINX__@${IMAGE_URI_NGINX}:${IMAGE_TAG} @" imagedef.json artifacts: files: - imagedef.json

When you run Terraform?

Once the necessary resources have been created and you access the ALB endpoint with a browser,
a page like the one below will be displayed.

Try running the deployment

Rewrite the contents of index.html to "terraform-fargate-deploy-test" and push it to Git.
Then, CodePipeline will be automatically executed via WebHook as shown below.

I'm trying to get the source code from GitHub.

Once completed, CodeBuild processing will begin.

It's running properly.
Once execution is complete, you can confirm that the Docker image is saved in ECR as shown below.

Once CodeBuild finishes running, deployment to Fargate will begin.

Looking at the number of tasks on Fargate, the number has increased by one, bringing the total to two.

Let's take a look at the state of the target group.

Two tasks are linked and you are trying to remove the older one.
After disconnecting, try accessing via ALB again.

It's updated properly.

CodePipeline also completed successfully from start to finish.

summary

What did you think?
When using containers in a production environment, including Fargate,
it is very important to think about how to deploy them, as is commonly referred to as CI/CD.
This time, I tried linking multiple AWS Code-based services, and
it's a very easy-to-use service, so I encourage everyone to give it a try.