Docker ECS Integration - InterWorks

Docker ECS Integration

Our team is focused on containerization using Docker and orchestration using Kubernetes. As we are a Cloud team, we wanted to explore different ways of deploying containers in the cloud.

Making research about deploying Docker containers on ECS we found out about Docker ECS Integration. As Docker says ‘The Docker ECS Integration enables developers to use native Docker commands to run applications in Amazon EC2 Container Service (ECS) when building cloud-native applications.’ they really mean it. ECS integration uses the same docker-compose.yml file for deployment in the cloud.

Why is this excellent?! If you want to deploy a solution to AWS ECS, for example using Code Build and Code Deploy, you need to set up these AWS services with additional permissions and configuration that takes time and slows down the process. As a developer, you need to configure ECS cluster, tasks, VPC, etc. And besides you already have a docker-compose.yml file, you’ll need to create an additional one buildspec.yml for CodeBuild setup that has a similar effect as docker-compose, but still differ in structure. So, you end up with additional files and configurations.

Docker ECS integration is an integration solution built from AWS and Docker teams, that provides quick deployment directly into AWS ECS (Elastic Container Service) using the Docker CLI and same docker-compose.yml.

Install Docker Desktop Edge

Docker ECS Integration is currently a beta release. For this blog post, I’m using Docker Desktop Edge v19.03.13-beta2. Yes, Docker Desktop Edge is different from Docker Desktop. Before you begin, install Docker Desktop Edge and make sure that under Command Line -> Enable experimental features are enabled.

Figure 1 Docker Desktop Edge

Check your installation by running the command: 

>docker ecs version.

Docker ECS Setup

The next step is to set up AWS profile and region by running:

>docker ecs setup

In the set-up, you define context, region, and AWS credentials. I’m using temporary credentials, so only the first time I enter the profile name with temporary credentials. After they expire, I just re-generate new ones under the same profile name on my local /.aws/credentials and no additional entries are required.

If you are going to use IAM user, make sure that you know AWS Access Key ID and AWS Secret Access Key with required permissions to support creating and managing several services in AWS: https://github.com/docker/ecs-plugin/blob/master/docs/requirements.md

  • ec2:DescribeSubnets
  • ec2:DescribeVpcs
  • iam:CreateServiceLinkedRole
  • iam:AttachRolePolicy
  • cloudformation:*
  • ecs:*
  • logs:*
  • servicediscovery:*
  • elasticloadbalancing:*

Run Compose Applications

Now, that you have integration with AWS, let’s check several things before run >docker ecs compose up:

  • check docker-compose.yml file and make sure that you are not using local images. Images can be places on DockerHub, ECR, but make sure that repo is accessible,
  • docker ecs compose up will run ECS services using the AWS Fargate serverless compute engine and create ECS Cluster, permissions, and VPC for you.

Run command in the folder where your docker-compose.yml file is located. After a minute or two in my case, the output is this:

Figure 2 Create ECS Cluster

The definitions for custom Laravel app, MySQL and nginx are in the docker-compose.yml file. Laravel app image is committed in Docker Hub. As you can see from the output, network, log group, service discovery, Load Balancer are completely managed by AWS. You can see real-time logs by running >docker ecs compose logs in a different CLI window from the same folder.

Figure 3 Logs

In the background, the Docker ECS integration plugin creates a CloudFormation file, defining the required resources for each service from the docker-compose file. It creates an ECS cluster and for each service, it registers a task definition and runs an instance of the service in the ECS Cluster, using AWS Fargate.

You can see CF stack from AWS console or by running >docker ecs compose convert and the CF file will be written out to your console.

AWS ECS Dashboard and Access Application

Let’s see what is created on AWS. Navigate to ECS. There are a cluster, tasks, and running services from definitions that are defined in docker-compose.yml file. 

Figure 4 ECS Cluster
Figure 5 List of services inside the cluster
Figure 6 Details about the service

Try to access your application. Under tab Tasks, click on Task ID number:

Figure 7 Running task details

Paste the Public IP address in Browser. For Laravel you’ll see a welcome screen:

For AWS service definitions, a complete CloudFormation stack is created. CloudFormation template is created by Docker for deploying applications on Amazon ECS as the description says.

If you explore the CloudFormation stack, you’ll find definitions for:

– Define and create ECS cluster with Task definitions and services:
AWS:ECS::Cluster,
AWS::ECS::Service
AWS::ECS::TaskDefinition

– Discover services using AWS Cloud Map API for ECS services:
AWS::ServiceDiscovery::Service
AWS::ServiceDiscovery::PrivateDnsNamespace

– Add Load balancers in front of App and database:
AWS::ElasticLoadBalancingV2::Listener
AWS::ElasticLoadBalancingV2::TargetGroup
AWS::ElasticLoadBalancingV2::LoadBalancer

– Create CloudWatch log group:
AWS::Logs::LogGroup

– Define Roles for task execution per task definition:
AWS::IAM::Role

– Add security groups for services:
AWS::EC2::SecurityGroup
AWS::EC2::SecurityGroupIngress

Clean up

Using >docker ecs compose down you can easily clean up the ECS cluster and everything that runs inside it. This command initiates CF stack delete, and clears up created resources.

Figure 8 Delete resources in progress

Conclusion

If you’ve already had docker-compose files locally, and wonder how to deploy an application on AWS Cloud, Docker ECS Integration is here for quick and easy deployment. Keep in mind that the plugin is still in beta version, so expect upgrades. As Docker says: Today this functionality is being made available as a beta UX using docker ecs to drive commands. Later this year when the functionality becomes generally available this will become part of our new Docker Contexts and will allow you to just run docker run and docker compose.


Katerina Pachkova Tashoska
Katerina Pachkova Tashoska