Keeping AWS ECS updated with Parameter Store
It has been a while since I have started using AWS ECS (Elastic Container Service) for hosting and maintaining Microservices. A 10,000 feet view of a simple ECS based Microservice Architecture would comprise of an ECS Cluster with a collection of one or more ECS service, each of which can have one or more containers running a Docker image.
While running on a EC2 backed ECS deployment, one of the biggest maintenance overhead comes into the picture when you have to keep your EC2 instances up-to-date with Amazons latest AMIs. Not doing this not only leaves us with VMs which need constant updates / looking after, but also an eventually out-dated version of Docker agent running on the instances. This generally gets flagged on the AWS ECS Console as shown in the image below:
This could either be resolved with manually updating the AMI ID in the CloudFormation / Terraform (or any IaC apart from these 2 which you may be using) OR by some custom script which you may have written using the AWS CLI to get the latest AMI ID for the specific region and updating your relevant autoscaling groups with them.
However, with the latest Amazons release of being able to query AMI IDs using parameter store, updating your stacks is more or less a cake walk. In this article, I will try my best to show everyone how I have done this.
STEP -1: The old way of querying images
Before the use of parameter store, you would have to query images per region by writing a fairly complex cli query as below:
aws ec2 describe-images --owners amazon --filters "Name=name,Values=amzn*" --query 'sort_by(Images, &CreationDate).Name' | grep amzn2
the above query would return all images published by AWS in a specific region. These images would then have a unique AMI ID (something like ami-12345qw) which refers back to the image.
Using the Parameter Store way!
The new parameter store approach allows us to query the latest AMI IDs without having to worry about custom queries. you can just use the above image names but without the dates in them. So an instance store call would look like this: /aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2. Running this query with the AWS cli ssm parameter:
aws ssm get-parameters --names /aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2 --region eu-west-1
You can further use
--query "Parameters.Value" to only get the AMI ID as the return value.
The best part I liked about the above approach is that the only thing we have to change is the region and the AWS published parameter store would return the relevant AMI IDs. Yes that is correct. No more Mappings needed within your CloudFormation templates.
Querying the ECS AMI Parameter Store
Before I show you how we can make best use of this approach using CloudFormation, let us try to query the ECS parameter store published by AWS.
aws ssm get-parameters --names /aws/service/ecs/optimized-ami/amazon-linux/recommended --region eu-west-1
The above call returns the latest recommended AMI which Amazon would have published and updated in their parameter store for people to use.
The above query helps us understand how we can query and view AWS published AMI information on the parameter store.
Using all the magic in CloudFormation
All the above techniques can then be used on an IaC tool like CloudFormation. Using this approach means, each time you run a new / update-stack operation, CloudFormation will retrieve the latest published ECS recommended AMI in that specific region (no need for static image references / region maps).
All we need to do is declare an SSM parameter type which will retrieve the AMI ID as shown below:
and then call that parameter in your AutoScalingGroup Launch Configuration by using
!Ref EcsAmiId as shown below:
For your CloudFormation template / IaC tool to successfully retrieve the SSM parameter, make sure you give the role / IAM user the ssm:GetParameters rights so that it can successfully retrieve these values from the parameter store in that specific AWS region.
That is it! this makes your ECS template fully reusable without any region maps / region dependencies and always have the latest recommended AWS ECS AMI in that specific region. All you have to do is run an update-stack function regularly which should keep your stacks up-to-date without having to worry about any warning messages on your clusters.
I would like to thank Mattia Lepri Berluti for helping me with this idea which made me go all the way to research, test, implement and write about how this can be done without having to dig through tons of articles!