Shutting down ECS node from an ASG
• Mark Eschbach
Annoyingly out of the box AWS doesn’t have a setup to include graceful shutdowns of ECS nodes running as apart of an ASG. Lucky for me this will just require a small set of Lambda expressions and hooking them up to an SQS queue. CircleCI published the basics of getting the ASG steup. This would be cool since I wouldn’t have to worry about this application living somewhere since it’s just an infrastructure task. After taking a few minutes though, I realized this will only get a message where I could ingest it. If I’m going to use Lambda to manage this aspect of the system then I’ll need to figure out this aspect.
Cloudonaut recommends using a timer mechanism to process as many messages as psosible within that period of time. Personally this feels a bit hacky as I now have ot manage timers which invoke other lambdas for the work to occur. Now I’ve got a mgaically (as in CloudWatch) setup translating between SQS and Lambda invocations. For a process as simple as this I don’t see the need for complex routing and random adaptions between synchronous and asynchronous code.
Given the timescale we’re working with, in a lot of ways it seems easier to scedule a container which will run an application which listens to several queues to process the information. Looking for details on how to prevent pulling lead me to a review of a coffee shop and scaling within AWS. The idea would be to use CloudWatch alarms to trigger scaling in on empty receives and out on
ApproximateNumberOfMessagesVisible growing large. I’m wondering if I could skip this and just schedule a Lambda function directly. Well, you can schedule one which is a good direction.
Within Terraform’s Lambda Permission resource example for SNS there is an example of triggering a Lambda. It’s close but not quiet what I’m looking for since it’s an active wiring from CloudWatch. Bingo! I think I found a workable example.