Healthchecks for AWS ECS

This note was created on 2023-03-02
This note was last edited on 2023-03-02

DoD: Slack/Email notification, when ECS task is stopped after failed healthchecks.

1. Create "AWS-Chatbot-NotificationsOnly-Policy" channel role for AWS Chatbot with following IAM policy (CloudWatchReadOnlyAccess):
~~~
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "autoscaling:Describe*",
                "cloudwatch:Describe*",
                "cloudwatch:Get*",
                "cloudwatch:List*",
                "logs:Get*",
                "logs:List*",
                "logs:Describe*",
                "logs:TestMetricFilter",
                "logs:FilterLogEvents",
                "sns:Get*",
                "sns:List*"
                ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}
~~~

2. Create Guardrail policy for AWS Chatbot:
~~~
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "autoscaling:Describe*",
                "cloudwatch:Describe*",
                "cloudwatch:Get*",
                "cloudwatch:List*",
                "logs:Get*",
                "logs:List*",
                "logs:Describe*",
                "logs:TestMetricFilter",
                "logs:FilterLogEvents",
                "sns:Get*",
                "sns:List*"
                ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}
~~~

3. Create standard SNS topic.
4. Create EventBridge rule with custom pattern:
~~~
{
  "source": ["aws.ecs"],
  "detail-type": ["ECS Task State Change"],
  "detail": {
    "lastStatus": ["STOPPED"],
    "stoppedReason": [{
      "prefix": "Task failed ELB health checks"
    }, "Essential container in task exited", "Task failed container health checks"]
  }
}
~~~
As "Target", choose "AWS service" and select SNS topic.