Cloud 18 min read

AWS CLI First #11: Elastic High-Availability: EC2 Auto Scaling Groups (ASG), Launch Templates & Self-Healing Fleets

Build self-healing, horizontally scalable cloud infrastructure with EC2 Auto Scaling Groups, versioned Launch Templates, Target Tracking scaling policies, and zero-downtime rolling instance refreshes purely via the AWS CLI.

Mohammad Rizky Prawira portrait
Mohammad Rizky Prawira
terminal Series Guide • Part 11 of 17

AWS with CLI: The Terminal-First Mastery Path

Part of the CLI-first comprehensive learning path from foundational setup to production cloud architectures.

View Full Learning Path arrow_forward

In Part 10, we deployed an Application Load Balancer (ALB) and manually registered two compute nodes (Node-A and Node-B) into a Target Group. While traffic was evenly distributed across Availability Zones, this static pattern introduces critical production liabilities:

  1. No Automated Recovery (Self-Healing): If an application process crashes or an underlying hypervisor suffers silent hardware degradation at 2:00 AM, the ALB marks the instance unhealthy—but no replacement node is created until an on-call engineer intervenes.
  2. Rigid Capacity & Overprovisioning: Static fleets either collapse during unexpected traffic spikes or drain your monthly budget by sitting 80% idle during off-peak night hours.
  3. Risky Fleet Deployments: Rolling out application code or updating base operating system AMIs requires tedious manual instance launches, target registrations, and manual deregistration.

In this guide, we transition from static server setups to elastic, self-healing, horizontally scalable cloud infrastructure. We will build an automated compute fleet using EC2 Launch Templates, Auto Scaling Groups (ASG), ELB Health Check Replacement, Target Tracking Scaling Policies, and Zero-Downtime Rolling Instance Refreshes—all orchestrated purely through the AWS CLI.

💰 Estimated Lab Cost & Resource Architecture

1. Cost Breakdown for this Lab

All resources in this lab are provisioned on-demand in the Asia Pacific (Singapore) ap-southeast-1 region and can be torn down immediately after completing the exercises:

ResourceQuantityUnit Cost (Hourly)Estimated Lab Cost (1 Hour)
Application Load Balancer (ALB)1 Load Balancer$0.0225 / hour + $0.008 / LCU-hr~$0.025
EC2 Graviton (t4g.micro)2 Baseline Nodes (up to 3 in test)$0.0084 / hour per instance~$0.017
Public IPv4 Addresses (AWS 2024 Rule)2 Attached Public IPs$0.005 / hour per IPv4~$0.010
EC2 Launch Template & ASG1 Template + 1 ASG$0.00 / hour (Free AWS Management)$0.000
Target Group & Scaling Policies1 Target Group + 1 Policy$0.00 / hour (Free)$0.000
Total Estimated Lab Cost~$0.052 USD

[!NOTE] AWS Auto Scaling Pricing: AWS Auto Scaling itself is completely free of charge. You pay only for the underlying EC2 instances, EBS storage, and Load Balancers spawned by the group.

2. Architecture Diagram

+----------------------------------------------------------------+
|                  Internet / End Users (HTTP)                   |
+-------------------------------+--------------------------------+
                                |
                                v
+-------------------------------+--------------------------------+
|           Application Load Balancer (Multi-AZ Ingress)         |
+-------------------------------+--------------------------------+
                                |
                                v
+-------------------------------+--------------------------------+
|       Target Group (HTTP Health Checks & Draining: 30s)        |
+-------------------------------+--------------------------------+
                                |
                +---------------+---------------+
                |                               |
                v                               v
+-------------------------------+ +------------------------------+
|     Availability Zone 1a      | |     Availability Zone 1b     |
|   (Subnet: 10.0.1.0/24)       | |   (Subnet: 10.0.2.0/24)      |
|                               | |                              |
| +---------------------------+ | | +--------------------------+ |
| |   Auto-Scaled Instance    | | | |   Auto-Scaled Instance   | |
| |   (AL2023 Graviton ARM64) | | | |  (AL2023 Graviton ARM64) | |
| +---------------------------+ | | +--------------------------+ |
+-------------------------------+ +------------------------------+
                                |
                                v
+----------------------------------------------------------------+
|    EC2 Auto Scaling Group (Min: 2, Desired: 2, Max: 5)         |
|  - Health Checks: ELB (Auto-replaces failed nodes in < 60s)    |
|  - Scaling Policy: Target Tracking (Avg CPU Utilization 50%)   |
|  - Lifecycle Hook: Connection Draining on Instance Terminate   |
+----------------------------------------------------------------+

🛠️ Phase 1: Environment & Foundation Setup

Before configuring an Auto Scaling Group, we require a Multi-AZ VPC and an Application Load Balancer with an active Target Group.

Option A: If continuing from Part 10 (ALB already running)

If you already have your Part 10 foundation running in your AWS account, re-export the dynamic resource IDs:

REGION="ap-southeast-1"
PROFILE="cloud-dev-admin"

VPC_ID=$(aws ec2 describe-vpcs --filters "Name=tag:Name,Values=Production-VPC" --region "$REGION" --profile "$PROFILE" --query "Vpcs[0].VpcId" --output text)
PUB_SUB_A=$(aws ec2 describe-subnets --filters "Name=vpc-id,Values=$VPC_ID" "Name=tag:Name,Values=Public-Subnet-A" --region "$REGION" --profile "$PROFILE" --query "Subnets[0].SubnetId" --output text)
PUB_SUB_B=$(aws ec2 describe-subnets --filters "Name=vpc-id,Values=$VPC_ID" "Name=tag:Name,Values=Public-Subnet-B" --region "$REGION" --profile "$PROFILE" --query "Subnets[0].SubnetId" --output text)
ALB_SG_ID=$(aws ec2 describe-security-groups --filters "Name=vpc-id,Values=$VPC_ID" "Name=group-name,Values=ALB-Public-SG" --region "$REGION" --profile "$PROFILE" --query "SecurityGroups[0].GroupId" --output text)
APP_SG_ID=$(aws ec2 describe-security-groups --filters "Name=vpc-id,Values=$VPC_ID" "Name=group-name,Values=App-Tier-SG" --region "$REGION" --profile "$PROFILE" --query "SecurityGroups[0].GroupId" --output text)
TG_ARN=$(aws elbv2 describe-target-groups --names "Production-App-TG" --region "$REGION" --profile "$PROFILE" --query "TargetGroups[0].TargetGroupArn" --output text 2>/dev/null || true)
ALB_ARN=$(aws elbv2 describe-load-balancers --names "Production-ALB" --region "$REGION" --profile "$PROFILE" --query "LoadBalancers[0].LoadBalancerArn" --output text 2>/dev/null || true)

Option B: Fast-Track 1-Click Multi-AZ & ALB Foundation (< 90 Seconds)

If starting fresh or running this lab independently, run this idempotent bootstrap script to build the entire Multi-AZ network, security group chaining, Application Load Balancer, and empty Target Group:

cat <<'EOF' > setup-asg-foundation.sh
#!/usr/bin/env bash
set -euo pipefail

REGION="ap-southeast-1"
PROFILE="cloud-dev-admin"

echo "=== 1. Multi-AZ Base Network ==="
VPC_ID=$(aws ec2 describe-vpcs --filters "Name=tag:Name,Values=Production-VPC" --region "$REGION" --profile "$PROFILE" --query "Vpcs[0].VpcId" --output text 2>/dev/null || true)
if [ -z "$VPC_ID" ] || [ "$VPC_ID" = "None" ]; then
  VPC_ID=$(aws ec2 create-vpc --cidr-block "10.0.0.0/16" --tag-specifications 'ResourceType=vpc,Tags=[{Key=Name,Value=Production-VPC}]' --region "$REGION" --profile "$PROFILE" --query "Vpc.VpcId" --output text)
  aws ec2 modify-vpc-attribute --vpc-id "$VPC_ID" --enable-dns-support '{"Value":true}' --region "$REGION" --profile "$PROFILE"
  aws ec2 modify-vpc-attribute --vpc-id "$VPC_ID" --enable-dns-hostnames '{"Value":true}' --region "$REGION" --profile "$PROFILE"
fi

IGW_ID=$(aws ec2 describe-internet-gateways --filters "Name=attachment.vpc-id,Values=$VPC_ID" --region "$REGION" --profile "$PROFILE" --query "InternetGateways[0].InternetGatewayId" --output text 2>/dev/null || true)
if [ -z "$IGW_ID" ] || [ "$IGW_ID" = "None" ]; then
  IGW_ID=$(aws ec2 create-internet-gateway --tag-specifications 'ResourceType=internet-gateway,Tags=[{Key=Name,Value=Production-IGW}]' --region "$REGION" --profile "$PROFILE" --query "InternetGateway.InternetGatewayId" --output text)
  aws ec2 attach-internet-gateway --vpc-id "$VPC_ID" --internet-gateway-id "$IGW_ID" --region "$REGION" --profile "$PROFILE"
fi

PUB_RTB_ID=$(aws ec2 describe-route-tables --filters "Name=vpc-id,Values=$VPC_ID" "Name=tag:Name,Values=Production-Public-RT" --region "$REGION" --profile "$PROFILE" --query "RouteTables[0].RouteTableId" --output text 2>/dev/null || true)
if [ -z "$PUB_RTB_ID" ] || [ "$PUB_RTB_ID" = "None" ]; then
  PUB_RTB_ID=$(aws ec2 create-route-table --vpc-id "$VPC_ID" --tag-specifications 'ResourceType=route-table,Tags=[{Key=Name,Value=Production-Public-RT}]' --region "$REGION" --profile "$PROFILE" --query "RouteTable.RouteTableId" --output text)
  aws ec2 create-route --route-table-id "$PUB_RTB_ID" --destination-cidr-block "0.0.0.0/0" --gateway-id "$IGW_ID" --region "$REGION" --profile "$PROFILE"
fi

PUB_SUB_A=$(aws ec2 describe-subnets --filters "Name=vpc-id,Values=$VPC_ID" "Name=tag:Name,Values=Public-Subnet-A" --region "$REGION" --profile "$PROFILE" --query "Subnets[0].SubnetId" --output text 2>/dev/null || true)
if [ -z "$PUB_SUB_A" ] || [ "$PUB_SUB_A" = "None" ]; then
  PUB_SUB_A=$(aws ec2 create-subnet --vpc-id "$VPC_ID" --cidr-block "10.0.1.0/24" --availability-zone "${REGION}a" --tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=Public-Subnet-A}]' --region "$REGION" --profile "$PROFILE" --query "Subnet.SubnetId" --output text)
  aws ec2 modify-subnet-attribute --subnet-id "$PUB_SUB_A" --map-public-ip-on-launch --region "$REGION" --profile "$PROFILE"
  aws ec2 associate-route-table --route-table-id "$PUB_RTB_ID" --subnet-id "$PUB_SUB_A" --region "$REGION" --profile "$PROFILE"
fi

PUB_SUB_B=$(aws ec2 describe-subnets --filters "Name=vpc-id,Values=$VPC_ID" "Name=tag:Name,Values=Public-Subnet-B" --region "$REGION" --profile "$PROFILE" --query "Subnets[0].SubnetId" --output text 2>/dev/null || true)
if [ -z "$PUB_SUB_B" ] || [ "$PUB_SUB_B" = "None" ]; then
  PUB_SUB_B=$(aws ec2 create-subnet --vpc-id "$VPC_ID" --cidr-block "10.0.2.0/24" --availability-zone "${REGION}b" --tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=Public-Subnet-B}]' --region "$REGION" --profile "$PROFILE" --query "Subnet.SubnetId" --output text)
  aws ec2 modify-subnet-attribute --subnet-id "$PUB_SUB_B" --map-public-ip-on-launch --region "$REGION" --profile "$PROFILE"
  aws ec2 associate-route-table --route-table-id "$PUB_RTB_ID" --subnet-id "$PUB_SUB_B" --region "$REGION" --profile "$PROFILE"
fi

echo "=== 2. Chained Security Groups ==="
ALB_SG_ID=$(aws ec2 describe-security-groups --filters "Name=group-name,Values=ALB-Public-SG" "Name=vpc-id,Values=$VPC_ID" --region "$REGION" --profile "$PROFILE" --query "SecurityGroups[0].GroupId" --output text 2>/dev/null || true)
if [ -z "$ALB_SG_ID" ] || [ "$ALB_SG_ID" = "None" ]; then
  ALB_SG_ID=$(aws ec2 create-security-group --group-name "ALB-Public-SG" --description "Public ingress for ALB" --vpc-id "$VPC_ID" --tag-specifications 'ResourceType=security-group,Tags=[{Key=Name,Value=ALB-Public-SG}]' --region "$REGION" --profile "$PROFILE" --query "GroupId" --output text)
  aws ec2 authorize-security-group-ingress --group-id "$ALB_SG_ID" --protocol tcp --port 80 --cidr "0.0.0.0/0" --region "$REGION" --profile "$PROFILE"
fi

APP_SG_ID=$(aws ec2 describe-security-groups --filters "Name=group-name,Values=App-Tier-SG" "Name=vpc-id,Values=$VPC_ID" --region "$REGION" --profile "$PROFILE" --query "SecurityGroups[0].GroupId" --output text 2>/dev/null || true)
if [ -z "$APP_SG_ID" ] || [ "$APP_SG_ID" = "None" ]; then
  APP_SG_ID=$(aws ec2 create-security-group --group-name "App-Tier-SG" --description "Inbound restricted to ALB" --vpc-id "$VPC_ID" --tag-specifications 'ResourceType=security-group,Tags=[{Key=Name,Value=App-Tier-SG}]' --region "$REGION" --profile "$PROFILE" --query "GroupId" --output text)
  aws ec2 authorize-security-group-ingress --group-id "$APP_SG_ID" --protocol tcp --port 80 --source-group "$ALB_SG_ID" --region "$REGION" --profile "$PROFILE"
fi

echo "=== 3. IAM Role for Systems Manager (SSM) ==="
aws iam get-role --role-name "SSMFleetRole" --profile "$PROFILE" >/dev/null 2>&1 || \
  aws iam create-role --role-name "SSMFleetRole" --assume-role-policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"ec2.amazonaws.com"},"Action":"sts:AssumeRole"}]}' --region "$REGION" --profile "$PROFILE"
aws iam attach-role-policy --role-name "SSMFleetRole" --policy-arn "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore" --region "$REGION" --profile "$PROFILE"

if ! aws iam get-instance-profile --instance-profile-name "SSMFleetProfile" --profile "$PROFILE" >/dev/null 2>&1; then
  aws iam create-instance-profile --instance-profile-name "SSMFleetProfile" --region "$REGION" --profile "$PROFILE"
  aws iam add-role-to-instance-profile --instance-profile-name "SSMFleetProfile" --role-name "SSMFleetRole" --region "$REGION" --profile "$PROFILE"
  echo "Waiting 10 seconds for IAM instance profile replication across AWS regions..."
  sleep 10
else
  aws iam add-role-to-instance-profile --instance-profile-name "SSMFleetProfile" --role-name "SSMFleetRole" --region "$REGION" --profile "$PROFILE" >/dev/null 2>&1 || true
fi

echo "=== 4. Target Group & Application Load Balancer ==="
TG_ARN=$(aws elbv2 describe-target-groups --names "Production-ASG-TG" --region "$REGION" --profile "$PROFILE" --query "TargetGroups[0].TargetGroupArn" --output text 2>/dev/null || true)
if [ -z "$TG_ARN" ] || [ "$TG_ARN" = "None" ]; then
  TG_ARN=$(aws elbv2 create-target-group \
    --name "Production-ASG-TG" \
    --protocol HTTP \
    --port 80 \
    --vpc-id "$VPC_ID" \
    --target-type instance \
    --health-check-protocol HTTP \
    --health-check-path "/" \
    --health-check-interval-seconds 15 \
    --health-check-timeout-seconds 5 \
    --healthy-threshold-count 2 \
    --unhealthy-threshold-count 2 \
    --matcher HttpCode=200 \
    --region "$REGION" \
    --profile "$PROFILE" \
    --query "TargetGroups[0].TargetGroupArn" \
    --output text)
  
  # Configure deregistration delay (connection draining) to 30 seconds for quick lab turnover
  aws elbv2 modify-target-group-attributes \
    --target-group-arn "$TG_ARN" \
    --attributes Key=deregistration_delay.timeout_seconds,Value=30 \
    --region "$REGION" \
    --profile "$PROFILE"
fi

ALB_ARN=$(aws elbv2 describe-load-balancers --names "Production-ALB" --region "$REGION" --profile "$PROFILE" --query "LoadBalancers[0].LoadBalancerArn" --output text 2>/dev/null || true)
if [ -z "$ALB_ARN" ] || [ "$ALB_ARN" = "None" ]; then
  ALB_ARN=$(aws elbv2 create-load-balancer \
    --name "Production-ALB" \
    --subnets "$PUB_SUB_A" "$PUB_SUB_B" \
    --security-groups "$ALB_SG_ID" \
    --scheme internet-facing \
    --type application \
    --ip-address-type ipv4 \
    --region "$REGION" \
    --profile "$PROFILE" \
    --query "LoadBalancers[0].LoadBalancerArn" \
    --output text)
  
  echo "Waiting for ALB to become active..."
  aws elbv2 wait load-balancer-available --load-balancer-arns "$ALB_ARN" --region "$REGION" --profile "$PROFILE"
  
  # Create HTTP Listener forwarding traffic directly to the ASG Target Group
  aws elbv2 create-listener \
    --load-balancer-arn "$ALB_ARN" \
    --protocol HTTP \
    --port 80 \
    --default-actions Type=forward,TargetGroupArn="$TG_ARN" \
    --region "$REGION" \
    --profile "$PROFILE"
fi

echo "=== Base Infrastructure Ready ==="
echo "VPC_ID=$VPC_ID"
echo "PUB_SUB_A=$PUB_SUB_A"
echo "PUB_SUB_B=$PUB_SUB_B"
echo "APP_SG_ID=$APP_SG_ID"
echo "TG_ARN=$TG_ARN"
echo "ALB_ARN=$ALB_ARN"
EOF

chmod +x setup-asg-foundation.sh
./setup-asg-foundation.sh

📜 Phase 2: Immutable EC2 Launch Templates

In legacy AWS architectures, Auto Scaling used Launch Configurations. AWS has officially deprecated Launch Configurations in favor of Launch Templates.

Launch Templates provide several foundational advantages:

  1. Version Control: Launch templates are immutable and versioned ($Default, $Latest, v1, v2). You can test a new configuration without breaking the existing fleet.
  2. Advanced Hardware Options: Support for Graviton (ARM64) processors, Spot Instances, T-family burst credits, and Mixed Instance Policies.
  3. IMDSv2 Enforcement: Mandating session tokens for instance metadata retrieval right at boot time.
                  +----------------------------------------------+
                  |         EC2 Launch Template (v1)             |
                  +----------------------------------------------+
                  |  AMI: Amazon Linux 2023 (ARM64 Graviton)     |
                  |  Instance Type: t4g.micro                    |
                  |  IAM Profile: SSMFleetProfile                |
                  |  Security Group: App-Tier-SG                 |
                  |  IMDSv2: HttpTokens=required (HopLimit=2)    |
                  |  UserData: Dynamic Nginx Web Server Bootstrap|
                  +----------------------------------------------+

1. The Multi-AZ Subnet Anti-Pattern

[!CAUTION] Never hardcode a Subnet ID inside an EC2 Launch Template used by an Auto Scaling Group! If you specify a single SubnetId inside the Launch Template’s network interface, the template becomes anchored to that single Availability Zone. When the Auto Scaling Group attempts to distribute compute nodes across ap-southeast-1a and ap-southeast-1b, EC2 throws an InvalidParameterCombination error.

The Golden Rule: Configure Security Groups and Public IP mappings in the Launch Template, but leave Subnet assignment strictly to the Auto Scaling Group’s VPCZoneIdentifier!

2. Creating the Bootstrap User-Data Script

We will craft a bash script that installs Nginx, queries the instance’s private IP and availability zone via IMDSv2, and renders a dynamic HTML dashboard:

cat <<'EOF' > userdata.sh
#!/bin/bash
dnf update -y
dnf install -y nginx

# Query IMDSv2 session token (HopLimit=2)
TOKEN=$(curl -s -S -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
INSTANCE_ID=$(curl -s -S -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id)
AZ=$(curl -s -S -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/placement/availability-zone)
PRIVATE_IP=$(curl -s -S -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/local-ipv4)

mkdir -p /usr/share/nginx/html
cat <<HTML > /usr/share/nginx/html/index.html
<!DOCTYPE html>
<html>
<head>
  <title>AWS CLI Auto Scaling Fleet</title>
  <style>
    body { font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; background: #0f172a; color: #f8fafc; display: flex; justify-content: center; align-items: center; height: 100vh; margin: 0; }
    .card { background: #1e293b; padding: 2rem; border-radius: 12px; border: 1px solid #334155; box-shadow: 0 10px 25px rgba(0,0,0,0.5); width: 450px; text-align: center; }
    h1 { color: #38bdf8; font-size: 1.5rem; margin-bottom: 0.5rem; }
    .badge { display: inline-block; padding: 0.25rem 0.75rem; border-radius: 9999px; background: #0284c7; color: white; font-size: 0.85rem; font-weight: 600; margin-bottom: 1rem; }
    .stat { display: flex; justify-content: space-between; padding: 0.5rem 0; border-bottom: 1px solid #334155; font-size: 0.9rem; }
    .stat-label { color: #94a3b8; }
    .stat-val { font-family: monospace; color: #f1f5f9; font-weight: bold; }
  </style>
</head>
<body>
  <div class="card">
    <h1>🚀 Auto-Scaled Node v1.0</h1>
    <div class="badge">Healthy & In-Service</div>
    <div class="stat"><span class="stat-label">Instance ID:</span><span class="stat-val">$INSTANCE_ID</span></div>
    <div class="stat"><span class="stat-label">Availability Zone:</span><span class="stat-val">$AZ</span></div>
    <div class="stat"><span class="stat-label">Private IP:</span><span class="stat-val">$PRIVATE_IP</span></div>
  </div>
</body>
</html>
HTML

systemctl enable --now nginx
EOF

3. Provisioning the Launch Template via CLI

We extract the latest official Amazon Linux 2023 ARM64 AMI from SSM Parameter Store, base64-encode our user-data script, and construct our launch template payload:

REGION="ap-southeast-1"
PROFILE="cloud-dev-admin"

# 1. Fetch latest Amazon Linux 2023 ARM64 AMI ID
AMI_ID=$(aws ssm get-parameter \
  --name "/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-arm64" \
  --region "$REGION" \
  --profile "$PROFILE" \
  --query "Parameter.Value" \
  --output text)

# 2. Base64 encode User Data (compatible across macOS and Linux)
USER_DATA_B64=$(base64 < userdata.sh | tr -d '\n')

# 3. Create Launch Template specification JSON
cat <<JSON > lt-config.json
{
  "ImageId": "$AMI_ID",
  "InstanceType": "t4g.micro",
  "IamInstanceProfile": {
    "Name": "SSMFleetProfile"
  },
  "NetworkInterfaces": [
    {
      "DeviceIndex": 0,
      "AssociatePublicIpAddress": true,
      "Groups": ["$APP_SG_ID"]
    }
  ],
  "UserData": "$USER_DATA_B64",
  "MetadataOptions": {
    "HttpTokens": "required",
    "HttpPutResponseHopLimit": 2,
    "HttpEndpoint": "enabled"
  },
  "TagSpecifications": [
    {
      "ResourceType": "instance",
      "Tags": [
        {"Key": "Name", "Value": "Production-ASG-Worker"},
        {"Key": "Environment", "Value": "Production"}
      ]
    }
  ]
}
JSON

# 4. Create the Launch Template
LT_ID=$(aws ec2 create-launch-template \
  --launch-template-name "Production-App-LT" \
  --version-description "v1.0-initial-nginx-arm64" \
  --launch-template-data file://lt-config.json \
  --region "$REGION" \
  --profile "$PROFILE" \
  --query "LaunchTemplate.LaunchTemplateId" \
  --output text)

echo "Launch Template Created: $LT_ID"
rm -f userdata.sh lt-config.json

Verify that the template has been registered and review its default version:

aws ec2 describe-launch-template-versions \
  --launch-template-id "$LT_ID" \
  --versions "$Default" \
  --region "$REGION" \
  --profile "$PROFILE" \
  --query "LaunchTemplateVersions[0].[LaunchTemplateName,VersionNumber,LaunchTemplateData.InstanceType,LaunchTemplateData.ImageId]" \
  --output table

⚖️ Phase 3: Multi-AZ Auto Scaling Group Creation

Now that our Launch Template defines how instances should be launched, the Auto Scaling Group defines where, how many, and when they should be provisioned.

                  +----------------------------------------------+
                  |         Auto Scaling Group Boundaries        |
                  +----------------------------------------------+
                  |  Min Size: 2                                 |
                  |  Desired Capacity: 2                         |
                  |  Max Size: 5                                 |
                  |  Subnets: Public-Subnet-A + Public-Subnet-B  |
                  |  Target Group: Production-ASG-TG             |
                  |  Health Check Type: ELB (Grace: 300s)        |
                  +----------------------------------------------+

1. Deploying the Auto Scaling Group

We execute aws autoscaling create-auto-scaling-group, binding our Launch Template directly to our Multi-AZ subnets and the ALB Target Group:

REGION="ap-southeast-1"
PROFILE="cloud-dev-admin"

aws autoscaling create-auto-scaling-group \
  --auto-scaling-group-name "Production-App-ASG" \
  --launch-template "LaunchTemplateName=Production-App-LT,Version=\$Default" \
  --min-size 2 \
  --max-size 5 \
  --desired-capacity 2 \
  --vpc-zone-identifier "$PUB_SUB_A,$PUB_SUB_B" \
  --target-group-arns "$TG_ARN" \
  --health-check-type "ELB" \
  --health-check-grace-period 300 \
  --default-cooldown 120 \
  --tags "ResourceType=auto-scaling-group,Key=Name,Value=Production-App-ASG,PropagateAtLaunch=false" \
  --region "$REGION" \
  --profile "$PROFILE"

Parameter Breakdown

  • --vpc-zone-identifier "$PUB_SUB_A,$PUB_SUB_B": Directs the ASG to balance compute nodes evenly across both Availability Zones (ap-southeast-1a and ap-southeast-1b).
  • --target-group-arns "$TG_ARN": Tells the ASG to automatically register newly booted EC2 instances with our Application Load Balancer Target Group upon launch, and automatically deregister them when terminated.
  • --health-check-type "ELB": Overrides the default EC2 hypervisor check. The ASG evaluates the application’s actual HTTP / response from the ALB Target Group.
  • --health-check-grace-period 300: Grants a 300-second (5-minute) grace window upon boot. The ASG will not terminate newly booted nodes while Nginx and cloud-init dependencies are still installing.

2. Monitoring Initial Fleet Convergence

Watch the Auto Scaling Group instantiate the desired capacity and query the Target Group’s health status:

echo "Monitoring ASG instance launches..."
aws autoscaling describe-auto-scaling-groups \
  --auto-scaling-group-names "Production-App-ASG" \
  --region "$REGION" \
  --profile "$PROFILE" \
  --query "AutoScalingGroups[0].Instances[*].[InstanceId,AvailabilityZone,LifecycleState,HealthStatus]" \
  --output table

Wait ~60 seconds for Nginx to finish booting, then inspect the Target Group registration:

aws elbv2 describe-target-health \
  --target-group-arn "$TG_ARN" \
  --region "$REGION" \
  --profile "$PROFILE" \
  --query "TargetHealthDescriptions[*].[Target.Id,Target.Port,TargetHealth.State,TargetHealth.Reason]" \
  --output table

Expected output once health checks pass:

---------------------------------------------
|           DescribeTargetHealth            |
+----------------------+----+----------+----+
|  i-01a2b3c4d5e6f7011 | 80 | healthy  | None|
|  i-09z8y7x6w5v4u3022 | 80 | healthy  | None|
+----------------------+----+----------+----+

3. Testing Traffic Distribution via ALB DNS

Fetch the public DNS name of your Load Balancer and send 6 consecutive HTTP curl requests:

ALB_DNS=$(aws elbv2 describe-load-balancers \
  --load-balancer-arns "$ALB_ARN" \
  --region "$REGION" \
  --profile "$PROFILE" \
  --query "LoadBalancers[0].DNSName" \
  --output text)

echo "Testing ALB DNS: http://$ALB_DNS"
for i in {1..6}; do
  curl -s "http://$ALB_DNS" | grep -o 'Private IP:</span><span class="stat-val">[^<]*' | sed 's/.*>//'
  sleep 1
done

You will observe alternating IP addresses, proving that incoming requests are dynamically balanced across compute nodes in separate Availability Zones!

🩺 Phase 4: Chaos Testing & Self-Healing Fleets

The hallmark of resilient cloud architecture is self-healing compute fleets. In this phase, we conduct a chaos engineering experiment: we deliberately kill the web application process on one of our running nodes and observe how AWS automatically evicts and replaces the faulty server.

1. Node A Nginx process killed (HTTP 500 / Connection Refused)
   |
   v
2. ALB Target Group marks Node A "unhealthy" (< 30 seconds)
   |
   v
3. ASG detects "ELB" health failure -> Initiates termination of Node A
   |
   v
4. ASG launches brand-new Node C -> Registers to Target Group -> Status: "healthy"

1. Inspecting the “EC2 vs. ELB” Health Check Trap

[!WARNING] The Silent Failure Disaster: If an ASG is configured with --health-check-type "EC2" (the default), the ASG only queries the EC2 hypervisor status check (StatusCheckFailed_System / StatusCheckFailed_Instance). If your web app crashes, runs out of memory, or encounters a software deadlock, the VM hypervisor remains 100% operational! The ALB will stop sending traffic to the broken node, but the ASG will never replace it. Your cluster capacity permanently degrades. Setting --health-check-type "ELB" links the instance lifecycle directly to application responsiveness.

2. Simulating Application Failure via AWS Systems Manager

Identify one of the active instance IDs and terminate its Nginx process using SSM Run Command (Zero-Trust execution):

REGION="ap-southeast-1"
PROFILE="cloud-dev-admin"

# Grab the first healthy instance ID
VICTIM_NODE=$(aws autoscaling describe-auto-scaling-groups \
  --auto-scaling-group-names "Production-App-ASG" \
  --region "$REGION" \
  --profile "$PROFILE" \
  --query "AutoScalingGroups[0].Instances[0].InstanceId" \
  --output text)

echo "Targeting instance for chaos test: $VICTIM_NODE"

# Stop the Nginx daemon remotely via SSM Run Command
aws ssm send-command \
  --instance-ids "$VICTIM_NODE" \
  --document-name "AWS-RunShellScript" \
  --parameters 'commands=["systemctl stop nginx"]' \
  --comment "Chaos Test: Stopping web server to trigger ELB health check eviction" \
  --region "$REGION" \
  --profile "$PROFILE" \
  --output text

3. Observing the Automated Self-Healing Reaction

Immediately monitor Target Group health:

watch -n 5 "aws elbv2 describe-target-health --target-group-arn $TG_ARN --region $REGION --profile $PROFILE --query 'TargetHealthDescriptions[*].[Target.Id,TargetHealth.State,TargetHealth.Reason]' --output table"

Within 30 seconds (2 failed health checks at 15s intervals):

  1. Target Group marks the victim node: unhealthy (Target.FailedHealthChecks).
  2. The ALB immediately ceases routing traffic to the victim node; 100% of client traffic seamlessly shifts to the remaining healthy instance.
  3. The Auto Scaling Group marks the instance’s HealthStatus as Unhealthy.
  4. The ASG launches a replacement instance to maintain the desired capacity of 2, and terminates the degraded node.

Check the ASG scaling activities log to see the audit trail:

aws autoscaling describe-scaling-activities \
  --auto-scaling-group-name "Production-App-ASG" \
  --max-items 3 \
  --region "$REGION" \
  --profile "$PROFILE" \
  --query "Activities[*].[StartTime,StatusCode,Description]" \
  --output table

You will see an entry matching: "Terminating EC2 instance: i-xxxxxx - an instance was taken out of service in response to an ELB health check failure."

📈 Phase 5: Dynamic Target Tracking Scaling Policies

Static fleets fail when incoming user traffic fluctuates. In traditional operations, engineers had to write complex CloudWatch alarms with high/low threshold step scaling rules.

Modern AWS architectures use Target Tracking Scaling Policies. You define a desired target metric (e.g. keep average CPU at 50%), and AWS automatically creates and tunes the underlying CloudWatch alarms, dynamically calculating how many instances to add or remove:

                      [ Incoming Web Traffic Spike ]
                                    |
                                    v
                 +--------------------------------------+
                 | Average Fleet CPU reaches 75%        |
                 +------------------+-------------------+
                                    |
                                    v
                 +--------------------------------------+
                 | Target Tracking Alarm Triggers       |
                 | Math: (Current: 75% / Target: 50%)   |
                 | Needs: 2 instances * 1.5 = 3 nodes   |
                 +------------------+-------------------+
                                    |
                                    v
                 +--------------------------------------+
                 | ASG Scales Out: Desired Capacity -> 3|
                 +------------------+-------------------+

1. Attaching a CPU Target Tracking Policy

We will instruct the ASG to maintain an average CPU utilization of 50% across the fleet:

REGION="ap-southeast-1"
PROFILE="cloud-dev-admin"

POLICY_ARN=$(aws autoscaling put-scaling-policy \
  --auto-scaling-group-name "Production-App-ASG" \
  --policy-name "Target-Tracking-CPU-50" \
  --policy-type "TargetTrackingScaling" \
  --target-tracking-configuration '{
    "PredefinedMetricSpecification": {
      "PredefinedMetricType": "ASGAverageCPUUtilization"
    },
    "TargetValue": 50.0,
    "DisableScaleIn": false
  }' \
  --region "$REGION" \
  --profile "$PROFILE" \
  --query "PolicyARN" \
  --output text)

echo "Target Tracking Policy configured: $POLICY_ARN"

AWS automatically provisions two CloudWatch alarms in the background:

  • An AlarmHigh rule that triggers Scale-Out when average CPU exceeds 50%.
  • An AlarmLow rule that triggers Scale-In when average CPU drops below 50% for sustained periods.

List the automatically provisioned alarms:

aws autoscaling describe-policies \
  --auto-scaling-group-name "Production-App-ASG" \
  --policy-names "Target-Tracking-CPU-50" \
  --region "$REGION" \
  --profile "$PROFILE" \
  --query "ScalingPolicies[0].Alarms[*].[AlarmName]" \
  --output table

2. Simulating Compute Stress via SSM Run Command

To test scale-out without manual SSH connections, we trigger an artificial CPU stress script across all active instances using SSM Run Command:

REGION="ap-southeast-1"
PROFILE="cloud-dev-admin"

ACTIVE_INSTANCES=$(aws autoscaling describe-auto-scaling-groups \
  --auto-scaling-group-names "Production-App-ASG" \
  --region "$REGION" \
  --profile "$PROFILE" \
  --query "AutoScalingGroups[0].Instances[?LifecycleState=='InService'].InstanceId" \
  --output text)

echo "Injecting CPU load into instances: $ACTIVE_INSTANCES"

# Run a CPU burn script for 240 seconds
aws ssm send-command \
  --instance-ids $ACTIVE_INSTANCES \
  --document-name "AWS-RunShellScript" \
  --parameters 'commands=["nohup sh -c \"cat /dev/urandom | gzip -9 > /dev/null\" > /dev/null 2>&1 & sleep 240; killall gzip || true"]' \
  --comment "Simulating CPU load spike for Target Tracking scaling test" \
  --region "$REGION" \
  --profile "$PROFILE" \
  --output text

Monitor the Auto Scaling Group capacity:

aws autoscaling describe-auto-scaling-groups \
  --auto-scaling-group-names "Production-App-ASG" \
  --region "$REGION" \
  --profile "$PROFILE" \
  --query "AutoScalingGroups[0].[DesiredCapacity,Instances[*].[InstanceId,LifecycleState]]" \
  --output json

As the average CPU metrics stream into CloudWatch, the policy will dynamically scale the group from 2 to 3 or 4 instances. Once the stress command terminates, CloudWatch detects the drop, waits for the cooldown period, and safely scales the cluster back down to 2.

🔄 Phase 6: Zero-Downtime Rolling Instance Refresh

How do you deploy a new application version or update base operating system AMIs across 50+ auto-scaled nodes without taking a single second of service downtime?

Legacy methods involved terminating instances manually or managing duplicate blue/green ASGs. Modern AWS CLI operations leverage EC2 Instance Refresh.

[ Current Fleet: v1.0 ] ---> [ Launch Template Updated to v2.0 ]
                                          |
                                          v
                         [ aws autoscaling start-instance-refresh ]
                                          |
+-----------------------------------------+-----------------------------------------+
| Step 1: Launch 1 new node (v2.0)        | Step 2: Terminate 1 old node (v1.0)     |
| Wait until Target Group is HEALTHY      | (Maintains MinHealthyPercentage: 50%)   |
+-----------------------------------------+-----------------------------------------+
                                          |
                                          v
                         [ Entire Fleet Converted to v2.0 with 0% Downtime ]

1. Creating Launch Template Version 2

Let’s publish Version 2 of our Launch Template with an updated application banner (v2.0-canary):

REGION="ap-southeast-1"
PROFILE="cloud-dev-admin"

# Create new UserData script for v2.0
cat <<'EOF' > userdata-v2.sh
#!/bin/bash
dnf update -y
dnf install -y nginx

TOKEN=$(curl -s -S -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
INSTANCE_ID=$(curl -s -S -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id)
AZ=$(curl -s -S -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/placement/availability-zone)
PRIVATE_IP=$(curl -s -S -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/local-ipv4)

mkdir -p /usr/share/nginx/html
cat <<HTML > /usr/share/nginx/html/index.html
<!DOCTYPE html>
<html>
<head>
  <title>AWS CLI Auto Scaling Fleet v2.0</title>
  <style>
    body { font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; background: #064e3b; color: #f8fafc; display: flex; justify-content: center; align-items: center; height: 100vh; margin: 0; }
    .card { background: #065f46; padding: 2rem; border-radius: 12px; border: 1px solid #047857; box-shadow: 0 10px 25px rgba(0,0,0,0.5); width: 450px; text-align: center; }
    h1 { color: #34d399; font-size: 1.5rem; margin-bottom: 0.5rem; }
    .badge { display: inline-block; padding: 0.25rem 0.75rem; border-radius: 9999px; background: #10b981; color: white; font-size: 0.85rem; font-weight: 600; margin-bottom: 1rem; }
    .stat { display: flex; justify-content: space-between; padding: 0.5rem 0; border-bottom: 1px solid #047857; font-size: 0.9rem; }
    .stat-label { color: #a7f3d0; }
    .stat-val { font-family: monospace; color: #ffffff; font-weight: bold; }
  </style>
</head>
<body>
  <div class="card">
    <h1>🎉 Canary Release v2.0 (Emerald)</h1>
    <div class="badge">Zero-Downtime Refresh</div>
    <div class="stat"><span class="stat-label">Instance ID:</span><span class="stat-val">$INSTANCE_ID</span></div>
    <div class="stat"><span class="stat-label">Availability Zone:</span><span class="stat-val">$AZ</span></div>
    <div class="stat"><span class="stat-label">Private IP:</span><span class="stat-val">$PRIVATE_IP</span></div>
  </div>
</body>
</html>
HTML

systemctl enable --now nginx
EOF

USER_DATA_V2_B64=$(base64 < userdata-v2.sh | tr -d '\n')

# Create Version 2 of the Launch Template
aws ec2 create-launch-template-version \
  --launch-template-name "Production-App-LT" \
  --version-description "v2.0-canary-release" \
  --source-version "1" \
  --launch-template-data "{\"UserData\":\"$USER_DATA_V2_B64\"}" \
  --region "$REGION" \
  --profile "$PROFILE"

rm -f userdata-v2.sh

# Update ASG to use the latest version automatically
aws autoscaling update-auto-scaling-group \
  --auto-scaling-group-name "Production-App-ASG" \
  --launch-template "LaunchTemplateName=Production-App-LT,Version=\$Latest" \
  --region "$REGION" \
  --profile "$PROFILE"

2. Triggering the Automated Instance Refresh

We trigger a rolling refresh. We specify a MinHealthyPercentage of 50, ensuring that at least half the fleet remains completely healthy and handling client requests at all times during the rollover:

REFRESH_ID=$(aws autoscaling start-instance-refresh \
  --auto-scaling-group-name "Production-App-ASG" \
  --preferences '{
    "MinHealthyPercentage": 50,
    "InstanceWarmup": 60
  }' \
  --region "$REGION" \
  --profile "$PROFILE" \
  --query "InstanceRefreshId" \
  --output text)

echo "Instance Refresh initiated: $REFRESH_ID"

3. Monitoring Rolling Canary Deployment

Monitor the refresh status in real time:

aws autoscaling describe-instance-refreshes \
  --auto-scaling-group-name "Production-App-ASG" \
  --instance-refresh-ids "$REFRESH_ID" \
  --region "$REGION" \
  --profile "$PROFILE" \
  --query "InstanceRefreshes[0].[Status,PercentageComplete,InstancesToUpdate]" \
  --output table

While the refresh progresses, test the ALB endpoint repeatedly:

for i in {1..8}; do
  curl -s "http://$ALB_DNS" | grep -E -o 'Auto-Scaled Node v1.0|Canary Release v2.0' || true
  sleep 2
done

You will witness the canary rollover in real time: initial requests return v1.0, followed by a split between v1.0 and v2.0, until 100% of responses transition to Canary Release v2.0 with zero HTTP 502/504 errors!

🪝 Phase 7: Lifecycle Hooks & Connection Draining

When an Auto Scaling Group terminates an instance during scale-in or an instance refresh, the default behavior is abrupt: the VM is immediately terminated, dropping any in-flight HTTP connections.

To prevent dropped requests, enterprise systems use two complementary mechanisms:

  1. ALB Connection Draining (Deregistration Delay): When an instance is removed, the ALB stops sending new requests and allows in-flight connections to complete gracefully (configured in Phase 1 to 30s).
  2. ASG Lifecycle Hooks: The ASG intercepts the termination event, placing the instance in a Terminating:Wait state while cleanup tasks (flushing logs, uploading metrics) complete.
REGION="ap-southeast-1"
PROFILE="cloud-dev-admin"

# Add a termination lifecycle hook with a 120-second timeout
aws autoscaling put-lifecycle-hook \
  --lifecycle-hook-name "GracefulShutdownHook" \
  --auto-scaling-group-name "Production-App-ASG" \
  --lifecycle-transition "autoscaling:EC2_INSTANCE_TERMINATING" \
  --heartbeat-timeout 120 \
  --default-result "CONTINUE" \
  --region "$REGION" \
  --profile "$PROFILE"

echo "Lifecycle Hook successfully attached to Production-App-ASG"

When an instance enters termination, your backend scripts can complete their teardown and signal the ASG to finalize termination immediately via CLI:

# Example signal command:
# aws autoscaling complete-lifecycle-action \
#   --lifecycle-hook-name "GracefulShutdownHook" \
#   --auto-scaling-group-name "Production-App-ASG" \
#   --lifecycle-action-result "CONTINUE" \
#   --instance-id "$INSTANCE_ID"

💡 Phase 8: FinOps: Scheduled Scaling Strategies

If your application serves business customers (B2B SaaS or internal enterprise tooling), traffic typically plummets outside of business hours (e.g. nights and weekends).

Running 4 or 6 compute nodes overnight is a waste of capital. Using Scheduled Actions, you can automate scale-down to 1 instance at 7:00 PM and scale back up to 2 instances at 7:00 AM every business day:

REGION="ap-southeast-1"
PROFILE="cloud-dev-admin"

# Scale Down to 1 instance every evening at 19:00 UTC
aws autoscaling put-scheduled-update-group-action \
  --auto-scaling-group-name "Production-App-ASG" \
  --scheduled-action-name "NightlyScaleDown" \
  --recurrence "0 19 * * MON-FRI" \
  --min-size 1 \
  --max-size 2 \
  --desired-capacity 1 \
  --region "$REGION" \
  --profile "$PROFILE"

# Scale Up to 2 instances every morning at 07:00 UTC
aws autoscaling put-scheduled-update-group-action \
  --auto-scaling-group-name "Production-App-ASG" \
  --scheduled-action-name "MorningScaleUp" \
  --recurrence "0 7 * * MON-FRI" \
  --min-size 2 \
  --max-size 5 \
  --desired-capacity 2 \
  --region "$REGION" \
  --profile "$PROFILE"

echo "Scheduled Scaling Rules registered."

[!TIP] FinOps Impact: Running 1 node instead of 2 during off-peak hours (12 hours/day + weekends) reduces your compute invoice by ~45% without impacting production performance during business hours!

🧹 Phase 9: 1-Click Bulletproof Lab Teardown

Auto Scaling Groups spawn dynamic compute and networking dependencies. Attempting to delete the VPC or Security Groups while the ASG is still terminating instances will trigger AWS DependencyViolation errors.

This hardened, idempotent teardown script scales the ASG to 0, waits for all compute instances to terminate completely, and systematically removes all load balancing, networking, and IAM dependencies:

cat <<'EOF' > destroy-asg-lab.sh
#!/usr/bin/env bash
set -euo pipefail

REGION="ap-southeast-1"
PROFILE="cloud-dev-admin"

echo "=== 1. Deleting Auto Scaling Group ==="
if aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names "Production-App-ASG" --region "$REGION" --profile "$PROFILE" --query "AutoScalingGroups[0].AutoScalingGroupName" --output text 2>/dev/null | grep -q "Production-App-ASG"; then
  echo "Scaling ASG down to 0 instances..."
  aws autoscaling update-auto-scaling-group \
    --auto-scaling-group-name "Production-App-ASG" \
    --min-size 0 \
    --desired-capacity 0 \
    --region "$REGION" \
    --profile "$PROFILE"

  echo "Force-deleting Auto Scaling Group..."
  aws autoscaling delete-auto-scaling-group \
    --auto-scaling-group-name "Production-App-ASG" \
    --force-delete \
    --region "$REGION" \
    --profile "$PROFILE"

  echo "Waiting for all ASG EC2 instances to fully terminate..."
  while true; do
    REMAINING=$(aws ec2 describe-instances \
      --filters "Name=tag:Name,Values=Production-ASG-Worker" "Name=instance-state-name,Values=pending,running,shutting-down,stopping" \
      --region "$REGION" \
      --profile "$PROFILE" \
      --query "Reservations[*].Instances[*].InstanceId" \
      --output text)
    if [ -z "$REMAINING" ]; then
      echo "All compute nodes terminated."
      break
    fi
    echo "Instances still terminating: $REMAINING (sleeping 10s)..."
    sleep 10
  done
fi

echo "=== 2. Deleting Launch Templates ==="
aws ec2 delete-launch-template --launch-template-name "Production-App-LT" --region "$REGION" --profile "$PROFILE" 2>/dev/null || true

echo "=== 3. Deleting ALB & Target Groups ==="
ALB_ARN=$(aws elbv2 describe-load-balancers --names "Production-ALB" --region "$REGION" --profile "$PROFILE" --query "LoadBalancers[0].LoadBalancerArn" --output text 2>/dev/null || true)
if [ -n "$ALB_ARN" ] && [ "$ALB_ARN" != "None" ]; then
  echo "Deleting ALB Listeners..."
  for lis in $(aws elbv2 describe-listeners --load-balancer-arn "$ALB_ARN" --region "$REGION" --profile "$PROFILE" --query "Listeners[*].ListenerArn" --output text 2>/dev/null || true); do
    aws elbv2 delete-listener --listener-arn "$lis" --region "$REGION" --profile "$PROFILE" 2>/dev/null || true
  done
  echo "Deleting ALB: $ALB_ARN..."
  aws elbv2 delete-load-balancer --load-balancer-arn "$ALB_ARN" --region "$REGION" --profile "$PROFILE"
  echo "Waiting for ALB to finish deleting..."
  aws elbv2 wait load-balancers-deleted --load-balancer-arns "$ALB_ARN" --region "$REGION" --profile "$PROFILE" 2>/dev/null || true
fi

for tg in "Production-ASG-TG" "Production-App-TG"; do
  TG_ARN=$(aws elbv2 describe-target-groups --names "$tg" --region "$REGION" --profile "$PROFILE" --query "TargetGroups[0].TargetGroupArn" --output text 2>/dev/null || true)
  if [ -n "$TG_ARN" ] && [ "$TG_ARN" != "None" ]; then
    echo "Deleting Target Group: $tg..."
    aws elbv2 delete-target-group --target-group-arn "$TG_ARN" --region "$REGION" --profile "$PROFILE" 2>/dev/null || true
  fi
done

echo "=== 4. Deleting Security Groups & Network ==="
VPC_ID=$(aws ec2 describe-vpcs --filters "Name=tag:Name,Values=Production-VPC" --region "$REGION" --profile "$PROFILE" --query "Vpcs[0].VpcId" --output text 2>/dev/null || true)

if [ -n "$VPC_ID" ] && [ "$VPC_ID" != "None" ]; then
  # Delete Security Groups
  for sg_name in "App-Tier-SG" "ALB-Public-SG"; do
    SG_ID=$(aws ec2 describe-security-groups --filters "Name=vpc-id,Values=$VPC_ID" "Name=group-name,Values=$sg_name" --region "$REGION" --profile "$PROFILE" --query "SecurityGroups[0].GroupId" --output text 2>/dev/null || true)
    if [ -n "$SG_ID" ] && [ "$SG_ID" != "None" ]; then
      echo "Deleting Security Group: $sg_name ($SG_ID)..."
      aws ec2 delete-security-group --group-id "$SG_ID" --region "$REGION" --profile "$PROFILE" 2>/dev/null || true
    fi
  done

  # Delete Subnets
  for sub in $(aws ec2 describe-subnets --filters "Name=vpc-id,Values=$VPC_ID" --region "$REGION" --profile "$PROFILE" --query "Subnets[*].SubnetId" --output text 2>/dev/null || true); do
    echo "Deleting Subnet: $sub..."
    aws ec2 delete-subnet --subnet-id "$sub" --region "$REGION" --profile "$PROFILE" 2>/dev/null || true
  done

  # Delete Route Tables
  for rtb in $(aws ec2 describe-route-tables --filters "Name=vpc-id,Values=$VPC_ID" "Name=tag:Name,Values=Production-Public-RT" --region "$REGION" --profile "$PROFILE" --query "RouteTables[*].RouteTableId" --output text 2>/dev/null || true); do
    echo "Deleting Route Table: $rtb..."
    aws ec2 delete-route-table --route-table-id "$rtb" --region "$REGION" --profile "$PROFILE" 2>/dev/null || true
  done

  # Detach & Delete Internet Gateway
  for igw in $(aws ec2 describe-internet-gateways --filters "Name=attachment.vpc-id,Values=$VPC_ID" --region "$REGION" --profile "$PROFILE" --query "InternetGateways[*].InternetGatewayId" --output text 2>/dev/null || true); do
    echo "Detaching IGW: $igw..."
    aws ec2 detach-internet-gateway --internet-gateway-id "$igw" --vpc-id "$VPC_ID" --region "$REGION" --profile "$PROFILE" 2>/dev/null || true
    echo "Deleting IGW: $igw..."
    aws ec2 delete-internet-gateway --internet-gateway-id "$igw" --region "$REGION" --profile "$PROFILE" 2>/dev/null || true
  done

  echo "Deleting VPC: $VPC_ID..."
  aws ec2 delete-vpc --vpc-id "$VPC_ID" --region "$REGION" --profile "$PROFILE" 2>/dev/null || true
fi

echo "=== 5. Cleaning up IAM Roles ==="
aws iam remove-role-from-instance-profile --instance-profile-name "SSMFleetProfile" --role-name "SSMFleetRole" --region "$REGION" --profile "$PROFILE" 2>/dev/null || true
aws iam delete-instance-profile --instance-profile-name "SSMFleetProfile" --region "$REGION" --profile "$PROFILE" 2>/dev/null || true

for pol in $(aws iam list-attached-role-policies --role-name "SSMFleetRole" --profile "$PROFILE" --query "AttachedPolicies[].PolicyArn" --output text 2>/dev/null || true); do
  aws iam detach-role-policy --role-name "SSMFleetRole" --policy-arn "$pol" --profile "$PROFILE" 2>/dev/null || true
done
aws iam delete-role --role-name "SSMFleetRole" --profile "$PROFILE" 2>/dev/null || true

rm -f setup-asg-foundation.sh destroy-asg-lab.sh userdata.sh userdata-v2.sh lt-config.json
echo "=== Part 11 Teardown Complete! Zero Resources Remaining. ==="
EOF

chmod +x destroy-asg-lab.sh
./destroy-asg-lab.sh

🏆 Key Takeaways & Enterprise Architecture Patterns

  1. Launch Templates over Configurations: Launch Configurations are legacy tech. Always use immutable, versioned Launch Templates with Graviton ARM64 architecture and enforced IMDSv2 metadata security.
  2. Never Bind Subnets in Templates: Keep network subnets inside the Auto Scaling Group’s VPCZoneIdentifier to ensure smooth Multi-AZ balancing without validation errors.
  3. Mandate ELB Health Checks: An ASG relying only on EC2 hypervisor health checks cannot detect deadlocked or crashed web applications. Always configure --health-check-type "ELB".
  4. Target Tracking Simplifies Operations: Instead of manual High/Low threshold alarm pairs, use Target Tracking policies (ASGAverageCPUUtilization or ALBRequestCountPerTarget) to let AWS handle dynamic capacity scaling math.
  5. Zero-Downtime Rollovers: Rolling out software updates with start-instance-refresh and a MinHealthyPercentage of 50% guarantees seamless canary deployments with 0% packet loss.

🔮 What’s Next in Part 12?

Now that we have mastered elastic compute clusters, how do we store state?

In Part 12, we dive deep into AWS Relational Database Service (RDS) & Multi-AZ Clustering. We will provision high-availability PostgreSQL/MySQL database engines, configure private DB Subnet Groups, manage automated snapshots, master IAM database authentication, and establish encrypted database connections from our auto-scaled compute fleet!

Related & Recommended Guides

Continue exploring related systems architectures and engineering field notes.

Cloud16 min read

AWS CLI First #17: Zero-Secret CI/CD: Automating AWS Deployments with GitHub Actions & OIDC

Eliminate long-lived access keys in CI/CD: configure AWS IAM OpenID Connect (OIDC) identity provider, craft least-privilege trust policies for GitHub Actions, build an automated deployment pipeline, validate templates with cfn-lint, and invalidate CloudFront CDN caches from the terminal.

Read article arrow_forward
Cloud18 min read

AWS CLI First #16: Infrastructure as Code: Deploying Multi-Tier Stacks with CloudFormation & SAM CLI

Transition from imperative scripts to declarative Infrastructure as Code: author production CloudFormation YAML templates, validate change sets, automate rollbacks, manage stack drifts, and build serverless apps with AWS SAM CLI.

Read article arrow_forward