AWS OpenTofu #113: Drift Detection & Resource Import

In an ideal DevOps universe, infrastructure is purely declarative: no human ever touches the AWS Web Console, all changes flow through Git pull requests, and automated CI/CD pipelines apply changes cleanly.

Then reality strikes.

At 11:30 PM on a Tuesday, an urgent production outage occurs. A database connectivity issue forces the on-call engineer to log into the AWS Web Console and manually edit a Security Group to restore service. The incident is resolved, but the engineer forgets to update the OpenTofu code.

Two weeks later, another engineer runs a routine tofu apply to add an S3 bucket. OpenTofu notices the discrepancy between the code and AWS, assumes the manual Security Group change was an error, and silently deletes the emergency firewall rule—triggering another massive outage.

This phenomenon is called Infrastructure Drift.

Furthermore, what happens when your company decides to adopt OpenTofu, but you already have 200 pre-existing AWS resources built years ago via ClickOps? You cannot delete and recreate active production systems.

In this lab, you will master the two most critical operational skills in Day-2 infrastructure management:

1. Drift Detection & Reconciliation: How to detect out-of-band AWS changes and decide whether to enforce code or adopt the drift.
2. Brownfield Adoption: How to use OpenTofu’s modern declarative import block and automated code generation (-generate-config-out) to bring legacy AWS assets under code control with zero downtime.

---

🎯 What You Will Learn

Drift Detection Mechanics	Drift Reconciliation Strategies	Modern Declarative Import
• How OpenTofu refreshes state against AWS APIs	• Strategy A: Overwrite & enforce the IaC baseline	• Declarative import blocks vs. legacy CLI tofu import
• The -refresh-only safe inspection workflow	• Strategy B: Adopt manual drift into Git code	• Auto-generating HCL code via -generate-config-out
• Detecting deleted out-of-band resources	• Handling unmanaged “shadow” rules in AWS	• Adopting legacy brownfield infrastructure
• Understanding the three-way state comparison engine	• CI/CD scheduled drift detection patterns	• Validating state import with zero planned changes

---

💰 Estimated Lab Cost

Resource	Configuration	Free Tier Cost	Standard Account (Past Free Tier)
Amazon S3 Storage	2 Standard Buckets (metadata storage only)	$0.00	$0.00 (Negligible < $0.001)
AWS Security Groups	1 Custom VPC Security Group	$0.00	$0.00 (Always 100% Free)
Compute & NAT Gateways	Zero EC2 instances or NAT Gateways	$0.00	$0.00
Total Estimated Cost	Complete Hands-On Session (~1 hour)	$0.00	$0.00

[!NOTE] All resources used in this lab (S3 buckets, VPC Security Groups) have zero hourly runtime fees. Total lab cost is $0.00 regardless of account age.

---

🔍 The Anatomy of Infrastructure Drift

Drift occurs whenever the Live Cloud Reality (AWS) diverges from the Declared Code (.tf) or the Recorded State (terraform.tfstate):

       ┌────────────────────────┐
       │   Codebase (.tf files) │
       └───────────┬────────────┘
                   │
                   │ OpenTofu Plan (Calculates Delta)
                   ▼
       ┌────────────────────────┐         Refreshes via         ┌────────────────────────┐
       │   terraform.tfstate    │ <───────────────────────────> │  AWS Cloud (Live API)  │
       │    (Cached State)      │          AWS API Calls        │   (Real Infrastructure)│
       └────────────────────────┘                               └────────────────────────┘

The Three Types of Drift

1. Attribute Modification: A human modifies a tag, changes an instance size, or adds an ingress rule directly in AWS.
2. Resource Deletion: Someone deletes an S3 bucket or terminates an instance via the AWS Console without updating IaC.
3. Shadow Assets: Someone spins up an unmanaged database or VPC alongside managed infrastructure.

The Three-Way Comparison Engine

When you run tofu plan, OpenTofu performs a three-way reconciliation:

1. Refresh: It calls Describe* APIs in AWS to update its internal view of live infrastructure.
2. Compare: It compares Live AWS against Cached State to detect drift.
3. Plan: It compares Desired Code (.tf) against Live AWS to determine what modifications (+, ~, -) are required.

---

🛠️ Step 1: Project Setup

Create a dedicated workspace directory for Lab 113:

mkdir -p ~/opentofu-labs/lab-113-drift-and-import
cd ~/opentofu-labs/lab-113-drift-and-import

We will start with two foundational files:

1. providers.tf — Standard AWS provider configuration.
2. main.tf — Our baseline managed infrastructure (an S3 bucket and a Security Group in the default VPC).

---

📄 Step 2: Baseline Configuration

Create providers.tf:

# providers.tf
terraform {
  required_version = ">= 1.8.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.50"
    }
    random = {
      source  = "hashicorp/random"
      version = "~> 3.6"
    }
  }
}

provider "aws" {
  region = "us-east-1"

  default_tags {
    tags = {
      Project     = "OpenTofu-Level-100"
      Lab         = "113-Drift-and-Import"
      ManagedBy   = "OpenTofu"
      Environment = "Lab"
    }
  }
}

Create main.tf:

# main.tf

# 1. Discover the AWS Default VPC
data "aws_vpc" "default" {
  default = true
}

# 2. Random ID for globally unique S3 bucket naming
resource "random_id" "suffix" {
  byte_length = 4
}

# 3. Baseline Managed S3 Bucket
resource "aws_s3_bucket" "app_storage" {
  bucket        = "lab113-app-storage-${random_id.suffix.hex}"
  force_destroy = true

  tags = {
    Owner = "DevOps-Team"
    Tier  = "Frontend"
  }
}

# 4. Baseline Managed Security Group
resource "aws_security_group" "web_firewall" {
  name        = "lab113-web-firewall-${random_id.suffix.hex}"
  description = "Managed web firewall"
  vpc_id      = data.aws_vpc.default.id

  # Allow HTTP only
  ingress {
    description = "HTTP ingress"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "lab113-web-firewall"
  }
}

---

🚀 Step 3: Deploy Baseline Infrastructure

Initialize and apply the baseline:

tofu init
tofu apply -auto-approve

Save the bucket name and security group ID to environment variables for quick reference:

BUCKET_NAME=$(tofu state show aws_s3_bucket.app_storage | grep 'bucket ' | awk '{print $3}' | tr -d '"')
SG_ID=$(tofu state show aws_security_group.web_firewall | grep '^    id ' | awk '{print $3}' | tr -d '"')

echo "Managed Bucket: $BUCKET_NAME"
echo "Managed Security Group: $SG_ID"

Verify that our infrastructure is completely in sync:

tofu plan

Output:

No changes. Your infrastructure matches the configuration.

---

⚡ Step 4: Simulating Out-of-Band ClickOps Drift

Now, imagine an engineer bypasses OpenTofu and makes manual changes directly in AWS using the AWS CLI (simulating ClickOps console actions).

Drift Event 1: Manually Inject an Insecure Ingress Rule

An engineer opens port 22 (SSH) from anywhere (0.0.0.0/0):

aws ec2 authorize-security-group-ingress \
  --group-id $SG_ID \
  --protocol tcp \
  --port 22 \
  --cidr 0.0.0.0/0

Drift Event 2: Manually Alter S3 Bucket Tags

Someone adds a billing tag to the S3 bucket in AWS:

aws s3api put-bucket-tagging \
  --bucket $BUCKET_NAME \
  --tagging 'TagSet=[{Key=Owner,Value=DevOps-Team},{Key=Tier,Value=Frontend},{Key=CostCenter,Value=Billing-9942}]'

Drift Event 3: Manually Terminate/Delete an Asset

What if an unapproved action completely deleted an S3 tag or resource? We now have two real drift events live in AWS.

---

🕵️ Step 5: Detecting the Drift

How do you find out what changed without applying changes?

Method 1: The Standard tofu plan

Run tofu plan:

tofu plan

OpenTofu queries the AWS API, refreshes state in memory, and displays the exact drift:

Note: Objects have changed outside of OpenTofu

OpenTofu detected the following changes made outside of OpenTofu since the
last "tofu apply" which may have affected this plan:

  # aws_s3_bucket.app_storage has changed
  ~ resource "aws_s3_bucket" "app_storage" {
        id                          = "lab113-app-storage-b4a1c2d3"
      ~ tags                        = {
          + "CostCenter"  = "Billing-9942"
            "Environment" = "Lab"
            "Lab"         = "113-Drift-and-Import"
            "ManagedBy"   = "OpenTofu"
            "Owner"       = "DevOps-Team"
            "Project"     = "OpenTofu-Level-100"
            "Tier"        = "Frontend"
        }
        # (9 unchanged attributes hidden)
    }

OpenTofu used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place

OpenTofu will perform the following actions:

  # aws_s3_bucket.app_storage will be updated in-place
  ~ resource "aws_s3_bucket" "app_storage" {
      ~ tags = {
          - "CostCenter"  = "Billing-9942" -> null
            # (6 unchanged elements hidden)
        }
    }

  # aws_security_group.web_firewall will be updated in-place
  ~ resource "aws_security_group" "web_firewall" {
      ~ ingress = [
          - {
              - cidr_blocks      = [
                  - "0.0.0.0/0",
                ]
              - description      = ""
              - from_port        = 22
              - protocol         = "tcp"
              - to_port          = 22
              # ...
            },
            {
              cidr_blocks      = [
                "0.0.0.0/0",
              ]
              description      = "HTTP ingress"
              from_port        = 80
              protocol         = "tcp"
              to_port          = 80
            },
        ]
    }

Plan: 0 to add, 2 to change, 0 to destroy.

Look at the surgical clarity of that plan:

1. Notice Section: “Objects have changed outside of OpenTofu.”
2. The S3 Plan: OpenTofu proposes removing "CostCenter" = "Billing-9942" because it is not declared in main.tf.
3. The Security Group Plan: OpenTofu detects the unauthorized port 22 SSH ingress rule and proposes deleting it!

---

Method 2: The Safe -refresh-only Inspection

What if you want to inspect or synchronize state with AWS without modifying any live infrastructure?

Run:

tofu plan -refresh-only

This updates state to match AWS without planning any additions or deletions:

Plan: 0 to add, 0 to change, 0 to destroy.
Note: You didn't use the -out option to save this plan, so OpenTofu can't guarantee that exactly these actions will be performed if you run "tofu apply" now.

If you ran tofu apply -refresh-only, OpenTofu would write the new AWS attributes into terraform.tfstate while leaving all cloud infrastructure untouched.

---

⚖️ Step 6: Reconciling the Drift

When drift occurs, you have two architectural choices:

                        DRIFT DETECTED
                              │
             ┌────────────────┴────────────────┐
             ▼                                 ▼
   [ Strategy A: Enforce ]           [ Strategy B: Adopt ]
   "The manual change was illegal.   "The manual change was intentional.
   Revert AWS back to our code."     Update our code to match AWS."

Strategy A: Enforce Code Baseline (Nuke the Unauthorized SSH Port)

The manual SSH port 22 was a security violation. We want OpenTofu to enforce our approved code baseline and strip out the rule.

Simply run:

tofu apply -auto-approve

OpenTofu calls the AWS API (RevokeSecurityGroupIngress) and closes port 22!

Verify with the AWS CLI:

aws ec2 describe-security-groups --group-ids $SG_ID --query "SecurityGroups[0].IpPermissions" --output table

Output:

--------------------------------------------------------
|                     IpPermissions                    |
+------------------------------------------+-----------+
|                 FromPort                 |  ToPort   |
+------------------------------------------+-----------+
|  80                                      |  80       |
+------------------------------------------+-----------+

Port 22 is completely gone! OpenTofu healed the infrastructure.

---

Strategy B: Adopt Drift into Code (Keep the Billing Tag)

Now suppose the finance team confirms that "CostCenter" = "Billing-9942" is required. Instead of overwriting it, we adopt it into our source of truth.

Open main.tf and update the aws_s3_bucket.app_storage tags:

# In main.tf
resource "aws_s3_bucket" "app_storage" {
  bucket        = "lab113-app-storage-${random_id.suffix.hex}"
  force_destroy = true

  tags = {
    Owner      = "DevOps-Team"
    Tier       = "Frontend"
    CostCenter = "Billing-9942" # << ADOPTED INTO CODE!
  }
}

Now run tofu plan:

tofu plan

Output:

No changes. Your infrastructure matches the configuration.

You have successfully reconciled both drift scenarios: one enforced, one adopted!

---

📦 Step 7: Brownfield Adoption with Declarative import

Now for the second major challenge: Importing unmanaged resources.

Imagine a legacy S3 bucket was created two years ago via ClickOps. It has live data. We need to bring it under OpenTofu management.

1. Create the Unmanaged Legacy Bucket via AWS CLI

Simulate the existing legacy asset:

LEGACY_BUCKET="legacy-corp-archive-$(openssl rand -hex 4)"
aws s3api create-bucket --bucket $LEGACY_BUCKET --region us-east-1
echo "Created Unmanaged Bucket: $LEGACY_BUCKET"

OpenTofu has zero knowledge of this bucket:

• It is not in main.tf.
• It is not in terraform.tfstate.

---

2. The Old Way vs. The Modern Way

In early versions of Terraform, you had to run an imperative command:

tofu import aws_s3_bucket.legacy legacy-corp-archive-xxxx

This was frustrating because it added the bucket to state, but did not write any .tf code. You had to manually guess the HCL attributes. If your code didn’t match the imported state, tofu plan would immediately try to modify or replace the bucket.

3. The Modern Declarative import Block (OpenTofu 1.8+)

Modern OpenTofu introduces the declarative import block combined with automatic code generation.

Create a new file called import.tf:

# import.tf
import {
  to = aws_s3_bucket.legacy_archive
  id = "REPLACE_WITH_YOUR_LEGACY_BUCKET_NAME"
}

Use sed or your editor to replace the bucket name with your actual $LEGACY_BUCKET:

sed -i '' "s/REPLACE_WITH_YOUR_LEGACY_BUCKET_NAME/$LEGACY_BUCKET/" import.tf

Verify import.tf:

cat import.tf

---

4. Automatically Generate the HCL Code!

Run tofu plan with the -generate-config-out flag:

tofu plan -generate-config-out=generated_resources.tf

Look at the terminal output:

aws_s3_bucket.legacy_archive: Preparing import... [id=legacy-corp-archive-xxxx]
aws_s3_bucket.legacy_archive: Refreshing state... [id=legacy-corp-archive-xxxx]

OpenTofu will perform the following actions:

  # aws_s3_bucket.legacy_archive will be imported
    resource "aws_s3_bucket" "legacy_archive" {
        arn                         = "arn:aws:s3:::legacy-corp-archive-xxxx"
        bucket                      = "legacy-corp-archive-xxxx"
        id                          = "legacy-corp-archive-xxxx"
        region                      = "us-east-1"
        # ...
    }

Plan: 1 to import, 0 to add, 0 to change, 0 to destroy.

Now inspect the newly generated file:

cat generated_resources.tf

OpenTofu inspected the live AWS bucket and automatically wrote the exact HCL resource block for you:

# __generated__ by OpenTofu
# Please review these resources and move them into your main configuration files.

resource "aws_s3_bucket" "legacy_archive" {
  bucket              = "legacy-corp-archive-xxxx"
  bucket_prefix       = null
  force_destroy       = null
  object_lock_enabled = false
  tags                = {}
  tags_all            = {
    "Environment" = "Lab"
    "Lab"         = "113-Drift-and-Import"
    "ManagedBy"   = "OpenTofu"
    "Project"     = "OpenTofu-Level-100"
  }
}

---

5. Finalize the Import

Apply the import:

tofu apply -auto-approve

Output:

aws_s3_bucket.legacy_archive: Importing... [id=legacy-corp-archive-xxxx]
aws_s3_bucket.legacy_archive: Import complete [id=legacy-corp-archive-xxxx]

Apply complete! Resources: 1 imported, 0 added, 0 changed, 0 destroyed.

Check your state list:

tofu state list

Output:

aws_s3_bucket.app_storage
aws_s3_bucket.legacy_archive
aws_security_group.web_firewall
random_id.suffix

The legacy bucket is now an officially managed citizen of your OpenTofu codebase!

Run tofu plan to confirm:

tofu plan

Output:

No changes. Your infrastructure matches the configuration.

---

🧹 Step 8: Clean Teardown

Because all resources—including the imported legacy bucket—are now fully tracked in state, tofu destroy will cleanly tear down everything:

tofu destroy -auto-approve

Output:

aws_s3_bucket.legacy_archive: Destroying...
aws_s3_bucket.app_storage: Destroying...
aws_security_group.web_firewall: Destroying...
random_id.suffix: Destroying...
Destroy complete! Resources: 4 destroyed.

Verify in AWS:

aws s3 ls | grep legacy-corp-archive

(Returns empty. Everything was destroyed cleanly).

---

🏋️ Hands-On Challenge

Solidify your drift and import skills:

1. Use the AWS CLI to create a standalone, unmanaged Security Group in your default VPC:

   aws ec2 create-security-group --group-name "legacy-db-sg" --description "Legacy database firewall"

2. Write an import block in import_sg.tf targeting aws_security_group.legacy_db.
3. Use tofu plan -generate-config-out=generated_sg.tf to generate its HCL configuration.
4. Apply the import and verify with tofu state list.
5. Clean up with tofu destroy.

---

📖 Summary & Key Takeaways

In this lab, you mastered Day-2 operational resilience in OpenTofu:

1. The Three-Way Comparison: tofu plan refreshes state from live AWS APIs, compares it against cached state to detect drift, and produces a diff against declared .tf code.
2. Drift Triage:
   • Enforce: Apply approved code over unapproved manual changes (e.g., closing rogue SSH ports).
   • Adopt: Merge intentional out-of-band updates into Git code (e.g., billing tags).
3. -refresh-only: The zero-risk command to synchronize state without making infrastructure modifications.
4. Modern Declarative import:
   • The import block declares what to import in code, keeping your workflow auditable in Git.
   • The -generate-config-out flag eliminates the painful manual guesswork of writing HCL for legacy infrastructure.
5. Zero Downtime Brownfield Migration: You can adopt pre-existing, mission-critical AWS assets into OpenTofu without a single second of service interruption.

---

🚀 What’s Next? (Lab 114)

Congratulations! You have completed Module 05: State Management & Operations.

Now, we arrive at the culmination of the entire course: Module 06: Capstone Projects.

In [AWS OpenTofu #114: Build a Mini AWS Environment], you will:

• Step into the shoes of a Lead DevOps Architect.
• Assemble every skill learned from Labs 101 to 113: Custom Multi-AZ VPC, Public & Private Subnets, Internet Gateways, Security Group Chaining, EC2 Web Servers with SSM, IAM Roles, S3 Storage, Variables, Locals, and State Management.
• Build a production-grade, multi-tier cloud environment from scratch!