Terraform TF_LOG and Debugging in Practice: How to Capture Verbose Logs

When you hit a mysterious failure or unexplained drift in Terraform, the first move is to enable verbose logging with the TF_LOG family of environment variables.

Log volume tends to balloon, though, and mishandling the configuration can drag out investigations or leak sensitive data. This article organizes safe log capture procedures, key reading techniques, and exam-relevant points from a practitioner's perspective.

TF_LOG Basics and Log Levels

Terraform enables verbose logging via the TF_LOG environment variable. Valid values are TRACE, DEBUG, INFO, WARN, and ERROR — the lower the level, the more verbose. For routine troubleshooting use DEBUG; reserve TRACE for when you need to dig deeper. Logs go to stderr by default, but TF_LOG_PATH writes them to a file.

For finer-grained control, use TF_LOG_CORE (Terraform core) and TF_LOG_PROVIDER (provider plugins). Anything you don't set falls back to TF_LOG as the default; per-component settings take precedence. Verbose logs may contain fragments of API requests, so handle them with care.

• Level guidance: DEBUG = general investigation, TRACE = digging into hard-to-reproduce or serious bugs
• Output destination: stderr by default, switch to a file with TF_LOG_PATH
• Scope: TF_LOG (everything), TF_LOG_CORE (core only), TF_LOG_PROVIDER (provider only)
• Caution: may contain sensitive information — always mask or remove before sharing

Flow of Terraform verbose logs (what gets emitted where)

Basic enablement (Linux/macOS, Windows)

# Linux/macOS
export TF_LOG=DEBUG
export TF_LOG_PATH=./terraform-debug.log
terraform plan

# Windows (PowerShell)
$env:TF_LOG = "DEBUG"
$env:TF_LOG_PATH = "./terraform-debug.log"
terraform plan

# Disable when finished
unset TF_LOG TF_LOG_PATH  # PowerShell: Remove-Item Env:TF_LOG, Env:TF_LOG_PATH

Capture Repro Logs Fast: Steps and Tips

Start with a minimal directory that reproduces the issue, capture DEBUG-level logs, and work through isolation. TRACE is a last resort — switch to it right before the section of interest to keep the output readable.

In CI/CD, give each job its own TF_LOG_PATH and clearly define log rotation and retention. Suppress color output with -no-color to improve log readability.

• Reproduce with a minimal config → capture at DEBUG → switch to TRACE only when needed
• Save to a file via TF_LOG_PATH and separate from stdout
• Use -no-color to suppress control characters (helpful in CI)
• Avoid long TRACE runs (performance and disk impact)

Standard repro log procedure (Bash example)

set -euo pipefail
export TF_LOG=DEBUG
export TF_LOG_PATH="./repro-$(date +%Y%m%d-%H%M%S).log"
terraform init -upgrade
terraform plan -no-color
# Only if the problem doesn't surface in plan: enable TRACE for the focused section before apply
export TF_LOG=TRACE
terraform apply -refresh-only -no-color  # Minimize side effects and verify diff only
# Cleanup
env -u TF_LOG -u TF_LOG_PATH true >/dev/null 2>&1 || true

How to Read DEBUG/TRACE Logs

Read logs in chronological order. Start from the WARN/ERROR right before the failure and trace backward through nearby DEBUG lines — that's the fastest way to pin down causality. For provider calls, correlate by resource address (e.g., module.db.aws_db_instance.this) and RPC direction (request/response).

When the diff doesn't match expectations, focus on how Unknown values are handled in plan (computed attributes) and on normalization that happens after read (refresh/re-running plan). Trim noisy TRACE output with grep and extract only the lines for the resource of interest to follow the thread.

• Focus on the 5-20 seconds right before the error
• Grep or split views by the target resource address
• Verify API call consistency in request → response order
• Follow the flow of Unknown/Computed attributes (plan → post-apply)

Log filtering examples (grep/awk)

# Filter by resource address
grep -F "module.db.aws_db_instance.this" terraform-debug.log | less

# Inspect surroundings by level and timestamp
awk '/ERROR|WARN|DEBUG/ {print}' terraform-debug.log | less

# Verify request/response pairs
grep -E "(request|response)" terraform-debug.log | grep aws_db_instance | less

Provider-Only Detail: Drill Down with Less Noise

When core seems fine but the API layer is suspect, bump TF_LOG_PROVIDER to DEBUG/TRACE while holding TF_LOG_CORE down at INFO or WARN — that cuts the noise. You suppress the diff computation details while still focusing on HTTP-equivalent exchanges and the flow of attribute application.

On Windows or in CI, temporarily flip the scope so only the moments right before and after the problem area run at a high level — this prevents log bloat.

• TF_LOG_PROVIDER=DEBUG/TRACE, TF_LOG_CORE=WARN/INFO
• Raise the level only right before the problem area, then restore it after apply
• In CI, adjust env vars per step to split logs

Per-scope log control (PowerShell and Bash)

# PowerShell
$env:TF_LOG_CORE = "WARN"
$env:TF_LOG_PROVIDER = "DEBUG"
$env:TF_LOG_PATH = "./provider-only.log"
terraform plan

# Bash
export TF_LOG_CORE=INFO
export TF_LOG_PROVIDER=TRACE
export TF_LOG_PATH=./provider-trace.log
terraform apply -refresh-only -no-color

Typical Issues: Where and How to Look

Authentication errors: when core succeeds but the provider's requests are rejected, provider-side DEBUG/TRACE reveals clues about credential detection, endpoints, and scopes. Also check the resolution order of profiles and environment variables.

Unstable diffs: suspect data source read order or drift from external changes, and compare stability via refresh-only and a re-run of plan. Also verify whether provider normalization logic is rounding attributes.

• Authentication/permissions → use provider DEBUG to verify credential detection and scope
• Drift/external changes → pull current state with refresh-only, then compare
• Attribute normalization → check whether re-reading after apply resolves the diff
• Sensitive handling → always re-audit for secrets before sharing logs

Minimum stability comparison procedure (Bash)

export TF_LOG=DEBUG
export TF_LOG_PATH=./stability-compare.log
terraform plan -no-color
terraform apply -refresh-only -no-color
terraform plan -no-color
# Compare the three steps to identify unstable areas

Exam Prep and Operational Tips

The Associate and Pro exams frequently test the roles of TF_LOG / TF_LOG_CORE / TF_LOG_PROVIDER / TF_LOG_PATH, the meaning of each log level, the fact that output goes to stderr, and combined use with -no-color. Be ready to articulate when to use which.

For Terraform Cloud/Enterprise remote runs, TF_LOG only affects local CLI behavior — the actual execution is in the remote run logs. In CI, standardize per-job file output, retention periods, and the secret-masking procedure.

• Exam prep: organize env var scopes and precedence
• In practice: logs go to stderr — file them with TF_LOG_PATH, handle rotation yourself
• Remote runs: client-side TF_LOG is limited — check the remote run logs

Safe capture in CI (e.g., GitHub Actions step excerpt)

- name: Terraform Plan (debug log)
  env:
    TF_LOG: DEBUG
    TF_LOG_PATH: terraform-${{ github.run_id }}.log
  run: |
    terraform init -input=false
    terraform plan -no-color -input=false
  # Store as an artifact and protect secret strings with a masking policy

Check Your Understanding

Frequently Asked Questions