Vault Auto Unseal Practical Guide: Stabilizing Operations with Cloud KMS Integration

Auto Unseal decrypts Vault's master key with an external KMS at startup, removing the need for manual Shamir unsealing. Combined with a cloud KMS, restart and failover operations become simpler and safer.

This article covers AWS KMS, Azure Key Vault, and GCP Cloud KMS, walking through setup steps, required permissions, and operational pitfalls with mitigations. It calls out points commonly tested on Ops exams in a form that maps directly to real-world practice.

Key Points of Auto Unseal and Cloud KMS Integration

Auto Unseal wraps Vault's master key with an external KMS and uses that KMS to decrypt it automatically at startup. This eliminates the operational burden of collecting multiple unseal keys and waiting on manual unseals across remote sites.

When Auto Unseal is enabled, initialization produces Recovery Keys instead of Shamir Unseal Keys. Recovery Keys are used only for recovery scenarios and certain operations, never for startup unsealing. Operationally, the KMS is called primarily at startup and during master key rekey/rotation, and is not involved in normal secret operations.

In availability design, reachability to the KMS determines whether startup succeeds. A running node keeps operating even if the KMS becomes temporarily unreachable, but restarts and scale-outs require KMS reachability. When the KMS is region-specific, align the Vault cluster topology with the KMS placement and redundancy policy.

• Initialization yields Recovery Keys (not used for manual unsealing)
• The KMS is invoked mainly at startup and during key rekey/rotation
• Vault cannot start when the KMS is unreachable (already-running instances continue to operate)
• All nodes must use the same seal configuration (same provider and key)

Auto Unseal (Raft + Cloud KMS) conceptual diagram

Basic seal stanza (example: AWS KMS)

seal "awskms" {
  region     = "ap-northeast-1"
  kms_key_id = "arn:aws:kms:ap-northeast-1:123456789012:key/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
  # 認証情報はインスタンスロール/タスクロール推奨（静的キーは避ける）
}

storage "raft" {
  path    = "/opt/vault/data"
  node_id = "vault-1"
}

listener "tcp" {
  address     = "0.0.0.0:8200"
  tls_disable = "0"
}

default_lease_ttl = "768h"
max_lease_ttl     = "768h"

AWS KMS: Setup Steps and Least Privilege

When using AWS KMS, grant KMS key-use permissions to the IAM role attached to the Vault runtime (EC2, ECS, EKS, etc.). Provision a symmetric customer master key (CMK) and place it in the same region as Vault nodes, or with equivalent redundancy.

At minimum, grant kms:Decrypt and kms:Encrypt, plus kms:DescribeKey for key-state checks. Also allow the IAM role as a Principal in the key policy to avoid common issues. Auto Unseal traffic is centered on startup, so KMS cost and rate-limit impact is limited, but frequent rolling restarts are still worth reviewing.

If KMS is used across regions, consider multi-region keys. If your cluster spans multiple regions, either replicate the same key (multi-region key) or ensure per-region keys and their configuration management stay consistent.

• Grant Decrypt/Encrypt/DescribeKey to the IAM role
• Allow the IAM role as a Principal in the key policy
• Because startup depends on KMS, restart planning and KMS reachability monitoring matter
• Simplify DR scenarios with multi-region keys

vault.hcl (AWS KMS) minimal example

seal "awskms" {
  region     = "ap-northeast-1"
  kms_key_id = "arn:aws:kms:ap-northeast-1:123456789012:key/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
}
# 認証は IAM ロール（EC2/EKS のメタデータ経由）を推奨
# 必要に応じて endpoint を明示（VPC エンドポイント等）
# endpoint = "https://kms.ap-northeast-1.vpce.amazonaws.com"

Azure Key Vault: Configuration and Permission Design

On Azure, you need wrapKey and unwrapKey (and get) permissions on the Key Vault key. In production, using Managed Identity simplifies secret management. When enabling network controls (Private Endpoint, Firewall), ensure connectivity from Vault.

Enabling soft-delete and purge protection on Key Vault hardens the design against accidental key deletion. When switching key versions, prepare a Vault-side rekey and configuration-update plan.

• Required permissions: get, wrapKey, unwrapKey
• Prefer Managed Identity authentication (credential-less)
• When using Private Endpoint, verify connectivity including DNS and routing

vault.hcl (Azure Key Vault) minimal example

seal "azurekeyvault" {
  tenant_id   = "00000000-0000-0000-0000-000000000000"
  vault_name  = "my-key-vault"
  key_name    = "vault-unseal"
  # Managed Identity を使う場合:
  # use_msi = true
  # システム割当またはユーザー割当 MI に鍵権限を付与
  # クライアントシークレットで行う場合（非推奨）:
  # client_id     = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
  # client_secret = "..."
}

GCP Cloud KMS: Configuration and Roles

On GCP, grant roles/cloudkms.cryptoKeyEncrypterDecrypter on the target CryptoKey to Vault's runtime service account. You must specify the project, location, key ring, and crypto key.

Prefer service-account metadata authentication (GCE/GKE) and minimize JSON key-file distribution. Co-locate the KMS and Vault nodes in the same or nearby regions to keep startup latency low and availability high.

• Required role: cloudkms.cryptoKeyEncrypterDecrypter
• Avoid distributing keys by leveraging Workload Identity on GCE/GKE
• Keep KMS location and Vault placement consistent

vault.hcl (GCP Cloud KMS) minimal example

seal "gcpckms" {
  project    = "my-gcp-project"
  region     = "asia-northeast1"  # KMS ロケーション
  key_ring   = "vault-ring"
  crypto_key = "vault-unseal"
  # 認証はデフォルト認証情報を推奨（GCE/GKE の SA）
  # credentials = "/path/to/service-account.json"  # 可能なら未使用
}

Ops Essentials: Availability, Monitoring, and Incident Response

A running Vault keeps operating even when the KMS is unreachable. The risk is at restart time. If KMS is unreachable during a restart or scale-out, unsealing fails. During planned maintenance or rolling upgrades, monitor KMS reachability and rate limits, and restart in stages.

Use the /v1/sys/health status for health checks. The status code differs across sealed, uninitialized, recovery mode, and other states, so tune your readiness/liveness conditions to match your orchestrator's needs.

Logs record the seal type and unseal success/failure. KMS reachability and authorization errors are valuable for first-response triage. Also, disabling, deleting, or changing permissions on KMS keys can trigger major incidents, so build change management and audit monitoring (CloudTrail / Activity Log / Cloud Audit Logs) into your process.

• If KMS reachability drops, avoid restarts and first restore network and permissions
• Use small batches for rolling restarts; respect KMS rate limits
• Define monitoring rules based on the health API status codes
• Always route KMS key state changes through change management

Health check example

curl -s -o /dev/null -w "%{http_code}\n" \
  http://127.0.0.1:8200/v1/sys/health
# 200: unsealed/active, 429: standby, 503: sealed など
# sealed 時は systemd/オーケストレータで自動再起動を抑止する設計が有効

Exam Tips and Best Practices

From an Ops perspective, common topics include when Auto Unseal runs (startup and rekey), the role of Recovery Keys, the impact of KMS reachability on cluster operations, and least-privilege design. Typical questions cover what to do when the KMS is unreachable, the impact of key rotation, and designs across multiple nodes and regions.

In practice, use roles or managed identities for credentials and avoid embedding static secrets in vault.hcl. Wire KMS audit logs into security operations, and explicitly manage KMS network reachability (Private Endpoints, VPC endpoints, firewalls, etc.) through IaC.

• Use the same seal configuration on every node (mixing is not allowed)
• Automatic rotation (under the same key ID) causes no downtime. Switching to a different key requires a planned rekey.
• During incidents, Recovery Keys can perform recovery operations but cannot replace unsealing
• KMS dependency is centered on startup; steady-state traffic is minimal

Configuration change deployment example (rolling)

# 例: 3 ノードを 1 台ずつローリング
for h in vault-1 vault-2 vault-3; do
  echo "Restarting $h"
  ssh $h 'sudo systemctl restart vault && sleep 10 && curl -sf http://127.0.0.1:8200/v1/sys/health >/dev/null'
  # KMS 到達性/ステータスを確認して次へ
done

Check Your Understanding

Frequently Asked Questions