Vault Kubernetes Secrets Engine: Dynamic Service Account Token Issuance

You want to keep Kubernetes cluster permissions minimal while still distributing short-lived credentials to automation infrastructure and workloads. Vault's Kubernetes secrets engine answers this need by calling Kubernetes's TokenRequest API to dynamically issue Service Account tokens.

Based on the stable behavior documented in the official guides, this article walks through the setup steps, RBAC design, operational patterns, and the differences exams love to ask about (specifically vs. the Kubernetes auth method).

Fundamentals and Flow: Short-Lived Tokens via TokenRequest

Vault's Kubernetes secrets engine calls the Kubernetes API's TokenRequest subresource against an existing Service Account (SA), obtains a JWT with a short expiration, and returns it to the client. Vault controls the Kubernetes connection info needed for this issuance, and uses roles to define which SAs and namespaces it is allowed to issue tokens for.

This approach uses standard SA tokens signed and verified by Kubernetes, so it offers high compatibility and fits operations that rely on natural TTL expiration (early forced revocation is generally not possible; the model assumes short lifetimes and frequent reissue).

• Benefits: short-lived tokens reduce leak risk and make distribution easy to automate
• Caveat: TTL may be truncated by Kubernetes's TokenRequest upper limit or other cluster settings
• Prerequisite: Vault needs RBAC permission to call TokenRequest against the Kubernetes API

End-to-end picture of dynamic issuance

Minimal example for obtaining a token from Vault (conceptual)

## 概念的な例（実際のパス名やフィールドはバージョンで差異あり。公式ドキュメントで要確認）
# 役割名 my-role で SA トークンを払い出し
vault read kubernetes/issue/my-role \
  service_account_name=my-app \
  namespace=apps \
  ttl=15m \
  audiences=https://kubernetes.default.svc

# 出力例（概略）
# token          : eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...
# expiration     : 2026-04-19T12:34:56Z
# namespace      : apps
# service_account: my-app

Required Permissions and Connection Setup: RBAC and Vault-side config

To let Vault execute TokenRequest, grant create permission on serviceaccounts/token on the Kubernetes side. Avoid overly broad permissions: the recommended pattern is to scope roles to a single namespace and a minimal set of Service Accounts.

On the Vault secrets engine side, configure the Kubernetes API server endpoint, the CA certificate, and a Service Account token (used by Vault to connect to Kubernetes) backed by the RBAC described above.

• Default to Role/RoleBinding (per-namespace) for RBAC. Use ClusterRoleBinding only when unavoidable
• Limit the create permission on TokenRequest to specific SAs or namespaces
• Set the CA, API endpoint, and token in Vault's kubernetes/config

Example: RBAC and the Vault secrets-engine connection config

# Kubernetes 側（最小権限の例: apps Namespace 内の TokenRequest のみ許可）
apiVersion: v1
kind: ServiceAccount
metadata:
  name: vault-k8s-se
  namespace: vault
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: tokenrequest-minimal
  namespace: apps
rules:
- apiGroups: [""]
  resources: ["serviceaccounts/token"]
  verbs: ["create"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: vault-k8s-se-binding
  namespace: apps
subjects:
- kind: ServiceAccount
  name: vault-k8s-se
  namespace: vault
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: tokenrequest-minimal

# Vault 側（概念例。実際のフィールド名は公式ドキュメントを確認）
# Kubernetes シークレットエンジン有効化
vault secrets enable kubernetes

# 接続設定（in-cluster を想定）
vault write kubernetes/config \
  kubernetes_host="https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT}" \
  kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt \
  token=@/var/run/secrets/kubernetes.io/serviceaccount/token

Role Design and Issuance: Tuning Namespace, SA, Audience, and TTL

Define what can be issued using roles. A role constrains the allowed namespaces and Service Accounts, the allowed audiences, and the upper bound on TTL. This keeps separation of duties and least privilege intact on the Vault side.

To issue a token, perform a read (or write) operation specifying the role name. The returned token is a JWT signed and verified by Kubernetes, and its TTL may be shortened depending on cluster-side settings.

• TTL may be shortened by Kubernetes's TokenRequest cap (cluster-dependent)
• Scope each role tightly by namespace and SA, and split horizontally into separate roles
• A Vault lease is created at issuance time, but the Kubernetes token itself generally cannot be revoked early

Conceptual commands for creating a role and issuing a token

# ロール作成（概念例。ご利用の Vault 版のロールパラメータ名は公式ドキュメントで確認）
vault write kubernetes/roles/ci-deployer \
  allowed_namespaces=apps,staging \
  allowed_service_accounts=deployer \
  allowed_audiences=https://kubernetes.default.svc \
  ttl=15m \
  max_ttl=1h

# 発行（ロールに許可された SA/Namespace の組み合わせで実行）
vault read kubernetes/issue/ci-deployer \
  service_account_name=deployer \
  namespace=apps \
  audiences=https://kubernetes.default.svc \
  ttl=10m

# 返却値（概略）: token, expiration, namespace, service_account など

Operational Patterns: Integrating with Vault Agent and CI/CD

For distributing tokens to workloads, a practical approach is to use Vault Agent (as a sidecar) to periodically reissue and write the token to a file that the process reads. For CI/CD, fetch once at the start of the job and size the TTL so it covers only the job's duration.

For observability, the bare minimum is monitoring lease expiration, exposing issuance-failure metrics, and collecting audit logs. Even when early revocation is not required, periodically verify that short TTLs and automated reissue actually work.

• Workloads: use a Vault Agent sidecar for automated reissue and rotation
• CI/CD: size the TTL to match job duration (idempotent retries and clear failure handling)
• Auditing: enable Vault's audit device alongside the Kubernetes API server audit log

Example: template output via Vault Agent (sidecar)

# Agent 設定（概念）: kubernetes/issue/ci-deployer から取得しファイル出力
exit_after_auth = false
pid_file = "/tmp/vault-agent.pid"

auth {
  method = "approle"  # 例: AppRole で Agent を認証
  mount_path = "auth/approle"
  config = {
    role_id_file_path   = "/vault/secrets/role_id"
    secret_id_file_path = "/vault/secrets/secret_id"
  }
}

template {
  destination = "/work/token"
  contents    = <<EOF
{{ with secret "kubernetes/issue/ci-deployer" (printf "service_account_name=%s namespace=%s ttl=%s" "deployer" "apps" "10m") }}{{ .Data.token }}{{ end }}
EOF
}

# ワークロードは /work/token を読み、期限前に Agent が置換する設計

Exam Focus: Auth Method vs. Secrets Engine, and Understanding TTL/Leases

Vault's Kubernetes auth method is the mechanism for Pods to log in to Vault. The Kubernetes secrets engine described here is the mechanism for Vault to issue Kubernetes credentials (SA tokens). Mixing up the direction of these two is a classic design mistake.

Also, tokens issued via Kubernetes's TokenRequest expire naturally with time. Revoking a Vault lease only affects Vault's internal bookkeeping; you generally cannot force a previously issued token to be invalid on the Kubernetes side. Short TTLs and automated rotation are therefore the assumed operating model.

• Kubernetes auth method: Pod-to-Vault authentication
• Kubernetes secrets engine: Vault-to-Kubernetes token issuance
• Early revocation: assumed impossible. Short TTLs and frequent reissue are the solution

Minimal exam-ready policy (allow read on the Vault issuance role)

# Vault ポリシー（例）: 指定ロールの発行のみ許可
auth "*" {
  # 認証方法は別途
}

path "kubernetes/issue/ci-deployer" {
  capabilities = ["read"]
}

# 実務では対象ロールごとに最小権限で切り分ける

Troubleshooting and Auditing: Common Pitfalls and What to Check

Missing RBAC for TokenRequest is by far the most common issue. Verify that create on serviceaccounts/token is granted on the target namespace/SA, and that resourceNames are not constraining things too tightly. If the TTL is shorter than expected, suspect the cluster-side upper limit.

For auditing, enable Vault's audit device so you can trace who issued tokens with which role and when. Being able to correlate this with Kubernetes API audit logs makes it even more useful.

• 403 Forbidden: Role/RoleBinding in the wrong namespace, or a resourceNames mismatch
• TTL truncation: hitting the cluster's TokenRequest cap. Raising Vault-side TTL cannot exceed it
• Audience mismatch: align with the target service's verifying audience (e.g. https://kubernetes.default.svc)

Practical commands for auditing and verification

# リース確認（Vault 側）
vault list sys/leases/lookup/kubernetes

# 直近の発行リクエスト（Vault 監査ログから grep）
# 例: file 監査デバイスを /var/log/vault_audit.log に設定済みとする
grep 'kubernetes/issue/ci-deployer' /var/log/vault_audit.log | tail -n 20

# Kubernetes 側の権限テスト（権限レビュー）
kubectl auth can-i create serviceaccounts/token --as=system:serviceaccount:vault:vault-k8s-se -n apps

Check Your Understanding

Frequently Asked Questions