Choosing SASL Mechanisms in Kafka: PLAIN / SCRAM / GSSAPI / OAUTHBEARER for Production and CCAAK

Kafka authentication is built on SASL, and you can choose from several mechanisms. In production you pick based on network boundaries, alignment with an existing IdP or Kerberos, operational burden, and performance. On the Confluent Certified Administrator for Apache Kafka (CCAAK) exam, you are typically tested on each mechanism's characteristics, listener design, and inter-broker configuration consistency.

This article covers the correct prerequisites, configuration, comparisons, and troubleshooting for PLAIN, SCRAM, GSSAPI, and OAUTHBEARER, following the stable specs in the official documentation and addressing both exam prep and real-world operations.

SASL and Kafka: Background

Kafka combines transport encryption (SSL/TLS) with authentication (SASL). security.protocol takes PLAINTEXT, SSL, SASL_PLAINTEXT, or SASL_SSL, and is assigned per listener. On the exam, a basic skill is recognizing the danger of sending PLAIN credentials in clear text (SASL_PLAINTEXT without TLS).

Inter-broker traffic uses its own listener and mechanism. sasl.mechanism.inter.broker.protocol is the mechanism used when a broker connects to another broker as a client; it must line up with listeners and inter.broker.listener.name. To enable multiple mechanisms, list them in sasl.enabled.mechanisms, and restrict per listener as needed with listener.name.<listener>.sasl.enabled.mechanisms.

• PLAIN is the simplest but transmits passwords in clear text, so TLS is mandatory
• SCRAM uses a hashed credential store plus challenge-response to avoid exposing passwords
• GSSAPI depends on Kerberos. It is the first choice if your organization already has a KDC
• OAUTHBEARER integrates with an external IdP. The crux is the broker-side token validation implementation
• Pick exactly one mechanism for inter-broker (SCRAM or GSSAPI are the production-grade defaults)

PLAIN and SCRAM: The Fastest Path and the Pragmatic Choice

PLAIN is the easiest to configure, but credentials travel in clear text so TLS is mandatory. On the exam you can generally treat PLAIN alone (SASL_PLAINTEXT) as an incorrect answer choice.

SCRAM (SCRAM-SHA-256/512) uses hashes stored securely on the brokers, so it cuts password-leak risk while remaining easy to operate. You attach user credentials to a user entity via kafka-configs.sh (stored securely in ZK on ZooKeeper-based clusters, and in the metadata quorum on KRaft clusters).

• A common production layout is EXTERNAL=SCRAM+TLS and INTERNAL=SCRAM (TLS optional but recommended)
• Picking SCRAM for inter-broker keeps things simple — create a dedicated user for the broker-to-broker connection
• Limit PLAIN to PoCs, isolated networks, or short-term use, and migrate to SCRAM as a rule

Typical configuration snippets (broker and client)

# server.properties（ブローカー）
listeners=INTERNAL://0.0.0.0:9092,EXTERNAL://0.0.0.0:9093
listener.security.protocol.map=INTERNAL:SASL_PLAINTEXT,EXTERNAL:SASL_SSL
inter.broker.listener.name=INTERNAL
sasl.enabled.mechanisms=PLAIN,SCRAM-SHA-512
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-512
# 必要に応じてリスナー単位で許可メカニズムを制限
listener.name.external.sasl.enabled.mechanisms=SCRAM-SHA-512

# （ブローカーの JAAS は別ファイルで指定: -Djava.security.auth.login.config=...）
# KafkaServer セクション例（SCRAM で inter-broker 用ユーザーを使用）
# KafkaServer {
#   org.apache.kafka.common.security.scram.ScramLoginModule required
#   username="broker" password="broker-secret";
# };

# ユーザー資格情報の登録例（実行ホストで）
# kafka-configs.sh --alter --entity-type users --entity-name alice \
#   --add-config 'SCRAM-SHA-512=[password=alice-secret]'
# kafka-configs.sh --alter --entity-type users --entity-name broker \
#   --add-config 'SCRAM-SHA-512=[password=broker-secret]'

# client.properties（SCRAM クライアント）
security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
  username="alice" password="alice-secret";

# client.properties（PLAIN クライアント：テスト用途）
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
  username="alice" password="alice-secret";

GSSAPI/Kerberos: The Enterprise Integration Standard

Kerberos (GSSAPI) is the first choice if you can integrate with an existing corporate KDC. It enables mutual authentication via ticket distribution and avoids managing clear-text passwords. In Kafka, you distribute a principal and keytab to every broker and client, and standardize sasl.kerberos.service.name (commonly kafka).

Use Krb5LoginModule in JAAS, and set useKeyTab=true and storeKey=true in the broker's KafkaServer section. Typical failure modes include mismatched principal.to.local short-name rules, clock skew, and reverse-DNS lookup failures.

• DNS and time synchronization (NTP) are mandatory. Clock skew is a leading cause of GSSAPI errors
• The principal must match the FQDN and realm (for example, kafka/[email protected])
• Clients can use a ticket cache (kinit), but brokers run more reliably with a keytab

OAUTHBEARER: Integrating with an External IdP and JWTs

OAUTHBEARER brings OAuth 2.0/OIDC token-based authentication into Kafka. Clients fetch an access token from a token endpoint and present it in the SASL initial response. The broker then runs a server callback (implementation class) to validate the token's signature, expiration, scopes, and so on.

Kafka itself provides token validation as a pluggable interface — the IdP endpoint and key handling (JWKS, issuer validation, and so on) are up to your implementation. The exam emphasizes that TLS is mandatory, that mechanisms should be isolated per listener, and that it is safer not to use OAUTHBEARER for inter-broker traffic.

• Client: configure sasl.login.callback.handler.class and sasl.oauthbearer.token.endpoint.url
• Broker: specify listener.name.<listener>.oauthbearer.sasl.server.callback.handler.class and implement validation
• Tokens are sensitive data. Never run OAUTHBEARER without TLS

Minimal client configuration example (OAUTHBEARER)

# client.properties（OAUTHBEARER）
security.protocol=SASL_SSL
sasl.mechanism=OAUTHBEARER
sasl.login.callback.handler.class=org.apache.kafka.common.security.oauthbearer.secured.OAuthBearerLoginCallbackHandler
sasl.oauthbearer.token.endpoint.url=https://idp.example.com/oauth2/token
# 必要に応じて clientId/clientSecret, scope など（LoginCallbackHandler 実装依存）

Listener Design and Mixing Mechanisms: Set a Pattern and Operate Safely

The standard production approach is to split listeners into external and internal: use a strong, TLS-required mechanism (SCRAM or OAUTHBEARER) on the external side, and an operationally light mechanism (SCRAM or GSSAPI) for internal and inter-broker traffic. Limit which mechanisms are exposed per listener with listener.name.<listener>.sasl.enabled.mechanisms to prevent misuse.

A broker can have multiple listeners, but inter.broker.listener.name is exactly one. So fix the inter-broker mechanism to a single choice and use a dedicated user (principal or SCRAM user) so that auditing and access control stay separated.

• External: EXTERNAL=SASL_SSL with SCRAM or OAUTHBEARER
• Internal: INTERNAL=SASL_PLAINTEXT or SASL_SSL with SCRAM or GSSAPI
• Point inter-broker at the INTERNAL listener; SCRAM or GSSAPI is the solid choice

Typical listener separation (external and internal)

Troubleshooting and Frequently Tested Points

When SCRAM throws Authentication failed, the usual causes are an unregistered user, a password mismatch, or a mechanism mismatch (for example, the client uses PLAIN while the broker is restricted to SCRAM). Check whether listener.name.<listener>.sasl.enabled.mechanisms is the restriction in play.

GSSAPI issues are most often KDC unreachable, clock skew, principal.to.local mapping mismatches, or reverse-DNS failures. Use klist, kvno, and nslookup to narrow it down, and re-verify the realm and KDC in krb5.conf. For OAUTHBEARER, turn up broker-side validation logs to pinpoint signing-key / JWKS fetch failures and aud/iss validation problems.

• Does sasl.mechanism match between client and broker?
• Did you create a dedicated inter-broker user (SCRAM) or principal (Kerberos)?
• Keep SASL_PLAINTEXT to a minimum. Always pick SASL_SSL for anything externally exposed

Frequently used diagnostic commands

# SASL/SCRAM: ブローカーの有効メカニズム確認（ログ）
# grep -i "Registered SASL mechanisms" server.log

# SCRAM: 登録ユーザーの確認（環境に応じて）
# kafka-configs.sh --describe --entity-type users --entity-name alice

# Kerberos: チケット/時刻/名前解決
# klist && date && nslookup broker1.example.com

# OAUTH: クライアント側でトークンを取得しデコードして検証
# curl -s -d 'grant_type=client_credentials' https://idp/oauth2/token | jq -r .access_token | cut -d. -f2 | base64 -d | jq

Check Your Understanding

Frequently Asked Questions