Kafka Articles

101 articles on Kafka certification prep and technical concepts

All Databricks Snowflake Terraform Vault Kafka dbt NVIDIA Azure Google Cloud AWS

Kafka

101 articles

Kafka Topics & Partitions: Distribution Fundamentals (2026)

How Kafka topics and partitions enable scale — ordering guarantees, partition keys, replication. The foundation of every Kafka exam.

CCDAK Exam Guide: Confluent Certified Developer (2026)

Complete prep for the CCDAK exam — Producer/Consumer API, Streams, Connect, Schema Registry. Scope, weighting, and tactics.

CCAAK Exam Guide: Confluent Certified Administrator (2026)

Pass the CCAAK exam — cluster management, partitions, security, monitoring, tuning. Operations-focused certification.

Kafka Replicas & ISR: Fault Tolerance Explained (2026)

Replica placement, in-sync replicas (ISR), leader election. The core of Kafka's durability and high availability.

Kafka Offsets: Commit Modes & Consumer Position (2026)

Offset semantics — auto vs. manual commit, __consumer_offsets, common reset strategies. Frequently on CCDAK.

Kafka Producer Basics: Send, Ack, Batch, Compression (2026)

Producer fundamentals — send semantics, batch/linger, key partitioning, error handling. CCDAK Producer domain coverage.

Kafka Consumer Basics: Poll Loop & Subscription (2026)

Consumer fundamentals — poll loop, subscribe/assign, deserializers, offset management. CCDAK Consumer domain.

Kafka Consumer Groups: Parallelism & Rebalance (2026)

Consumer groups — partition assignment, rebalances, group coordinator. The backbone of Kafka consumer parallelism.

Kafka Partition Assignors: Range, RoundRobin, Sticky (2026)

Partition assignment strategies compared — when each helps, trade-offs, and how exam questions probe them.

Kafka Rebalance: Eager vs Cooperative (2026)

Rebalance protocols — eager vs. cooperative. Why cooperative is the default and what changes for consumers.

Cooperative Rebalance: Incremental Reassignment (2026)

Cooperative incremental rebalance internals — how partitions move incrementally, configuration, and operational impact.

Kafka Message Keys: Ordering & Co-Partitioning (2026)

Why keys matter — per-key ordering, partition co-location, hot-key risks. Patterns from production and exam.

Kafka acks Configuration: Durability vs Latency (2026)

acks=0/1/all explained — durability guarantees, latency trade-offs, interaction with min.insync.replicas.

Idempotent Producer: No-Duplicate Sends (2026)

Idempotent producer mechanics — producer ID, epoch, sequence numbers. Foundation for exactly-once.

Kafka Transactions: Multi-Partition Atomicity (2026)

Transactional API — initTransactions, sendOffsetsToTransaction, commit/abort. The path to exactly-once across partitions.

Kafka Exactly-Once (EOS): Guarantees & Configuration (2026)

EOS in Kafka — what's actually guaranteed, idempotent + transactional config, Streams EOS_V2. Frequently on CCDAK.

Kafka Retention Policy: time vs size, segment behavior (2026)

Retention policy mechanics — retention.ms, retention.bytes, segment rolling. How log size is actually controlled.

Kafka Log Compaction: Key-Based Snapshot (2026)

Log compaction — tombstones, segment lifecycle, when compaction beats time-based retention. Common in CCDAK.

Kafka cleanup.policy: delete, compact, and both (2026)

Topic cleanup policies — delete, compact, and the combined policy. Selection guide and operational implications.

Kafka Compression Comparison: gzip, snappy, lz4, zstd (2026)

Compression codec trade-offs — CPU vs. ratio, latency impact, choosing for batch size. Tuning patterns the exam tests.

Producer Batching & Linger: Throughput Tuning (2026)

batch.size and linger.ms interplay — when batching wins, latency cost, common misconfigurations.

min.insync.replicas: Durability Threshold (2026)

How min.insync.replicas interacts with acks=all, common safe defaults, failure modes when set too high.

Unclean Leader Election: Data Loss Trade-Off (2026)

Unclean leader election explained — when to disable, data loss vs. availability, real-world incidents.

Replication Factor: Choosing the Right Number (2026)

Replication factor selection — durability, cost, rack awareness. Patterns for production and exam.

Kafka Controller: Cluster Coordinator (2026)

Controller responsibilities, election, KRaft vs ZooKeeper differences. The brain of the Kafka cluster.

KRaft Mode: Kafka Without ZooKeeper (2026)

KRaft fundamentals — Raft-based metadata, migration from ZooKeeper, sizing controllers. The current standard.

ZooKeeper to KRaft Migration: Steps & Risks (2026)

Migrate from ZooKeeper to KRaft — preflight, rolling steps, rollback considerations. Required CCAAK reading.

Confluent Schema Registry: Subjects, Versions, Compatibility (2026)

Schema Registry fundamentals — subject naming, version evolution, compatibility levels. CCDAK schema domain.

Avro Schemas in Kafka: Evolution & Defaults (2026)

Avro for Kafka — schema evolution rules, default values, reader/writer schemas. The most common Kafka serialization.

Protobuf with Kafka & Schema Registry (2026)

Protobuf serialization in Kafka — proto file management, compatibility, Confluent integration patterns.

JSON Schema with Kafka & Schema Registry (2026)

JSON Schema validation in Kafka — Confluent's JSON Schema serializer, compatibility rules, when to choose over Avro.

Schema Registry Compatibility Modes: BACKWARD, FORWARD, FULL (2026)

Each compatibility mode explained — what changes are allowed, transitive variants, real evolution patterns.

Kafka Schema Evolution: Safe Field Changes (2026)

Safe schema evolution rules — adding fields, removing fields, renaming, defaults. Critical for production pipelines.

Schema Registry Subjects: Naming Strategies (2026)

Subject naming strategies — TopicName, RecordName, TopicRecordName. When each fits and gotchas.

Schema References: Composing Avro/Protobuf Schemas (2026)

Schema references for reuse and composition — registration, evolution, common architectural patterns.

Topic-Schema Binding: How They Actually Connect (2026)

How Kafka topics bind to Schema Registry subjects — serializer config, version selection, common pitfalls.

Schema Registry Security: Auth, RBAC, TLS (2026)

Lock down Schema Registry — auth providers, RBAC, TLS. Patterns for production-grade installs.

Kafka Connect: Distributed Data Movement (2026)

Connect architecture — workers, tasks, distributed vs. standalone, REST API. The path for source/sink connectors.

Kafka Connect Source Connectors: Ingest Patterns (2026)

Source connector internals — offset tracking, transformation, common JDBC and CDC patterns.

Kafka Connect Sink Connectors: Delivery Patterns (2026)

Sink connector internals — exactly-once delivery, error handling, target system back-pressure.

Connect: Distributed vs Standalone Mode (2026)

Distributed vs. standalone Connect — when each fits, failover behavior, REST API differences.

Connect SMTs: Single Message Transforms (2026)

SMT fundamentals — built-in transforms, predicates, common cleanup patterns. Stay before custom code.

Connect Converters: Avro, JSON, String, Protobuf (2026)

How Connect converters handle ser/deser — key vs. value converter, Schema Registry integration, gotchas.

Debezium CDC: Database Change Capture for Kafka (2026)

Debezium architecture, supported databases, snapshot vs. streaming, common operational patterns.

JDBC Source/Sink Connectors: Polling vs CDC (2026)

JDBC connector patterns — bulk vs. incrementing vs. timestamp modes, when to switch to Debezium.

Kafka Connect Dead Letter Queue (DLQ) (2026)

DLQ configuration — error topic, headers, reprocessing patterns. Required for production reliability.

Kafka Connect REST API Reference (2026)

Connect REST API — config management, status checks, task restart. Operational basics for CCAAK.

Kafka Streams: KStream and KTable Patterns (2026)

Kafka Streams fundamentals — KStream, KTable, GlobalKTable. The model behind every Streams app.

KStream vs KTable: Choosing the Right Abstraction (2026)

When to model as KStream vs. KTable — semantics, joins, aggregations. The most common Streams design decision.

Kafka Streams Stateful Operations: State Stores (2026)

Stateful processors — aggregate, reduce, count, joins. State store fundamentals and the changelog topic.

Kafka Streams Windowing: Tumbling, Hopping, Session (2026)

Window types explained — boundary semantics, grace period, suppress for emit control. CCSD coverage.

Kafka Streams State Stores: RocksDB & Changelog (2026)

State store internals — RocksDB, changelog topics, restoration. The tuning surface for stateful Streams apps.

Kafka Streams Joins: Inner, Left, Outer (2026)

Stream-stream, stream-table, table-table joins — semantics, co-partitioning, foreign-key joins.

Exactly-Once in Kafka Streams (EOS_V2) (2026)

EOS in Streams — what it actually guarantees, configuration, performance impact. Behavior on rebalance and restart.

ksqlDB: SQL on Kafka Streams (2026)

ksqlDB architecture — streams, tables, queries, materialized views. SQL-first stream processing on Kafka.

ksqlDB Push vs Pull Queries (2026)

Push vs. pull queries in ksqlDB — semantics, latency, use cases. The two query modes explained.

Kafka SASL Auth: PLAIN, SCRAM, OAUTHBEARER, GSSAPI (2026)

SASL mechanism comparison — configuration, security trade-offs, when each fits. CCAAK security domain.

Kafka SSL/TLS Encryption: Wire Security (2026)

Kafka TLS configuration — keystore, truststore, mutual TLS, common rollout patterns.

Kafka OAuth/OIDC Authentication (2026)

OAuth/OIDC support in Kafka — token issuance, validation, broker configuration. Modern auth for Kafka clusters.

Kafka ACLs: Topic and Group Authorization (2026)

Kafka ACL model — principals, resources, operations. Common policy patterns and CCAAK exam coverage.

Kafka Quotas: Producer, Consumer, Request Quotas (2026)

Quota mechanics — throttling behavior, scope, configuration. Prevent noisy neighbors at the broker.

Kafka Rack Awareness: AZ-Spread Replica Placement (2026)

Rack awareness configuration — broker.rack, replica placement, fetch from follower. Required for multi-AZ clusters.

Kafka Broker Sizing: Memory, Disk, Network (2026)

Broker capacity planning — heap, page cache, disk IOPS, network. Sizing guidance from production experience.

Kafka Monitoring: Key Metrics & Alerts (2026)

The critical Kafka metrics to monitor — under-replicated partitions, ISR shrink, consumer lag. CCAAK observation domain.

Kafka JMX Metrics Reference (2026)

JMX metric catalog — broker, producer, consumer, Streams. The names you need for monitoring queries.

Kafka Monitoring with Prometheus & Grafana (2026)

Prometheus exporters, common Grafana dashboards, alert rules. Standard observability stack for Kafka.

Tuning Kafka Log Retention: Storage Cost Control (2026)

Retention tuning patterns — size vs. time, per-topic overrides, segment behavior, infinite retention.

Kafka Cluster Upgrade: Rolling Restart Procedure (2026)

Safe Kafka cluster upgrades — inter.broker.protocol.version, log.message.format.version, rollback plan.

Kafka Disaster Recovery: Multi-Region Patterns (2026)

DR patterns — MirrorMaker 2, Cluster Linking, stretched cluster. Trade-offs and decision criteria.

MirrorMaker 2: Cross-Cluster Replication (2026)

MM2 architecture — topic and offset translation, prefix patterns, common operational gotchas.

Confluent Replicator: Enterprise Cross-Cluster Sync (2026)

Confluent Replicator vs. MM2 — feature differences, when each fits, licensing considerations.

Cluster Linking: Mirror Topics Without MM2 (2026)

Cluster Linking architecture — mirror topics, offset preservation, when it beats MirrorMaker 2.

Confluent Cloud: Managed Kafka Overview (2026)

Confluent Cloud capabilities — Stream Governance, Stream Sharing, billing. What you trade for the managed experience.

Confluent Cloud vs Self-Managed Kafka (2026)

Honest comparison — cost, ops burden, feature parity. The decision framework for greenfield deployments.

Kafka Tiered Storage: Hot/Cold Log Segments (2026)

Tiered storage architecture — remote storage manager, cost impact, supported configurations.

Stretched Kafka Cluster: Multi-AZ Synchronous Replication (2026)

Stretched cluster design — RTT requirements, replica placement, common pitfalls. Compared to MM2.

AWS MSK vs Confluent Cloud (2026)

MSK and Confluent Cloud compared — features, cost, vendor lock-in, operational maturity.

Kafka Private Networking: VPC Peering, Private Link (2026)

Private networking options for managed Kafka — VPC peering, AWS PrivateLink, Azure Private Link.

Kafka Cost Optimization: Brokers, Storage, Network (2026)

Practical Kafka cost reduction — partition consolidation, tiered storage, retention right-sizing, instance choice.

Event Sourcing with Kafka: Patterns & Trade-Offs (2026)

Event sourcing on Kafka — append-only design, snapshots, replay. When Kafka is the right event store.

CQRS with Kafka: Command/Query Separation (2026)

CQRS implementation patterns using Kafka — command topics, projection consumers, common gotchas.

Transactional Outbox Pattern with Kafka (2026)

Outbox pattern with Kafka — dual-write avoidance, Debezium for outbox, common implementation choices.

Saga Pattern with Kafka: Distributed Transactions (2026)

Saga pattern implementation — choreography vs. orchestration, compensation, common architectural choices.

Kafka Error Handling Patterns (2026)

Production-grade error handling — retry topics, DLQ, poison pill detection, manual reprocessing.

Handling Kafka Poison Pills (2026)

Poison-pill detection and handling — deserialization errors, malformed payloads, skip vs. quarantine.

Kafka Consumer Lag: Diagnose & Reduce (2026)

Consumer lag diagnosis — burndown rate, partition skew, rebalance impact. Patterns for resolving lag.

Kafka Producer Tuning: Throughput & Latency (2026)

Producer tuning checklist — batch, linger, compression, in-flight requests. Real production trade-offs.

Kafka Topic Naming Conventions That Scale (2026)

Topic naming strategies — environment, domain, version. Patterns that survive years of evolution.

Kafka Capacity Planning: Brokers, Disk, Network (2026)

Capacity planning for Kafka — throughput-driven sizing, headroom, growth modeling. CCAAK operations.

Kafka Audit Logs: Authorization & Access Tracking (2026)

Kafka audit log sources — Confluent audit logs, broker logs, CloudWatch/CloudTrail for managed. Compliance patterns.

Kafka Engineer Career & Salary (2026)

Career value of Kafka certifications — market demand, salary, role progression for CCDAK and CCAAK holders.

Confluent Kafka Certifications: CCDAK & CCAAK Explained (2026)

Compare the two Confluent Kafka certifications — scope, difficulty, prerequisites, and which to take first.

Kafka Cert Difficulty: CCDAK vs CCAAK Compared (2026)

Honest difficulty comparison between CCDAK and CCAAK — study time, prerequisites, exam style differences.

Kafka Certification Cost: Fees, Vouchers, Retakes (2026)

Confluent certification fees, voucher options, retake rules. Total spend planning for CCDAK + CCAAK.

Confluent Kafka Exam: Test-Day Experience (2026)

What to expect on CCDAK/CCAAK test day — online proctoring, time pressure, the question types that surprise candidates.

Confluent Cloud Free Tier: Hands-On Learning Setup (2026)

Use Confluent Cloud's free tier for cert study — what's free, common practice scenarios, transition to paid.

Kafka Streams Materialized Views: KTable as Cache (2026)

Materialize stream state for low-latency reads — KTable + interactive queries, common architectural patterns.

Confluent RBAC: Role-Based Access Beyond ACLs (2026)

Confluent's RBAC model — roles, role bindings, scopes. Where it goes beyond Kafka ACLs.

Confluent Kafka Sample Questions: Format Walkthrough (2026)

Real-format CCDAK/CCAAK sample questions with detailed answer reasoning. Calibrate before paid prep.

Confluent Kafka Study Guide: CCDAK + CCAAK Plan (2026)

Week-by-week study plans for CCDAK and CCAAK — official materials, hands-on, mock exam strategy.

Kafka Cert Study Time: CCDAK vs CCAAK (2026)

Realistic study-time estimates for CCDAK and CCAAK based on background. What changes with prior Kafka experience.