sdlc.dev

Kafka path · Keeping the data · Topic 8 of 12

KRaft: metadata without ZooKeeper

Not started

A Kafka cluster has two kinds of state: the records in partitions, and the metadata about them (which topics exist, where each partition's replicas are, who leads, who is in sync, configuration and ACLs). For most of Kafka's history the metadata lived in ZooKeeper, a separate system to deploy and secure. KRaft moves it into Kafka: a small set of controller nodes replicate a metadata log among themselves with a Raft-style consensus protocol, and brokers follow that log. KRaft became production-ready in 3.3, and Kafka 4.0 removed ZooKeeper mode entirely.

Why it matters: Every Kafka 4.x cluster runs KRaft; ZooKeeper support is gone. Knowing what the controllers do explains how leaders are elected, what a quorum loss does, and why upgrading an old cluster takes a migration first.

  1. 1Controllers own the metadata

    Topics, partitions, leaders, ISRs, configs and ACLs live in a metadata log, __cluster_metadata, replicated among controllers with the Raft-based KRaft protocol.

  2. 2A majority of controllers must be up

    Run 3 controllers to tolerate one failure, or 5 to tolerate two. Without a majority, no metadata can change, including leader elections.

  3. 3Brokers follow the log

    Brokers fetch the metadata log and apply it, so each has a current local copy instead of asking a separate system.

  4. 4ZooKeeper was removed in 4.0

    Kafka 4.0 (March 2025) supports only KRaft. ZooKeeper clusters must migrate to KRaft on 3.x (3.9 is the last 3.x release) before upgrading.

1

How it works

The explanation, with examples you can run.

Roles

# a dedicated controllerprocess.roles=controllernode.id=1listeners=CONTROLLER://controller1.example.com:9093controller.listener.names=CONTROLLERcontroller.quorum.bootstrap.servers=controller1.example.com:9093,controller2.example.com:9093,controller3.example.com:9093

process.roles can be broker, controller, or broker,controller (combined). Combined servers are convenient for development and small clusters; the documentation's caution is that a controller sharing a machine with a broker is less isolated from broker load. Every node is formatted once with a cluster id (kafka-storage.sh random-uuid, then kafka-storage.sh format); Kafka no longer formats storage automatically.

The quorum

On the three-node test cluster (combined mode):

bin/kafka-metadata-quorum.sh --bootstrap-server kafka1:9092 describe --status
ClusterId:              anD7Csf_SH-oHi5O2uMR7wLeaderId:               1LeaderEpoch:            1HighWatermark:          42MaxFollowerLag:         0MaxFollowerLagTimeMs:   0CurrentVoters:          [{"id": 1, ...}, {"id": 2, ...}, {"id": 3, ...}]

One controller is the active leader of the metadata log; the others are hot standbys with the log already applied, so failover is quick. A majority of voters must be alive: 3 controllers tolerate one failure, 5 tolerate two. Lose the majority and existing partition leaders keep serving, but nothing in the metadata can change, which includes electing new partition leaders when a broker fails.

The features command shows what the cluster has switched on. On 4.1.0:

Feature: eligible.leader.replicas.version  ... FinalizedVersionLevel: 1Feature: group.version                     ... FinalizedVersionLevel: 1Feature: kraft.version                     ... FinalizedVersionLevel: 0Feature: metadata.version                  ... FinalizedVersionLevel: 4.1-IV1Feature: share.version                     ... FinalizedVersionLevel: 0Feature: transaction.version               ... FinalizedVersionLevel: 2

group.version 1 enables the new consumer protocol on the broker, transaction.version 2 the strengthened transaction protocol, and kraft.version 0 means a static controller set; Kafka 4.1 added an upgrade to dynamic controller membership, so controllers can be added and removed.

Upgrading from ZooKeeper

A ZooKeeper-mode cluster cannot be upgraded straight to 4.x. It must first be migrated to KRaft on a 3.x release that supports migration (3.9, the last 3.x release, is the usual choice), with the controllers copying the metadata from ZooKeeper and the brokers then restarted in KRaft mode. Some configurations and metrics that only existed for ZooKeeper are gone. Java requirements also changed in 4.0: brokers need Java 17, clients Java 11.

2

Check yourself

3 questions. Get them all right to finish this topic.

  1. 1.A cluster has 3 dedicated controllers. Two of them fail. What still works?
  2. 2.Why can't you upgrade a ZooKeeper-based 3.6 cluster straight to Kafka 4.0?
  3. 3.What is a combined server (process.roles=broker,controller)?

0 of 3 right so far. Get all three to finish this topic.

Go deeper