Broker Cluster, Replication, Leaders, Followers, and ISR
Learn how Kafka achieves durability and fault tolerance through distributed broker architecture, replication, and partition leadership.
Inside this chapter
- Why Kafka Runs as a Cluster
- Partition Leaders and Followers
- Replication Factor
- In-Sync Replica Set
- Failover and Leader Election
- Operational Perspective
Series navigation
Study the chapters in order for the clearest path from Kafka basics and local setup to stream processing, platform operations, cloud usage, and advanced event-driven architecture thinking. Use the navigation at the bottom to move smoothly through the full tutorial series.
Why Kafka Runs as a Cluster
Kafka is designed for distributed operation. A cluster with multiple brokers provides fault tolerance, scale, and operational flexibility. Students should understand that a single-node demo is useful for learning, but real Kafka strength comes from clustered deployment.
Partition Leaders and Followers
Each partition has a leader broker and may have follower replicas. Producers and consumers normally interact with the leader, while followers replicate the log for availability and recovery.
Replication Factor
The replication factor determines how many broker copies of a partition exist. Higher replication improves fault tolerance but increases storage and replication overhead.
In-Sync Replica Set
The ISR, or in-sync replica set, contains replicas that are caught up closely enough with the leader to be considered reliable for acknowledgement and failover purposes.
Failover and Leader Election
If a broker fails, Kafka can promote a suitable follower to leader so the partition remains available. This is a core part of Kafka’s durability story.
Operational Perspective
In a payments platform, losing access to critical event streams during node failure could disrupt order fulfilment and reporting. Replication strategy and failover behavior are therefore business-level concerns, not just infrastructure details.