How to load balance the Kafka Leadership?

Question

My kafka version is kafka_2.9.2-0.8.1.1. I have two brokers in the cluster, 4 topics and each topic has 4 partitions.

When I run

sh kafka-topics.sh --describe --zookeeper rhost:2181

for all the topics/partitions, I see broker 1 as Leader.

How can I load balance the leader?

For example, for topic 1 and topic 2 have broker 1 as leader and for topic 3 and topic 4 have broker 2 as leader.

Solution 1

The partitions should be automatically rebalanced, since the default value of the broker configuration parameter auto.leader.rebalance.enable is true. (see documentation)

However, by default this rebalance occurs every 5 minutes, as defined by the leader.imbalance.check.interval.seconds parameter. If you wish this to occur more frequently, you will have to modify this parameter.

Solution 2

You can use the Preferred Replica Leader Election Tool:

sh kafka-preferred-replica-election.sh --zookeeper zklist

This guarantees that the leadership load across the brokers in a cluster is evenly balanced.

Solution 3

I know it is a bit late, maybe you already have the answer, but to balance leaders, first you need to make brokers equally preferred between all partitions, for a broker to be a "preferred leader" it has two criteria, first, it needs to be in sync replica, second, it has to be the first element on the replicas list. So, if you have a small enough number of topics/partitions, you can do that manually, it would be easier, otherwise you need to reassign partitions with distributing the first element (preferred replica) among all brokers, then kick off preferred leader election tool which will make sure that the preferred leader is actually the leader.

Comments

[kumar]

,I have enabled this "auto.leader.rebalance.enable=true" but it's not working.using 2 kafka with brokerId 0 and 1.if broker 0 stopped then broker 1 not act as a load balancing.Error I got is "java.net.ConnectException: Connection refused"

[Jeff Widman]

According to the current docs, the auto.leader.rebalance.enable value defaults to true, not false

[kisna]

Only v0.8 suggests setting it to true, 0.10 and above have that set to true by default. See Balancing leadership in v0.8 docs: kafka.apache.org/081/documentation.html Either way, when you describe a topic: leader "X" and first replica in the list (Y, Z, X) which has the preferred replica order should be same for all partitions for correct leader elections, some times this can be missing and sometimes, same node can be leader for many partitions - i.e., leaders in ISR can also be not balanced. You can always run the preferred replica election script to fix this problem.