Kafka: How to achieve Round Robin Partition in Kafka

Question

I am new in kafka. My requirement is, I have two partition for example Partition-0 and Partition-1 and I have list of values which also contains KEY value. I want to store data according to my key like key-1 will goes to Partition-0, key-2 will goes to Partition-1. With old API there is way to achieve like we need to implement Partition interface but how I can do this with new API. thank you

Solution 1

With the new producer you may also implement Partitioner interface (https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/producer/Partitioner.java) to achieve round-robin distribution.

You can use DefaultPartitioner for reference - https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/producer/internals/DefaultPartitioner.java

Solution 2

If you want round-robin behaviour, just do not pass key when writing to Producer and DefaultPartitioner will do the job for you. You do not need to write a custom implementation. From the javadocs:

/**
 * The default partitioning strategy:
 * <ul>
 * <li>If a partition is specified in the record, use it
 * <li>If no partition is specified but a key is present choose a partition based on a hash of the key
 * <li>If no partition or key is present choose a partition in a round-robin fashion
 */