Can single Kafka producer produce messages to multiple topics and how?
Never tried it myself, but I guess you can. Since the code for producer and sending the record is (from here https://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html):
Producer<String, String> producer = new KafkaProducer<>(props);
for(int i = 0; i < 100; i++)
producer.send(new ProducerRecord<String, String>("my-topic", Integer.toString(i), Integer.toString(i)));
So, I guess, if you just write different topics in the ProducerRecord
, than it should be possible.
Also, here http://kafka.apache.org/081/documentation.html#producerapi it explicitly says that you can use a method send(List<KeyedMessage<K,V>> messages)
to write into multiple topics.
Comments
-
Shankar almost 2 years
I am just exploring
Kafka
, currently i am using Oneproducer
and One topic to produce messages and it is consumed by oneConsumer
. very simple.I was reading the Kafka page, the
new Producer API is thread-safe
and sharing single instance will improve the performance.Does it mean i can use single Producer to publish messages to multiple topics?
-
miguno over 7 yearsFWIW, the latest Kafka docs for the Producer API are at kafka.apache.org/documentation.html#producerapi (the link above is for the old 0.8.1 version of Kafka.
-
Jacek Laskowski over 7 yearsWhere exactly does the producerapi section says about the method?
-
miguno over 7 yearsThe producerapi section says: "Examples showing how to use the producer are given in the javadocs.". That page has a code example that shows how you can define the destination topic when sending a message.
-
Novemberland about 5 yearsThe matter of fact doing so meke the writing process of the producer slow. Has anybody checked the performance of the producer which writes on multiple topics?
-
Kuldeep Jain over 4 years@RadioLog, so can we produce message to multiple topics all having different jaas.conf (keytab and pricipal) ? Pls see my questions here: stackoverflow.com/q/58313628/948268