can't connect to kafka server

Question

I try to use Kafka with version 0.9.0 with port 9092. If I use telnet, I successfully connect to this address, but I fail to connect to Kafka server with Java API

Here is my Java example exactly use the official supplied documentation:

Properties props = new Properties();
props.put("bootstrap.servers", "192.168.174.128:9092");
props.put("acks", "all");
props.put("retries", 0);
props.put("batch.size", 16384);
props.put("linger.ms", 1);
props.put("buffer.memory", 33554432);
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

Producer<String, String> producer = new KafkaProducer(props);
for(int i = 1; i < 10; i++) ｛
  producer.send(new ProducerRecord<String, String>("test", Integer.toString(i), Integer.toString(i)));

｝

producer.close();

I use vmware virtual machine to build my Kafka server and Zookeeper. My virtual machine IP address is 192.168.174.128. I successfully create and consume message with Kafka client.

And here is the exception full stack

2016-01-03 09:44:38.049 [kafka-producer-network-thread | producer-1] DEBUG [] [] Selector - Connection with localhost/127.0.0.1 disconnected
java.net.ConnectException: Connection refused: no further information
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_51]
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) ~[na:1.7.0_51]
    at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:54) ~[kafka-clients-0.9.0.0.jar:na]
    at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:79) ~[kafka-clients-0.9.0.0.jar:na]
    at org.apache.kafka.common.network.Selector.poll(Selector.java:274) ~[kafka-clients-0.9.0.0.jar:na]
    at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:270) [kafka-clients-0.9.0.0.jar:na]
    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:216) [kafka-clients-0.9.0.0.jar:na]
    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:141) [kafka-clients-0.9.0.0.jar:na]
    at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
2016-01-03 09:44:38.049 [kafka-producer-network-thread | producer-1] DEBUG [] [] NetworkClient - Node 0 disconnected.
2016-01-03 09:44:38.078 [kafka-producer-network-thread | producer-1] DEBUG [] [] NetworkClient - No node found. Trying previously-seen node with ID 0
2016-01-03 09:44:38.078 [kafka-producer-network-thread | producer-1] DEBUG [] [] NetworkClient - Give up sending metadata request since no node is available
2016-01-03 09:44:38.099 [kafka-producer-network-thread | producer-1] DEBUG [] [] NetworkClient - Initiating connection to node 0 at localhost:9092.
2016-01-03 09:44:38.681 [kafka-producer-network-thread | producer-1] DEBUG [] [] Metrics - Removed sensor with name connections-closed:client-id-producer-1
2016-01-03 09:44:38.681 [kafka-producer-network-thread | producer-1] DEBUG [] [] Metrics - Removed sensor with name connections-created:client-id-producer-1
2016-01-03 09:44:38.681 [kafka-producer-network-thread | producer-1] DEBUG [] [] Metrics - Removed sensor with name bytes-sent-received:client-id-producer-1
2016-01-03 09:44:38.681 [kafka-producer-network-thread | producer-1] DEBUG [] [] Metrics - Removed sensor with name bytes-received:client-id-producer-1
2016-01-03 09:44:38.682 [kafka-producer-network-thread | producer-1] DEBUG [] [] Metrics - Removed sensor with name bytes-sent:client-id-producer-1
2016-01-03 09:44:38.682 [kafka-producer-network-thread | producer-1] DEBUG [] [] Metrics - Removed sensor with name select-time:client-id-producer-1
2016-01-03 09:44:38.682 [kafka-producer-network-thread | producer-1] DEBUG [] [] Metrics - Removed sensor with name io-time:client-id-producer-1
2016-01-03 09:44:38.682 [kafka-producer-network-thread | producer-1] DEBUG [] [] Metrics - Removed sensor with name node--1.bytes-sent
2016-01-03 09:44:38.683 [kafka-producer-network-thread | producer-1] DEBUG [] [] Metrics - Removed sensor with name node--1.bytes-received
2016-01-03 09:44:38.683 [kafka-producer-network-thread | producer-1] DEBUG [] [] Metrics - Removed sensor with name node--1.latency
2016-01-03 09:44:38.683 [kafka-producer-network-thread | producer-1] DEBUG [] [] Metrics - Removed sensor with name node-0.bytes-sent
2016-01-03 09:44:38.683 [kafka-producer-network-thread | producer-1] DEBUG [] [] Metrics - Removed sensor with name node-0.bytes-received
2016-01-03 09:44:38.683 [kafka-producer-network-thread | producer-1] DEBUG [] [] Metrics - Removed sensor with name node-0.latency
2016-01-03 09:44:38.683 [kafka-producer-network-thread | producer-1] DEBUG [] [] Sender - Shutdown of Kafka producer I/O thread has completed.
2016-01-03 09:44:38.683 [main] DEBUG [] [] KafkaProducer - The Kafka producer has closed.

Solution 1

I had this problem. To resolve this problem you should uncomment following config in server.properties file and fill it with IP of your zookeeper.

advertised.host.name = zookeeper IP

As kafka document says this config is

Hostname to publish to ZooKeeper for clients to use

Because in new kafka consumer (0.9) consumer asks zookeeper IP from broker.

Solution 2

This happened to me due to a different issue:

The 0.10.1.0 Java API seems not to be compatible with a 0.10.0.1 server.

I hope that can solve someone else's problem.

Comments

[Thanos]

Having the same issue with a 0.10.1.0 broker and the 0.10.1.0 apis, any luck for you?

[ssice]

I am using the 0.10.0.1 stack currently, but if get to update it, I'll reply back. I hope someone else who can see this topic can help you.