Unable to start Zookeeper server in Apache Kafka

Question

I am in kafka_home/bin/windows (as suggested here). Then I get the same problem as here, where it suggests to launch the program from the kafka home. I get this error when launching from the windows folder:

INFO Reading configuration from: config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2014-08-21 11:53:55,748] FATAL Invalid config, exiting abnormally (org.apache.zookeeper.server.quorum.QuorumPeerMain)
org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error processing config/zookeeper.properties
    at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:110)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:99)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:76)
Caused by: java.lang.IllegalArgumentException: config/zookeeper.properties file is missing
    at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:94)
    ... 2 more
Invalid config, exiting abnormally

Does anybody know how to solve this?

Solution 1

The default location of the configuration file is specified with a relative path, config/zookeeper.properties.

The directory tree of Kafka looks somewhat like this:

bin/
  |
  +-- zookeeper-server-start.sh
  |
  +-- windows/
        |
        +-- zookeeper-server-start.bat
config/
  |
  +-- zookeeper.properties

So if you are literally inside the bin/windows/ directory and then run zookeeper-server-start.bat from there, the ZK startup script will effectively look for its configuration at bin/windows/config/zookeeper.properties, which doesn't exist.

Take a look at the existing answer (Apache Kafka error on windows - Couldnot find or load main class QuorumPeerMain) for the correct command to launch the ZK startup script, where the example command line explicitly defines where to find the configuration file:

$ cd bin/windows
$ zookeeper-server-start.bat ../../config/zookeeper.properties

# The below will NOT work (explanation is above)
$ cd bin/windows
$ zookeeper-server-start.bat

Solution 2

The problem here is the windows bat scripts are not properly tested. Also the documentation has not mentioned the prerequisites like installing scala, java and setting the ENV variables that zookeer and Kafka depends up on.

Here is what I did to resolve the problem and then zookeeper , kafka server has started properly, running fine. 1. Install scala and set the env variables SCALA_HOME, SCALA_VERSION , SCALA_BINARY_VERSION. Add the SCALA_HOME\bin to PATH. 2. Install java and set JAVA_HOME and add JAVA_HOME\bin to PATH. 3. CORRECT THE FILEs \bin\windows\zookeeper-server-start.bat AND \bin\windows\kafka-run-class.bat BY REPLACING ALL THE ENTRIES THE STRING %~dp0 WITH %CD% 4. CD AND RUN THE COMMAND TO START ZOOKEER : bin\windows\zookeeper-server-start.bat config\zookeeper.properties eg -> D:\apps\kafka_2.12-0.11.0.1>bin\windows\zookeeper-server-start.bat config\zookeeper.properties Zookeeper shoould start properly 5 CD AND RUN THE COMMAND TO START KAFKA SERVER : bin\windows\kafka-server-start.bat config\server.properties eg -> D:\apps\kafka_2.12-0.11.0.1>bin\windows\kafka-server-start.bat config\server.properties kafka server/broker should start properly 6. We can then test createing a topic and list the topics as mentioned in the kafka quick start guide her : https://kafka.apache.org/quickstart

D:\apps\kafka_2.12-0.11.0.1>bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test2
Created topic "test2".

D:\apps\kafka_2.12-0.11.0.1>bin\windows\kafka-topics.bat --list --zookeeper localhost:2181
test
test2

enter image description here

Solution 3

I Find answers from @MiChael G Noll much helpful. I would advise following with respect to windows:

Start Zookeeper by [Remember to navigate to root of kafka-directory. The contents of root directory in windows is bin, config, libs, logs etc]:

.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties

Start kafka Server by .\bin\windows\kafka-server-start.bat .\config\server.properties

While you are doing all this, notice that for windows we navigated to .bin\windows from where we called the respective .bat files.

Similarly when you want to create topics or partitions remember that in your cmd terminal you are calling these .bat files from correct location[All while your root is kafka directory hence the use of .\bin The dot represents the root directory, in this root where kafka is stored].

Other relevant links and commands can be found here: https://www.javaer101.com/en/article/12057273.html

Also I would like to point out for beginners like me- the parallel between Linux and windows execution

In Linux to write message to console you call: kafka-console-producer.sh as follows: .sh is the extension for shell execution in linux

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test

It's parallel in windows is observable through .bat extension, which is executable script on windows console[cmd]: kafka-console-producer.sh

.\bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic test

Also notice the location of .bat and .sh in kafka. in windows it is located at bin\windows while in Linux it is available directly under bin for the kafka directory. Make sure to call accordingly !

Comments

[sirdan]

In the first line of the quickstart guide in the Apache Kafka website, it is clearly written that for Windows users you have to be in the root folder, then execute bin\windows\... and the rest of the command. Anyway, thanks.

[miguno]

Yes, but in your question above you explicitly said that you did something different: you were not in the root folder and then executed bin\windows\zookeeper-server-start.bat (which would have worked), but instead you were in the wrong directory: "I am in kafka_home/bin/windows".

[sirdan]

I know, my comment was to clarify that I figured out a mistake of mine.