Python error: Cannot import name KafkaConsumer

Question

I installed the kafka-python package for Python. I have a kafka producer and consumer running. I want the python code to read the kafka topic and print out the messages.

My python code is below:

import sys
from kafka import KafkaConsumer

def kafkatest():
    print "Step 1 complete"
    consumer=KafkaConsumer('test',bootstrap_servers=['localhost:9092'])
    for message in consumer:
        print "Next message"
        print message


if __name__=="__main__":
    kafkatest()

I get the following error:

C:\Python27>python.exe kafka.py Traceback (most recent call last):   File "kafka.py", line 2, in <module>
    from kafka import KafkaConsumer   File "C:\Python27\kafka.py", line 2, in <module>
    from kafka import KafkaConsumer ImportError: cannot import name KafkaConsumer

Any suggestions on what I am missing here?

Solution

You have named your file kafka.py. Now when you run it, python encounters the following statement inside it:

from kafka import KafkaConsumer

So it must find a module called kafka. How does it known where to look? Well it looks in the directories on sys.path, the first of which is initialised to be the directory of the input script.

Thus, the file it finds and attempts to look in is your module called kafka, which does not define KafkaConsumer and hence the error.

Effectively, you are importing yourself.

Moral: Stop naming your scripts with identical names to system libraries or external packages that you're hoping to use.