zipimport.ZipImportError: can't decompress data; zlib not available in spark(ubuntu 16.04 LTS)

Question

I am trying to use the countbyvalue() function and it gave me the following error:

File "/usr/local/lib/python3.5/runpy.py", line 174, in _run_module_as_main
      mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
    File "/usr/local/lib/python3.5/runpy.py", line 109, in _get_module_details
      __import__(pkg_name)
  zipimport.ZipImportError: can't decompress data; zlib not available

I see solutions for this as to configure python again but while configuring it says --with-zlib-dir is an unrecognized option.

I am not sure if it would change any, on proceeding further, even it couldn't recognize the option

How can I solve it?

Solution

Answer to a similar question raised here suggested installing zlib1g-dev in order to solve this error:

sudo apt-get install zlib1g-dev

More info about zlib1g-dev

Note: If you have a spark cluster, you might need to install the package on every node in the cluster.

Comments

[David Foerster]

How exactly did you or are you trying to install Python 3.5?

[Jiteesh]

I have downloaded the package and configured it, install and make install

[David Foerster]

We need more detail. What exactly did you download? How exactly did you configure it and what was the output of the configuration run? If you didn't record it, please try to replicate it. Could you please edit your post when you want to clarify something or add information? It’s best to have everything relevant in one place. Additionally, comments may be deleted for various reasons. Thanks.

[Jiteesh]

Hey David. it worked out. Thank you for the response. Yes I will be including the details from next time. Appreciate your time.

[Jiteesh]

Hello Yaron, Thank you for the response. Appreciate it. I am using stand alone mode for spark and using it in jupyter notebook. I have already installed it but its the same error I am facing.

[Yaron]

Did you try to restart after you installed zlib package? Also, it might be a problem with the iPython/spark installation

[Jiteesh]

Yes. I have, just now, installed the package and restarted the system. Same error though

[Jiteesh]

I have also tried to configure zlib by editing comments in setup file in Modules. still not able to make any changes

[Jiteesh]

It seems what you thought is right. The problem is with ipython. unable to recognize zlib. in terminal the commands like countbyvalue are working fine. Can you suggest how to fix this. in ipython(jupyter notebook)... Thanks in advance

[Elder Geek]

Direct link to answer: unix.stackexchange.com/a/334103/67643