Hadoop: Python client driver for HiveServer2 fails to install
26,549
Solution 1
pyhs2 is great, it packages several needed python packages together for a dbapi, among them sasl. Note if you pip install sasl you'll get the same error.
sasl (the python package) depends on libsasl2-dev (on a Debian/Ubuntu machine). The error you're seeing is that the compiler can't find the libraries that sasl is supposed to wrap.
If you sudo apt-get install libsasl2-dev and rerun, it will likely be fixed.
edit: Based on your comment, here's a robust way of getting this to work:
curl -O -L ftp://ftp.cyrusimap.org/cyrus-sasl/cyrus-sasl-2.1.26.tar.gz
tar xzf cyrus-sasl-2.1.26.tar.gz
cd cyrus-sasl-2.1.26
./configure && make install
Solution 2
Pyhs2 has the following dependencies:
gcc-c++
python-devel.x86_64
cyrus-sasl-devel.x86_64
So, just run this
sudo yum install gcc-c++ python-devel.x86_64 cyrus-sasl-devel.x86_64
sudo pip install pyhs2
Solution 3
If you have an error in Windows when installing "sasl" with
pip install sasl
you can install pre-compiled packet.
Download sasl packet for python36 in x64 from link: https://www.lfd.uci.edu/~gohlke/pythonlibs/#sasl
Then run pip install sasl-0.2.1-cp36-cp36m-win_amd64.whl
Author by
dokondr
Updated on July 27, 2022Comments
-
dokondr almost 2 years
I am trying to install a Python client driver for HiveServer2: https://cwiki.apache.org/confluence/display/Hive/Setting+Up+HiveServer2#SettingUpHiveServer2-PythonClientDriver
Installations says that: "A Python client driver for HiveServer2 is available at https://github.com/BradRuderman/pyhs2 It includes all the required packages such as SASL and Thrift wrappers".
Yet, running
pip install pyhs2fails on SASL compilation (see below). I have Hadoop 2.2.0 installed and working on localhost. Please help, to install Python client.[root@localhost /]# pip install pyhs2 Requirement already satisfied (use --upgrade to upgrade): pyhs2 in /usr/lib/python2.6/site-packages Downloading/unpacking sasl (from pyhs2) Downloading sasl-0.1.3.tar.gz Running setup.py (path:/tmp/pip_build_root/sasl/setup.py) egg_info for package sasl Downloading/unpacking thrift (from pyhs2) Downloading thrift-0.9.1.tar.gz Running setup.py (path:/tmp/pip_build_root/thrift/setup.py) egg_info for package thrift Installing collected packages: sasl, thrift Running setup.py install for sasl building '_saslwrapper' extension gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -Isasl -I/usr/include/python2.6 -c sasl/saslwrapper.cpp -o build/temp.linux-x86_64-2.6/sasl/saslwrapper.o sasl/saslwrapper.cpp:21:23: error: sasl/sasl.h: No such file or directory sasl/saslwrapper.cpp:51: error: ‘sasl_interact_t’ has not been declared sasl/saslwrapper.cpp:54: error: ‘sasl_conn_t’ has not been declared sasl/saslwrapper.cpp:54: error: ‘sasl_secret_t’ has not been declared sasl/saslwrapper.cpp:57: error: ISO C++ forbids declaration of ‘sasl_conn_t’ with no type sasl/saslwrapper.cpp:57: error: expected ‘;’ before ‘*’ token sasl/saslwrapper.cpp:58: error: ‘sasl_callback_t’ does not name a type sasl/saslwrapper.cpp:71: error: ISO C++ forbids declaration of ‘sasl_secret_t’ with no type sasl/saslwrapper.cpp:71: error: expected ‘;’ before ‘*’ token sasl/saslwrapper.cpp: In constructor ‘saslwrapper::ClientImpl::ClientImpl()’: sasl/saslwrapper.cpp:35: error: class ‘saslwrapper::ClientImpl’ does not have any field named ‘conn’ sasl/saslwrapper.cpp:35: error: class ‘saslwrapper::ClientImpl’ does not have any field named ‘secret’ sasl/saslwrapper.cpp: In destructor ‘saslwrapper::ClientImpl::~ClientImpl()’: sasl/saslwrapper.cpp:36: error: ‘conn’ was not declared in this scope sasl/saslwrapper.cpp:36: error: ‘sasl_dispose’ was not declared in this scope sasl/saslwrapper.cpp:36: error: ‘conn’ was not declared in this scope sasl/saslwrapper.cpp: In member function ‘void saslwrapper::ClientImpl::lastCallback()’: sasl/saslwrapper.cpp:49: error: ‘SASL_CB_LIST_END’ was not declared in this scope sasl/saslwrapper.cpp: In member function ‘bool saslwrapper::ClientImpl::init()’: sasl/saslwrapper.cpp:83: error: ‘sasl_client_init’ was not declared in this scope sasl/saslwrapper.cpp:84: error: ‘SASL_OK’ was not declared in this scope sasl/saslwrapper.cpp:85: error: ‘sasl_errstring’ was not declared in this scope sasl/saslwrapper.cpp:91: error: ‘SASL_CB_GETREALM’ was not declared in this scope sasl/saslwrapper.cpp:93: error: ‘SASL_CB_USER’ was not declared in this scope sasl/saslwrapper.cpp:94: error: ‘SASL_CB_AUTHNAME’ was not declared in this scope sasl/saslwrapper.cpp:97: error: ‘SASL_CB_PASS’ was not declared in this scope sasl/saslwrapper.cpp:99: error: ‘SASL_CB_PASS’ was not declared in this scope sasl/saslwrapper.cpp:107: error: ‘SASL_NEED_PROXY’ was not declared in this scope sasl/saslwrapper.cpp:109: error: ‘callbacks’ was not declared in this scope sasl/saslwrapper.cpp:109: error: ‘conn’ was not declared in this scope sasl/saslwrapper.cpp:109: error: ‘sasl_client_new’ was not declared in this scope sasl/saslwrapper.cpp:110: error: ‘SASL_OK’ was not declared in this scope sasl/saslwrapper.cpp:111: error: ‘sasl_errstring’ was not declared in this scope sasl/saslwrapper.cpp:115: error: ‘sasl_security_properties_t’ was not declared in this scope sasl/saslwrapper.cpp:115: error: expected ‘;’ before ‘secprops’ sasl/saslwrapper.cpp:117: error: ‘secprops’ was not declared in this scope sasl/saslwrapper.cpp:124: error: ‘SASL_SEC_PROPS’ was not declared in this scope sasl/saslwrapper.cpp:124: error: ‘sasl_setprop’ was not declared in this scope sasl/saslwrapper.cpp:125: error: ‘SASL_OK’ was not declared in this scope sasl/saslwrapper.cpp:127: error: ‘sasl_dispose’ was not declared in this scope sasl/saslwrapper.cpp:133: error: ‘SASL_AUTH_EXTERNAL’ was not declared in this scope sasl/saslwrapper.cpp:134: error: ‘SASL_OK’ was not declared in this scope sasl/saslwrapper.cpp:136: error: ‘sasl_dispose’ was not declared in this scope sasl/saslwrapper.cpp:141: error: ‘SASL_SSF_EXTERNAL’ was not declared in this scope sasl/saslwrapper.cpp:142: error: ‘SASL_OK’ was not declared in this scope sasl/saslwrapper.cpp:144: error: ‘sasl_dispose’ was not declared in this scope sasl/saslwrapper.cpp: In member function ‘bool saslwrapper::ClientImpl::setAttr(const std::string&, const std::string&)’: sasl/saslwrapper.cpp:163: error: ‘secret’ was not declared in this scope sasl/saslwrapper.cpp:164: error: ‘sasl_secret_t’ was not declared in this scope sasl/saslwrapper.cpp:164: error: expected primary-expression before ‘)’ token sasl/saslwrapper.cpp:164: error: expected ‘;’ before ‘malloc’ sasl/saslwrapper.cpp: In member function ‘bool saslwrapper::ClientImpl::start(const std::string&, saslwrapper::output_string&, saslwrapper::output_string&)’: sasl/saslwrapper.cpp:199: error: ‘sasl_interact_t’ was not declared in this scope sasl/saslwrapper.cpp:199: error: ‘prompt’ was not declared in this scope sasl/saslwrapper.cpp:205: error: ‘conn’ was not declared in this scope sasl/saslwrapper.cpp:205: error: ‘sasl_client_start’ was not declared in this scope sasl/saslwrapper.cpp:206: error: ‘SASL_INTERACT’ was not declared in this scope sasl/saslwrapper.cpp:208: error: ‘SASL_INTERACT’ was not declared in this scope sasl/saslwrapper.cpp:209: error: ‘SASL_OK’ was not declared in this scope sasl/saslwrapper.cpp:209: error: ‘SASL_CONTINUE’ was not declared in this scope sasl/saslwrapper.cpp: In member function ‘bool saslwrapper::ClientImpl::step(const std::string&, saslwrapper::output_string&)’: sasl/saslwrapper.cpp:222: error: ‘sasl_interact_t’ was not declared in this scope sasl/saslwrapper.cpp:222: error: ‘prompt’ was not declared in this scope sasl/saslwrapper.cpp:227: error: ‘conn’ was not declared in this scope sasl/saslwrapper.cpp:227: error: ‘sasl_client_step’ was not declared in this scope sasl/saslwrapper.cpp:228: error: ‘SASL_INTERACT’ was not declared in this scope sasl/saslwrapper.cpp:230: error: ‘SASL_INTERACT’ was not declared in this scope sasl/saslwrapper.cpp:231: error: ‘SASL_OK’ was not declared in this scope sasl/saslwrapper.cpp:231: error: ‘SASL_CONTINUE’ was not declared in this scope sasl/saslwrapper.cpp: In member function ‘bool saslwrapper::ClientImpl::encode(const std::string&, saslwrapper::output_string&)’: sasl/saslwrapper.cpp:244: error: ‘conn’ was not declared in this scope sasl/saslwrapper.cpp:244: error: ‘sasl_encode’ was not declared in this scope sasl/saslwrapper.cpp:245: error: ‘SASL_OK’ was not declared in this scope sasl/saslwrapper.cpp: In member function ‘bool saslwrapper::ClientImpl::decode(const std::string&, saslwrapper::output_string&)’: sasl/saslwrapper.cpp:265: error: ‘conn’ was not declared in this scope sasl/saslwrapper.cpp:265: error: ‘sasl_decode’ was not declared in this scope sasl/saslwrapper.cpp:266: error: ‘SASL_OK’ was not declared in this scope sasl/saslwrapper.cpp: In member function ‘bool saslwrapper::ClientImpl::getUserId(saslwrapper::output_string&)’: sasl/saslwrapper.cpp:282: error: ‘conn’ was not declared in this scope sasl/saslwrapper.cpp:282: error: ‘SASL_USERNAME’ was not declared in this scope sasl/saslwrapper.cpp:282: error: ‘sasl_getprop’ was not declared in this scope sasl/saslwrapper.cpp:283: error: ‘SASL_OK’ was not declared in this scope sasl/saslwrapper.cpp: In member function ‘bool saslwrapper::ClientImpl::getSSF(saslwrapper::output_int*)’: sasl/saslwrapper.cpp:294: error: ‘conn’ was not declared in this scope sasl/saslwrapper.cpp:294: error: ‘SASL_SSF’ was not declared in this scope sasl/saslwrapper.cpp:294: error: ‘sasl_getprop’ was not declared in this scope sasl/saslwrapper.cpp:295: error: ‘SASL_OK’ was not declared in this scope sasl/saslwrapper.cpp: In member function ‘void saslwrapper::ClientImpl::addCallback(long unsigned int, void*)’: sasl/saslwrapper.cpp:311: error: ‘callbacks’ was not declared in this scope sasl/saslwrapper.cpp: In member function ‘void saslwrapper::ClientImpl::setError(const std::string&, int, const std::string&, const std::string&)’: sasl/saslwrapper.cpp:323: error: ‘conn’ was not declared in this scope sasl/saslwrapper.cpp:324: error: ‘sasl_errdetail’ was not declared in this scope sasl/saslwrapper.cpp:326: error: ‘sasl_errstring’ was not declared in this scope sasl/saslwrapper.cpp: At global scope: sasl/saslwrapper.cpp:338: error: variable or field ‘interact’ declared void sasl/saslwrapper.cpp:338: error: ‘sasl_interact_t’ was not declared in this scope sasl/saslwrapper.cpp:338: error: ‘prompt’ was not declared in this scope error: command 'gcc' failed with exit status 1