how to add a jar file in hive
Solution 1
- upload the JAR file into hdfs path
- Add the JAR File using Add command and HDFS full PATH
Example:
hadoop fs -put ~/Downloads/hive.jar /lib/
open hive shell
add jar hdfs:///lib/hive.jar
Solution 2
I see following issues with your approach. Before adding make sure you are able to list the file on Local file system or hdfs where ever it exists.
The jar you are trying to add is by default in hive class path as its part of $HIVE_HOME/lib (on local file system where ever you have hive client/service installed)
on the other hand in regards to your question about how to add jars in hive, we can add using local file system or hadoop distributed file system (HDFS)
Add jar file:///root/hive-contrib-0.10.0.jar (Given that you copied this jar on LFS root directory)
Add jar hdfs://<namenode_hostname>:8020/user/root/hive-contrib-0.10.0.jar (Given that you copied to HDFS root home)
Solution 3
if you want to permanently add the jars you need to do the following. 1. Hive-site.xml ( /etc/hive/conf )
<property>
<name>hive.aux.jars.path</name>
<value>file:///mnt1/hive-jars/hive-contrib-2.1.1.jar</value>
</property>
- add hive-contrib-2.1.1.jar to the path "/mnt1/hive-jars" configured in hive-site.xml
This should ideally work after restarting hive-server2. 3. sudo stop hive-server2 4. sudo start hive-server2
But sometimes it does not work. i am not sure why so you can use the following dirty way.
put your jar file in the following path so that hive automatically picks it up while restart.
- add hive-contrib-2.1.1.jar to /usr/lib/hive-hcatalog/share/hcatalog
- sudo stop hive-server2
- sudo start hive-server2
Solution 4
I have read these answers above which was very useful. And I combined all into one solution:
put jars into local disk and give read/write permission
chmod -R 777 /tmp/json.jar
upload to hdfs file system and give permissions too:
hdfs dfs -put /tmp/json.jar hdfs://1.1.1.1:8020/jars/ hdfs dfs -chmod -R 777 hdfs://1.1.1.1:8020/jars/
add jar into hive env.
add jar hdfs://1.1.1.1:8020/jars/json.jar
Solution 5
- You have to give the full path to the jar JAR and not only its name.
- Don't guess the location. Check the file system to see that it is there, before trying to add it.
Avinash Jadhav
Updated on February 08, 2020Comments
-
Avinash Jadhav over 4 years
I'm trying to add hive-contrib-0.10.0.jar in hive using ADD JAR hive-contrib-0.10.0.jar command but it always saying hive-contrib-0.10.0.jar does not exist.
I'm using HDP 2.1 version right now. I also added this jar file into
/user/root
folder using hue and run the commandADD JAR hdfs:///hive-contrib-0.10.0.jar
but it giving me same error
jar file doesn't exist
.Is there any way to solve this problem. Where should I keep this jar file so that it will run successfully and what will be the command to be used?