How to include hbase-site.xml in the classpath

Question

I am currently trying to get my HBase code to use the settings specified in my hbase-site.xml. It seems to use default settings instead of what is specified in the hbase-site.xml config file. I have restarted the HBase cluster since updating the files, but it is still not using config files that I updated.

The cluster I am using is 2 nodes, one of which is the master. The config files on both of the nodes specify the IP of the master node as the zookeeper quorum. I believe the problem is that my settings specified in hbase-site.xml are not being used because the code runs fine if I set the zookeeper quorum to the same value as in my hbase-site.xml via code, but the second node cannot contact the master if the quorum is not specified via code.

config = HBaseConfiguration.create();
config.set("hbase.zookeeper.quorum", masterNodeIP);

I would greatly appreciate instructions or a link on how to include hbase-site.xml into my code's classpath. I develop with Eclipse on a Windows machine and have the HBase environment installed on a Linux cluster. I usually use Eclipse to compile the code, due to dependencies.

Ideally, I want each node in the cluster to use its own config file.

Thanks in advance!

Solution

If it's using the default regardless of what you put in your hbase-site.xml, it probably means it is being overriden by another file in your classpath. This is very possible because there is already a conf-site.xml in the hbase jar.

To fix this, edit your classpath to add the directory containing your hbase-site.xml at the end of the classpath to be sure nothing overrides it. Something like:

java -cp $CLASSPATH:/usr/lib/hbase/conf path.to.your.Mainclass

If it gets too obscure, I would advise running this from command line and not in Eclipse so you can be exactly sure of what you have in your classpath.

Hope that helps.

Comments

[Alex Vertlieb]

Thanks for the reply. How would you suggest I take the fairly complex set of project and library references from Eclipse and incorporate them as part of the command you suggested on the Linux machine. Any idea if there is a way to half-compile the program in Eclipse and then do the command you suggested to link a mostly finished jar up with that classpath addition?

[Charles Menguy]

@user1777358 You can find the jars you need by typing hbase classpath on command line which should give you all jars you need to work with hbase.

[Alex Vertlieb]

Alright, I presume it is similar for hadoop. I can probably look up how to link other projects into the compile statement. I will work with this. Thanks again.

[Charles Menguy]

Yeah you can do hadoop classpath too and it will give you the set of required jars. I do this for all my projects when I need to create a shell script to run the job, and it's been working fine so far.

[kumarhimanshu449]

Hi @CharlesMenguy and @Alex Vertlieb , I am trying to connect HBase (located inside VM on my windows machine) from Scala code (located on my windows machine). How can I set the hbase-site.xml in my Java CLASSPATH inside my windows machine?