zookeeper unable to open socket to localhost/0:0:0:0:0:0:0:1:2181

Question

I am using zookeeper ensemble for hbase. Zookeeper is running on 3 machines. While HBase is also in fully distributed mode. I have Nutch 2.x version. When I start nutch to crawl some data, it gives following buggs in nutch log file.

ERROR zookeeper.ClientCnxnSocketNIO - Unable to open socket to localhost/0:0:0:0:0:0:0:1:2181
2015-01-23 16:34:21,956 WARN  zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.SocketException: Network is unreachable
        at sun.nio.ch.Net.connect0(Native Method)
        at sun.nio.ch.Net.connect(Net.java:457)
        at sun.nio.ch.Net.connect(Net.java:449)
        at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:647)
        at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:266)
        at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:276)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:958)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
2015-01-23 16:34:22,063 WARN  zookeeper.RecoverableZooKeeper - Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid

When I run a command on three zookeepers instances

echo ruok | nc 1.1.1.1 2181 it says imok

What is wrong with it? My hbase version is 0.94.14 and zookeeper version is 3.4.5 , solr version 4.10.3 ( for indexing )and Nutch version is 2.2.3

Solution 1

( CentOS ) Check your /etc/hosts file and if it conatins like

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

change it to

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         ip6-localhost ip6-localhost.localdomain localhost6 localhost6.localdomain6

Solution 2

I don't know zookeeper, but the two IP addresses that you're posting don't have anything to do with each other:

1.1.1.1 is an IPV4 address. 0:0:0:0:0:0:0:1 is an IPV6 address (and it's the one for localhost). The shorthand for the IPV6 localhost address is ::1

Thus, if you're expecting a server on localhost, make sure that it binds to IPV6 as well and not only to 127.0.0.1 (which is the IPV4 address of localhost). I've seen problems like this with servers that only listened to 127.0.0.1, or with firewalls that only allowed access to localhost on IPV4, but not on IPV6.

I'm not sure what you expect to hit when you're going to 1.1.1.1. Are you using this address locally? AFAIK it's a publically routed address, but I guess it gets a lot of this kind of unsolicited traffic that it's more or less unusable.