Elasticsearch server discovery configuration

Question

I've installed ElasticSearch server, that i'm running by:

$ ./elasticsearch -f
 {0.18.2}[11698]: initializing ...
 loaded [], sites []
 {0.18.2}[11698]: initialized
 {0.18.2}[11698]: starting ...
 bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/192.168.1.106:9300]}
 new_master [Stingray][ocw4qPdmSfWuD9pUxHoN1Q][inet[/192.168.1.106:9300]], reason: zen-disco-join (elected_as_master)
 elasticsearch/ocw4qPdmSfWuD9pUxHoN1Q
 recovered [0] indices into cluster_state
 bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.1.106:9200]}
 {0.18.2}[11698]: started

How I can configure Java client to connect to this server? I have just:

node.client=true

but, after trying to connect i'm receiving:

org.elasticsearch.discovery.MasterNotDiscoveredException: 
    at org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(TransportMasterNodeOperationAction.java:162)

If i'm configuring java client as:

node.data=false

I'm getting following logs:

INFO main node:internalInfo:93 - [Stark, Tony] {0.18.2}[13008]: starting ...
INFO main transport:internalInfo:93 - [Stark, Tony] bound_address {inet[/0:0:0:0:0:0:0:0:9301]}, publish_address {inet[/192.168.1.106:9301]}
INFO elasticsearch[Stark, Tony]clusterService#updateTask-pool-13-thread-1 service:internalInfo:93 - [Stark, Tony] new_master [Stark, Tony][WkNn96hgTkWXRnsR0EOZjA][inet[/192.168.1.106:9301]]{data=false}, reason: zen-disco-join (elected_as_master)

As I understood it means that this new node (supposed to be client node) made itself a new master node. And I don't from log that it's found and connect to any other node.

Both server and client are started on same machine. 192.168.1.106:9200 are accessible from browser.

And I can't find any good documentation about discovery config. Where I can read more about ElasticSearch configurations? And how to configure Java client?

Solution 1

Faced the same issue where the nodes were not able to elect a master on nodes restart.

The problem lies in the communication of nodes among themselves.

Please ensure in your elastic search logs, whether the node restart says

  publish_address {127.0.0.1:9200}
  or
  publish_address {0.0.0.1:9200}

This means the current node is not publishing its IP address to other nodes and hence the nodes won't recognise this node even though the node IP might be present in the discovery.zen.ping.unicast.hosts

Solution Make the following changes in elasticsearch.yml. Add

 network.host: _non_loopback:ipv4_
 and restart the node.
 Ensure that the bound address now shows the <IP address>:<port no> and not the localhost.

This means that now your node is discoverable. The second step to make it discoverable in the cluster is to add the ip address of the node in the unicast hosts lists of all the master nodes, so that whenever we have a new master, the node is discoverable to the new master.

   Add the node IP to the discovery.zen.ping.unicast.hosts 
   list of hosts of all the masters to make it disoverable. A masterpings all the 
   nodes present in the unicast list.

Solution 2

The most likely reason for this failure is firewall on your machine that blocks multicast discovery traffic on port 54328. Both client and master are broadcasting on this port during initial discovery and they don't hear back from each other. That's why when you specify node.client=true client node (that cannot be a master) fails with MasterNotDiscoveredException and node with no data elects itself as a master.

Solution 3

I ran into the same problem and by using IP numbers in the config file resolved it for me.

in /config/elasticsearch.yml

uncomment and change the network.host setting to:

network.host: 127.0.0.1

You can also change this to your machine IP number in ifconfig.

Solution 4

I had the same issue. Eventually, it turned out I had a firewall issue, with my firewall (on Ubuntu) blocking the ports of ElasticSearch. I'm using the default firewall on Ubuntu, ufw.

So, to open up the ports, I ran these commands on the terminal:

sudo ufw allow proto tcp to any port 9200:9400
sudo ufw allow proto tcp to any port 54328

My cluster runs locally on 9200, and all my clients open up on 9300+. So, I just opened the range 9200-9400 for them. 54328 is for the multicast broadcast.

Just to be complete: I also used the TransportClient, which works, but I added hardcoded my localhost to the address the TransportClient will work on. Not a good thing for production code :-)

Solution 5

Something like this should work:

        Settings s = ImmutableSettings.settingsBuilder()
                .put(this.settings)
                .build();
        TransportClient client = new TransportClient(s);
        client.addTransportAddress(new InetSocketTransportAddress(
                "localhost",
                9300)
        );

What fouled me up was I originally tried connecting the client to 9200, not 9300. Guidance for settings above can be found from http://www.elasticsearch.org/guide/reference/java-api/client.html

View more solutions

Comments

[Igor Artamonov]

Actually i've started using TransportClient too, with preconfigured addr. But I thought that ElasticClient must discover existing nodes automatically. Or I've misunderstood something?

[digitalsanctum]

The auto discovery occurs by giving each node the same cluster name as explained here: elasticsearch.org/guide/reference/setup/configuration.html

[Igor Artamonov]

yes, sure, i've same cluster name on both parts. At least configured it to have.

[harperville]

This actually helped me with another issue I was having so upvote for you, sir. :)

[Keith]

The multicast broadcast is UDP, not TCP