how to run multiple instances of elasticsearch on one host

Question

I have several machines each with 128 GB of ram, each host is running a single instance of Elasticsearch. I would like to run another data node on each host and allocate around 30 GB to the jvm heap.

I know I have to create a separate config file .yml and data directory..etc. My question is do I need to modify the service wrapper so that each node will be started/ stopped seperatly?

I am running ES version 1.3 on Centos 6.5

thank you

Solution

You need to prepare two elasticsearch.yml config files to configure settings accordingly and specify these files when startup up the two nodes.

bin/elasticsearch -Des.config=$ES_HOME/config/elasticsearch.1.yml
bin/elasticsearch -Des.config=$ES_HOME/config/elasticsearch.2.yml

At least the following should be set differently for the two nodes:

http.port
transport.tcp.port
path_data
path_logs
path_pid
node.name

The following needs to point to the other in both files to allow the nodes to find each other:

discovery.zen.ping.unicast.hosts: '127.0.0.1:9302'

EDIT: the property is now deprecated, look at : https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery-settings.html

See this blog and this discussion

Comments

[M4rk]

why should the cluster.name be set differently?

[centic]

you are right, that is not useful, I have adjusted the answer.