Elasticsearch read and write consistency

Question

Elasticsearch doesn't have "read consistency" param (like Cassandra). But it has "write consistency" and "read preference".

Documentation says the following about Write Consistency

Write Consistency
To prevent writes from taking place on the "wrong" side of a network partition, by default, index operations only succeed if a quorum (>replicas/2+1) of active shards are available. This default can be overridden on a node-by-node basis using the action.write_consistency setting. To alter this behavior per-operation, the consistency request parameter can be used.

Valid write consistency values are one, quorum, and all.

Note, for the case where the number of replicas is 1 (total of 2 copies of the data), then the default behavior is to succeed if 1 copy (the primary) can perform the write.

The index operation only returns after all active shards within the replication group have indexed the document (sync replication).

My question is about the last paragraph:

The index operation only returns after all active shards within the replication group have indexed the document (sync replication).

If write_consistency=quorum (default) and all shards are live (no node failures, no network-partition), then:
1) Does index operation return as soon as quorum of shards have finished indexing? (even though all shards are live/active)
2) Or does index operation return when all live/active shards have finished indexing? (i.e. quorum is considered only in case of failures/timeouts)

In the first case - read may be eventual-consistent (may get stale data), write is quicker.
In the second case - read is consistent (as long as there are no network-partitions), write is slower (as it waits for the slower shard/node).

Does anyone know how it works?

Another thing that I wonder about - is why the default value for 'preference' param (in get/search request) is randomized but not _local (which must have been more efficient I suppose)

Solution 1

I think I can answer my own question now :)

Regarding the first question, by re-re-reading the documentation (this and this) a few times :) I realized that this statement should be right:

Index operation return when all live/active shards have finished indexing, regardless of consistency param. Consistency param may only prevent the operation to start if there are not enough available shards(nodes).

So for example, if there are 3 shards (one primary and two replicas), and all shards are available - the operation will be waiting for all 3 (considering that all 3 are live/available), regardless of consistency param (even when consistency=one)
This makes the system consistent (at least the document-api part); unless there is a network-partition. But, I didn't have a chance to test this yet.

UPDATE: by consistency here, I don't mean ACID-consistency, it is just the guarantee that all replicas are updated at the moment when request is returned.

Regarding the second question: The obvious answer is - it is randomized to spread the load; on the other hand, a client can pick a random node to talk to, but probably it is not 100% efficient as a single request may need multiple shards.

Solution 2

Write: I am not sure if above is true for ES 6.1 https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#index-wait-for-active-shards says that index operation returns if the primary shard is active and can be changed to other values.

Searches are random so setting wait for active shard field to all will ensure write is successful if it is performed on all shards.

Read: Preference can still be used but it is marked as deprecated.

Comments

[animageofmine]

great job answering your own question!