How to configure "search.max_open_scroll_context" in elasticsearch?
13,820
Solution 1
I was able to fix this issue by increasing the maximum number of scroll contexts:
curl -x "" -X PUT localhost:9200/_cluster/settings -H 'Content-Type: application/json' -d'{
"persistent" : {
"search.max_open_scroll_context": 1024
},
"transient": {
"search.max_open_scroll_context": 1024
}
}'
Solution 2
I have met the same problem, my solution is clear the scroll explicitly after every use
ClearScrollRequest clearScrollRequest = new ClearScrollRequest();
clearScrollRequest.addScrollId(scrollId);
restHighLevelClient.clearScroll(clearScrollRequest, RequestOptions.DEFAULT);
and do not need to change the max_open_scroll_context
Author by
huangjs
Updated on July 14, 2022Comments
-
huangjs almost 2 years
I an using
scroll
to get data in elasticsearch (version: 7.0.0). However, when I used it, the exception was thrown.- Request:
GET /index-name/_search?scroll=1m { "size": 100, "query": { "match_all" : {} } }
- Response
{ "error": { "root_cause": [ { "type": "exception", "reason": "Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting." } ], "type": "search_phase_execution_exception", "reason": "all shards failed", "phase": "query", "grouped": true, "failed_shards": [ { "shard": 0, "index": "nr-v1", "node": "jVqXndodRtqsZ4Srh9eHSg", "reason": { "type": "exception", "reason": "Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting." } } ] }, "status": 500 }
And I check the node status using
GET /_nodes/stats/indices/search
, the response:{ "_nodes": { "total": 3, "successful": 3, "failed": 0 }, "cluster_name": "bprc", "nodes": { "AdT9tX_jRqiuRyNnYunRdg": { "timestamp": 1557036722528, "name": "es2", "transport_address": "192.168.99.74:9300", "host": "192.168.99.74", "ip": "192.168.99.74:9300", "roles": [ "master", "data", "ingest" ], "attributes": { "ml.machine_memory": "16819453952", "ml.max_open_jobs": "20", "xpack.installed": "true" }, "indices": { "search": { "open_contexts": 502, "query_total": 3171, "query_time_in_millis": 2490, "query_current": 0, "fetch_total": 538, "fetch_time_in_millis": 951, "fetch_current": 0, "scroll_total": 2020, "scroll_time_in_millis": 1948250008, "scroll_current": 502, "suggest_total": 0, "suggest_time_in_millis": 0, "suggest_current": 0 } } }, ......
My question: 1. How to fix the problem about the exception that "Trying to create too many scroll contexts. Must be less than or equal to: [500]."
"scroll_total": 2020, "scroll_time_in_millis": 1948250008, "scroll_current": 502
what is the function of the three attributes?
-
Vicky over 3 yearsI dint have permission to change the cluster settings, clearing the scroll context solved my problem, thank you for the answer.