Elasticsearch - Want to sort by field in all indices where that particular field available or not if not then avoid it

Question

Currently, Getting result based on scoring but what i want to do is i want a result based on scoring + Field Status with value true/false.

If value is true then needed that results in priority but there is possibility that status field is not exist in all indices.

           "query" => [
                  'bool' => [
                     'filter' => $filter,
                     'must' => [
                     "multi_match" => [
                        'query' => "$string",
                        "type" => "cross_fields",
                        'fields' => ['field1','field2','field3'],
                        "minimum_should_match" => "80%"
                         ]
                    ]
                  ]
            ],
            "sort" => [
                    "_score",
                    [ "status" => ["order" => "desc","unmapped_type" => "boolean"] ]
            ],

But getting error below :

[type] => illegal_argument_exception
[reason] => Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [status] in order to load field data by uninverting the inverted index. Note that this can use significant memory.

Anyone help me out to ignore for indices where that field not available or any other solution with this problem?

Solution

As discussed in the chat, the issue happened due to @jilesh

forget to delete the old index mapping and only upate the data that's what this thing was occurring.

Below answer is relevant when you get below error with proper setup

Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [status] in order to load field data by uninverting the inverted index. Note that this can use significant memory.

In that case, please enable the field data on the field if you want to get rid of the error but beware it can cause performance issues.

Read more about the field data on official site.

You can enable it in your order field in your mapping as shown.

{
  "properties": {
    "order": { 
      "type":     "text",
      "fielddata": true
    }
  }
} 

Comments

[jilesh]

I am trying to sort by status field with descending order but main issue is status field not available in every indexes and issue causing with only indexes where this field not available. If something wrong with my query then please help me out to correct

[Amit]

@jilesh, then you should use exist query with must clause to search only on those documents where this order field present refer elastic.co/guide/en/elasticsearch/reference/current/… for more details

[Amit]

@jilesh, but the error which you mentioned is different, can you provide your index mapping as well?

[jilesh]

order is not a field field name is status i am trying to set status field in descending order.

[jilesh]

Not customize mapping from my end it's generated by default and i think it's different for every indices as per the fields available into it.

[Amit]

@jilesh, oh yes, my bad , yeah as its generated default it would be text and that's the reason of error, can get the generated mapping using elastic.co/guide/en/elasticsearch/reference/current/… and provide(formatted) in the question as it might be big

[jilesh]

Let us continue this discussion in chat.