How to filter out fields that do not exist in elastic search?

Question

I would like to check if a field exists, and return results for documents where it does not exist. I am using the Golang library Elastic: https://github.com/olivere/elastic

I tried the following but it does not work:

e := elastic.NewExistsFilter("my_tag")
n := elastic.NewNotFilter(e)
filters = append(filters, n)

Solution 1

Ok, I wont go deep in your language query API. Since you want to search on a field not existing (null), use an exists filter inside a must_not (if you use bool filters):

{
  "query": {
    "filtered": {
      "filter": {
        "bool": {
          "must_not": [
            {
              "exists": {
                "field": "your_field"
              }
            }
          ]
        }
      }
    }
  },
  "from": 0,
  "size": 500
}

Hope this helps!

Thanks

Solution 2

You can use exist query with bool query must_not:

GET /_search
{
    "query": {
        "bool": {
            "must_not": {
                "exists": {
                    "field": "your_field"
                }
            }
        }
    }
}

Tested in Elasticsearch 6.5

Solution 3

You can create a bool query for not exists like this:

existsQuery := elastic.NewExistsQuery(fieldName)
existsBoolQuery := elastic.NewBoolQuery().MustNot(existsQuery)

Solution 4

I won't try to provide a complete solution, being that I'm not really familiar with the library your using (or, indeed, the go language).

However, Lucene doesn't support pure negative querying as you have here. Lucene needs to be told what to match. Negations like this serve strictly to prohibit search results, but do not implicitly match everything else.

In order to do what you are looking for, you would want to use a boolean query to combine your not filter with a match all (which I see is available in the library).

Note: As with anytime you use a match all, performance may suffer.

View more solutions