How to get all field names in elasticsearch index

Question

I just start using elasticsearch 5.2 .

I am trying to get all keys in the index if I have the following mapping:

"properties": {
         "name": { "type": "text" },
         "article": {
          "properties": {
           "id": { "type": "text" },
           "title":  { "type": "text"},
           "abstract": { "type": "text"},
            "author": {
             "properties": {
              "id": { "type": "text" },
              "name": { "type": "text" }
}}}} } }

is it possible to get all fields full name ? like this:

 name,
 article.id ,
 article.title ,
 article.abstract ,
 article.author.id,
 article.author.name

how can I get that ?

Solution 1

You may use _field_names field.

The _field_names field indexes the names of every field in a document that contains any value other than null.

GET _search
{
  "size"0,
  "aggs": {
    "Field names": {
      "terms": {
        "field": "_field_names", 
        "size": 100
      }
    }
  }
}

Update : from ES 5 onwards

the _field_names field has been locked down and is only indexed, it doesn't support fielddata (memory intensive) or doc values,

Ref : https://github.com/elastic/elasticsearch/issues/22576

As an alternative, you may getMapping API

The get mapping API can be used to get more than one index or type mapping with a single call. General usage of the API follows the following syntax: host:port/{index}/_mapping/{type}

$ curl -XGET 'http://localhost:9200/index/_mapping?pretty'

You may then process the response to extract all the field names in the index

Solution 2

The mapping API also allows querying the field names directly. Here is a python 3 code snippet which should do the work:

import json
import requests

# get mapping fields for a specific index:
index = "INDEX_NAME"
elastic_url = "http://ES_HOSTNAME:9200"
doc_type = "DOC_TYPE"
mapping_fields_request = "_mapping/field/*?ignore_unavailable=false&allow_no_indices=false&include_defaults=true"
mapping_fields_url = "/".join([elastic_url, index, doc_type, mapping_fields_request])
response = requests.get(mapping_fields_url)

# parse the data:
data = response.content.decode()
parsed_data = json.loads(data)
keys = sorted(parsed_data[index]["mappings"][doc_type].keys())
print("index= {} has a total of {} keys".format(index, len(keys)))

# print the keys of the fields:
for i, key in enumerate(keys):
    if i % 43 == 0:
        input()
    print("{:4d}:     {}".format(i, key))

Very convenient indeed. Do note that keys which contain "." in their name may confuse you a bit in how cascaded they are in the document...

Comments

[igx]

tried that but I am getting : "root_cause" : [ { "type" : "illegal_argument_exception", "reason" : "Fielddata is not supported on field [_field_names] of type [_field_names]" } ],

[igx]

IIUC I cannot use the _field_names in aggregations as suggested

[Rahul]

Yes, you may only query for the existence of a field but can't aggregate on the same.

[Rahul]

you can also use get _mapping api but that would require some coding on your part

[Red-Tune-84]

The get mappings approach will not work if you are using the join functionality. If you want to get the fields from a strictly child document this is not possible, since you cannot match your join field value.

[Red-Tune-84]

I found a solution, which I think may be suggested but not obvious in redlus's answer. See my answer in another thread: stackoverflow.com/a/48724411/3453043