ElasticSearch - How to display an additional field name in aggregation query

Question

How can I add a new key called 'agency_name' in my output bucket.

I am running an aggregation code as shown below

{
  "aggs": {
    "name": {
      "terms": {
        "field": "agency_code"
      }
    }
  }
}

I will be getting the out put as

"aggregations": {
    "name": {
        "doc_count_error_upper_bound": 130,
        "sum_other_doc_count": 39921,
        "buckets": [
            {
                "key": "1000",
                "doc_count": 105163
            },
            {
                "key": "2100",
                "doc_count": 43006
            }
        ]
    }
}

While displaying I need to show the agency name, code and doc_count

How can I modify the aggregation query so that I could get the below format. I am new to ElasticSearch, not sure how to fix this

"aggregations": {
    "name": {
        "doc_count_error_upper_bound": 130,
        "sum_other_doc_count": 39921,
        "buckets": [
            {
                "key": "1000",
                "doc_count": 105163,
                "agency_name": 'Agent 1'
            },
            {
                "key": "2100",
                "doc_count": 43006,
                "agency_name": 'Agent 2'
            }
        ]
    }
}

Sample Data in ElasticSearch (fields are analysed)

{

    "_index": "feeds",
    "_type": "news",
    "_id": "22005",
    "_version": 1,
    "_score": 1,
    "_source": {
        "id": 22005,
        "name": "Test News",
        "agency_name": "Agent 1",
        "agency_code": "1000",
    }

}

Solution 1

You can use the top hits aggregation like in the link below. The format will be slightly different since creating the extra aggregation will embed the agency name under another 'hits' key.

Adding additional fields to ElasticSearch terms aggregation

{
  "aggs": {
    "name": {
      "terms": {
        "field": "agency_code"
      },
      "aggs": {
        "agency_names" : {
           "top_hits": {
                size: 1, 
                _source: {
                    include: ['agency_name']
                }
            }
         } 
       }
    }
  }
}

Solution 2

I think you would need to add another "aggs" to it. But it would not be in the format in which you want but as another field in the output , reason being currently you are aggregating based on "agency_code" and the doc_count shows how many times the particular agency code occurs. Now when you want to aggregate it based on "agency_name" the field might in different documents than "agency_code" and in different numbers as well , if they always exist in pair than this parent-child indexing might be of some help.

https://www.elastic.co/guide/en/elasticsearch/guide/current/indexing-parent-child.html

Solution 3

ES has no way of knowing agency_name and agency_code map one-to-one. Therefore I would recommend a number of possible strategies.

• Don't analyze agency_name and use the term agg over that field. I would be surprised if you actually need to do tokenization of the agency_name.
• Store the id to name mapping in a relational database or a flat file cache and do the join client side
• Store the agency documents as another type and make two calls. The first to get the ids and then a second to lookup the agencies by id

As Aditya Patel mentioned above, parent child relationships may help out as well but I believe you will still have to use one of the above strategies to resolve the id->name mapping.

Solution 4

This is old post, however, I ran into same issue and I followed what is given at https://www.elastic.co/guide/en/elasticsearch/reference/current/agg-metadata.html. Add metadata details and it is return as part of the result above bucket. Hope it will help someone in future.

View more solutions