Kibana time delta between two fields

Question

I have two fields as part of a log message saved in our ELK cluster:

"EventTime": "2015-07-28 17:03:20",
"EventReceivedTime": "2015-07-28 17:03:22"

Is there a way to get the time difference between this fields (in this case 2 sec.) in each log message and display it trough Kibana3?

If its not possible a direct elasticsearch query would also work.

Thanks in advance!

Solution 1

Yes, I just did it with some test data in Kibana using a scripted field. In Kibana, go to Settings, click on your index pattern in the upper left corner.

You should see 2 tabs "Fields" and "Scripted fields".

Click on the "Scripted fields" tab. Then "Add scripted field".

Enter a "Name" and in the Script field enter something like

doc['EventReceivedTime'].value - doc['EventTime'].value

Click "Create Field" at the bottom. Now you should see that new scripted field in Discover and can use it in visualizations. My timestamps were in milliseconds and my delta_time was in milliseconds.

Solution 2

If the values are numeric, you're supposed to be able to make scripted fields in kibana (using the enabled "elasticsearch scripting" feature). This would have to be computed for each event when it is displayed.

I would recommend doing it in logstash as the events come through. You can drop use the ruby{} filter to compute the difference before writing to elasticsearch, so it's available in queries and for display with no additional processing at that time.

Solution 3

If the fields are both date fields you can first convert them to milliseconds and then subtract them. E.g.

doc['@timestamp'].value.getMillis() - doc['lastUpdatedDate'].value.getMillis()

This works for elastic/6.2.2

Comments

[James]

how about for DateOptionalTime fields ?