How can I capture Java exceptions, including the stack trace, from a log file using grep?

Question

Summary

I am trying to fetch logs from a log file using the grep command. However, I can match time stamps, but am not getting the full stack trace I need.

Log File Sample

[1/10/16 23:55:33:018 PST] 00000057 ServerObj E   SECJ0373E: Exception message
at com.own.ws.wim.util.UniqueNameHelper.formatUniqueName(UniqueNameHelper.java:102)
at com.own.ws.wim.ProfileManager.getImpl(ProfileManager.java:1569)

What I've Tried

I am able to fetch log entries, but I want the stack trace as well. I've tried:

$ grep -i '^[[:space:]]*at' --before-context=2 SystemOut.log |
    grep "1/13/16 7:[1-60]" 
[1/10/16 23:55:33:018 PST] 00000057 ServerObj E   SECJ0373E: Exception message

Any idea how this can be achieved?

Solution 1

(From my answer here: https://stackoverflow.com/a/16064081/430128)

Here is a quick-and-dirty grep expression... if you are using a logger such as log4j than the first line of the exception will generally contain WARN or ERROR, the next line will contain the Exception name, and optionally a message, and then the subsequent stack trace will begin with one of the following:

1. "\tat" (tab + at)
2. "Caused by: "
3. "\t... <some number> more" (these are the lines that indicate the number of frames in the stack not shown in a "Caused by" exception)
4. An Exception name (and perhaps message) before the stack

We want to get all of the above lines, so the grep expression is:

grep -P "(WARN|ERROR|^\tat |Exception|^Caused by: |\t... \d+ more)"

It assumes an Exception class always contains the word Exception which may or may not be true, but this is quick-and-dirty after all.

Adjust as necessary for your specific case.

Solution 2

Use Context for Fixed-Length Traces

In your original post, you show a single three-line entry. If you know that each exception message with a stack trace is exactly three lines long, then you can use one of the --after-context flags (if supported by your grep) to get all three lines. For example, to pull all exceptions along with the stack trace:

$ fgrep -A2 'Exception message' SystemOut.log
[1/10/16 23:55:33:018 PST] 00000057 ServerObj E   SECJ0373E: Exception message
at com.own.ws.wim.util.UniqueNameHelper.formatUniqueName(UniqueNameHelper.java:102)
at com.own.ws.wim.ProfileManager.getImpl(ProfileManager.java:1569)

Use Multi-Line Expressions for Variable-Length Stack Traces

However, if you don't know how many lines are in the stack trace, then you need a multiline regex with a stop-pattern. For this, you need a grep with the Perl-compatible regular expression (PCRE) library compiled in. For example, with grep -PM or pcregrep -M:

$ pcregrep -M 'Exception message[^\[]+' SystemOut.log
[1/10/16 23:55:33:018 PST] 00000057 ServerObj E   SECJ0373E: Exception message
at com.own.ws.wim.util.UniqueNameHelper.formatUniqueName(UniqueNameHelper.java:102)
at com.own.ws.wim.ProfileManager.getImpl(ProfileManager.java:1569)

This will print each line with an exception, using the square bracket that starts a new timestamp as the stop-pattern. You can certainly adjust the regular expression to suit your needs, or pipe the results to another grep to filter specific timestamps in or out.

This worked for me given the corpus you originally posted. Your mileage may vary.