`tail -f` until text is seen

Question

I've got a CI server with a command-line interface that allows me to remotely kick-off a job (jenkins CI server and the jenkins-cli.jar tool).

After I kick the job off I tail -f the log (sorry for the messy command):

ssh -t my-jenkins-host.com "tail -f \"/var/lib/jenkins/jobs/$job_name/builds/\`ls -ltr /var/lib/jenkins/jobs/$job_name/builds/ | grep '^l' | tail -n 1|awk '{print \$9}'\`/log\""

After the job successfully completes, usually after at least 5 minutes, I get the following line on the output:

Finished: SUCCESS

Is there a good way to stop tailing the log at this point? i.e. is there like a tail_until 'some line' my-file.log command?

BONUS: extra credit if you can supply an answer that returns 0 when SUCCESS is matched, 1 when FAILURE is matched, and your solution works on mac! (which i believe is bsd based)

Solution 1

You can pipe the tail -f into sed, telling it to quit when it sees the line you're searching for:

tail -f /path/to/file.log | sed '/^Finished: SUCCESS$/ q'

sed will output each line it processes by default, and exit after it sees that line. The tail process will stop when it tries to write the next line and sees its output pipe is broken

Solution 2

tail -f my-file.log | grep -qx "Finished: SUCCESS"

-q, meaning quiet, quits as soon as it finds a match

-x makes grep match the whole line

For the second part, try

tail -f my-file.log | grep -m 1 "^Finished: " | grep -q "SUCCESS$"

-m <number> tells grep to stop after number matches

and the grep -q exit status will only be 0 if SUCCESS is found at the end of the line

If you want to see all the output, you can't use grep -q, but you can still do

tail -f my-file.log | grep -m 1 "^Finished: "

which does everything except set the exit status to 1 if FAILURE appears.

Solution 3

I didn't like any of the answers here, so decided to roll my own. This bash script meets all the criteria, and includes the BONUS for exit of 1 on failure.

#!/bin/bash
while IFS= read -r LOGLINE || [[ -n "$LOGLINE" ]]; do
    printf '%s\n' "$LOGLINE"
    [[ "${LOGLINE}" == "Finished: SUCCESS" ]] && exit 0
    [[ "${LOGLINE}" == "Finished: FAILURE" ]] && exit 1
done < <(timeout 300 tail -f my-file.log)
exit 3

Also included is a timeout feature, which will result in an exit code of 3. If you don't have the timeout command on your system, grab the timeout.sh script from Anthony Thyssen:

https://antofthy.gitlab.io/software/ (search for "Timeout:")

Per the comments below, I updated the log print to stop escape character expansion and included all the features of a standard 'read'. See https://stackoverflow.com/a/10929511 for complete 'read' details. The EOF check isn't required here, but is included for completeness.

Solution 4

A variation on @Mikel's answer with @Mrozek's comments (I would have replied on the comment but I think i don't have enough privileges yet)

tail -f my-file.log | tee >( grep -qx "Finished: SUCCESS" )

would allow you to use @Mikel's solution and still see the output on the screen

Solution 5

You also try

 grep -q 'App Started' <(tail -f /var/log/app/app.log)

View more solutions

Comments

[aaronstacy]

update: tail seems to do the same buffering, so i'm guessing it's not something that's worth trying to work around.

[aaronstacy]

booh yea! perfect. ...so by any chance is there a way to exit 0 if i match one thing (say 'SUCCESS') and 1 if i match something else (like maybe 'FAILURE')?

[Michael Mrozek]

@aaronstacy If you're using GNU grep, the q command takes an optional exit code. So the sed command would be sed '/^Finished: SUCCESS$/ q0; /^Finished: FAILURE$/ q1'

[lk-]

This might not work if Finished: SUCCESS is the last line of output

[aaronstacy]

@Michael Mrozek aaaand of course i'm not b/c i'm using friggin mac

[Michael Mrozek]

I used grep in my answer originally, but if he's using tail -f he probably wants to see the file output; grep isn't going to show all the intermediate lines

[kiltek]

Can we add a timeout to this, like: "if its not read between 60 to 120 seconds, then abort the tail and give an error exit code in the shell" ?

[Phate]

This solution has a major flaw: in my case the log ends by the searched line. No more lines will be written so the process would stay stuck as tail has no way to break :(

[roaima]

Very nice. Consider using while IFS= read -r LOGLINE to prevent the shell performing whitespace splitting on the lines from tail.

[dave_thompson_085]

@roaima: read doesn't split when there is only one variable (and it's not an array with -a), but you do need -r if the input data contains backslash. But if the input data contains backslash, then echo "$var" may also screw up depending on your shell and/or system, so better printf '%s\n' "$line"

[roaima]

@dave_thompson_085 Understanding "IFS= read -r line"

[Martín Coll]

Thanks, this works great on MacOS too.

[ArtOfWarfare]

This does print everything up to and including the desired line, but it seems that tail never actually exits and instead just hangs. I see this same behavior both on my Linux server (distro unknown) running Jenkins as well as on my development Mac.

[Admin]

one can use timeout for this i.e. timeout 15 sh -c "tail -f my-file.log | tee >( grep -qx 'Finished: SUCCESS' )"