How do I get both STDOUT and STDERR to go to the terminal and a log file?

Question

I have a script which will be run interactively by non-technical users. The script writes status updates to STDOUT so that the user can be sure that the script is running OK.

I want both STDOUT and STDERR redirected to the terminal (so that the user can see that the script is working as well as see if there was a problem). I also want both streams redirected to a log file.

I've seen a bunch of solutions on the net. Some don't work and others are horribly complicated. I've developed a workable solution (which I'll enter as an answer), but it's kludgy.

The perfect solution would be a single line of code that could be incorporated into the beginning of any script that sends both streams to both the terminal and a log file.

EDIT: Redirecting STDERR to STDOUT and piping the result to tee works, but it depends on the users remembering to redirect and pipe the output. I want the logging to be fool-proof and automatic (which is why I'd like to be able to embed the solution into the script itself.)

Solution 1

Use "tee" to redirect to a file and the screen. Depending on the shell you use, you first have to redirect stderr to stdout using

./a.out 2>&1 | tee output

or

./a.out |& tee output

In csh, there is a built-in command called "script" that will capture everything that goes to the screen to a file. You start it by typing "script", then doing whatever it is you want to capture, then hit control-D to close the script file. I don't know of an equivalent for sh/bash/ksh.

Also, since you have indicated that these are your own sh scripts that you can modify, you can do the redirection internally by surrounding the whole script with braces or brackets, like

  #!/bin/sh
  {
    ... whatever you had in your script before
  } 2>&1 | tee output.file

Solution 2

Approaching half a decade later...

I believe this is the "perfect solution" sought by the OP.

Here's a one liner you can add to the top of your Bash script:

exec > >(tee -a $HOME/logfile) 2>&1

Here's a small script demonstrating its use:

#!/usr/bin/env bash

exec > >(tee -a $HOME/logfile) 2>&1

# Test redirection of STDOUT
echo test_stdout

# Test redirection of STDERR
ls test_stderr___this_file_does_not_exist

(Note: This only works with Bash. It will not work with /bin/sh.)

Adapted from here; the original did not, from what I can tell, catch STDERR in the logfile. Fixed with a note from here.

Solution 3

The Pattern

the_cmd 1> >(tee stdout.txt ) 2> >(tee stderr.txt >&2 )

This redirects both stdout and stderr separately, and it sends separate copies of stdout and stderr to the caller (which might be your terminal).

• In zsh, it will not proceed to the next statement until the tees have finished.

• In bash, you may find that the final few lines of output appear after whatever statement comes next.

In either case, the right bits go to the right places.

---

Explanation

Here's a script (stored in ./example):

#! /usr/bin/env bash
the_cmd()
{
    echo out;
    1>&2 echo err;
}

the_cmd 1> >(tee stdout.txt ) 2> >(tee stderr.txt >&2 )

Here's a session:

$ foo=$(./example)
    err

$ echo $foo
    out

$ cat stdout.txt
    out

$ cat stderr.txt
    err

Here's how it works:

1. Both tee processes are started, their stdins are assigned to file descriptors. Because they're enclosed in process substitutions, the paths to those file descriptors are substituted in the calling command, so now it looks something like this:

the_cmd 1> /proc/self/fd/13 2> /proc/self/fd/14

2. the_cmd runs, writing stdout to the first file descriptor, and stderr to the second one.

3. In the bash case, once the_cmd finishes, the following statement happens immediately (if your terminal is the caller, then you will see your prompt appear).

4. In the zsh case, once the_cmd finishes, the shell waits for both of the tee processes to finish before moving on. More on this here.

5. The first tee process, which is reading from the_cmd's stdout, writes a copy of that stdout back to the caller because that's what tee does. Its outputs are not redirected, so they make it back to the caller unchanged

6. The second tee process has it's stdout redirected to the caller's stderr (which is good, because it's stdin is reading from the_cmd's stderr). So when it writes to its stdout, those bits go to the caller's stderr.

This keeps stderr separate from stdout both in the files and in the command's output.

If the first tee writes any errors, they'll show up in both the stderr file and in the command's stderr, if the second tee writes any errors, they'll only show up only in the terminal's stderr.

Solution 4

the to redirect stderr to stdout append this at your command: 2>&1 For outputting to terminal and logging into file you should use tee

Both together would look like this:

 mycommand 2>&1 | tee mylogfile.log

EDIT: For embedding into your script you would do the same. So your script

#!/bin/sh
whatever1
whatever2
...
whatever3

would end up as

#!/bin/sh
( whatever1
whatever2
...
whatever3 ) 2>&1 | tee mylogfile.log

Solution 5

EDIT: I see I got derailed and ended up answering a different question from the one asked. The answer to the real question is at the bottom of Paul Tomblin's answer. (If you want to enhance that solution to redirect stdout and stderr separately for some reason, you could use the technique I describe here.)

---

I've been wanting an answer that preserves the distinction between stdout and stderr. Unfortunately all of the answers given so far that preserve that distinction are race-prone: they risk programs seeing incomplete input, as I pointed out in comments.

I think I finally found an answer that preserves the distinction, is not race prone, and isn't terribly fiddly either.

First building block: to swap stdout and stderr:

my_command 3>&1 1>&2 2>&3-

Second building block: if we wanted to filter (e.g. tee) only stderr, we could accomplish that by swapping stdout&stderr, filtering, and then swapping back:

{ my_command 3>&1 1>&2 2>&3- | stderr_filter;} 3>&1 1>&2 2>&3-

Now the rest is easy: we can add a stdout filter, either at the beginning:

{ { my_command | stdout_filter;} 3>&1 1>&2 2>&3- | stderr_filter;} 3>&1 1>&2 2>&3-

or at the end:

{ my_command 3>&1 1>&2 2>&3- | stderr_filter;} 3>&1 1>&2 2>&3- | stdout_filter

To convince myself that both of the above commands work, I used the following:

alias my_command='{ echo "to stdout"; echo "to stderr" >&2;}'
alias stdout_filter='{ sleep 1; sed -u "s/^/teed stdout: /" | tee stdout.txt;}'
alias stderr_filter='{ sleep 2; sed -u "s/^/teed stderr: /" | tee stderr.txt;}'

Output is:

...(1 second pause)...
teed stdout: to stdout
...(another 1 second pause)...
teed stderr: to stderr

and my prompt comes back immediately after the "teed stderr: to stderr", as expected.

Footnote about zsh:

The above solution works in bash (and maybe some other shells, I'm not sure), but it doesn't work in zsh. There are two reasons it fails in zsh:

1. the syntax 2>&3- isn't understood by zsh; that has to be rewritten as 2>&3 3>&-
2. in zsh (unlike other shells), if you redirect a file descriptor that's already open, in some cases (I don't completely understand how it decides) it does a built-in tee-like behavior instead. To avoid this, you have to close each fd prior to redirecting it.

So, for example, my second solution has to be rewritten for zsh as {my_command 3>&1 1>&- 1>&2 2>&- 2>&3 3>&- | stderr_filter;} 3>&1 1>&- 1>&2 2>&- 2>&3 3>&- | stdout_filter (which works in bash too, but is awfully verbose).

On the other hand, you can take advantage of zsh's mysterious built-in implicit teeing to get a much shorter solution for zsh, which doesn't run tee at all:

my_command >&1 >stdout.txt 2>&2 2>stderr.txt

(I wouldn't have guessed from the docs I found that the >&1 and 2>&2 are the thing that trigger zsh's implicit teeing; I found that out by trial-and-error.)

View more solutions

Comments

[Sergey Kotyushkin]

I did not know you could bracket commands in the shell scripts. Interesting.

[Ed Brannin]

I also appreciate the Bracket shortcut! For some reason, 2>&1 | tee -a filename wasn't saving stderr to the file from my script, but it worked fine when I copied the command and pasted it into the terminal! The bracket trick works fine, though.

[NVRAM]

You'll lose grouping of arguments containing whitespace with $1 $2 $3 ..., you should use (w/quotes): "$@"

[Flimm]

Note that the distinction between stdout and stderr will be lost, as tee prints everything to stdout.

[Flimm]

Note that the distinction between stdout and stderr will be lost, as tee prints everything to stdout.

[Flimm]

Note that the distinction between stdout and stderr will be lost, as tee prints everything to stdout.

[Sam]

FYI: The 'script' command is available in most distributions (it's part of the util-linux package)

[jarno]

@Flimm stderr could be redirected to different tee process which again could be redirected to stderr.

[Gabriel]

@Flimm, is there a way (any other way) to retain the distinction between stdout and stderr?

[Artyom]

Works in Powershell also. It is *>&1 | tee filename to redirect all.

[MatrixManAtYrService]

@Gabriel I believe this reserves the distinction you're talking about: stackoverflow.com/a/53051506/1054322

[MatrixManAtYrService]

@Flimm, I wrote up jarno's suggestion here: stackoverflow.com/a/53051506/1054322

[Agi Hammerthief]

This looks really useful and what I want. I'm not sure how to replicate the use of brackets (as shown in the first line) in a Windows Batch Script, though. (tee is available on the system in question.) The error I get is "The process cannot access the file because it is being used by another process."

[Don Hatch]

This solution, like most other solutions proposed so far, is race-prone. That is, when the current script completes and returns, either to the user's prompt or some higher-level calling script, the tee, which is running in background, will still be running, and may emit the last few lines to the screen and to the logfile late (that is, to the screen after the prompt, and to the logfile after the logfile is expected to be complete).

[Don Hatch]

This solution, like most other solutions proposed so far, is race-prone. That is, when the current script completes and returns, either to the user's prompt or some higher-level calling script, the tee, which is running in background, will still be running, and may emit the last few lines to the screen and to the logfile late (that is, to the screen after the prompt, and to the logfile after the logfile is expected to be complete).

[Don Hatch]

@MatrixManAtYrService it does it at the cost of flaky timing problems; see the comment I added. I don't know whether it's possible to robustly keep stdout and stderr separate while keeping the timing well behaved; I suspect it's not possible.

[pylipp]

@DonHatch Can you propose a solution that amends this issue?

[MatrixManAtYrService]

I'd also be interested in a test case that makes the race apparent. It's not that I'm doubtful, but it's hard to attempt to avoid it because I haven't seen it happen.

[Don Hatch]

@pylipp I don't have a solution. I would be very interested in one.

[Don Hatch]

@MatrixManAtYrService To demonstrate, in an interactive bash shell: { ( echo "to stdout"; echo "to stderr" >&2 ) > >(sleep 1; tee stdout.txt); } 2> >(sleep 2; tee stderr.txt >&2 ) . The shell prompt comes back immediately; "to stdout" 1 second later, "to stderr" 2 seconds later. In a correct solution, the prompt would not appear until after the "to stderr".

[MatrixManAtYrService]

@DonHatch Thanks for the test. I find it interesting that it passes with this answer in zsh, but fails in bash. Anyhow, I took another try at it: gist.github.com/mattrixman/e796abd41a086f2990926b27bea5c4ce Do you think that fixes the race problem? If so I may post it as a separate answer (since this one works in the zsh case).

[MatrixManAtYrService]

A kind soul in the unix/linux stackexchange helped me understand the zsh difference: unix.stackexchange.com/a/558315/146169

[Don Hatch]

@MatrixManAtYrService I looked at your gist, with the flocks to communicate from the tee processes that they are done. That's clever. It seems to work, and I don't see any holes in the logic. It does seem very fiddly, though. Perhaps it can be regarded as an existence proof while we keep thinking about whether there's a neater recipe.

[Don Hatch]

@pylipp , I think I finally figured out a good solution. See the answer I just now posted.

[MatrixManAtYrService]

@DonHatch Thanks for the feedback. I agree that it's fiddly. I may tinker with how much of that fiddlyness can be hidden under a rug off in my .bashrc. I'm usually a defender of bash and its ilk, but it shouldn't be this hard...

[MatrixManAtYrService]

I played around with this in bash and it works well. Just a warning for zsh users with a habit of assuming compatibility (like myself), it behaves differently there: gist.github.com/MatrixManAtYrService/…

[Don Hatch]

@MatrixManAtYrService I believe I got a handle on the zsh situation, and it turns out there's a much neater solution in zsh. See my edit "Footnote about zsh".

[Don Hatch]

@PaulTomlin I looked back over the question's revision history, and found that it's actually clear from the beginning that what was wanted was a way to redirect the entire script's output. (The edit emphasizes that, but didn't change it.) You give a great answer to that, at the end, but I almost missed it because you started by answering a different question. I'd like to upvote this but I can't, quite, because of this.

[Paul Tomblin]

@DonHatch Considering I answered this and it was accepted as an answer 11 years ago, I'm not going to lose any sleep over it. Also, you misspelled my name.

[Don Hatch]

@PaulTomblin I hope you don't object that I removed the word "now" from your answer, so that your answer no longer implies the OP changed the meaning of their question, which (in my perception) was inappropriately putting them in a negative light. I'm not looking for an edit war here, but I am interested in improving your answer by making it more respectful to the OP.

[Paul Tomblin]

Personally, I think the only disrespect being shown here is to "improve" an answer and downvote it at the same time.

[pylipp]

Thanks for explaining the solution in such detail. Do you also know how to retrieve the return code when using a function (my_function) in the nested stdout/stderr filtering? I did { { my_function || touch failed;} 3>&1 1>&2 2>&3- | stderr_filter;} 3>&1 1>&2 2>&3- | stdout_filter but it feels weird to create a file as indicator of failure...

[Don Hatch]

@PaulTomblin I've removed my downvote. I'm not giving your answer an upvote, since the problem I described earlier hasn't been fixed. I'm not sure what to do about the fact that you're perceiving disrespect from me; none has been intended, in anything I've said or done here.

[Don Hatch]

@pylipp I don't offhand. You might ask that as a separate question (perhaps with a simpler pipeline).

[antred]

Has anyone been able to get this to work in GNU Make? I'm using GNU Make 4.3 on Windows 10, and while doing the tee thingy shown by Paul Tomblin directly in CMD works fine, when I try to feed it to GNU Make's shell function ( gnu.org/software/make/manual/html_node/… ), it bombs out. I added some debug output to Make and found out that GNU Make somehow interprets the output of the command before the pipe as another command that it should execute, which of course fails.

[Paul Tomblin]

@antred that would depend entirely on what shell GNU Make is invoking. Is it bash?

[antred]

@PaulTomblin Turns out the problem was my incorrect use of GNU Make's shell function. In the place in which I used it, the shell function was unnecessary (and apparently even caused problems). Running the recipe directly, without the shell function, worked fine. P.S. And the shell I used was the standard cmd shell in Windows.

[Lance Kind]

I love this solution. But it affects the scripting with apt-get (it can't connect to servers, or the apt-get early isn't finished updating before the second apt-get). I don't why. (Seems like a race condition is happening.) All I know is that when I comment out the line, the script works consistently. It's not just this solution but others, with or without tee. So it comes down to this: some commands are sensitive to this. I don't know why.

[aff]

If you need to preserve the exit status of the script, you can combine this with Jason Sydes/MatrixManAtYrService's answer, to become: { ... } 1> >(tee -a output.file) 2>&1. The output order of the contained scripts will be preserved, but the writes to output.file will lag a bit.

[wisbucky]

This a great pattern, as it allows you to easily configure any combination of which stdout/stderr to write to file vs console.