using tee to output intermediate results to stdout instead of files

Question

I know that to capture a pipeline's contents at an intermediate stage of processing, we use tee as ls /bin /usr/bin | sort | uniq | tee abc.txt | grep out , but what if i don't want to redirect the contents after uniq to abc.txt but to screen(through stdout, ofcourse) so that as an end result , i'll have on screen, the intermediate contents after uniq as well as the contents after grep.

Solution 1

sometimes /dev/tty can be used for that...

ls /bin /usr/bin | sort | uniq | tee /dev/tty | grep out | wc

Solution 2

ls /bin /usr/bin | sort | uniq | tee /dev/fd/2 | grep out | wc

On a linux system you can use the the /dev/fd/[num] links like named pipes in many cases. This will duplicate stdout to stderr, which, typically, is your terminal screen, but doesn't need to be.

Solution 3

This command worked for me.

ls /bin /usr/bin | sort | uniq | tee /dev/pts/0 | grep out

You could check what is your terminal using the command tty and replace the tee to redirect the output to that terminal.

References

https://stackoverflow.com/a/18025293/1742825

Solution 4

How to do it (example):

exec 3>&1; ( ls |( tee >&3 ) >/dev/null ); exec 3>&-

Which will show the ls result and send it to nirvanah.

To grok the key part, 3>&1, you may read I/O Redirection and esp. this example.

In short: >somefileis short for 1>somefile, which in turn means Assign the file handle of somefile to file descriptor 1 (and drop the former value of that descriptor, for the scope of this process).

So, 3>&1 means: Assign the file descriptor 1 (which may but need not be tty) to the (until now unused) file descriptor 3. We're effectively using &3 as a temporary variable.

Solution 5

mkfifo myfifo
cat myfifo& ls /bin /usr/bin | sort | uniq | tee myfifo | grep out

mkfifo creates a FIFO (first in, first out) special file, a.k.a. a named pipe.  Start an asynchronous cat to read from the fifo, and then run your pipeline, teeing the intermediate result to the fifo.

This will produce a [1]+ Done cat myfifo message at the end.  You can suppress that with this magic trick:

(cat myfifo&); ls /bin /usr/bin | sort | uniq | tee myfifo | grep out

For a long term, robust solution, you might want to create a permanent fifo (e.g., $HOME/myfifo) rather than creating a new one every time.  But that will fail if you may be running multiple instances of this simultaneously.  Alternatively,

• Generate a unique name (e.g., with mktemp).
• Create the fifo in a directory that's guaranteed to be writable (e.g., /tmp).
• Remove the fifo at the end of the command.

View more solutions

Comments

[Joe Sewell]

The problem is stdout is a pipe to grep out. It's not your terminal anymore. Do you have any sort of guarantee that this will always be run from a terminal?

[Aman]

@JoeSewell In my case, yes from a terminal, but please tell what are the other sources from apart from terminal(are you talking about simply shell-scripting or something else). Just curious. I'm new to Linux and would love to explore what you are talking in detail.

[Aman]

@JoeSewell thanks, i'll do some research on them :)

[Joe Sewell]

That assumes the command runs on the terminal /dev/pts/0.

[Joe Sewell]

This assumes the command is run from a terminal.

[Ramesh]

@JoeSewell, you could see which is the running terminal using tty. I have updated with that information.

[Joe Sewell]

Nice update, but why not use /dev/tty instead? That assumes, of course, that the command even has access to the terminal. If it's run from, for example, inside gvim, there may be no terminal involved at all. Doing this within gmake could become even more complex with multiple jobs, since not all processes can get to "the terminal."

[Daniel Alder]

Only works with bash, not dash /bin/sh on Ubuntu. I use this command for docker build: ID=$(docker build "$dockercontext" 2>&1 | tee /dev/fd/3 - | tail -1)

[mikeserv]

@DanielAlder - it works fine with dash. but you need to direct it somewhere. where does &3 go? the fd/2 above assumes the command is run from a terminal and &2 goes to the tty. for example: dash -c '{ echo hey; read v; echo "${v#?}" >/dev/fd/2; } <>/dev/fd/1 |:'

[Daniel Alder]

You are right. Didn't realize that /dev/fd/* are now also supported in dash. fd/3 is redirected to stdout in my case. I use this command line now, the other variant was overkill: ID=$(docker build "$1" 2>&1 | tee /dev/fd/2 - | tail -1)

[mikeserv]

@DanielAlder - those dont really have anything to do with the shell. theyre just files. or... links to file-descriptors anyway. but the shell doesnt put them there or control their behavior at all - those are handled by the kernel. the shell just does open() and dup2() with file-descriptors and it treats those exactly the same as it does any other file.

[Daniel Alder]

You don't know the history, and I didn't know the current situation. Not many years ago, there was no /dev/fd/ in the system. It was compiled directly in bash like /dev/tcp/ too which can be used to open tcp connections. Actually, bash still supports the emulation in case it can't find /dev/fd. Obviously I missed some developments of /proc/self, udev and the kernel so I didn't know that this is now system default.

[mikeserv]

@DanielAlder - it was ksh, actually, not bash that started all that - especially the /dev/tcp socket biz. in fact | isnt even a pipe at all in ksh - its a socket, and /dev/fd/[anything] fails for it. guess they came full circle.

[Scott - Слава Україні]

(1) It seems to me that this does not do what the question asks for.  (2) TLDP is not a good source of information; please do not use it.  (3) I don’t see much correlation between the page you linked to and your answer.  Why are you giving TLDP credit (/ blame)?  (4) The question is about a pipeline involving ls, sort, uniq and grep.  It’s good manners to answer the question that was asked — and yet your answer shows a pipeline involving cat (a UUOC, BTW) and xsel. … (Cont’d)

[Scott - Слава Україні]

(Cont’d) … (5) Please don’t clutter our site with answers that you know to be wrong, unless you have a good reason to do so.  Instead of adding a second answer (and, inexplicably, calling it an “earlier answer”), just delete the wrong answer and replace it with the one that’s less bad.

[user7543]

Or use the output of the tty command? tee `tty` etc.

[user7543]

ls /bin /usr/bin | sort | uniq | tee /dev/tty | grep out | wc > out only contains the output from wc.

[Colin Pitrat]

The exact command provided doesn't work for me: $ 3>&1 ls |( tee >&3 ) >/dev/null gives bash: 3: Bad file descriptor

[geek-merlin]

Thanks @ColinPitrat, i seem to have missed something, but using exec it works for me.

[Colin Pitrat]

Cool, this makes it a great answer because it's now more portable and conceptually simpler than all the others.

[Admin]

On Linux at least you can also use tee /dev/stdout.