redirecting to /dev/null

Question

I'm reading an example bash shell script:

#!/bin/bash

# This script makes a backup of my home directory.

cd /home

# This creates the archive
tar cf /var/tmp/home_franky.tar franky > /dev/null 2>&1

# First remove the old bzip2 file.  Redirect errors because this generates some if the archive
# does not exist.  Then create a new compressed file.
rm /var/tmp/home_franky.tar.bz2 2> /dev/null
bzip2 /var/tmp/home_franky.tar

# Copy the file to another host - we have ssh keys for making this work without intervention.
scp /var/tmp/home_franky.tar.bz2 bordeaux:/opt/backup/franky > /dev/null 2>&1

# Create a timestamp in a logfile.
date >> /home/franky/log/home_backup.log
echo backup succeeded >> /home/franky/log/home_backup.log

I'm trying to understand the use of /dev/null 2>&1 here. At first, I thought this script uses /dev/null in order to gracefully ignore errors, without causing the script to crash (kind of like try catch exception handling in programming languages). Because I don't see how using tar to compress a directory into a tar file could possibly cause any type of errors.

Solution 1

No, this will not prevent the script from crashing. If any errors occur in the tar process (e.g.: permission denied, no such file or directory, ...) the script will still crash.

This is because of using > /dev/null 2>&1 will redirect all your command output (both stdout and stderr) to /dev/null, meaning no outputs are printed to the terminal.

By default:

stdin  ==> fd 0
stdout ==> fd 1
stderr ==> fd 2

In the script, you use > /dev/null causing:

stdin  ==> fd 0
stdout ==> /dev/null
stderr ==> fd 2

And then 2>&1 causing:

stdin  ==> fd 0
stdout ==> /dev/null
stderr ==> stdout

Solution 2

To understand "redirecting to /dev/null" easily, write it out explicitly. Below is an example command that tries to remove a non-existent file (to simulate an error).

rm nonexisting.txt 1> /dev/null 2> /dev/null

1. is for stdout. Sends info logs to /dev/null
2. is for stderr. Sends error logs to /dev/null

Below are couple of enhancements.

Enhancement 1: You can replace 1> with just >. This is because 1 is the default stdout and you can ignore mentioning defaults.

rm nonexisting.txt > /dev/null 2> /dev/null

Enhancement 2: You can replace the 2nd file redirect (> /dev/null) with a file descriptor duplication (>& 1). This is because /dev/null is already pointed to by stdout 1.

rm nonexisting.txt 1> /dev/null 2>& 1

Enhancement 3: This is such a common operation, that many shells have a shortened form of this as a single &> operator.

rm nonexisting.txt &> /dev/null

My suggestion: Stick to the first option. Write out the commands explicitly instead of using pointers. Takes little to no extra effort but much easier to understand and explain.

Solution 3

I'm trying to understand the use of "> /dev/null 2>&1" here.

(note that I added the redirection before /dev/null in your question.)

The above would redirect the STDOUT and STDERR to /dev/null. It works by merging the STDERR into the STDOUT. (Essentially all the output from the command would be redirected to the null device.)

... without causing the script to crash (kind of like try catch exception handling in programming languages).

It's not quite like a try/catch or anything. It simply silences any sort of output (including error) from the command.

Because I don't see how using tar to compress a directory into a tar file could possibly cause any type of errors.

It could result in errors for a number of reasons, including:

• Inadequate permissions on the file(s) you're attempting to archive or on the file that you're attempting to write to
• Lack of disk space in order to create the archive

Solution 4

When you run CMD > /dev/null 2>&1

STDOUT redirects to /dev/null, and then STDERR redirects to THE ADDRESS of STDOUT, which has been set to /dev/null , consequently both STDOUT and STDERR point to /dev/null

Oppositely, when you run CMD 2>&1 >/dev/null

STDERR redirects to THE ADDRESS of STDOUT (File descriptor 1 in that moment, or /proc/self/fd/1), and then STDOUT redirects to /dev/null , but STDERR keeps redirecting to fd1!! As a result the normal output from STDOUT is discarded, but the errors coming from STDERR are still being written onto the console.

Solution 5

cat nonexistantfile.txt &>/dev/null

This redirects both STDOUT both STDIN
And is equivalent to

cat nonexistantfile.txt >/dev/null 2>&1

View more solutions

Comments

[terdon]

Just a trick to avoid unnecessary output. As for why tar can cause errors: because the target directory does not exist, because the source doesn't, because you don't have write access to the target, or read to the source, because tar is not in your $PATH, because tar crashed (you never know), because there's no space left on the device, because the tar version changed and now requires different syntax, because the disk caused an I/O error. I'm sure you could find more.

[Weishi Z]

As of understanding > /dev/null 2>&1 , this command result in stderr ==> stdout, so stderr still get printed to stdout??

[cuonglm]

@WeishiZeng: Yes, but at that time, stdout was pointed to /dev/null.

[Oscar]

why mention & before fd 1 (stdout) and not before fd 2 (stderr) ?

[kev]

probably not too important, but what is fd?

[Alex Bitek]

@kev en.wikipedia.org/wiki/File_descriptor

[dbmikus]

Why does: CMD > /dev/null 2>&1 work but CMD 2>&1 > /dev/null still give me STDERR?

[Henk Langeveld]

Recommended: Use 2>& 1 in code examples to stress that the number and the ampersand are considered to be part of the redirection operator. It is common for redirection to a file to have a space between > and /path/to/file, redirection to a file descriptor is essentially the same thing.

[cuonglm]

@HenkLangeveld Check pubs.opengroup.org/onlinepubs/9699919799/utilities/… [n]>&word is the general form. There's no need for space here.

[Antonio Mano]

fd mens File Descriptor. Linux treats all devices as files.

[törzsmókus]

@dbmikus You should have asked that as a separate question :) Nevertheless, the first one says "redirect writes to file descriptor 1 to /dev/null, and then redirect writes to file descriptor 2 to the same place as the writes to file descriptor 1 are going" . The second one says "redirect writes to file descriptor 2 to the same place as the writes to file descriptor 1 are going, and then redirect writes to file descriptor 1 to /dev/null". I also struggled with this. Idea from a coworker and from unix.stackexchange.com/a/497215/12428 for a detailed description.

[cuonglm]

@törzsmókus In CMD 2>&1 >/dev/null, at the time 2>&1 occurs, file descriptor 1 still point to standard out, so standard error will go to standard out. > /dev/null takes no effect.

[törzsmókus]

@cuonglm > /dev/null does take effect; the stdout (fd 1) of CMD goes there (i.e. disappears), and the stderr (fd 2) of CMD goes to the standard out.

[cuonglm]

@törzsmókus I mean take no effect to previous standard error redirection.

[Kusalananda]

No, redirections are handled left to right. In your example, standard output is redirected to filename, then standard error is redirected to wherever standard output is currently going (to filename). If it was the opposite way around, standard error would end up on the terminal while only standard output was redirected to filename. Also, in command >file1 2>file2, file2 would not be created if file1 could not be created (no matter whether file1 and file2 were actually totally different pathnames).

[TNT]

A more brief way to redirect both stdout and stderr to /dev/null is: command &> /dev/null.

[vstepaniuk]

&> /dev/null did not work for me in a script file, outputting the whole stderr after the script is completed, but > /dev/null 2>&1 worked well.

[Fritz]

"... the script will still crash." -- No it will not crash, because set -o errexit is not set. Only the tar command will fail and the rest of the script will happily continue on trying to execute nonsense commands on non-existing files.

[NONONONONO]

this is actually quite the better answer, as it is a "this means that", rather than a generic "this does that"

[Meir Gabay]

Superb, I've been struggling to understand this topic for ages, finally I know how to use it. Explicit commands are the best way to explain and to memorize.

[Crescent Fresh]

Love this answer, thank you

[thomp45793]

"Stick to the first option" is good advice. "Enhancement 3" is substantially less portable.

[Hunter Kohler]

@thomp45793 Portability? Readability? __no __my __attribute__((cringe)) __code __looks __attribute__((allways_inline)) __like __this

[Hunter Kohler]

I've gotta say, including anything from google as a response hurts my soul because I already Googled it and then came here (╥_╥)

[Kusalananda]

The question does not use wget at all.

[rickard]

i change my question. instead of using >/dev/null 2>&1 Could you use : -O /dev/null -o /dev/null . is this the same?

[Kusalananda]

In the question, >/dev/null 2>&1 is used with both tar and with scp. You can't use -O /dev/null -o /dev/null with both of those commands. Neither of those commands would understand -q -spider.

[G-Man Says 'Reinstate Monica']

If you have a new question, please ask it by clicking the Ask Question button. Include a link to this question if it helps provide context. - From Review