Standard Input, Output, and Error

Two kinds of output

A program usually produces two kinds of output: its results, and messages about how it’s getting on, especially errors. They go to two different places ¹:

• Standard output (stdout) for results.
• Standard error (stderr) for errors and status messages.

Programs also read from standard input (stdin). By default, input comes from the keyboard and both outputs go to your screen, which is why they look like one stream ^{1 2}.

Internally the shell numbers them as file descriptors: 0 for input, 1 for output, 2 for error ¹. Redirection changes where each one points.

Output to a file

> sends standard output to a file instead of the screen ^{1 2}:

me@linuxbox:~$ ls -l /usr/bin > programs.txt
me@linuxbox:~$ less programs.txt

Two things to know about >:

1. It empties the file first, before the command even runs. If the command produces nothing, or fails, you’re left with an empty file ¹. > file on its own, with no command, is a quick way to empty a file or create a blank one.
2. It overwrites silently. A reader of Shotts’ book, working as root in /usr/bin, typed ls > less to “see what happens”: it replaced the less program with a text listing ¹. > points at a file; |, next lesson, points at a program.

>> appends instead, creating the file if it doesn’t exist ¹:

me@linuxbox:~$ date >> worklog.txt
me@linuxbox:~$ echo "replaced the furnace filter" >> worklog.txt

Bash can refuse to overwrite with > if you turn on set -o noclobber; then >| forces it ³. Some people put that in their startup file.

Errors to a file

Redirect standard error with its number, 2> ¹:

me@linuxbox:~$ ls -l /bin/usr 2> errors.txt

To catch both streams in one file, there are two ways ¹:

me@linuxbox:~$ ls -l /bin/usr > all.txt 2>&1     # the classic way
me@linuxbox:~$ ls -l /bin/usr &> all.txt         # bash's shorthand
me@linuxbox:~$ ls -l /bin/usr &>> all.txt        # shorthand, appending

2>&1 means “send stream 2 wherever stream 1 is going right now.” Redirections are applied left to right, so the order matters: > all.txt 2>&1 points output at the file, then errors at the same place. Written the other way, 2>&1 > all.txt, errors are pointed at the screen (where output was at that moment), and only then does output move to the file ^{1 3}.

Throwing output away

/dev/null is a special file that swallows anything written to it ¹. Use it to silence what you don’t want:

me@linuxbox:~$ find / -name "*.conf" 2> /dev/null    # hide the "Permission denied" noise
me@linuxbox:~$ some-noisy-command > /dev/null         # keep only the errors
me@linuxbox:~$ some-command &> /dev/null              # silence everything

The first is something you’ll type constantly: searching the whole system as an ordinary user produces a wall of permission errors, and 2> /dev/null leaves just the results.

Input from a file

< feeds a file to a program’s standard input ^{1 2}:

me@linuxbox:~$ wc -l < /etc/passwd
34

Many programs take a file name anyway, so < matters most for programs that only read standard input. Notice the difference, though: wc -l /etc/passwd prints the name too; wc -l < /etc/passwd prints just the number, because wc never saw a file name.

cat with no arguments copies standard input to standard output, so cat > note.txt, then typing a few lines and pressing Ctrl-D, makes a quick file. Ctrl-D means “end of input” ¹.

Here documents

A here document feeds several lines of text, written right in the command, to standard input. Everything up to a line holding only the end marker is the input ^{1 3}:

me@linuxbox:~$ cat > grocery.txt << EOF
oats
apples
coffee
EOF

You’ll use these in scripts to write out config files and messages. A here string, <<<, feeds a single line: wc -w <<< "how many words here" ³.

Grouping commands

To send the output of several commands to one place, group them in braces. The spaces and the final semicolon are required ¹:

me@linuxbox:~$ { date; uptime; df -h /; } > status.txt

Your turn

So

Programs read standard input and write two streams, output and errors. > and >> send output to a file (and > empties it first), 2> sends errors, > file 2>&1 or &> sends both, /dev/null discards, and < and here documents supply input. The shell does all of this before the program runs.