Standard input/standard output/standard error and redirection

In Linux, there are two ways to view the output of a shell command, one is to display the output on the terminal screen, and the other is to save the output to a file.

In this tutorial, we will use examples to illustrate the essential knowledge in shell scripting, namely standard input, standard output and redirection.

What is standard input/standard output/standard error

Everything is a file in Linux, including input and output. And each process can open nine file descriptors at the same time.

The reserved file descriptors are 0, 1, 2. When the command is run, the process started by the command will automatically open the three reserved file descriptors.

0  standard input, the full name is standard input, referred to as stdin, the default is keyboard input, and the corresponding file descriptor is /proc/self/fd/0.

1 standard output, the full name is standard output, referred to as stdout, the default is to output to the screen, and the corresponding file descriptor is /proc/self/fd/1.

2 Standard error, the full name of standard error, referred to as stderr, is output to the screen by default, and the corresponding file descriptor is /proc/self/fd/2.

These file descriptors can be used to control the input and output of command programs or scripts. You need to fully understand these three concepts.

Because it is the backbone of shell scripts or programs. Therefore, we will describe each of them in detail in the form of examples to help you understand.

Standard input

STDIN Standard input, that is, file descriptor 1, defaults to the keyboard. A file can be specified as standard input using the redirection symbol <.

If you replace standard input with a file, the redirection symbol < will pass the file data as standard input to the specified command.

Example commands cat < archive.tar | gzip -c > archive.tar.gz use the redirection symbol < to pass the archive.tar file as standard input to the cat command.

After the cat command receives the standard input, it writes the contents of the archive.tar file to the standard output, and finally passes it through the pipeline to the gzip command for compression.

cat < archive.tar | gzip -c > archive.tar.gz

Standard output

STDOUT Standard output, that is, file descriptor 2, defaults to the screen. Standard output can be redirected to a file using the redirection symbols >>, >.

Data can also be appended to a specified file using the redirection symbol >>. For example, the command pwd >> log will append the standard output of the pwd command to the log file.

The redirection symbol > overwrites the specified file with standard output. For example the command > filename will clear the file filename.

pwd >> log
> filename

Standard error

ls file2 > file The command attempts to redirect standard output to a file using the redirection > notation . If there is no file2 in the current directory.

The ls command will print an error message ls: cannot access 'file2': No such file or directory, which is commonly referred to as stderr.

By default, the shell sends standard error to screen. If you need to redirect standard error to a log file, you can use the redirection symbols > / >> to redirect errors.

ls file2

ls: cannot access 'file2': No such file or directory

Redirect error

As mentioned in the introduction, 2 is standard error, so we redirect the error by putting file descriptor 2 before the redirect symbol.

Assuming that the xfile file does not exist in the current directory, now running the command ls -l xfile 2> log will redirect the error to the file log, which 2> means redirecting the standard error.

As you can see the error message is not printed on the screen, but written to a file, you can run the cat command to view the contents of the log file.

ls -l xfile 2> log
cat log

redirect error and standard output

To redirect standard error and standard output, each redirection symbol must be preceded by the correct file descriptor. Multiple redirection symbols can be used in the same command to redirect standard output and standard error.

Suppose the file file1 exists in the current directory, but the file file2 does not exist. Running the command now ls flie1 file2 will produce both standard error and standard output.

For some reason, you may want standard error to be written to the error.log file and standard output to be written to the access.log file.

At this point, you can use the file description 1 and the redirection symbol > to write to the access.log file, and the combined symbol is 1>.

The file descriptor 2 and the redirection symbol are > written to the error.log file, and the combined symbol is 2>. So the final command is ls file1 file2 2> error.log 1> access.log.

ls file1 file2 2> error.log 1> access.log

You can also use the &> symbol to redirect standard output and standard error to the same file. For example commands ls file1 file2 &> log.

You may see commands that redirect stderr and stdout differently than you usually see in other tutorials.

In order to make it easy to understand how to redirect standard error and standard output, another command that you often use is not explained here, which will be mentioned in the last section of this tutorial.

ls file1 file2 2> error.log 1> access.log
ls file1 file2 &> log

Permanent redirect

There are two ways of output redirection: temporary redirection and permanent redirection. For temporary redirection , the > or >> symbols can be used.

If you have a lot of data to redirect, you can use the exec command for permanent redirection.

Permanent redirects are not really the only type of redirection that can be modified at any time using the exec command.

Permanent redirection is equivalent to creating a file descriptor for the process, and the standard output, standard error, and standard input of the next run command all use the same file descriptor.

For example, the std.sh script will redirect all standard output after the exec command, which is the standard output generated by echo.

If we run the cat command to view the log file, we will see the standard output of the echo command.

You can use the exec command multiple times to redirect different file descriptors, for example exec 2> error.log the command redirects standard error to the error.log file.

It is still difficult to understand when you read this, please try to run the std.sh script, understand the running sequence, and run the cat command to view the two files error.log and log.

#!/bin/bash
exec 1> log #Permanent redirect stdout
echo "Permanent redirection"
echo "from a shell to a file."
echo "without redirecting every line"

exec 2> error.log #Permanent redirect stderr
echo "Script Begining ..."
echo "Redirecting Output"

In addition to using the exec command to redirect standard output, standard error. Standard input can also be redirected using the exec command. The default standard input is STDIN, which is file descriptor 1, usually the keyboard.

exec 0< log The command uses the log file as standard input instead of the default keyboard. Typically Linux system administrators use this technique to read log files and process them.

The stdin.sh script is simple. You should know how to use the read command to get user input. If a file is redirected to standard input, the read command will attempt to read the contents of the file.

#!/bin/bash
exec 0< testfile
total=1
while read line; do
	echo "#$total: $line"
	total=$(($total + 1))
done

Nohup command redirects standard error and standard output

Sometimes you may not want to see any output. We redirect all output to the black hole, the empty device file /dev/null. This situation is common when background processes are started with the nohup command .

You may have seen symbols 2>&1 when starting background processes with the nohup command , but rarely used this way to 2> /dev/null 1> /dev/null redirect to an empty device file /dev/null.

Actually ls -al file1 file2 2> /dev/null 1> /dev/null and ls -al file1 file2 > /dev/null 2>&1 are not equal commands.

2> /dev/null 1> /dev/null Instead > /dev/null 2>&1 of first redirect standard error to standard output and then to an empty device file.

ls -al file1 file2 2> /dev/null 1> /dev/null
ls -al file1 file2 > /dev/null 2>&1

If you 2>&1 were redirecting standard error messages to standard output, you might wonder 2>1 what the result would be, we can run the command ls -al file1 file2 2>1 to verify this.

After running the command, you may notice that file 1 exists in the current directory, and you will find that the standard error is written to file 1 when you run the command cat 1. ​

That is to say 2>1, the standard error will be redirected to file 1, so 2>&1 the &1 refers to the standard output.

ls -al file1 file2 2>1
cat 1

ls: cannot access 'file2': No such file or directory

Conclusion

Now you understand standard input, standard output, standard error and how to redirect them.