Piping and Redirection in Unix


bash scripting and running a bash script

Something new, yet not unfamiliar, in yesterday’s Unix Systems Programming lecture about Piping and Redirection. This is another of the skills which we learnt in Web Systems, but I confess I didn’t really completely understand it back then, although I knew how to produce the result I needed for the assessed exercises all the same.

First, some terminology

In Unix, commands usually receive input via the keyboard and the output is produced on the screen. These commands are referred to as filters, and is the default behaviour, however this input and output can be redirected elsewhere.

When a command or program begins to execute, three I/O streams are already open:

  • “Standard” input (STDIN, 0) from keyboard
  • “Standard” output (STDOUT, 1) to screen
  • “Standard” error (STDERR, 2) to the screen (error messages)

Piping can redirect STDOUT to the STDIN of anther though, and usually uses the character |. It does not also redirect the STDERR. Each command is executed as a separate process, however, and done in parallel.

To redirect all of these I/O streams, you can use several commands:

  • (command) > file.txt will save output from command to specified file (file.txt, in this case), but won’t save errors. Explicitly written 1>.
  • (command) < file.txt redirects input of command to come from the specified file (file.txt here). Explicitly written 0<.
  • (command) >> file.txt appends output from command to existing or non-existent file (will create).
  • To redirect STDERR explicitly: 2>
  • To redirect STDOUT and STDERR explicirly: 2>&1, though the numbers can be reversed. The & is important otherwise bash will think it is a file name.

Using these produces different results, so can be useful in manipulating data… though I would suggest you try anything step by step, and always have redundancies, since it can be easy to screw up (for me, anyway).

Some examples

Just some examples of classwork that may be interesting to remember. In the first example we want to see who else is logged in on UTS’ rerun server, but print in out in a more readable manner, ie. User x is logged in from (address) at (date and time). The solution as follows:

-bash-4.0$ who | awk '{ print "User "$1" is logged from "$6" at "$3 $4 $5
> }'

Where $1 - $6 are different columns, awk intelligently sorts the columns without having to tell it a delimiter.

To list the files in one’s home directory, but only show the inode number (or the pointer), plus sort this list of numbers in ascending order, we can do the following:

-bash-4.0$ ls -i ~ | awk {'print $1'} | sort -n
83204
106559
107270
108172
108186
108187
108188
108426
108533
108538
108539
108541
108542
108543
658830

And finally, just an example of a simple calculator, based on a specified output. It took me a while to realise the calculation needed to be within $(( calculation )), rookie mistake.. Certainly this could be written better with elif, I have seen, but this was just the first attempt.

-bash-4.0$ cat calc.sh 
#!/usr/bin/env bash

# Accept two numbers as arguments
# Arithmetic calculation will be executed on these
# Allows user to select arithmetic sign and calculate result

echo PLEASE SELECT ARITHMETIC SIGN:
echo '1) +'
echo '2) -'
echo '3) *'
echo '4) /'
read sign
echo "YOUR SELECTION: "$sign
if [ $sign -eq 1 ]
then
   calcresult=$(($1 + $2)) 
else if [ $sign -eq 2 ]
     then
        calcresult=$(($1 - $2))
     else if [ $sign -eq 3 ]
          then
             calcresult=$(($1 * $2))
          else if [ $sign -eq 4 ]
               then
                  calcresult=$(($1 / $2))
               fi
          fi
     fi
fi
echo "Let's see....? The answer is: "$calcresult

Anything else you think I should know for the big, bad Unix quiz next week? :)

Related Posts

Kirinyan
Bash Scripting Adventures at UTS