Background Jobs on the Linux Workstations
This document covers these subjects
Policies on background jobs:
- If you are actually sitting at a machine, there are no limits on
the number of background jobs you may have on that machine.
- You are limited to one unattended background job, on one machine,
at a time.
- That job must be `nice'd (that is, it must be set to run at a low
priority) to minimize the impact on other users of the machine.
- Locking the screen and leaving the machine is not an acceptable
way of running unattended jobs.
- If you violate these rules, your background jobs will be killed
and you may lose your computing privileges.
Foreground vs. background jobs:
When you type a command at a UNIX shell prompt, normally the shell
will wait until that command has completed before prompting you for
another command. This is called foreground processing. Only one
command at a time can be running in the foreground in any window,
because the shell won't prompt for another until the command that
is running has finished.
If you don't want to wait for a command to finish before you see
the prompt again, you can run the command in the background. Any
shell command can be run in the background by typing an ampersand
(&) at the end of the command line. The following command:
wc /hp-ux &
will count the number of bytes in the UNIX kernel, and it will take
a second or two to complete. Try it with and without the ampersand
and notice how the shell doesn't wait for the command to complete
when you execute it in the background. When you start a background
job, you'll get a message that looks like this:
The number in brackets is a `job number' assigned by the shell to the
background job, and the second number is a `process number' assigned
by the operating system. When a background job is completed,
you'll get a notice from the shell that looks like this:
 Done wc /hp-ux
Output and input for background jobs:
When background jobs have output, they write it to the screen.
When the current foreground job and one or more background jobs are
all writing the the screen at once, the outputs are mingled
together and the result can be very confusing.
When background jobs require input from the keyboard, they stop
running. You must bring them into the foreground in order to
supply them with input from the keyboard.
Because of this, it is usually a good idea to redirect the input
and output of background jobs.
Redirecting input and output:
The output of any shell command can be redirected to a file
(instead of the screen) by using a greater than (
followed by the file name. The less than (
<) character is an input
redirection operator - it will cause the command to read its input
from a file rather than from the keyboard.
Output redirection can be used for any command, not just background
commands. For instance, the command
ls -R ~ >listing
will do a directory listing of your home directory and all of its
subdirectories and save the output in a file called listing. You
do not need to create the output file in advance---it will be
created if it does not already exist.
Caution! When you redirect output to an existing file, that file
is overwritten. The previous contents will be lost. If you wish
to append the output of a command to a file, use the redirection
operator >> instead of >.
To use input redirection, you must first create the file and put
some appropriate input into it. You can do this with an editor
such as emacs or vi. For instance, if the file testrun contains a
series of Matlab commands, then the command
matlab <testrun >testrun.out
will cause Matlab to take its commands from the file testrun and to
put the results in the file testrun.out.
Unattended background jobs:
When you logout, any jobs you had running in the background will
continue to run until they are done. If you redirected the output
to a file, you will be able to look at that output later. This is
very convenient for jobs involving lots of numerical calculations,
or any other task which takes minutes or hours to complete, as you
don't have to sit and wait for the result. These jobs are called
unattended background jobs, because you are not sitting at the
computer, waiting on the result.
However, it is a rule of this computing site that unattended jobs
must be nice'd. This means that they run at a lower priority. The
jobs will still run to completion, but they will not interfere with
other people using the machine. For example,
nohup nice matlab < inputfile > outputfile &
will run Matlab in the background at a lower priority (with input
and output redirected to files). csh and
tcsh have a built in nice command that follows the
syntax listed above. If you are using a different shell, or if
you are explicitly invoking /bin/nice, see the man
page for the correct syntax.
If you forget to nice a background job when you start, or if the
job takes longer than you anticipated and you decide to log out and
go to lunch, you can use the command
renice -n 19 to lower the
priority of already-running jobs. The argument to the renice
command is a process number (the second of the two numbers reported
when you started the job.) For example, the command:
renice -n 19 8397
will lower the priority for the job whose process number is 8397.
If you've forgotten the process number, you can use the command
to list all of your jobs and their process numbers. Note: the jobs
command only knows about jobs started from the same window. See
the manual page for the ps command to learn how to find out about
jobs started from a different window.
Running multiple unattended background jobs is not allowed:
It is a rule of this computer site that you can only run one
unattended background job, on only one machine, at a time.
However, often it's necessary to run several calculations, or run
the same lengthy calculation several times with different inputs.
In this case, you must arrange for the calculations to run one at a
time instead of starting up many separate background jobs.
To run your commands one at a time, create a file containing a list
of the commands you want to run. This is called a `shell script'.
Then you can run the shell script, and each of the commands in it
will be executed in turn. You run a shell script by typing its
file name at the shell prompt (don't forget the & if you want it to
run in the background and nice if you plan to leave before it
finishes.) You may need to set execute permission on the file so
that the script is runnable---use the chmod command, like this:
chmod u+x filename
Commands for manipulating jobs:
The shell provides a number of commands for manipulating background
|(Press Z while
holding down the Ctrl key.) Suspends the current foreground job. You
can type this any time and you'll get a shell prompt. You can then
put the current job into the background with the bg command or resume
running it in the foreground with the fg command.|
current background jobs in the current window.|
|Brings a background
job to the foreground.|
|Restarts a stopped
or suspended background job.|
|Stops a background
job. This doesn't kill the job, but it will not continue executing
until it is restarted.|
|Waits for all
background jobs to terminate.|
|Runs jobs at lower
csh manual page for more details on how to use these commands. The
ps commands have their own manual pages.