Run linux commands via job queue

Often we have build long running jobs under linux which then call other shell scripts when they are finished.
Sometimes it would be possible to run such tasks partially concurrently, but for this we need some sort of job queue for the linux shell.
Unfortunally the cron/crontab does not allow such dependencies.

The simplest solution is to install the task spooler from this site.
You should however first look if your distribution has a binary package for it.

In debian you can install it via:
apt-get install task-spooler

After this, you can just put new jobs in the queue via the ts command.
Please note that in debian the command is named tsp to prevent name conflicts with the moreutils package.

When you just type ts(p) the system shows you all running/pending/completed tasks.

To put a new job in the queue you just type this:
ts(p) /home/user/mytask.sh

The job is then startet as soon as possible.
Per default the task spooler does start one task and waits until it is finished.
When the first task is finished it executes the next one from the queue and so on, until the queue is empty.

You can tell the task spooler to run multiple tasks concurrently,
you just set the maximum number of allowed concurrent executions with this command:
ts(p) -S 2
This would allow the task spooler to run up to two tasks concurrently.

You can also set priority on tasks, remove them  from the queue an so on.
Look at the man page for more details.

As of 0.7.3 the man page does not list all options, to see all available options, use the -h option.

The output of the jost is per default stored in files in the /tmp folder.
You can view them, or use ts(p) to send them via email to some destination.