Parallel Processing with Bash

There is often the need to run several bash scripts in parallel. To achieve this, one need to keep track of the running processes. Here’s my solution:

#!/bin/bash

# parallel child processes
PP=5

if [ -n "$1" ]; then
        sleep $1
        exit
fi

reaper () {
        echo "waiting to finish" >&2
        wait
        exit 0
}
trap reaper 0 1 2 3 11 15

while :;
do      J=$(jobs -p)
        echo "running: "$J
        N=$(echo $J|wc -w)
        if [ -z "$N" -o "$N" -lt $PP ]; then
            $0 $[ $RANDOM % 8 ] &
        fi

        sleep .2
done

Paste this snippet to a file, and set the executable flag.

$ cat > parallel.sh
$ chmod +x parallel.sh

The output will look like this:

$ ./parallel.sh
running:
running: 25431
running: 25431 25438
running: 25431 25438 25445
running: 25431 25438 25445 25452
running: 25438 25445 25452 25459
running: 25445 25459 25466
running: 25445 25459 25466 25473
running: 25445 25459 25466 25480
running: 25445 25459 25466 25480 25487
running: 25459 25466 25480 25487
running: 25459 25466 25480 25487 25499
running: 25459 25466 25487 25499
running: 25459 25466 25487 25499 25511
running: 25466 25487 25499 25511
running: 25466 25487 25499 25511 25524
running: 25466 25487 25499 25511 25524
running: 25466 25487 25499 25524
running: 25487 25499 25524 25541
running: 25487 25499 25524 25541 25548
running: 25487 25524 25541 25548
running: 25524 25541 25560
running: 25524 25560 25567
running: 25524 25560 25567 25574
running: 25560 25567 25581
running: 25560 25567 25581 25588
running: 25560 25567 25581 25588 25595
^Cwaiting to finish
waiting to finish
$

Comments are welcome.

See also a similar solution of Grumbel: Parallel Bash