Concurrent BASH: Utilize your Dual-Core or Quad-Core processors

We can use batch command to send a series of tasks into a queue, and let it runs by itself. It monitors the load average and admit more tasks when it is low. However, the load average is not changing fast enough for many purposes. In addition, it lacks synchronization facility which is important when you want to make sure some concurrent operations complete before you proceed. Making it user-unfriendly, we have to write the script into a command stream for the batch command.

Hence, I'm trying to design a system to overcome problems as mentioned. Your BASH program can do like the following:

prun.bash:
#!/bin/bash
cd /tmp/xman
date
for i in *.ogg ; do
    spawn oggdec *.ogg
done
synchronize
echo "Done"
date
exit 0

xman@sai xman $ time ./prun.bash
Sun Aug 27 21:44:13 SGT 2006
Done
Sun Aug 27 21:44:26 SGT 2006
real    0m13.279s
user    0m0.020s
sys     0m0.072s

Isnt that's cool? Multiple oggdec which decode ogg files will run in parallel on your dual-core or quad-core machine but with limited max number of threads at a time. The Dual-Core machine completes the decoding in merely 13.3s.

Now let me decode using serial script. The serial script:

xman@sai xman $ cat run
#!/bin/bash
date
for i in *.ogg ; do
    oggdec "$i" &> /dev/null
done
echo "Done"
date
exit 0

xman@sai xman $ time ./run
Sun Aug 27 21:38:26 SGT 2006
Done
Sun Aug 27 21:38:46 SGT 2006
real    0m20.625s
user    0m19.273s
sys     0m1.004s

Serial script is unable to utilize two cores simultenously. Hence, it's important to use concurrent facility in order to fully utilize the processors that you are using.