[mdlug] UNIX tips: Learn 10 good UNIX usage habits

[Robert Citek]

On Mon, Mar 10, 2008 at 1:54 PM, Aaron Kulkis <akulkis3 at hotpop.com> wrote:
>  You write a shell script to do some task...
>
>  And then someone else writes a shell-script, which has a giant
>  for loop and calls your script in their script....
>
>  It's ridiculous cpu cycle waste...

Figured I'd try to replicate the authors work

$ yes and | head -20000000 > tmp/a/longfile.txt

$ time grep -c and tmp/a/longfile.txt
20000000

real    0m2.142s
user    0m1.340s
sys     0m0.036s

$ time cat tmp/a/longfile.txt | grep -c and
20000000

real    0m1.383s
user    0m1.344s
sys     0m0.108s

$ yes and | head -2 > tmp/a/longfile.txt

$ time grep -c and tmp/a/longfile.txt
2

real    0m0.004s
user    0m0.004s
sys     0m0.000s

$ time cat tmp/a/longfile.txt | grep -c and
2

real    0m0.005s
user    0m0.008s
sys     0m0.000s

$ yes and | head -200000000 > tmp/a/longfile.txt

$ time grep -c and tmp/a/longfile.txt
200000000

real    0m24.169s
user    0m13.549s
sys     0m0.576s

$ time cat tmp/a/longfile.txt | grep -c and
200000000

real    0m20.297s
user    0m12.769s
sys     0m1.352s

20 million records, about 2 seconds.
2 records, about 5 milliseconds
200 million records, 25 seconds.

Most of my work is well under 200 million records.

BTW, this is my laptop running Ubuntu 7.10 with dual 2GHz cpus and 1
GB RAM.  Not a screamer by any stretch.

Regards,
- Robert