'xargs' utility, 0setup speed

I have upgraded to the full 'xargs' utility from the Busybox applet. This is in the 'findutils' package.
I was experiementing with using 'xargs' in 0setup script in Woof, and was getting "input too long" message, then decided that the full 'xargs' has far more useful commandline options.

I have experimented with using 'xargs' in the '0setup' utility to try and speedup the conversion of the Ubuntu/Debian databases to Puppy standard format. Previously the code used a pipe into a while-loop, which is very slow. Results:

main universe
xargs method: 4 minutes 2 sec 12 minutes 21 sec
pipe into while-loop: 6 minutes 58 sec 31 minutes 42 sec

Then I did some lateral thinking, devised a technique that involves rotating the generated database file so that rows become columns.

file rotation: 2 minutes 25 sec 12 minutes 57 sec

However, the file rotation method has turned out to be less reliable. Also, I wrote a little compiled utlity in Genie to perform the most time-critical part of the code, potentially reducing the total conversion time to a minute or two, however it segfaults partway through. So I have put that method on ice for now.

Why be concerned about the speed of 0setup?
0setup is a script in Woof, however it is also used in a running Puppy when the user wants to update the package database. The database files have to be downloaded and in the case of Ubuntu that is several megabytes, so that can take awhile if you are on dialup. The conversion to Puppy standard format will be on top of that, and the figure shown above of just under 32 minutes is I think a bit to long to wait.
12 minutes is more tolerable I suppose, given that a database update is something that will be done occasionally.

Anyway, haven't resolved it, this is all on hold for now.

Tags: woof