site  contact  subhomenews

Woof: Making 0setup faster

October 23, 2009 — BarryK
I reported recently on the super-fast code that Wosh has created:

Wosh sent me version 0.0.1 of PPKG today, and it is nice but not yet quite what I need for building Karmic Puppy.

To fill the immediate need, I thought that I would take another look at the Debian/Ubuntu-to-Puppy database conversion code in the '0setup' script.

Testing the Ubuntu "main" repository db, the script took about 11 minutes. I applied some lateral thinking, and have now got it down to about 2 minutes.

Here is the new code:

while read DB_ONELINE
case $DB_ONELINE in
DB_fullfilename="`echo -n "$Filename" | rev | cut -f 1 -d '/' | rev`"
DB_path="`echo -n "$Filename" | rev | cut -f 2-9 -d '/' | rev`"
DB_version="`echo -n "$Version" | sed -e 's%^[0-9]:%%' -e "$v1PATTERN" -e "$v2PATTERN"`"
DB_pkgrelease="`echo -n "$Version" | sed -e 's%^[0-9]:%%' -e "s%${DB_version}%%" -e 's%^\-%%' -e 's%^\+%%'`"
DB_category="`${FIND_CAT} $Package "$Description"`"
DB_dependencies="`echo -n "$Depends" | sed -e 's%, %n%g' | cut -f 1 -d ' ' | tr 'n' ' ' | sed -e 's% $%%' -e 's% %,+%g' -e 's%,$%%'`"
[ "$DB_dependencies" != "" ] && DB_dependencies='+'"$DB_dependencies"
echo "$DB_pkgname|$Package|$DB_version|$DB_pkgrelease|$DB_category|${InstalledSize}K|$DB_path|$DB_fullfilename|$DB_dependencies|$Description|" >> $ONE_PKGLISTS_COMPAT
echo -n "$Package "
$(grep -E '^Package:|^Installed-Size:|^Architecture:|^Version:|^Depends:|^Filename:|^Description:' ${ONE_PKGLISTS_COMPAT}pre | tr '[t"|]' ' ' | tr -s ' ' | sed -e 's%^Installed-Size%InstalledSize%' -e 's%: %="%' -e 's% $%%' -e 's%$%"%')

Of course the big one is the "universe" repo. The database file to be processed is 24MB, a lot to churn through. Universe now takes 9 minutes, so all-up the conversion time will be about 12 minutes, apart from the time to download the Debian/Ubuntu db files.
The Debian repo is a bit smaller so conversion will be quicker.

This faster script is just a temporary fix, as I expect to move onto PPKG soon.


Viva Karmic Woof

0setup faster
Username: BarryK
"Oh yeah, one other thing. I got those faster speeds by also changing the first line to use the Ash interpreter: #!/bin/ash ...makes a huge difference.

Tags: woof