Woof: slacko version problem

01micko alerted me to a problem with building Slacko in Woof, migrating from Slackware 13.37 to 14.0.

On ibiblio.org, there is a directory 'pet_packages-slacko' with PETs.
There is also a matching database file 'Packages-puppy-slacko-official'.

The problem, basically, is that building Slacko from Slackware 14.0 binary packages is currently accessing the same PET repository (as above paragraph) and the current release of Slacko (based on Slackware 13.37 binary packages).

DB entries to have two fields that identify the specific release compatibility, "...|slackware|13.37|..." or "...|slackware|14.0|...", so in theory pet_packages-slacko can have PETs for 13.37 and 14.0 -- the Woof build system and PPM will choose the right one.

However, 01micko has reported that a great many of the existing 13.37-based PETs in the repo are not compatible with the 14.0-based Slacko, so upgraded PETs would have to be created for all.
And of course the old and new PETs can remain in the repo.

The problem described above is specific to Slacko. Most other pups use a different name for each release. For example, for Ubuntu-based builds, there was Lucid Puppy, which has repo directory pet_ackages-lucid and db file Packages-puppy-lucid-official, then my later Ubuntu-based Precise Puppy, with repo dir pet_packages-precise and db file Packages-puppy-precise-official.

It is keeping the same name "slacko" (variable DISTRO_FILE_PREFIX in file DISTRO_SPECS is actually the problem) that is causing the problem for Slacko.

I am proposing a solution. A new variable in file DISTRO_SPECS, let's call it 'DISTRO_DB_SUBNAME', and this will be set differently for each release of Slacko.

For compatibility with existing puppies and all other builds, if DISTRO_DB_SUBNAME does not exist then it will be set to same as DISTRO_FILE_PREFIX.
If it does exist, it will be set to same as DISTRO_FILE_PREFIX.

Unless specifically set to something else. In the case of 14.0-based Slacko, choose a new name, say "slacko2" or "slacko14".
Woof and PPM will then access repo dir pet_packages-slacko14 and db file Packages-puppy-slacko14-official.

I think that the Mageia-based pup also has this problem.

I am posting this tonight, if anyone wants to comment, please do. I intend to implement it in Woof tomorrow.


Posted on 1 Nov 2012, 19:02


Comments:

Posted on 1 Nov 2012, 21:24 by 01micko
slacko fix
Sounds ok, I'll test it out tomorrow night

Thanks for looking at the issue.


Posted on 2 Nov 2012, 20:01 by BarryK
Re Woof Slacko fix
I have made the changes in Woof.

I have created puppylinux/pet_packages-slacko14 directory on ibiblio.org, and shortly will be copying all the PETs out of pet_packages-slacko that are needed for the 14.0-based build.

I won't delete any from pet_packages-slacko. That will remain as the repo for the 13.x-based Slackos, and 01micko can restore that appropriately.

There will also be two db files:
Packages-puppy-slacko-official
Packages-puppy-slacko14-official

...currently the (local, in Woof) content of the first is as it was before in Woof, the second is the new Packages-puppy-slacko-official (renamed) that 01micko has sent me.

These db files will need to be modified to match the content in the dirs. The first for 13.x-based Slacko, second for 14.x-based Slacko.

Right now I am running the '1download' script, which creates a list of all the needed PETs that are currently missing from dir pet_packages-slacko14.
When the script completes, I will copy those PETs across, then rerun 1download.



Posted on 3 Nov 2012, 6:30 by Iguleder
Faster Woof
Barry, I have fast package list converters for about 15 distros here, all written in awk.

It takes 7 minutes and 9 seconds for my 0setup equivalent to process the package lists of all repositories, of all of these distributions, under my Eee PC. The result package list has a format very similar to the PPM one.

I licensed this code under a BSD license, so you can use it to speed up Woof. I don't understand why it has to take half an hour for Woof to process the Debian package lists.


Posted on 3 Nov 2012, 8:25 by BarryK
DISTRO_DB_SUBNAME implemented
01micko,
OK, it is done. Woof commits:

http://bkhome.org/fossil/woof2.cgi/info/f92926df2b
http://bkhome.org/fossil/woof2.cgi/info/6e52193790
http://bkhome.org/fossil/woof2.cgi/info/adb11a55d1

I ran the scripts to do a Slacko build, but couldn't do it properly as not all PETs are uploaded. Also, one of them has a broken pinstall.sh script.

It all seems sane though.

One thing, sandbox3/0builtin_files_slacko-5.* is broken. That will need to be fixed, that's in script 3builddistro I think.



Posted on 3 Nov 2012, 19:26 by 01micko
running 1download
I am starting with an empty pets dir and running 1download right now so I can reconcile the repo.

I notice the GUI borks the DISTRO_SPECS file. That will catch a few out!

Let you know how it goes..


Posted on 3 Nov 2012, 20:46 by 01micko
pets uploaded
I have finished 1download and 95% reconciled the repo. The 5% that I didn't upload are either not going to be included or will have upgrades in my next build so after about Tuesday everything should be sane. I say Tuesday because (hopefully) by then I will have released beta3.

Thanks


Posted on 4 Nov 2012, 19:05 by BarryK
Re Slacko beta3
01micko,
let's see, it is now Sunday night. Right now I am fixing a serious bug, well, two bugs and have fixed one of them, when save to entire partition.
Should have that done by Monday night.

I will also check out that borking of DISTRO_SPECS.



Posted on 6 Nov 2012, 14:00 by 01micko
Slacko PPM problem
Hi Barry

Well, I built my distro from latest woof and first thing I decided to test was PPM... yep failed.. it shows the old slacko repo by default even though "slacko14" is the greyed out checkbox (default).

I had a hunch that might be a problem.

Other than that everything else seems fine.. might run accross a few more DISTRO_FILE_PREFIX bugs..

Looks like beta3 might be a couple more days.


Posted on 6 Nov 2012, 14:11 by 01micko
udate on repo
Well on second run it's ok.. good enough for beta! A bit more testing and I'll release it maybe tonight (late).. the family gets upset when I upload stuff! Hogs my feeble bandwith.


Posted on 6 Nov 2012, 14:26 by 01micko
udate on repo #2
Ok, in 3buildistro line 1497 we have :

enabledrepos="Packages-puppy-slacko-official Packages-puppy-noarch-official "

What if I make it :

enabledrepos="Packages-puppy-${DISTRO_DB_SUBNAME}-official Packages-puppy-noarch-official "

?

I'll try it..


Posted on 6 Nov 2012, 15:39 by 01micko
repo #3
enabledrepos="Packages-puppy-${DISTRO_DB_SUBNAME}-official Packages-puppy-noarch-official "

Yes, that works..this is nearly RC but since woof has undergone a lot of dev in the last few days I'll let it go as beta3


Posted on 6 Nov 2012, 18:55 by BarryK
Re repo #3
01micko,
Good, I have made the change in 3builddistro.



Posted on 8 Nov 2012, 12:03 by 01micko
gconv-modules.cache
Well.. slackware does not ship gconv-modules.cache in it's glibc package, or glibc-solibs.. I searched my slackware-14 installation.. no such file.

Ok, it's easy to generate, just run iconvconfig..

It certainly does make a small difference on my fast box, can't imagine the difference on a slow machine..

So.. how to automate generation? I can do it at startup or it could be done in woof at the 3builddistro chroot stage. I guess I'll hack rc.sysinit and put it in there, on condition gconv-modules.cache doesn't exist.



Posted on 8 Nov 2012, 12:53 by 01micko
re -gconv-modules.cache
It's too much headache generating the file on the fly at boot (we need to consider frugal, frugal upgrade, full, full upgrade... bah!) so I hacked 3builddistro and posting the code. (untested, report soon)

At line 2485 I added the code, I'll post preceding and next lines.
if [ "$WOOF_HOSTARCH" = "$WOOF_TARGETARCH" ];then

if [ -d rootfs-complete/usr/share/glib-2.0/schemas ];then
[ -e rootfs-complete/usr/bin/glib-compile-schemas ] && chroot rootfs-complete /usr/bin/glib-compile-schemas /usr/share/glib-2.0/schemas
fi
#120523 seems need this too...
if [ -d rootfs-complete/usr/lib/gio/modules ];then
[ -e rootfs-complete/usr/bin/gio-querymodules ] && chroot rootfs-complete /usr/bin/gio-querymodules /usr/lib/gio/modules
fi
fi

# ***NOTICE*** cross-build, this will have to be executed at first boot... #01micko 121108
if [ "$WOOF_HOSTARCH" = "$WOOF_TARGETARCH" ];then
[ ! -f rootfs-complete/usr/lib/gconv ] && chroot rootfs-complete /usr/sbin/iconvconfig
fi

#120618 raspi debian squeeze, links as main browser, helpsurfer html viewer, gtkmoz missing...
#120620 problem, helpsurfer cannot display /usr/share/doc/index.html, so do not point gtkmoz at basichtmlviewer...
if [ ! -e rootfs-complete/usr/local/bin/gtkmoz ];then
ln -s defaultbrowser rootfs-complete/usr/local/bin/gtkmoz
fi





Posted on 8 Nov 2012, 12:56 by 01micko
doh
didn't paste the vital bit!

***NOTICE*** cross-build, this will have to be executed at first boot... #01micko 121108

if [ "$WOOF_HOSTARCH" = "$WOOF_TARGETARCH" ];then
[ ! -f rootfs-complete/usr/lib/gconv/gconv-modules.cache ] && chroot rootfs-complete /usr/sbin/iconvconfig
fi




Posted on 8 Nov 2012, 13:12 by 01micko
no
Forget above.. added to support/slacko/woof_base_hacks


Posted on 8 Nov 2012, 16:46 by BarryK
Re iconvconfig
01micko,
There is some history with this. Some time ago I was using iconvconfig, but it created a broken cache file.

I may have put some code into Woof, then removed it. I think it was the glibc template FIXUPHACK.

I probably posted about it on my old blog. There should be something in the Woof Fossil history also, and in the Forum.



Posted on 8 Nov 2012, 20:26 by 01micko
re Re iconvconfig
Yeah read all that stuff before posting the original.

My addition to the woof_base_hack worked as intended so I'll post you an update soon.

The problem was that slackware doesn't ship the file, and they are probably right in doing that. How can you have a pre-constructed .cache file?

Anyway, problem solved.