Woof standardised pkg databases

In the discussion after release of Woof alpha1 it became apparent that there is a problem with some packages not getting downloaded. I tested a Slackware build so that one at least seems ok.

As I looked into fixing the bug, I realised that to do it properly I really needed to implement what I had planned to do next, which is to pre-process package database files from the various distros into a standardised compact format.

So, I have implemented this in the '0setup' script. Testing it now. Each line of the standardised format looks like this:

pkgname|nameonly|version|pkgrelease|category|size|path|fullfilename|dependencies|description|

Example:
abiword-1.2.4|abiword|1.2.4|5|Document|999K|slackware/ab|abiword-1.2.4-5-i486.tgz|+aiksausus,+gtk2|a nice wordprocessor|

Pre-processing all the distros package database files into this same simple format has a number of advantages. For a start, it will greatly simplify the other scripts in Woof -- I'll do that today.

The standardised format is also very good for the package manager. It will be very easy, for example, to add the Ubuntu 'main', 'universe' and 'multiverse' repositories in the GUI window of the package manager.

Good, just finished testing Arch. Here is an example from 'Packages-arch-200902-core':

automake-1.10.2|automake|1.10.2|1|BuildingBlock||core/os/i686|automake-1.10.2-1ARCHTYPE.pkg.tar.gz|+perl,+bash,+texinfo|A GNU tool for automatically creating Makefiles|

Note that 'ARCHTYPE' would normally be '-i686' or in the case of architecture-independent packages it would be nothing, however I found the package naming to be inconsistent. For example, I found that the 'aterm' package, which does have a binary executable in it, is named 'aterm-1.0.1-1.pkg.tar.gz'. So, later scripts have to test for the existence of 'automake-1.10.2-1-i686.pkg.tar.gz' or 'automake-1.10.2-1.pkg.tar.gz'.

NOTICE:
To anyone who would like to try alpha1, just in case you haven't picked up this information from reading the feedback. The Arch build is currently broken, due to the problem mentioned at start of this post. I have only verified the Slackware build works. Should be fixed for alpha2, coming soon.

---

Comments

Unleashed
Username: BarryK
There were a couple of posts on the forum: [i]As I understand it you can build a standard Dingo-style Puppy using Woof and the packages from Pet_Packages-4, just like you can with Unleashed. Where the extra value arises is you can also choose to quickly add Debian, Slackware, Ubuntu or Arch compatibility for access to the vast repositories of those distributions.[/i] [i]Barry, while it is fresh in mind, why isn't there a "DISTRO_PKGS_SPECS-puppy" option?[/i] Yes, Woof can behave just like Unleashed, if provided with an appropriate DISTRO_PKGS_SPECS-puppy. So, you could use it to build Puppy 4.1.2, or 4.2 if the pet pkgs are available.

x86-64bit?
Username: Michalis
"Barry I'm wondering whether through woof will be possible to build entirely 64bit or multiprocessor, multithread puplets? At least will it be possible to add 64 packages along with the 32bit puppy packages? If woof could support 64bit, then arch's 64bit packages are good condensers. I know that with puppy you weren't planing to, but what about woof?

Vertical slash
Username: BarryK
"The vertical slash was chosen after considerable thought, as one of the characters less likely to cause any conflicts. Only developers are likely to be looking at the pkg database files, and they will know what it's about. I wanted the path and package name to be in separate fields, but if you do want them together: one-line-from-database | cut -f 7,8 -d '|' | tr '|' '/'

"Vertical slash"
Username: BarryK
"Probably that character has a generic name, apart from what we call it in the context of programming. Ah yes, "vertical bar": http://en.wikipedia.org/wiki/Vertical_bar ...interesting, the "broken bar" is a different character, but keyboards confuse the two.

Bars on my keyboard
Username: nic2109
"On my Dell laptop the Vertical bar is to the left on the number 1 but requires the 'AltGr' key held down to use it. The broken bar is to the left of the Z and requires only the Shift key. Posting to this blog from Puppy 4.2 (beta 1) shows both keys just the same, so you are spot on th say that keyboards confuse them. Here is a string of Vertical bars ||||||, and these are all Broken bars |||||||. Do they look the same to the readers? It might just be the mapping, but where does that get specified and then fixed?

Arch package names
Username: BarryK
"An update for this thread. I originally posted about being uncertain what the Arch package name is: automake-1.10.2-1ARCHTYPE.pkg.tar.gz, where 'ARCHTYPE' could be '-i686' or -x86-64' or nothing. The reason for the uncertainty is that the package database that I had downloaded (for example 'core.as.tar.gz') did not contain the actual package name. However, Arch package database has three different database tarballs, and in the case of the 'core' group of packages I found that 'core.db.tar.gz' does have the package filenames. So, problem solved. Woof has to download both tarballs to extract enough information to create the new standardised database format file.

Tags: woof