Build-order, dependencies

The Quirky build system, that forked from Puppy's Woof distro build system sometime ago, creates the root filesystem in sandbox3/rootfs-complete.
This is done by the '3builddistro' script.

There is a list of packages that 3builddistro reads, variable PKGS_SPECS_TABLE, which is in file DISTRO_PKGS_SPECS-${DISTRO_BINARY_COMPAT}-${DISTRO_COMPAT_VERSION} (ex: DISTRO_PKGS_SPECS-ubuntu-wily).

Up until now, the list in variable PKGS_SPECS_TABLE has been in alphabetical order, which is convenient for us, to quickly see what is there. 3builddistro reads PKGS_SPECS_TABLE and installs the packages into sandbox3/rootfs-complete in the order of entries in the variable.
Whatever order they are in, 3builddistro installs them in that order.

Package post-install scripts
This is not necessarily the best though, as a package may conceivably have a post-install script that assumes the dependencies of the package are already installed.

Looking at the Debian/Ubuntu/Raspbian/Devuan family, that all use DEB packages, these may have 'preinst' and 'postinst' scripts. However, 3builddistro totally ignores these.
3builddistro does not run DEB pre-install or post-install scripts, yet we still get a sane distro -- which might seem incredible, but that's how we have done it up to now.

3builddistro, and in some case the templates in folder 'packages-temples', do perform some hacks to fix some packages. For example, the gcc DEBs are missing some symlinks.

I would like to go the extra step and run those 'preinst' and 'postinst' scripts. It requires that sandbox3/rootfs-complete be chroot-able, and 'dpkg' needs to be installed.
Which means that the order of installation of packages into rootfs-complete becomes an issue.

Install order
I am tackling this order of installation issue first. I created script support/build-order, that is called from support/findpkgs, which in turn is called from 1download or 2createpackages.
Script 'build-order' creates file status/findpkgs_GENERIC_NAMES-${DISTRO_BINARY_COMPAT}-${DISTRO_COMPAT_VERSION} (ex: findpkgs_GENERIC_NAMES-ubuntu-wily) which has the package names in order so that a packages dependencies install first. 3builddistro now reads this.

Finding missing deps
While I was at it, I thought about the hard work we have to put in when creating the package list for a new release of Ubuntu (etc.). New packages, new dependencies.
So, I added code into script 'build-order' that finds all missing dependencies, and creates a list of extra entries that can be inserted into variable PKGS_SPECS_TABLE.

To do
Finding all deps and adding them to the package-list is nice. Already very useful, but this is a work-in-progress. Some improvements needed, and I have not yet tackled execution of the 'preinst' and 'postinst' scripts.

Note, it is too much work for me to put this into the Woof project, however, I am documenting it and a Woof developer should be able to extract the new functionality and insert into Woof. That's if they want it of course.

---

Comments

I have now implemented the final part of this exercise.

Script 2createpackages retains the 'DEBIAN' folder for each package, out of the .deb files.

Script 3builddistro creates a minimal chroot-able sandbox3/rootfs-complete, with the 'dpkg' package in it (as some postinst scripts need it), then as it fetches each package that 2createpackages created, it runs any postinst script it finds.

I am only now testing this. At this stage, I don't know how dpgk will behave in the minimal rootfs. Also, some of those postinst sripts are pretty weird, very "Debian", and I don't know if I am shooting myself in the foot by running them.

...it might be better if I replace the dpkg utilities with do-nothing scripts, at least during execution of the postinst scripts.

Tags: linux