Finding XBPS to be very frustrating

The XBPS package manager in Void Linux, and used in QV, is great, except when the database becomes broken. Then, fixing can be inpossible, or so it seems.

I'm running QV with a deliberately broken package database. This can happen to someone running Void and they haven't updated or synced for a long time. Try and do anything, maybe install a package with 'xbps-install', and get messages with "broken, unresolvable shlib".

The official advice is to run "xbps-install -Su", which is to sync to latest database and update everything ... except, that doesn't work, gives lots of "broken, unresolvable shlib".

Here is an example:

# xbps-install --repository=https://repo-fi.voidlinux.org/current --force --verbose libndp
Found NetworkManager-1.48.10_1 in repository https://repo-fi.voidlinux.org/current
Found libndp-1.8_1 in repository https://repo-fi.voidlinux.org/current
Found openresolv-3.13.2_1 in repository https://repo-fi.voidlinux.org/current
Found wpa_supplicant-2.10_5 in repository https://repo-fi.voidlinux.org/current
Found libmm-glib-1.18.12_1 in repository https://repo-fi.voidlinux.org/current
Found libnm-1.48.10_1 in repository https://repo-fi.voidlinux.org/current
Found newt-0.52.21_7 in repository https://repo-fi.voidlinux.org/current
Found libcurl-8.10.1_1 in repository https://repo-fi.voidlinux.org/current
newt-0.52.21_7: broken, unresolvable shlib `libpopt.so.0'
NetworkManager-1.48.10_1: broken, unresolvable shlib `libpsl.so.5'
libcurl-8.10.1_1: broken, unresolvable shlib `libpsl.so.5'
newt-0.52.21_7: broken, unresolvable shlib `libslang.so.2'
Transaction aborted due to unresolved shlibs.

So, perhaps we can work down recursively and update all the packages. Yes, except 'xbps-install' is too dumb, cannot do that. It is possible, see this:

# xbps-install --repository=https://repo-fi.voidlinux.org/current --force sox
sox-14.4.2_8: broken, unresolvable shlib `libao.so.4'
Transaction aborted due to unresolved shlibs.
# xbps-install --repository=https://repo-fi.voidlinux.org/current --force libao
libcurl-8.10.1_1: broken, unresolvable shlib `libpsl.so.5'
Transaction aborted due to unresolved shlibs.

We have traced down, it looks like maybe can install libpsl...

# xbps-install --repository=https://repo-fi.voidlinux.org/current --force libpsl

Name          Action    Version           New version            Download size
public-suffix install   -                 2024.08.25_1           130KB 
libpsl        downgrade 0.21.5_2          0.21.5_1               72KB 

Size to download:              203KB
Size required on disk:         485KB
Space available on disk:        43GB

Do you want to continue? [Y/n] 

OK, now go back up, try libao...

# xbps-install --repository=https://repo-fi.voidlinux.org/current --force --ignore-file-conflicts libao

Name         Action    Version           New version            Download size
libid3tag    update    0.15.1_2          0.15.1b_8              28KB 
libmad       update    0.15.1_2          0.15.1b_10             53KB 
libwavpack   install   -                 5.7.0_1                89KB 
opusfile     install   -                 0.12_3                 44KB 
sox          update    14.4.2_2          14.4.2_8               385KB 
libcurl      install   -                 8.10.1_1               368KB 
vorbis-tools update    1.4.2_2           1.4.2_3                206KB 
libao        update    1.2.0_2           1.2.2_2                29KB 

Size to download:             1206KB
Size required on disk:        3076KB
Space available on disk:        43GB

Do you want to continue? [Y/n] 

...yay!

I just happened to find these two able to install, but there is no tool to do it, to fix an entire installation. 'xbps-install' is too dumb, or too smart for its own good.    

Tags: quirky

QV version 241007 released

Here it is:

https://distro.ibiblio.org/quirky/quirky-void/amd64/releases/base/

Read about QV in the "quirky" news tag:

https://bkhome.org/news/tag_quirky.html

You are most welcome to play with QV!

Bootup QV on a USB Flash drive, you will find 'qv-installer' which is an easy way to install to internal drive.

Forum feedback here:

https://forum.puppylinux.com/viewtopic.php?p=132484#p132484   

Tags: quirky

Update PET for QV

QV is a quasi-full install that manages packages with the Void xbps package manager, with PKGget as a GUI frontend. Updating is tricky, as it is not just a matter of installing the latest packages from the Void repository. There are also .pet and .tar.xz (from Scarthgap) packages, plus all of the support scripts.

Regarding the support scripts, look in /usr/local for example and you will see lots of them. These are all in woofQV, the QV build system.

I have tackled creating an update .pet package. It will be named qv-update-<date>.pet and is about 91MB -- it will always be about that size. The full drive-image .img file, which is about 1.5GB, after being installed from thereon it should only be required to install 91MB update PETs.

Here is the script that creates both the devx and qv-update PETs:

https://github.com/bkauler/woof-quantum-vis/blob/main/buildquantumvis/rootfs/8create-extra-pets

As QV is based upon snapshots, installing either of those, devx or qv-update PET package, will only apply to the current snapshot. Probably a good strategy would be to take a snapshot of the one that you want to update, boot into it, then install qv-update. If you don't like the update, for whatever reason, no problem, just delete the snapshot and go back to the older one.

May be able to release QV this evening. Playing with this is going to be very interesting.   

Tags: quirky

Attempt to disable pam

I think that PAM is an acronym for "Permission Access Management". In QV, it has caused trouble, and we want to disable it.

QV has 'pam-libs', not 'pam-base' and 'pam' packages that are also in the Void repository.

I have put a hack into the 'init' script in the initrd:

#20240423 attempt disable pam, this will zeroize these files...
#for aF in /mnt/${WKG_DEV}/${QV}/etc/pam.d/*; do >$aF; done
#20241006 try this instead...
#for aF in /mnt/${WKG_DEV}/${QV}/etc/pam.d/*; echo 'skip-authentication' >$aF; done
#or, if delete /etc/pam.d then pam will read /etc/pam.conf instead...
rm -rf /mnt/${WKG_DEV}/${QV}/etc/pam.d 2>/dev/null
#have pre-created this in rootfs-skeleton...
#echo 'skip-authentication' > /mnt/${WKG_DEV}/${QV}/etc/pam.conf

...there was an earlier attempt, 20240423, deleting everything in /etc/pam.d

From online reading, file /etc/pam.conf is ignored if folder /etc/pam.d exists. So the hack now trying is to delete /etc/pam.d entirely, and just have a global "skip-authentication" line in /etc/pam.conf

Haven't tried it yet.   

Tags: quirky

QV basic handling kernel version changes

Did some online reading about this; it is an awkward situation. With btrfs, there may be lots of snapshots, even hundreds. Some may be old, setup in a certain way, with certain apps, and working fine -- if you have updated the kernel, you may not want to bootup one of those old snapshots with the new kernel.

There is also the problem that the old snapshot will have kernel modules in /usr/lib/modules, for the old kernel, not for the new kernel. /usr/lib/firmware may also be a problem; may have correct firmware for the old kernel, not for the new.

A lot on online dscussion about the best way to handle this.

I have made a start, some basic handling:

https://github.com/bkauler/woof-quantum-vis/commit/cf5d3036d237bac510674133c431f50a2b963bfb

So, when you choose to bootup a different snapshot from the default, the 'init' script will look to see if /usr/lib/modules/<default kernel version> exists. If not, will offer to update the snapshot to use the default kernel.

Or, will offer to change the default kernel to the kernel version that is in the chosen snapshot. That may be more than one to choose from.

It can be fleshed out a bit more. Haven't thought about /usr/lib/firmware. Not tested yet. The code would be more elegant if could use kexec.     

Tags: quirky

Kexec fail

Utility 'kexec' is in the 'kexec-tools' package. It enables to change the kernel, without having to perform a complete reboot. It can run a new kernel, new initrd and new kernel commandline parameters.

I posted a few days ago about compiling the kernel to support kexec:

• The pink and grey galahs have returned! — September 28, 2024

Both EasyOS and QV have a menu in the initrd, with one entry to run the Xorg Wizard instead of starting Xorg. A problem with that, is that ideally I want the kernel to boot with the "nomodeset" parameter, so as to start Xorg Wizard with a very basic video setup. This is where kexec would be useful. Unfortunately, cannot get it towork.

Here is the menu in the initrd. Item 8 is the one we want:

I put in code to execute kexec; however, it refused to load the kernel, with this error message:

Symbol: '__auxval' in common section

Hunted all over the place online, could not find anyone reporting that error. The kexec utility is compiled statically with klibc in OpenEmbedded/Yocto, using kexec-tools version 2.0.18. So, I bumped to version 2.0.28; now, the kernel loads.

There are two steps, kexec has to be run twice, first to load the kernel, second to execute the loaded kernel. There are usually more commandline parameters required, but this is the basic idea:

# kexec -l <path to new kernel>
# kexec -e

What happens now, is the second execution of kexec runs, a cursor appears on the screen, then ...nothing happens.

My knowledge of how kexec works is very superficial. Played around with various parameters, but no joy. Have commented-out the code in the 'init' script in the initrd:

https://github.com/bkauler/woof-quantum-vis/commit/877731dc37ae55313defb405b475c31d35d893c4

...have left the code in there, as maybe will figure out what is wrong sometime in the future.

Note, I had hoped to get kexec working for another reason. There needs to be a mechanism to manage change of kernel version in QV. Need to put some thought into that.    

Tags: quirky

QV now has encrypted folders

I posted early this year about fscrypt patch for btrfs, see related posts:

• btrfs-progs with fscrypt take-2 — March 20, 2024
• btrfs-progs with fscrypt patches — March 19, 2024
• Kernel 6.8.1 with btrfs fscrypt — March 16, 2024
• Improved support for btrfs — March 15, 2024
• btrfs-progs compiled statically in OE — March 14, 2024
• Linux kernel btrfs supports fscrypt — March 14, 2024

However, back then had to abandon fscrypt for btrfs, as the kernel crashed at bootup.

Unfortunately, the btrfs developers have not shown any sign of being interested in progressing with this. The "v5" patch from March 2024 is still the latest. Anyway, I decided to take another look at it, applied the patches to the 6.8.12 kernel (as the patches were developed on the 6.8.x kernel source). This time booted up without having folder encryption, just to see if the patches have some degree of sanity -- nope, got to the desktop, then the btrfs filesystem shortly afterward changed automatically to read-only -- btrfs does that if it detects an error, to avoid further damage.

I booted up a few times, and it consistently got to the desktop and then went to read-only.

So, giving up on that. I really do want encrypted folders, so time for some lateral thinking...

What I have done is create a sparse file with an ext4 filesystem. After bootup, this is what is in the working-partition, showing hidden folders and files:

File '.sparse-ext4' is a sparse file with an ext4 filesystem. It is mounted on folder '.sparse-mnt'. Inside, it contains a folder hierarchy like this:

.sparse-mnt
files  home
       1  2  3 ...

What happens at bootup in the initrd, that 'files' folder gets mounted on '/files', and one of those number-folders, depending what snapshot booting into, gets mounted on '/home'. After bootup, this is what you see:

...looks normal, but now, '/files' and '/home' are encrypted folders, that got unlocked by the password at bootup.

Here are the github commits:

https://github.com/bkauler/woof-quantum-vis/commit/370ddf467637f0e2a5dbe36ce0ca097ee5d8ac35

https://github.com/bkauler/woof-quantum-vis/commit/9c0a7ffdce5210b7f3dba6484652aeba79f4a6be

From the user's point of voew, QV works as before. In the initrd menu you can choose which snapshot to boot into, create a new snapshot, or delete a snapshot.

As before, '/files' is not snapshotted -- it is for your personal storage and thus appropriate to just accumulate whatever you put into it.

There is a different '/home' folder for each snapshot, but as this is in an ext4 partition, if a new snapshot is created, the entire content of the current '/home' gets copied to the new snapshot.

There are some finishing-off details, will tackle those tomorrow.

Regarding the sparse file; that is a large file that only occupies as much storage as there is content in the ext4 filesystem. So, store more files, it grows. However, if delete files, the sparse file will shrink, as it is mounted with the "discard" option. That is an interesting topic.

Note that the change to an ext4 sparse file is determined at build-time in woofQV, by means of a variable QV_FILES_SPARSE -- in running QV, this variable can be found in /root/.packages/build-choices. Decided to do it this way as didn't want to "burn the bridges", in case the btrfs developers do eventually come up with a working fscrypt patch.

EDIT 2024-10-01:
Decided not to have '/home' in a ext4 sparse file. It is enough if anything that a user wants to keep private, can be kept in '/files'. Besides, I am hoping that one day the btrfs developers will make progress with fscrypt.      

Tags: quirky