site  contact  subhomenews

easyVoid problem with user-installed packages

March 04, 2024 — BarryK

Here is the previous blog post in this easyVoid and woofV project:

https://bkhome.org/news/202403/xbpsget-cli-utility-to-install-xbps-packages.html

In a nutshell, woofV builds an easyVoid distribution with XBPS package manager handling all package installation, update, removal, etc. woofV creates 'easy.sfs' with all packages populated in it by XBPS.

However, there is an elephant in the room. Actually, there are a couple of elephants; considering just one of them here. The problem is, how to handle user-installed packages when there is a version update?

Here is the situation...

When XBPS installs a package, its details get added to file /var/db/xbps/pkgdb-0.38.plist. This is an XML text file, and information about all installed packages are in here, in alphabetical order. If, for example, I install 'abiword', various details about it, such as the description and size, get put into that file. Also, another file gets created, /var/db/xbps/.abiword-files.plist, XML also, that lists all the files in that package.

Say that the current version of easyVoid is "240304". Now, I used woofV and create a new release, say "240405". A user of "240304" can click on the "update" icon and download the new version. The three files 'vmlinuz', 'initrd' and 'easy.sfs' are extracted from the downloaded file, and after a reboot, the user has updated.

The problem is that 'easy.sfs' has /var/db/xbps/pkgdb-0.38.plist, which is the up-to-date packages in this new release. However, because of the layered filesystem, the old /var/db/xbps/pkgdb-0.38.plist will be "on top".

...which is just the start of the problems.

That particular problem is solved by the 'init' script in the initrd, which will delete /mnt/wkg/.session/var/db/xbps, that is, the folder on the top read-write layer. So, the /var/db/xbps files in easy.sfs will appear "on top".

The real problem is that this new 'pkgdb-0.38.plist' will not contain the information about any of the user-installed packages, nor will there be any '.<package name>-files.plist' files. XBPS will no longer know that they are installed. That is one of the elephants.

I thought of two possible solutions. Firstly, use XBPS to uninstall the user-installed packages before the reboot, then reinstall them after booting up the new release. Secondly, extract all the user-installed information out of pkgdb-0.38.plist, then merge it into the new pkgdb-0.38.plist after rebooting to the new release.

I decided on the latter solution. To implement that, wrote two scripts, 'pkgdb-extract-users' and 'pkgdb-merge-users'. Here they are:

https://github.com/bkauler/woofq/blob/main/easyos/easy-code/rootfs-skeleton/usr/local/woofV/support/pkgdb-extract-users

https://github.com/bkauler/woofq/blob/main/easyos/easy-code/rootfs-skeleton/usr/local/woofV/support/pkgdb-merge-users

...probably they could be implemented more efficiently using 'xmllint', 'xml2' and/or 'awk'.

When you click on the "update" desktop icon, the script /usr/local/easy_version/easy-update runs, and this will now run 'pkgdb-extract-users'. At reboot, the 'init' script in the initrd will delete the old pkgdb-0.38.plist and the hidden files, then /etc/rc.d/rc.update will run 'pkgdb-merge-users'. The changes to those three scripts are here:

https://github.com/bkauler/woofq/commit/9cb568dc618f00ea9393ac1d8fe81ae00a07b9bb

Could get XBPS to do an automatic update of user-installed packages also; will probably add that.

This is not yet tested.

As an extra note, I found it an interesting read; wiak, rockedge, jamesbond and peebee deliberating on the early design of VoidPup:

https://forum.puppylinux.com/viewtopic.php?t=4741

Hopefully one elephant has been negotiated out of the back door. Or rather, I had to demolish a wall to get it out. The other elephant is handling of the 'devx' SFS, which will be considered soon.

I would like to add one extra comment. It is awkward getting XBPS to function with a layered filesystem, aufs or overlayfs; however, it does bring a major advantage, support for snapshots. XBPS knows nothing about snapshots, but this is inherent in the design of EasyOS, and indeed in all layer-based pups such as those built from Woof-CE.

In the case of Easy, the previous 'vmlinuz', 'initrd' and 'easy.sfs' files will still exist. So, perchance you don't like the latest version, easy-peasy can roll back to the previous. So, we are adding major functionality to XBPS.        

Tags: easy

xbpsget CLI utility to install .xbps packages

March 03, 2024 — BarryK

Continuing the woofV project, this is the previous post:

https://bkhome.org/news/202403/pkgget-working-in-easyvoid-version-240229.html

I also posted about an updated help page for PKGget (PPM):

https://bkhome.org/news/202403/pkgget-help-page-updated.html

The PPM, right from the early Puppy days, also has a CLI utility, 'petget', that you can run in a terminal; for example:

# petget abiword-1.2.3-amd64.pet

Or, it can be run by right-button click on a package in ROX-Filer and choose "petget" in the menu.

Not, just PETs, petget will install many kinds of packages, such as .deb and .rpm.

'petget' is still in EasyOS; however, for .xbps packages I have created a new utility, 'xbpsget'. Same thing, it is a CLI utility, that can be invoked from a terminal or via right-click menu. Here it is:

https://github.com/bkauler/woofq/blob/main/easyos/easy-code/rootfs-skeleton/usr/local/petget/xbpsget

Normally, you would run PKGget and install .xbps packages from the Void respository. However, there may be a situation sometimes were you have a local .xbps package that you want to install.

'xbpsget' can also uninstall, by appending a "-", for example:

# xbpsget -abiword-1.2.3_1.x86_64.xbps

Note that unlike 'petget', 'xbpsget' will install any missing dependencies, and if uninstall will also uninstall any dependencies that are no longer needed. Note also, have written it but not yet tested.   

Tags: easy

PKGget help page updated

March 02, 2024 — BarryK

The PKGget (PPM) window has a help button, that brings up a HTML page. I clicked on it, and realised that it is way out of date. It was last edited in 2013. Updated, and here it is:

https://github.com/bkauler/woofq/blob/main/easyos/easy-code/rootfs-skeleton/usr/local/petget/help.htm

...um, to see it as a web page, will have to download the raw file, then open in browser.

I used Google Translate and created de, fr, es and tr translations:

https://github.com/bkauler/woofq/tree/main/easyos/easy-code/rootfs-skeleton/usr/local/petget  

Tags: easy

PKGget working in easyVoid version 240229

March 01, 2024 — BarryK

Continuing the woofV project, here is the previous post:

https://bkhome.org/news/202402/woofv-built-easyvoid-240224-working-well.html

The significance of easyVoid with version numbering yymmdd, that is, year-month-day, is that it is built by woofV and populated entirely by the XBPS package manager.

PKGget, the package manager in EasyOS, is a descendent of PPM, the Puppy Package Manager. PKGget will now recognise the existence of XBPS and become a GUI frontend for it. Starting it up:

img1

EasyOS defaults to the "classical" UI of earlier pups; I think that recent pups have a different UI, but functionality is basically the same. The Void "current" repository has "abiword" entry, so I clicked on that to install it. Not that I would use Abiword (past experience was not good), but OK for this demonstration. There is a confirmation window:

img2

After clicking the "Install abiword" button, there is a drop down to XBPS:

img3

After installation, the installed packages show up in PKGget, and there is a menu entry:

img4

PKGget has sophisticated dependency-checking; you cannot uninstall a package if it is a dependency of another. For example, when I click on the "goffice" entry in "Installed packages", uninstall is refused:

img5

So, have to uninstall "abiword" and "libabiword" first. When I click on the "abiword" entry in "Installed packages", it drops down to XBPS:

img7

...need to fix that title in the terminal window. Abiword is removed, and PKGget provides some useful information about dependencies that are no longer required:

img8

PKGget, like its ancestor PPM, is a chameleon, able to install many different types of packages, such as .deb, .rpm, .pet, .tar.xz, etc. And now .xbps. However, so far only .xbps and .pet have been implemented to work with XBPS.

So, if a .pet is installed, PKGget will convert it to .xbps and then use the underlying XBPS to install it. In the "pet-noarch" repository, I chose to install "desk_background_after-sunset-reddish-clouds-lake", by clicking on the entry:

img9

The PET package installs OK, and in a terminal can confirm that XBPS handled the actual install:

img11

There are a lot of things still to be fixed and implemented. There are some caveats and gotchas also, and details still to be thought through. This blog post is to give a taste of what to expect. It is basically working, quite well actually.

Family gathering all day today. Need a couple more days to work on it, then hopefully can release a pre-alpha for anyone interested in testing it.  And of course another interesting feature will be the inbuilt "woofV" -- to be described later.    

Tags: easy

Overlay filesystem is still a disaster

February 25, 2024 — BarryK

Have tried to use it many times since the early days of EasyOS in 2017. Here are recent posts:

https://bkhome.org/news/202205/tried-overlay-filesystem-again.html

https://bkhome.org/news/202304/overlayfs-is-still-pathetic.html

I have remained hopeful. After all, other Linux developers are using Overlay, so why is it so awful for me? Yesterday, I compiled the 5.15.149 kernel, and this time ticked all these boxes:

img1

Booted, but Xorg failed:

(EE) 
Fatal server error:
(EE) Cannot move old log file "/var/log/Xorg.0.log" to "/var/log/Xorg.0.log.old"

So many basic operations fail. For example, cannot do a recursive remove:

# rm -rf /var/log
rm: cannot remove '/var/log' directory: not empty

Cannot move:

# mv -f /var /varBAK
mv: cannot move '/var' to a subdirectory of itself, '/varBAK'

Try remove a folder that has files in it, and no sub-directories:

# rm -rf /root/Web-Server

...removed the files, but not directory 'Web-Server'

I tried various combinations of turning off those default options, still got the same failures. At least didn't get that "invalid cross-device link" from previous tests.

I just have to give up. When Dr. Okajima retires from developing Aufs, I will also retire from Linux development.  

Tags: easy

woofV-built easyVoid 240224 working well

February 24, 2024 — BarryK

Previous post in the woofV project:

https://bkhome.org/news/202402/woofv-drive-image-get-desktop.html

...as posted before, woofV now builds a drive-image that can be written to a usb-stick and it boots to a desktop.

I have worked on a few bugs, and the "240224" build is looking real good. Normal-looking desktop:

img1

There are still some bugs, but so far only found minor ones. A couple of strange entries in the menu. A couple of minor broken things.

Not a bug, but undesired; XBPS has installed 'lxterminal' and 'cellwriter' -- these are not in the woofV package list, but they must be dependencies of some packages, and XBPS has automatically installed all required deps.

Xorg has chosen the 'intel' driver, which on my Lenovo desktop causes font rendering issues. I have to exit to the commandline and run "xorgwizard", and it offers to change from "sna" to "uxa" rendering, which fixes it.

'easy.sfs' is big, 844MB, and the drive-image file, 'easy-240224-amd64.img', is 905MB. Compare with easyVoid 6.0.3, easy.sfs is 798MB. The latter is built with WoofQ, the former with woofV. The woofV build populates 'easy.sfs' using the XBPS package manager, which pulls in extra dependencies -- such as 'lxterminal' and 'cellwriter'.

I'm very keen to start converting PKGget to become a GUI frontend to XBPS, but first have to work on the various issues mentioned above.

Should mention; XBPS works great in the running easyVoid 240224. I ran 'xbps-install' in a terminal and installed a couple of audio players.    

Tags: easy

pMusic updated to version 6.9.3.1

February 23, 2024 — BarryK

pMusic is an audio player created by zigbert; see forum:

https://forum.puppylinux.com/viewtopic.php?t=5801

The EasyOS "noarch" PET repository had pMusic version 6.9.3; however, 6.9.3.1 has some fixes.

I have modified the PET to make it more compatible with EasyOS, and merged 'pmusic-NLS.pet' into it. Also added translations to the menu entry. Uploaded here:

https://distro.ibiblio.org/easyos/noarch/packages/pet/pet_packages-noarch/

EasyOS has Audacious 3.x audio player, which requires gtk2. However, the Void repository has Audacious version 4.x, which requires qt6 -- actually, I think that it is a compile choice, gtk3, qt5 or qt6, and Void has chosen the latter. I don't want to bring in all the qt6 packages, so have decided to have pMusic as the builtin audio player for easyVoid. Note, could have used Audacious from the Kirkstone package repository; however, want to minimize using Kirkstone packages, due to possible dependency mismatches.  

Tags: easy

woofV drive-image get desktop

February 22, 2024 — BarryK

Continuing the woofV project, here is the previous post:

https://bkhome.org/news/202402/woofv-post-process-populated-rootfs.html

I have worked on '7create-easy-sfs', '8create-drive-img' and some support scripts, and have built a easyVoid drive image file, which I wrote to a USB-stick and booted it, and got a desktop. The scripts are here:

https://github.com/bkauler/woofq/tree/main/easyos/easy-code/rootfs-skeleton/usr/local/woofV/rootfs

There are some things broken, but it is good to get the desktop.

The 'rootfs' folder, that became 'easy.sfs', is very big, and that is something that I need to examine -- why is it so much bigger than 'easy.sfs' 6.0.2 -- the easyVoid built with WoofQ (and not using XBPS). Yes, XBPS did pull in more dependencies, but didn't expect that much size growth.

I'm still in two minds about Void's rolling release model. It seems that an update to the repository can happen at any time. So, in woofV I download the repo database, then start installing packages to 'rootfs'. In the middle of doing that, the repo changes, and suddenly a package that attempt to install isn't in the repo.

To work around this, the Void documentation advises to sync with the repo everytime install a package. I haven't done that, but looks like should do so. Likely, that will slow down installation. If the repo does change then XBPS may have to reinstall some already-installed package. I guess it will work; it just bothers me.

I was reading reports that if attempt to install a package in the middle of a repo change, there will be failures downloading, even if trying to sync with the database; the advice is to wait awhile, then retry.

Then there is another concern; a fundamental question about using XBPS to install the packages, versus using PKGget (PPM). There are so many potential issues, that I am attempting to work around. Forum member dimkr posted a couple of these fundamental concerns to the forum:

https://forum.puppylinux.com/viewtopic.php?p=110287#p110287

PKGget is very simple, and works OK with the devx; if a choose to install a package that has a dependency on the devx SFS, PKGget will popup a window warning of this.

However, PKGget is too simple in some respects; it may have a problem with the rolling release model. At startup, PKGget syncs with the Void repo database; however, if install a package and an already-installed dependency has become too old, PKGget isn't aware of this. However, the plan is that the woofV-built easyVoid, PKGget will be changed to be a GUI frontend to XBPS, so repo changes will be handled auromatically.

Anyway, just cogitating over the issues.    

Tags: easy