Exploring debootstrap

This is fascinating...

Recently, Caramel has been testing woofQ, the build system for EasyOS, see this forum thread:

https://forum.puppylinux.com/viewtopic.php?t=14493

Caramel commented that the '2createpackages' script is very slow. Yeah, the scripts in woofQ are ancient; I have hacked on them, but really, the entire thing needs a rewrite. Which is what has been done with Woof-CE, in particular due to forum member dimkr.

I actually did do a complete rewrite, substantially anyway, which is woofQV. The decision in that case was to base the build of QV on Void Linux binary packages, also went for a new underlying structure; btrfs and snapshots. The result is not EasyOS, it is a different distribution; QV, "Quantum Vis".

Note that QV still has PKGget, which derives from the Puppy PPM, but substantially rewritten as a frontend to the Void XBPS package manager. It still retained the chameleon-like nature of PPM, able to install many package types, including Debian .deb and .pet packages. It achieves an integration of these disparate repositories.

Thinking about a rewrite of woofQ, I do not want to build from packages of a rolling-release distro, as that is anathema to the principle of building EasyOS with read-only squashfs files; in the case of EasyOS, the main guy is 'easy.sfs'.

Easy Daedalus is built with Devuan Daedalus (Debian Bookworm) .deb packages. Yes, it looks like this is the way to go, moving forward. So, rewrite woofQ, let's call it "woofQ2", based on building with 'dpkg' and 'apt'. So they will be the underlying package management. There could still be a PKGget, but as done with QV, a wrapper around apt.

woofQ2 needs to be able to build easy.sfs fast, and this is where debootstrap comes in. This is a script that builds a rootfs; that is, a folder with apt inside it, that can be chrooted into and any required packages installed.

Forum member fredx181 has used debootstrap for his DebianDog and UbuntuDog distros. I wrote about BookwormDog last year:

• BookwormDog build script — September 23, 2024

A couple of days ago, I downloaded his latest, TrixieDog; the 'mklive-trixie' script. See forum:

https://forum.puppylinux.com/viewtopic.php?t=11771

I ran it in Easy Daedalus. Had to install 'debootstrap', 'dpkg' and 'isolinux' beforehand. Note, Easy has the 'dpkg' applet for busybox, but the full package is required. It went through, created an ISO which I wrote to a USB-stick and booted; got a desktop!

Create rootfs with debootstrap

So, how to go about using debootstrap to create a chrootable rootfs, that could become easy.sfs? I wrote a script to do that...

The script is 'create-devuan-rootfs'. Yeah, decided to go with Devuan. The upcoming Debian Trixie will be Devuan Excalibur. Here is the script:

#!/bin/sh
#created 20250611

#just in case abnormal exit debootstrap...
mount | grep -q '/rootfs-devuan/'
if [ $? -eq 0 ];then
 busybox umount rootfs-devuan/proc 2>/dev/null
 busybox umount rootfs-devuan/sys 2>/dev/null
 busybox umount rootfs-devuan/dev/shm 2>/dev/null
 busybox umount rootfs-devuan/dev/pts 2>/dev/null
 busybox umount rootfs-devuan/dev 2>/dev/null
fi

DPKG="$(which dpkg)"
if [ $? -ne 0 ];then
 echo "Utility 'dpkg' not installed."
 exit 1
else
 if [ -L "$DPKG" ];then
  echo "The busybox 'dpkg' is installed. Need the full 'dpkg' package."
  exit 1
 fi
fi

if [ ! -f Packages ];then
 wget http://packages.devuan.org/devuan/dists/excalibur/main/binary-amd64/Packages.xz
 unxz Packages.xz
fi

which debootstrap >/dev/null
if [ $? -ne 0 ];then
 DEBfnd="$(grep '^Filename: pool/main/d/debootstrap/debootstrap_' Packages)"
 DEBfnd="${DEBfnd##*/}" #ex: debootstrap_1.0.141devuan1_all.deb
 wget --no-check-certificate http://packages.devuan.org/devuan/pool/main/d/debootstrap/${DEBfnd} -O ${DEBfnd}
 dpkg -i ${DEBfnd}
fi
which debootstrap >/dev/null
if [ $? -ne 0 ];then
 echo "Failed to install debootstrap"
 exit 1
fi

if [ ! -f /usr/share/keyrings/devuan-archive-keyring.gpg ];then
 DEBfnd="$(grep '^Filename: pool/main/d/devuan-keyring/devuan-keyring_' Packages)"
 DEBfnd="${DEBfnd##*/}" #ex: devuan-keyring_2023.10.07_all.deb
 wget --no-check-certificate http://packages.devuan.org/devuan/pool/main/d/devuan-keyring/${DEBfnd} -O ${DEBfnd}
 dpkg -i ${DEBfnd}
fi
if [ ! -f /usr/share/keyrings/devuan-archive-keyring.gpg ];then
 echo "failed to install package 'devuan-keyring'"
 exit
fi

#debootstrap 1.0.141devuan1 needs a patch:
#excalibur is a symlink to ceres, not working correctly...
if [ -L /usr/share/debootstrap/scripts/excalibur ];then
 rm -f /usr/share/debootstrap/scripts/excalibur
 cp -a /usr/share/debootstrap/scripts/ceres /usr/share/debootstrap/scripts/excalibur
 PTN1='s%"\$MERGED_USR" = "yes" -o "\$CODENAME" != "daedalus"%"X" == "Y"%'
 sed -i -e "${PTN1}" /usr/share/debootstrap/scripts/excalibur
fi

if [ -d rootfs-devuan ];then rm -rf rootfs-devuan; sync; fi
mkdir rootfs-devuan

mkdir -p rootfs-devuan/etc/dpkg/dpkg.cfg.d
mkdir -p rootfs-devuan/etc/apt/apt.conf.d

echo 'path-exclude=/usr/share/doc/*
path-exclude=/usr/include/*
path-exclude=/usr/share/man/*
path-exclude=/usr/share/bash-completion/*
path-exclude=/usr/share/common-licenses/*
path-exclude=/usr/share/info/*' > rootfs-devuan/etc/dpkg/dpkg.cfg.d/50excldoc

echo 'APT::Install-Recommends "false";
APT::Install-Suggests "false";' > rootfs-devuan/etc/apt/apt.conf.d/50norec


#######
debootstrap --arch=amd64 --variant=minbase --include=busybox-static,locales,dialog,pulseaudio excalibur rootfs-devuan
sync
#######

#why does dpkg in the rootfs ignore dpkg.cfg.d?...
#rm -rf rootfs-devuan/usr/share/doc/*
#rm -rf rootfs-devuan/usr/include/*
#rm -rf rootfs-devuan/usr/share/locale/*
#mkdir rootfs-devuan/usr/share/locale/C
#mkdir rootfs-devuan/usr/share/locale/en
#mkdir rootfs-devuan/usr/share/locale/de
#rm -rf rootfs-devuan/usr/share/man/*
#rm -rf rootfs-devuan/usr/share/bash-completion/*
#rm -rf rootfs-devuan/usr/share/common-licenses/*
#rm -rf rootfs-devuan/usr/share/info/*
#sync
#note: /usr/share/locale will be populated as more pkgs installed.

#tell apt all sources:
#ref: http://packages.devuan.org/devuan/dists/
echo -n 'deb http://deb.devuan.org/merged excalibur main contrib non-free
deb http://deb.devuan.org/merged excalibur-updates main contrib non-free
deb http://deb.devuan.org/merged excalibur-security main
' > rootfs-devuan/etc/apt/sources.list

#/proc has been left mounted. add these, for more chrooting...
busybox mount --rbind /dev rootfs-devuan/dev
busybox mount --bind /sys rootfs-devuan/sys

chroot rootfs-devuan localedef -f UTF-8 -i en_US --no-archive en_US.utf8
#######

echo
echo "Basic rootfs created!"
echo -n "ENTER to install more: "
read WANTMORE

######
echo
chroot rootfs-devuan apt update #coz those extra repos added.

PKGLIST='ca-certificates
cups
file
firefox-esr
firmware-linux-free
geany
geany-plugins
gnupg-utils
gvfs
gxmessage
jwm
kmod
l3afpad
libgdk-pixbuf2.0-bin
librsvg2-bin
mingetty
network-manager
obconf
openbox
pavucontrol
psmisc
sakura
x11-apps
xinit
yad'

for aPKG in ${PKGLIST}
do
 [ "$aPKG" == "" ] && continue
 chroot rootfs-devuan apt install ${aPKG}
 echo
 sync
 echo -n "ENTER to install more: "
 read WANTMORE
done

####
echo
echo "...finished"
echo -n "Press ENTER if you want to umount proc, sys and dev: "
read YESUM
if [ "$YESUM" == "" ];then
 busybox umount rootfs-devuan/proc 2>/dev/null
 busybox umount rootfs-devuan/sys 2>/dev/null
 busybox umount rootfs-devuan/dev/shm 2>/dev/null
 busybox umount rootfs-devuan/dev/pts 2>/dev/null
 busybox umount rootfs-devuan/dev 2>/dev/null
fi
###end###

...not too long to put all of it into this blog post. Have uploaded the script, with false ".gz" appended, here.

Next-up, save the downloaded .deb packages in a local repository, which will be accessed on subsequent builds.

This script builds folder rootfs-devuan from .deb packages, but EasyOS is also built with lots of .pet and OE Kirkstone/Scarthgap .tar.xz packages. The apt management knows nothing about these. In woofQV, these foreign packages get converted to .xbps packages in a local repository, so that the XBPS manager can install them. However, I don't know if want to go to all that trouble again; thinking of doing it a different way.    

Tags: easy

Orange menu entries fixed in QV

In the release announcement for QV 250605, I posted a list of issues to work on.

One of those is the "orange ball" menu entries. Choose one of those, and it failed to install.

Fixed script /usr/local/orange/orange.install, see github:

https://github.com/bkauler/woof-quantum-vis/commit/27b859dfba605c748014d20212a1388613a1deeb   

Tags: quirky

Change snapshot management

As mentioned in previous posts, one of the menu items in the initrd is to bootup a different snapshot than the default. It is item-6:

QV is unique in that each snapshot is the complete filesystem, everything under "/", not just a folder such as "/home" as other distros implement snapshots.

So, each snapshot is a complete and self-contained installation of QV. An older snapshot may work with a particular kernel version, and you might have to be careful with booting that snapshot with a later kernel. The kernel version that have been used with a particular snapshot are found in /boot, for example /boot/vmlinuz-6.6.68

If I have upgraded to kernel 6.12.32 in the default snapshot, then want to bootup the older one that is only tested with kernel 6.6.68, then we need to be able to manage this. For example, you might decide to test it with the 6.12.32, but if not happy then you will want to revert to 6.6.68.

If you choose to update the kernel, /usr/lib/modules will also need updating, as will /usr/lib/firmware

Or, you might choose to bootup the older snapshot, but switch back to the kernel that you know works.

If have booted with the default kernel, then choose a snapshot that has only used an older kernel, and you choose to switch to the older kernel, then 'kexec' is required.

If the kernel version in a snapshot is changed a lot, which may happen over time, proliferation of old versions and old folders in /usr/lib/modules, needs to be curtailed.

I had a coding marathon last night and today, and all of this has been implemented. Not yet tested. Here are the changes to the 'init' script in the initrd:

https://github.com/bkauler/woof-quantum-vis/commit/18b418fa1439c7e1362c753648d8f03eb9ca5295

Updated translations:

https://github.com/bkauler/woof-quantum-vis/commit/1f7f52149a02f29c1d9da903411e9abef1459cd7

This enhanced snapshot-change management not yet tested!   

Tags: quirky

Linux kernel 6.12.32 compiled

Compiled in Easy Daedalus, but intend to use the same kernel also for Easy Scarthgap and QV.

The reason, to make less work for myself.

QV 250605 has the 6.15.1 kernel, which I chose so as to get the most recent bcachefs. However, I have gone cold on bcachefs. Apart from it seeming to be still immature, one of the things that attracted me to it is native encryption; but that is not really suitable.

The bcachefs encryption encrypts the entire partition. You cannot pick and choose to encrypt parts of the partition, as fscrypt offers with ext4 and f2fs. QV is a frugal install, with the intention that it can co-exist with anything else in the rest of the partition; encrypting the entire partition is not what I want.

So, there is no need to be at the most recent kernel version. Will stay with the same 6.12.x as used in the Easy OSs.

The kernel has both aufs and overlayfs, but neither are required in QV.

A couple of people have tested QV 250605 and reported video issues. I wonder if the 6.15.1 kernel is responsible? That would be a good reason to go back to 6.12.x.     

Tags: easy

Fix menu broken video in initrd woofQV

I applied this fix in woofQ, for Easy Scarthgap and Easy Daedalus, see previous blog post:

https://bkhome.org/news/202506/fix-broken-video-menu-in-initrd-woofq.html

Have done the same in woofQV. This applies to QV. Here are fixes in the 'init' script in the initrd:

https://github.com/bkauler/woof-quantum-vis/commit/8a8682dbe45f7ac1e4f54d7cd5314cfd1f12d1a9

Also, translations:

https://github.com/bkauler/woof-quantum-vis/commit/89d34694aa39ef0d44381f6ecf4794fb4c16da5b

There are other situations in the initrd that also require kexec, planning to tackle that next.    

Tags: quirky

Fix broken video menu in initrd woofQ

There is a menu in the initrd, you can choose things like filesystem check and version rollabck. Have now added "Run Xorg Wizard to fix broken video".

Previously couldn't do that, because wanted to boot with the kernel commandline parameter "nomodeset". This has to be on the commandline, can't be specified after the kernel has loaded. When the enu is displayed in the initrd, the kernel has already loaded.

The only way to do is to restart the kernel, and there is a utility to do this, 'kexec'. However, previously I couldn't get kexec to work. Made some changes,and it now works. So can now have that entry in the initrd.

Here are the changes in the 'init' script in the initrd:

https://github.com/bkauler/woofq/commit/f7d606e957afcb978a63f7c2e41980a288a38e10

And translations for the new text:

https://github.com/bkauler/woofq/commit/a5fd0b8e6d892ab543c4e0f70ae338e8213c2132

This applies to both Easy Scarthgap and Daedalus, both being built with woofQ.

The fix has also been done in woofQV, will post separately about that.       

Tags: easy

Mini-tutorial creating a snapshot in QV

The boot menu in QV, in the initrd, has an item to choose to create a new snapshot. Scroll down this blog post, there is a photo; it is item number 4 in the menu, "Take a snapshot". What it does is take a snapshot of the current default snapshot.

To understand what is going on with this snapshot feature of btrfs, you can create one manually. This is a bit of a learning exercise about btrfs and how QV implements it.

The previous blog post, announcing the 250605 version of QV, also has a brief overview of QV:

• QV version 250605 released — June 05, 2025

...overview needs to be fleshed out a lot more.

You guys who are familiar with Puppy Linux, Puppy-derivatives and Puppy-like distributions, will know about a "frugal" install. QV is a specific type, it is a "full frugal". That is, it looks like a normal Linux installation, with folders /dev, /sys, /proc, /etc, /usr, etc., except in a folder, not at "/". In the case of QV in a USB-drive, it looks like this:

...that path needs explanation...

sdc2 is the btrfs partition, folder 'quantumvis' is the frugal installation of QV. Then there are sub-folders "4/@qv" ... that needs explaining!

Back-tracking, after having written 'qv-250605-amd64.img' file to the USB-drive, and boot, there are only two folders; /mnt/sdc2/quantumvis/1 and /mnt/sdc2/quantumvis/2. "1" and "2" are just folder names. They each hold one snapshot, so the folder naming 1, 2, etc., is an organizational thing.

Folder "@qv" is also just a folder name, except it has mounted on it a btrfs subvolume. The "@" prefix is just a folder-naming convention, so that we know this folder has a btrfs filesystem on it.

On a subsequent bootup of the USB-drive, I chose to create a new snapshot; that is item-4 "Take a snapshot", and that automatically became number 3. So, looking with ROX-Filer, we see there is now a folder "3":

What would be very good is to create a snapshot manually, so as to understand better how it works. I am also doing it to refresh my memory, having left QV and btrfs back in November 2024.

So, current situation is QV is running, there are three snapshots, and the current one is "1". So, let us create a snapshot of "1", and number it "4":

Firstly, although we know in my case that the working-partition is sdc2 and frugally-installed in folder 'quantumvis', it can also be found like this:

# . /etc/rc.d/PUPSTATE 
# echo $WKG_DEV
sdc2
# echo $WKG_DIR
quantumvis/
# 

Creating a snapshot is really easy:

# mkdir /mnt/${WKG_DEV}/${WKG_DIR}4
# btrfs subvolume snapshot /mnt/${WKG_DEV}/${WKG_DIR}1/@qv /mnt/${WKG_DEV}/${WKG_DIR}4

Now there is folder "4":

Now look in folder "4":

the btrfs subvolume filesystem "1/@qv" has been "copied" to "4/@qv", except there is no actual file copying involved; the snapshot happens in the blink of an eye. "4" behaves completely different, and is completely independent of "1" ...well, this is the magic of the btrfs copy-on-write filesystem.

Look at the photo at top of this blog post, it shows what is inside 4/@qv

Note: don't be confused by the word "subvolume". It is just a nested btrfs filesystem inside another btrfs filesystem. Normally, we have, say, ext4, with a single filesystem occupying the entire partition. Nesting a filesysystem may take awhile to get your head around, but ultimately you will see it as a simple concept.

To make this into a bonafide new snapshot, the file 'snapshot-record' needs to be edited (see above photo). Append a line, like this, in format number-date-description:

1|250605|Original Snapshot
2|250605|Snapshot of pristine first bootup
3|250605|test snapshot
4|250606|manually created snapshot of 1

Next thing I did was reboot, to see if can change into this new snapshot...

...this is the menu in the initrd. I selected item-6, "Choose a snapshot to boot into". Then got this:

...I chose item-4, clicked on "OK", then got a desktop:

...see the text at top of the screen, letting you know what snapshot is currently running. This is now the default.

We could keep going, making more snapshots. Some users of btrfs have hundreds of them. Interesting discussion on this, one person suggesting keep it no more than about 100:

https://unix.stackexchange.com/questions/140360/practical-limit-on-the-number-of-btrfs-snapshots

...yes, deleting a snapshot that has children-snapshots (grandchildren etc.) can apparently take awhile.

That reminds me, I wrote about deleting snapshots in QV, in 2024:

• QV snapshot deletion — April 15, 2024

...commented on the potential issue with children and grandchildren.

A clarification, if you have btrfs snapshot experience on other Linux distros:

Some other distros may only snapshot some folders, especially the /home folder. QV is different, it snapshots the entire "/" filesystem. With the exception that folder /files is a separate encrypted ext4 filesystem in a sparse file; the same /files appears in all snapshots.

Look at an above photo, there is file '.sparse-ext4' -- that is the ext4 sparse file. It is mounted on folder '.sparse-mnt' -- .sparse-ext4 is loop-mounted on that, then bind-mounted to /files in all snapshots.

This little mini-tutorial is a diversion. I've been rethinking some structure of QV, got kexec to work; so getting back onto that. Well, tomorrow.

If you would like to provide feedback, there is a forum topic to discuss btrfs snapshots here (hint: woofwoof):

https://forum.puppylinux.com/viewtopic.php?t=14548    

Tags: quirky

QV version 250605 released

"QV" is an acronym for "Quirky Void" or "Quantum Vis". It is built with woofQV. The github project page has a brief introduction to QV:

https://github.com/bkauler/woof-quantum-vis

I worked on it for several months in 2024, then went back to working on EasyOS; so QV has lapsed since then. It is a unique distro, fascinating concepts, and really does deserve to have ongoing attention. So, back on it.

You can read about the development of QV in the "quirky" tag of this blog. Here are the most-recent several posts:

• woofQ changes merged into woofQV — June 04, 2025
• bcachefs utility crash in initrd — June 02, 2025
• Reconsidering QV — June 01, 2025
• Finding XBPS to be very frustrating — October 11, 2024
• QV version 241007 released — October 07, 2024
• Update PET for QV — October 07, 2024
• Attempt to disable pam — October 06, 2024
• QV basic handling kernel version changes — October 02, 2024
• Kexec fail — October 02, 2024
• QV now has encrypted folders — September 30, 2024
• KDE Dolphin works in QV — September 28, 2024
• The pink and grey galahs have returned! — September 28, 2024

...you can look through those to get basic ideas what QV is all about. Though, I do need to write some documentation.

Here is a screenshot, taken in an earlier release:

A couple of things don't work...

One of the above links describes an "update PET package"; that currently doesn't work. The intention is that it will be like EasyOS; a small download. QV is a very big initial download, "qv-250605-amd64.img", about 1.8GB; however, the update PET packages are expected to be about 90MB.

The menu has some "orange ball" entries. You would know about these if you have used EasyOS. They are applications in the menu, but not yet installed, and choosing from the menu, it will download and install. Currently broken.

Well, that's all the bugs I can think of now, but of course we will find more.

The usual thing, just like EasyOS; you write the drive-image file to a USB drive, then boot it. You must use a good quality drive, and fairly big, at least 32GB. Personally, I use SSDs in USB-caddies; big, fast and last indefinitely. Currently using a "small" SSD of 500GB.

Brief introduction to QV concepts

Unlike EasyOS, QV does not use squashfs files, nor aufs/overlayfs layers. It is "frugal" though, with everything in a folder; that is, looks like a normal Linux installation, except in a folder and not under "/"

Another big difference is that the working partition is a btrfs filesystem, which brings something great to the table; snapshots. Snapshots are managed via a menu in the initrd. Photo:

Snapshots are great. You could, for example, create a "compile environment" snapshot, and install the 'devx-1.0.pet' into it. Snapshot creation is virtually instantaneous. This article introducing btrfs principles, including snapshots, looks good:

https://fedoramagazine.org/working-with-btrfs-general-concepts/

I mentioned above, minimum size of 32GB; but if you really get into snapshots, like want to compile packages, or want to download woofQV and build your own QV, then you are going to need a lot more space. An SSD in USB-caddy is a good solution, or you can install to a btrfs partition in an internal drive -- QV has "QV Installer" to make that easy.

Notice the above photo, a password to decrypt the working-partition. Currently though, only the /files folder is encrypted. This folder is where you keep all your personal stuff, and you can also setup apps to store their stuff in there, like, for example a video editor project.

An important point about /files is that it is outside the snapshot mechanism, so all snapshots will see the same content in /files.

The thing is though, btrfs does not have native encryption. What has been done is rather sneaky; /files is actually a ext4 filesystem, which is in a sparse file. ext4 supports fscrypt, per-folder encryption.

Unlike EasyOS, QV does not offer containers. However, like EasyOS, QV offers each application running as its own user. This provides fantastic isolation between apps. A further security feature can be seen in the above boot menu, "Turn ON lockdown"; this will prevent the user from mounting any partitions -- the btrfs partition in which QV is running, is mounted, but the user cannot fiddle with it, nor even see outside the folder in which QV is installed.

I am intending to experiment with using bcachefs instead of btrfs. Stay tuned for that.

URL links

Primary download:

https://distro.ibiblio.org/quirky/quirky-void/amd64/releases/base/

Fast mirror in Europe, courtesy of NLUUG:

https://ftp.nluug.nl/os/Linux/distr/quirky/quirky-void/amd64/releases/base/

The EasyOS documention explains how to write a drive-image file to a USB drive, in case all you know about are ISO files:

https://easyos.org/install/how-to-write-easyos-to-a-flash-drive.html

Feedback is welcome in the QV section of the Puppy Forum:

https://forum.puppylinux.com/viewforum.php?f=236

Testers welcome!

Do bear in mind that QV is very experimental, much more than EasyOS. Report bugs, suggestions welcome.

TODO list

Lots of things to work on. Thinking, off the top of my head...

See item above "6 Choose snapshot to boot into". Each snapshot may have a particular Linux kernel, and if, for example, you switch back to a much older snapshot, that requires an older kernel, currently you will be asked if you want to update that snapshot to the current kernel (and modules), or not. If you decide to boot with the older kernel, a reboot is required, which is clumsy. I intend to get "kexec" working, which is a fast method to change kernel.

Fix the orange-ball menu entries.

Fix updating.

Write documentation.

Evaluate bcachefs.

EDIT 2025-06-06:
I have written a mini-tutorial on btrfs snapshots and how QV implements them:

• Mini-tutorial creating a snapshot in QV — June 06, 2025

Hope to write more docs in the future, though as QV is a moving target they may get out of date quick.   

Tags: quirky