site  contact  subhomenews

Things are happening

February 24, 2017 — BarryK
Just realised, the last blog post was Feb. 10, and it is now 24th.

So, a very brief post, in case anyone is wondering what is happening. Lots of things! Working on Easy Linux, containers, and more.

Still expecting to have something to upload mid-March.

Tags: easy

More stability issues with E200HA

February 07, 2017 — BarryK
Ha ha, it has got worse. Running the new "Easy Linux" Quirky on an overlay filesystem, stability has become worse. I was getting a very occasional freeze with the full installation of Quirky, but it has become more frequent when running on an overlay f.s.

Apart from seemingly random freezing, I have been getting a degradation. I suddenly notice the degradation, such as apps not starting, even htop, and xload shows CPU usage has gone through the roof. Requires holding down the power button to shutdown.

I wondered if the overlay f.s. was to blame, so patched the 4.4.47 kernel with aufs and compiled. First time ran it with aufs, it ran for an hour, no problem. Next bootup, got the UI degradation after about half an hour.

I read on the internet that this kernel boot parameter will fix freezing on the E200HA:


Rebooted, got a freeze after only a few minutes! Still using aufs. Rebooted, now it has been running for one hour and 7 minutes. Hmmm.

Using it right now, posting from SeaMonkey.


Sorting version numbers

February 04, 2017 — BarryK
I have a need to sort version numbers in the Easy Linux initramfs. The 'sort' utility in Coreutils has the "-V" option, however my initramfs only has busybox sort, which does not have that option.

I recall some years ago, trying to figure out a script to do it. Now that the need has come up again, I did a search with google, and found this:

...yes, a simple one-liner, works great.

A note on progress with Easy Linux, the next-gen Quirky. Coming along very nicely, and lots of fun. When will there be a release? ...not sure, want to get it pretty rock-solid first, even though it will be an alpha or pre-alpha. Probably early-March.

Tags: easy, linux

Kernel 4.4.45 compiled

January 31, 2017 — BarryK
I recently compiled the 4.4.44 kernel, with namespaces and cgroups enabled (though only "devices" cgroups).

A bit more online reading and I found more configuration settings that look good when working with containers. So have now compiled the 4.4.45 kernel with these turned on.

Typing from my handwritten notes, here they are:

Device Drivers ->

[*] Support for multiple instances of devpts CONFIG_DEVPTS_MULTIPLE_INSTANCES
[*] Network device support ->
-*- Network core driver support ->
<*> Virtual ethernet pair device CONFIG_VETH

In my notes, I have written that for the multiple devpts, mount with "mount -o newinstance ..."
CONFIG_VETH supports a local ethernet tunnel.

A progress note: my "next generation" Quirky, codenamed "Easy", boots to the desktop. Details to follow.


Easy Containers

January 25, 2017 — BarryK
I might as well give this effort a name, calling it "Easy Containers". I am having a go at creating containers for Quirky, using grassroots utilities, rather than the heavy-duty packaged techniques such as LXC or Docker.

Thanks to jamesbond and heaps of posts on the Internet, the basics of Easy Containers is looking good. Well, it is fun anyway.

The container is to be a layered filesystem, using aufs or overlayfs, with the operation system SFS file underneath, and the rw layer either a folder, tmpfs or a partition. The SFS would be your Quirky, Puppy, or Fatdog .sfs, straight from the live-CD.

Layered root filesystem
Setting up the layers is straightforward, for example using overlayfs:
# mount -t squashfs -o loop,noatime q.sfs q_ro/ro1
# mount -t overlay -o lowerdir=q_ro/ro1,upperdir=q_rw/rw1,workdir=q_rw/work1 overlay q_top1

Where q_ro/ro1, q_rw/rw1, q_rw/work1 and q_top1 are just folders.

After doing this, I made some changes on top, that is, in ./q_top1:

With Quirky, /sys and /run do not exist, so created them.

I want to avoid any "mount -bind ..." or "mount -rbind ...", so instead of doing that for /dev, I created static device nodes in q_top1/dev -- you can copy these out of initrd-tree/dev in Woof.
Don't copy /dev/* from the host system, it has all sorts of stuff mounted. I also manually created folders /dev/pts and /dev/shm.

The online advice is to execute "mount --rbind /dev ./q_top1/dev" prior to the chroot, and I don't know what the downside is with just having static device nodes. A device node that is in-use in the host, what of that? -- having a copy of the node in the rootfs, is that unusable? In what situation would this matter?

chroot into rootfs
To chroot into q_top1, just do this:
# cp -f /etc/resolv.conf ./q_top1/etc/resolv.conf
# unshare -piumU --fork --mount-proc --map-root-user --setgroups=deny env -i TERM=xterm DISPLAY=:0 /bin/busybox chroot ./q_top1 /bin/sh

Then inside the chroot, do this:
# mount -t proc proc /proc
# mount -t tmpfs shmfs /dev/shm

You can then run leafpad, geany, rox, seamonkey, etc. However, there is a bug, the first time an X app is started, it crashes with this error message:
"BadShmSeg (invalid shared segment parameter)"

Thereafter, X apps start OK.

The CLI full-screen 'mp' text editor, that uses ncurses, works. As it is not an X app, it does not have that crashing problem.

It is interesting that ":0" works inside the chroot, some online docs state that won't work, and you need to use other methods, such as the following...

This is probably a better way to do it, as I aim to make the chroot environment more isolated. It is odd, when I tested this, the above-mentioned "BadShmSeg" crash did not occur, yet just now did it the same way, and that error is back. So, there is a mystery here.

I read if you remove the "-nolisten tcp" on the Xorg invocation line in /usr/bin/xwin, and run "xhost +localhost", you can then set DISPLAY=localhost:0 inside the chroot filesystem.
Two problems with this. Firstly, it is considered to be a security risk, and xhost is considered to not be a very secure safeguard. Second, I could not get it to work, the value of DISPLAY is reported as invalid inside the chroot environment.

However, there is another, secure way of doing it. You need the 'socat' utility, which is available as a DEB, if you are running Ubuntu or Debian based puplet. I found out how to do it here:

Create a little script, I have called it 'ec-chroot':
if ! pidof socat; then
# -ly causes log to syslog.
socat -ly -d -d TCP-LISTEN:6000,fork,bind=localhost UNIX-CONNECT:/tmp/.X11-unix/X0 &
#needed for internet access...
cp -f /etc/resolv.conf ./q_top1/etc/resolv.conf
#--map-root-user needed if have -U, otherwise env will fail.
unshare -piumU --fork --mount-proc --map-root-user --setgroups=deny env -i TERM=xterm DISPLAY=localhost:0 /bin/busybox chroot ./q_top1 /bin/sh

Once inside the chroot, run these:
# mount -t proc proc /proc
# mount -t tmpfs shmfs /dev/shm
# leafpad

...and leafpad starts without crashing, or rather, it did first time I tested this, not today!

SeaMonkey works too, and accesses the Internet, except that churns out dbus errors. I tested with starting udevd beforehand, but no good. SM still runs. I could compile SM without dbus support, to avoid the problem. Or, could figure out why dbus doesn't work.

Multiple containers
I have tested running two containers simultaneously, both with the underlying q.sfs. Got them running right now, one is running geany, the other leafpad, no problem with these windows.

Just an observation, not quantified, but I think there is a slight startup delay of the X app when using the DISPLAY=localhost:0 method.

What to do about that "BadShmSeg"? This question has come up many times on the Internet, but haven't found a definitive solution. My wild guess is that there is some shared-memory that Xorg is using in the host, that is not valid in the chroot environment, and Xorg detects that when the first X app is run. There are many online posts of a fix for Qt-based apps, using "export QT_X11_NO_MITSHM=1", however, that is not a fix, it is just avoiding the problem.

A less-than-satisfactory solution would be to run a little do-nothing X app on first entry to the chroot environment.

It is most pertinent to now ask, how secure is this? I have used 'unshare' to unshare everything, have not run "mount --bind" or "mount --rbind", "env -i" has removed most environment variables. What more can we do to improve security?

Here is something interesting to think about. Inside the container:
sh-4.3# mkdir /mnt/sda2
sh-4.3# busybox mount -t ext4 /dev/sda2 /mnt/sda2
mount: permission denied (are you root?)
sh-4.3# whoami

...which is good from a security viewpoint. But why doesn't it work, and can it be got around?

This is such an interesting topic, so I will start a Forum thread for feedback.


Grassroots containers in Fatdog

January 24, 2017 — BarryK
Ha ha, I thought that jamesbond, one of the main guys behind Fatdog, would have already studied containers and simple ways to implement them!

I have already conducted simple experiments on "grassroots", or build-your-own" containers, using 'env', 'unshare' and 'chroot', haven't posted to the blog yet, as need to do more investigation.

However, jamesbond has already done all the work on a "grassroots" implementation. This is an email he sent me today:

RE: Container - you may want to see how Fatdog supports containers, here:

I have merged the standard sandbox into Woof-CE, so newer puppies should have built-in.

As you are probably aware by now, "container" is full of hype and stuff. For Windows users these kind of things are probably "new" and "interesting", but for Linux they're re-packaged old stuff. A container is just "chroot on steroid" and there are many ways to achieve it.

The most basic way you can just use "unshare" command from recent core-utils (in fact, from Fatdog uses this if it's available instead of just standard "chroot"). If you want to run a process inside an existing namespace, you can use "nsenter" (also from core-utils).

Of course, the basic tools for Linux container is LXC, and this is what I use for

This script and all other Fatdog scripts are available here:


For reference, there is a webpage that info on a grassroots build-your-own approach, using 'unshare', 'env' and 'chroot':

Note, 'unshare', 'env' and 'chroot' are all busybox applets, though "full" versions are available elsewhere.

Tags: easy, linux

Getting started with LXC

January 24, 2017 — BarryK
My venture into containers continues. Yesterday I posted some preliminary notes, and a quick look at Firejail:

LXC is, as far as I can determine, the officially supported Linux kernel mechanism for containers. The website for LXC is here:

Kernel config
None of my previous kernels for Quirky have namespaces and cgroups support. For my experiments with namespaces and cgroups, I have compiled the 4.4.44 kernel with these settings:
# CONFIG_CPUSETS is not set
# CONFIG_MEMCG is not set
# CONFIG_BLK_CGROUP is not set

Note that for cgroups, I only enabled CONFIG_CGROUP_DEVICE, as that is the only one that intersts me, for now. Was that a bad decision? Maybe should have enabled more, but can do so in future.

LXC 2.0.6
The Ubuntu DEB has a huge number of dependencies, so I compiled lxc version 2.0.6 from source, using this very cutdown config:
# ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var --build=x86_64-pc-linux-gnu --with-distro=slackware --disable-lua --enable-bash --disable-python --disable-selinux --disable-apparmor

This is the report:
- compiler: gcc
- distribution: slackware
- init script type(s):
- rpath: no
- GnuTLS: yes
- Bash integration: yes
Security features:
- Apparmor: no
- Linux capabilities: yes
- seccomp: no
- SELinux: no
- cgmanager: no
- lua: no
- python3: no
- examples: yes
- API documentation: no
- user documentation: no
- tests: no
- mutex debugging: no
- Logs in configpath: no

After installation of LXC, I ran "lxc-checkconfig", and it reported:
# lxc-checkconfig
--- Namespaces ---
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: enabled
Network namespace: enabled
Multiple /dev/pts instances: missing
--- Control groups ---
Cgroup: enabled
Cgroup namespace: required
Cgroup device: enabled
Cgroup sched: missing
Cgroup cpu account: missing
Cgroup memory controller: missing
Cgroup cpuset: missing
--- Misc ---
Veth pair device: missing
Macvlan: missing
Vlan: missing
Bridges: missing
Advanced netfilter: enabled
FUSE (for use with lxcfs): enabled
--- Checkpoint/Restore ---
checkpoint restore: missing
File capabilities: enabled

What is not shown above, is that the line "Cgroup namespace: required" has the "required" in red colour text. Indicating that something is amiss.

After googling around, I couldn't determine the exact cause of this "required", except for lots of people asking the same question, but did find this statement "This one should be fine to ignore", here:

Anyway, a quick little test, to see if can create a basic container. It has to be created from files from the host system, and for this there are templates:
# ls /usr/share/lxc/templates
lxc-alpine lxc-centos lxc-fedora lxc-oracle lxc-sshd
lxc-altlinux lxc-cirros lxc-gentoo lxc-plamo lxc-ubuntu
lxc-archlinux lxc-debian lxc-openmandriva lxc-slackware lxc-ubuntu-cloud
lxc-busybox lxc-download lxc-opensuse lxc-sparclinux

I will give the busybox template a go:
# lxc-create -n mycontainer -t busybox
lxc-create: lxccontainer.c: do_create_container_dir: 972 No such file or directory - failed to create container path /var/lib/lxc/mycontainer
lxc-create: tools/lxc_create.c: main: 318 Error creating container mycontainer

...hmmm. I manually created /var/lib/lxc, then tried again:
# lxc-create -n mycontainer -t busybox
setting root password to "root"
'dropbear' ssh utility installed

Yes, works, and it is even chrootable:
# chroot /var/lib/lxc/mycontainer/rootfs /bin/sh
# ls
bin etc lib mnt root selinux tmp var
dev home lib64 proc sbin sys usr
# exit

However, there are a lot of lxc-* utilities, so can use those to get into my container.
Using this webpage as a getting-started guide:

It seems that we have to "start" a container first, before can log into it:
# lxc-start -n mycontainer -d
lxc-start: tools/lxc_start.c: main: 360 The container failed to start.
lxc-start: tools/lxc_start.c: main: 362 To get more details, run the container in foreground mode.
lxc-start: tools/lxc_start.c: main: 364 Additional information can be obtained by setting the --logfile and --logpriority options.

Using the fireground option:
# lxc-start -n mycontainer -F
lxc-start: cgroups/cgfs.c: cgfs_init: 2359 cgroupfs failed to detect cgroup metadata
lxc-start: start.c: lxc_spawn: 1093 Failed initializing cgroup support.

...cgroups. I am at the bottom of the learning curve with cgroups, so had better readup on that next. Obviously, there is something I have to initialize.

Well, at the bottom of the learning curve with namespaces and containers also!
But, one step at a time, will get there.


Containers for Quirky

January 22, 2017 — BarryK
I am thinking about the next-generation Quirky, and through 2016 have played with various software technologies, such as device-mapper and btrfs.

Another one on the list is containers, and I did a bit of reading during 2016, wasn't impressed. A Docker container has an absolute minimum size of 3.6MB. I saw one post where the person was proud that he had reduced the size of his Docker container to 25MB.
This idea of putting everything into a container, is so wasteful, and contrary to our Puppy philosophy.

It is all very well that these days we have heaps of storage, but with containers it is very easy to use it up. I am thinking of Quirky running on a Flash stick. Ideally, I would like Quirky to run happily on a 4GB stick.

I was trying to think how containers can be implemented efficiently, and thought, why not use an overlay filesystem, such as overlayfs or aufs? This is Puppy's home turf! This seems like a very good idea, and so I expected the same idea will have occurred to many other people -- which it has, lots of posts, including using overlayfs for Docker.

Firstly though, what is a container? A filesystem that you can chroot into is a container. An environment that you can run an application with isolation from the rest of the system, is a container. The basic requirement is to provide security, such as prevent a rogue app from getting out, or an outside intrusion from getting out.
Containers go some steps beyond the basic chroot, with extra levels of isolation.

This idea of using overlayfs. Instead of copying everything, libraries, etc., into a container, mount it as an overlayfs, with the read-only system as an SFS file underneath. This SFS file can be shared by all containers. This is extremely space efficient, but it also solves some implementation problems, for example as described here:

Right now, I am just reading and learning more. Nowhere near formulating what the proposed "next gen" Quirky will look like.

Currently studying this, a simple implementation of containers, very interesting: