site  contact  subhomenews

EasyOS and Quirky development hiatus

June 29, 2018 — BarryK

I have decided to put the development of EasyOS and Quirky on-hold for awhile, as I want to reconsider some of the basic architecture.

Various thoughts have been brewing in the back of my mind for awhile, and it has come to a head as I have been implementing EasyPak, my concept for "universal packages".

Universal packaging systems include Flatpak and Snaps, and the more I looked into these, the more troubled I became. The idea, basically, is that a universal package will run on "any Linux distro", however, I started to realise that there are caveats to this. Some assumptions have to be made about the underlying OS, such as existence of pulseaudio, systemd, dbus, and so on. In other words, the "universal app" will actually only run on certain distros.

Then there is the size. A universal package is just about an entire distribution. Yep, just about every underlying package has to be included in the universal package. Hence, when I installed the Krita flatpak, the Qt-based paint program, from Flathub, the download was 566MB.

I began to question the value of the whole philosophy of universal packages.

I am also troubled by the philosophy underlying containers. Yeah, yeah, universal packages, containers, flavour of the month, flavour of the year, whatever.

There are some aspects of Android that I like. Android runs each app as a separate user. This is actually a very simple isolation mechanism, that uses Unix groups/users architecture that has been there right from the start of Unix and Linux.

I have never liked Java, yet the JVM does provide abstraction from the hardware (HAL), which has one important principle, that packages can be shipped as Java code and will run on all Android phones and tablets. Well, the Java API provided with Android does change. You can download an app, size say 10 - 50MB, and it will run on all phones that have the minimum specified Android version. Yes, these are universal packages, a fraction of the size of those coming from Flatpak and Snap.

Anyway, there are a few lines of thought that I want to follow, will try and post here when there is an interesting development. 

Tags: easy, quirky

Packages recompiled in oe-qky-src

June 23, 2018 — BarryK

Finalised, for now, the backporting of packages from the Sumo release of OpenEmbedded, in my fork 'oe-qky-src', see commits (June 23, 2018):

https://github.com/bkauler/oe-qky-src/commits/master

Did a build overnight, and have now imported the binary packages, all 724 of them, into woofQ, for future builds of EasyOS and Quirky. The packages will be available in the "oe-pyro" repository in the PPM (Package Manager), and so are uploaded.

Here they are:

http://distro.ibiblio.org/easyos/amd64/packages/compat/oe/pyro/

Now, I am keen to get back to EasyPak, an exciting new idea for "universal packages"...

Tags: oe, easy, quirky

How to give super-powers to zeus

June 20, 2018 — BarryK

This is very interesting! I have a user named 'zeus', your normal underprivileged user. How can I give zeus admin-privileges, without actually becoming root -- because, that is what 'sudo' does, can bump up to 'root' to perform admin operations.

I want to perform some admin operations, while still being zeus. Never mind why I want to do this, I just do.

The 'capsh' utility, in the 'libcap' package, can do it. I wrote about "Linux capabilities" recently:

http://bkhome.org/news/201805/improving-linux-capabilities.html

...however, I am not interested in the cap_sys_mount patch anymore.

Puppy Linux and derivatives such as Easy and Quirky, run as 'root', with the ability to run Internet applications as user 'spot', and in Easy in containers with unprivileged-root -- the latter is achieved by using 'capsh' to drop privileges when chroot into a container.

Anyway, running as root, it would seem that capsh could be used to switch to a normal user, yet keep any privileges that we want to keep. In Easy, there is a user named 'zeus', that I created especially for this experiment.

I thought that capsh would work (using "--secbits"), however, it didn't. I am using libcap version 2.25, which the original author stopped work on some years ago. I discovered that some further work has been done on libcap, to add that missing/broken feature:

https://git.kernel.org/pub/scm/linux/kernel/git/morgan/libcap.git/commit/

...thanks Andrew!

I modified the source slightly, copied from the kernel source /usr/src/linux-4.14.44/include/uapi/linux/capability.h, prctl.h, and securebits.h, to libcap-2.25/libcap/include/uapi/linux/, and changed the "DYNAMIC..." line in Make.Rules to this:

DYNAMIC := $(shell echo yes)

...so as to get dynamically liked executables.

Then just ran the usual:

# make
# new2dir make install

Running "capsh --print" prints out all of the capabilities. Now, if I want to change to user zeus and keep all of those capabilities:

# capsh --keep=1 --user='zeus' --inh='cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,cap_wake_alarm,cap_block_suspend,cap_audit_read' --addamb='cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,cap_wake_alarm,cap_block_suspend,cap_audit_read' --
# whoami
zeus
# rm -f NewFile1
#

'NewFile1' was owned by root, and a user would not be able to delete it, which I checked was the case when I just did a normal "su zeus". Yippee, zeus has super-powers!

Note, the order is important:

capsh --keep=1 --user='zeus' --inh='...' --addamb='...' -- 

The "--" causes bash to run, so you have a new shell, and get back to root by typing "exit".

Tags: easy, linux, quirky

Devuan will be repo for EasyPak

June 18, 2018 — BarryK

I posted yesterday about the genesis of EasyPak, my take on "universal packages":

http://bkhome.org/news/201806/applying-the-kiss-principle-to-universal-packages.html

I decided to use the Devuan "ascii" repository for building the universal packages. I have created script /usr/local/EasyPak/create_db and a repository spec file /usr/local/EasyPak/DISTRO_COMPAT_REPOS-devuan.

It works, creates the Puppy-format package files in /usr/local/EasyPak, for "main", "contrib" and "non-free". Script create_db can be rerun at any time in a running EasyOS to update these db files.

EasyOS "Pyro" series is built from packages compiled in my fork of OpenEmbedded. That is, a distro compiled entirely from source. However, the "universal packages" will be constructed from Devuan DEBs. This does not matter, as that is the whole idea of universal packages, that they will run on any Linux distro. Implementing this will be "proof of the pudding".

The next step... taking a break, will think about that later.

Tags: easy

Applying the KISS principle to universal packages

June 17, 2018 — BarryK

The main contenders for "universal packages", that is, apps that will run on any Linux distribution, are AppImages, Flatpaks and Snaps.

I have been studying them. AppImages seem a bit too limited, though a simple concept. Flatpaks and Snaps are very complicated, and after much reading, I think that there is a much easier way to achieve similar goals.

Thinking about a very simple mechanism for universal packages, it would seem that I need a proper CLI package manager, instead of PPM (PETget Package Manager, or Puppy Package Manager). Then I remember, there is such as project, named Pkg.

Pkg is a mammoth effort by Puppy Forum member sc0ttman. The project is described here:

http://murga-linux.com/puppy/viewtopic.php?p=985654

There is older discussion here:

http://murga-linux.com/puppy/viewtopic.php?t=111468

The project is online, at gitlab:

https://gitlab.com/sc0ttj/Pkg

I downloaded:

# git clone git://gitlab.com/sc0ttj/Pkg.git --depth 1

My idea for a universal package system, is to use what is already there, the DEB and RPM repositories, and their superb versioning and updating handling. The idea is simple: choose a package from a Debian/Ubuntu/Devuan/whatever repo, in a container in Easy, download it and all deps -- the secret is to download most deps, overwriting many packages that are already in the underlying q.sfs. A few details to fill in, but that's it, essentially.

The important point here, is that containers in Easy are complete, containing the entire Easy filesystem. This is because every container has q.sfs as a read-only layer in it. Other distros do not have this, they construct cut-down containers.

Each container in Easy has its own complete package manager. Which is where sc0ttman's Pkg may have a roll to play. I need to be able to do many operations from scripts.

So, although Easy now supports Flatpaks, from the commandline anyway, I might not use it.

Of course, my idea needs a name, so I reckon EasyPaks is appropriate! That's it, I officially name my universal packaging system EasyPak. With due acknowledgement of these guys!:

https://www.easypak.net/

Tags: easy

EasyOS built with Flatpak support, take-1

June 16, 2018 — BarryK

I have posted about compiling the 'flatpak' package and its dependencies:

http://bkhome.org/news/201806/testing-flatpak-in-easyos.html

And generation of the signed-key pair needed to use Flatpak:

http://bkhome.org/news/201806/auto-generated-gnupg-signed-key-pair.html

I had compiled 'flatpak' and several dependencies in a running EasyOS, so yesterday and today I ported them to my fork of OpenEmbedded. See the commits for June 16 and June 14, 2018:

https://github.com/bkauler/oe-qky-src/commits/master

The packages are:

flatpak, libseccomp, npth, libassuan, pinentry, appstream-glib, rng-tools, gnupg1-static (statically compiled 'gpg' utility, used in initrd), gpgme, gnupg, gcab, ostree, polkit105 (polkit version 0.105), libsoup, glib-networking

These were imported into woofQ and EasyOS 0.9.4.2 built.

Now for a sanity test. Booting 0.9.4.1 from a USB-stick, pristine first-boot, typed this in a terminal (requires Internet access):

# flatpak remote-add flathub https://flathub.org/repo/flathub.flatpakrepo
# flatpak install https://flathub.org/repo/appstream/org.kde.krita.flatpakref

...it asked to confirm, I entered "y", then off it went. This time I watched the download, it was 566MB ...yikes!

Now to run it:

# flatpak run org.kde.krita

...success!

What I want to do next, is create a nice GUI. Probably will replace "petget" icon on the desktop with "appget", to run the Flatpak GUI.
When a Flatpak app is chosen, I want it to automatically install as an Easy Container, with icon on the desktop. So, each Flatpak app will run in its own container. This already happens, with Bubblewrap, but EasyContainers is more efficient, and from a tentative examination even looks more secure.

...fun times ahead!

Tags: easy

Auto-generated gnupg signed-key pair

June 15, 2018 — BarryK

I posted yesterday about getting Flatpaks working in EasyOS. This required a signed-key pair in /root/.gnupg, which is created using the 'gpg' utility.

I wanted to automate this, so that it is something that the user will not have to bother with setting up. I want Flatpak apps installation to "just work".

To this end, I compiled 'gpg' statically in OE, with the musl library. I used version 1.4.23, not the 2.x series of gnupg. The old 1.4.x series is still being maintained, due to its usefulness in embedded systems. The 'gpg' utility in gnupg 1.4.23 has less dependencies, but the static stripped executable was still big, at 850KB.

In the gnupg source package, the 'configure' script has the '--enable-minimal' option, to build a very small executable, however, it was so cut-down as to be broken. Instead, I configured as follows, which actually created a smaller executable:

# ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var --build=x86_64-pc-linux-gnu --with-tar=/bin --disable-nls --disable-keyserver-path --disable-mailto --disable-finger --disable-hkp --disable-ldap --disable-keyserver-helpers --disable-photo-viewers --disable-bzip2 --disable-twofish --disable-blowfish --disable-idea --disable-agent-support --disable-card-support --disable-gnupg-iconv --disable-asm --enable-dev-random --enable-static-rnd=linux --without-readline --without-libusb --disable-camellia

Now for something contentious...

Prior to compiling, I edited 'config.h' and set NAME_OF_DEV_RANDOM to "/dev/urandom" instead of "/dev/random". The reason is, the kernel feeds activity to /dev/random, and when gpg is generating the signed-keys, it can suck everything from /dev/random and then block, waiting for the kernel to feed more random activity to /dev/random. On a running distro, this random activity is generated by usage, such as running an application, using the mouse, just about anything.

At bootup in the initrd, the situation is bleak if blocking happens. The way around it is for gpg to use /dev/urandom, which is like /dev/random except that it does not stop. If it runs out of random data, the kernel repeats whatever has already been sent. So no blocking occurs, and gpg generates the keys fast.

However, it must be understood that the generated keys are not as secure.

At first bootup, after the password is asked for, to encrypt folders in the working-partition, the same password is used as the passphrase to generate signed keys. The "real name" of the user is auto-generated as "EasyOS user $RANDOM". After bootup, look in /root/.gnupg and there they are, the signed-keys.

Note, there is /dev/hwrng, which apparently generates random data from the hardware, however, I do not know if it will work on all PCs, nor whether it is usable so early in the bootup.

Tags: easy

Testing flatpak in EasyOS

June 14, 2018 — BarryK

As the Purism guys might be using flatpaks for the Librem 5 phone, I decided to check it out. They could also be used in EasyOS, as there is a nice collection available online.

Flatpaks, like Ubuntu's snaps, are self-contained applications, with everything bundled to run in any Linux distro. They may also be configured to run in a sandbox, basically, a container. Flatpak uses a container mechanism called Bubblewrap.

I thought that it might be possible to use Easy Containers, as they are far more efficient to setup a container, compared with the way Flatpak/Bubblewrap does it.

It took many hours chasing dependencies, and I compiled it all, and finally got Flatpak to work. I won't document it here, yet, as need to sort out some details, where I was messing around.

I installed Krita, a KDE app, from this Flatpak repository:

https://flathub.org/home

...and it was a massive download, 359.7MB, actually, it might have been more than that, I went away for awhile, and it downloaded some more stuff.

Finally, ran it:

# flatpak run org.kde.krita

...surprise, it actually worked. However, I saved a file, at "/root", but I haven't got a clue where that "/root" is actually located. Somewhere in /var/lib/flatpak probably, but I couldn't find it. It was there somewhere, was able to open it.

Updates, apparently, are smaller, as it uses a delta difference system.

However, the initial download is too big. I don't think that this is suitable for a phone, as is.

Tags: easy