site  contact  subhomenews

Revisiting pup_event

February 15, 2018 — BarryK


Due to dissatisfaction with available service managers, I am thinking of writing my own, as an extension of "pup_event".

In June 2013, I wrote pup_event_frontend_d.bac, BaCon code. /usr/local/pup_event has this utility, plus some more scripts. The main objective was to manage hotplugging, for example to detect when a USB drive is plugged in or removed.

I announced pup_event_frontend_d.bac here:

http://bkhome.org/archive/blog2/201306/new-pupeventfrontend.html

It was very creative, totally different from any other Linux distro. Not long after that, I retired from the Puppy development "front line".

Actually, the concept of pup_event started earlier, circa 2008, see this post:

http://bkhome.org/archive/blog2/200805/drive-icons-on-desktop.html

...the post has links to some documentation, which has moved, now here:

http://bkhome.org/archive/puppylinux/technical/module-loading.htm

http://bkhome.org/archive/puppylinux/technical/event-management.htm

Fast forward to today, I wondered what has happened to pup_event in woof-CE. There has been a lot done to it, mostly by "wdlkmpx". For example, he rewrote pup_event_frontend_d in C:

https://github.com/puppylinux-woof-CE/woof-CE/blob/4439fcc63d3eedb839fa625bb8f983d3b2461e3d/woof-code/rootfs-skeleton/usr/local/pup_event/pup_event_frontend_d.c

...that page is an earlier version. Since then, it got moved, and there have been more commits.

For myself, I like BaCon, and will probably stay with it.

I think that pup_event can be easily extended to be a service manager. My tentative thinking is to remove /usr/local/pup_event_frontend_d (which is launched from /root/.xinitrc when X starts), replace with pup_event_backend (that will be launched from /etc/rc.d/rc.sysinit).

This new pup_event_backend will do the same as pup_event_frontend_d, but a couple of extra things, including service management.

There are a couple of old scripts in /sbin:

pup_event_backend_firmware
pup_event_backend_modprobe
pup_event_backend_modprobe_protect

These were developed back around 2008 - 2012, in collaboration with "rerwin" (Richard Erwin).

There are still in woof-CE, untouched since being imported from Woof2, four years ago:

https://github.com/puppylinux-woof-CE/woof-CE/tree/testing/woof-code/rootfs-skeleton/sbin

I don't know if we need them anymore. Perhaps not. They were developed in the days of 'module-init-tools', before 'kmod', when the kernel did not have automatic /dev node creation and automatic firmware loading.

That early backend stuff is described in the above link, here it is again:

http://bkhome.org/archive/puppylinux/technical/module-loading.htm 

Tags: easy, quirky

Experimenting with Busybox Runit

February 14, 2018 — BarryK

Refer to these posts yesterday:

http://bkhome.org/news/201802/busybox-1251-runit-applets.html

http://bkhome.org/news/201802/thinking-about-service-managers.html

I have been wondering more about why I find daemontools and derivatives to be unsatisfactory. I think that the developers are system adminstrators, and have a particular mindset. They are developing for systems that have fairly fixed software and interfaces, not for generic desktop systems.

Also, I have struggled with the documentation. Regarding Runit, one person commented:

"Runits documentation is so bad that I had to learn it by experimentation, even though I know its ancestor daemontools inside and out."

I was also hampered by the documentation of the Busybox implementation of Runit. "sv --help" showed the commandline options, which lead me to believe it was only a subset of the full 'sv' utility. Options such as "check" and "start" were missing. Until I looked at the Busybox 'sv.c' source, and saw that those options are supported.

Anyway, my simple requirement for some services to be dependent on a network up. It was not explained in the official docs, found some info after much hunting...

I created a service named "network", by creating a folder '/etc/sv/network'.
Inside, created two scripts, 'run' and 'check'. Contents of 'run':

#!/bin/ash
#sleep forever...
sleep 9999d

Contents of 'check':

#!/bin/ash
exec 2>&1
#exit with 0 if network is up (exclude 'lo')...
UP_IFS="$(grep -l '^0$' /sys/class/net/[^l]*/dormant | cut -f 5 -d '/')"
#...ex: /sys/class/net/eth0/dormant, cut 'eth0'
if [ "$UP_IFS" ];then
 exit 0
else
 exit 1
fi

Now, if another service specifies "network" as a dependency, the 'network/check' script will be run, and if returns "0" then yep it is running.

So, I created '/etc/sv/test_net', with script 'run':

#!/bin/ash
exec 2>&1
sv -w10 check network
if [ $? -eq 0 ];then
 echo 'up' >> /tmp/zz-net
 sleep 99999d
else
 echo 'down' >> /tmp/zz-net
 exit 1
fi

...this is just a dummy script. A real service might be to start a daemon instead of that "sleep 99999d".

The key line in that script is "sv -w10 check network". This will run the 'network/check' script. If 'check' returns non-0, the above 'run' will exit with non-0. The Runit supervising utility 'runsvdir' will wait one second, then try again, that is, rerun 'test_net/run', and will keep at it until the network is up.

This is the official way to do it, and at this stage I was thinking "Oh my goodness gracious me, surely not!" (or words to that effect). But yep, that's it, a polling loop. Furthermore, if several services have the same "network" dependency, they will all independently be calling 'network/check'.

Then I recalled something that I had read in the 's6' docs, at https://skarnet.org/software/s6/why.html:

"Neither System V init, daemontools, runit or perp provides any hooks to wait for a service to go up or down. runit provides a waiting mechanism, but it's based on polling, and the ./check script has to be manually written for every service."

It is a pity that I am not happy with Runit, as the Busybox implementation is tiny. I reported yesterday that I recompiled Busybox statically, adding the Runit applets and three extra network applets. The size of the stripped binary grew from 933KB to only 965KB.

Tags: easy, quirky

Ninit: Minit on steroids

February 13, 2018 — BarryK

Ah, maybe I have found "the one"!

As I lamented in the previous blog post, I found the service managers derived from daemontools to be very restricted:

http://bkhome.org/news/201802/thinking-about-service-managers.html

Minit is one of them:

https://www.fefe.de/minit/

Ninit was forked from Minit, by Nikola Vladov, and considerable thought has gone into extending the usefulness. Dependency options are now looking very good.

Minit and Ninit are both intended to be compiled statically with dietlibc, and I have done that. Host is Quirky Xerus64 8.4, which has dietlibc in the "devx" PET.

A quick note on how to compile Ninit. I edited the 'Makefile', set "DIET=/usr/bin/diet", and had to create a symlink:

# ln -s lib /usr/lib/diet/lib-x86_64
# make
# make install

Back in the mists of time (2012 actually), I created a PET for Minit:

http://distro.ibiblio.org/puppylinux/pet_packages-common/minit-0.10-i486.pet

http://distro.ibiblio.org/puppylinux/pet_packages-common/minit_DOC-0.10-i486.pet

It has etc/minit, which which is a quick hack to make Minit work in an unchanged Puppy. It sets up a replacement for /etc/inittab and runs the usual /etc/rc.d/rc.sysinit.

I copied etc/minit/* to /etc/ninit/. Then inserted "init=/sbin/ninit" into the kernel boot parameters, then rebooted. Hey, it's working, back at the usual desktop!

This sets up the infrastructure for service management. One thing that I am keen to do is implement services that are dependent on a network connection.

Now, about Ninit. It is a dead project, however has been preserved here:

https://gitlab.com/notklaatu/ninit

If you download this, it is version 0.14.

More info is here:

http://without-systemd.org/wiki/index.php/Ninit_instructions 

Tags: easy, quirky

Thinking about service managers

February 13, 2018 — BarryK

These are just "out loud" thoughts...

There are many service managers that derive from, or were inspired by, a package called 'daemontools'. Runit is one of them.

I have been sifting so many of the service managers, with maybe init replacement, but have always been left discontented.

Trying to pin down why this discontent...

The emphasis on automatic restarting of daemons. Why? If a daemon crashes for some reason, I don't want it to be automatically restarted.

The lack of support for one-shot services.

Incomplete dependency management. For example, I want a service to wait until there is a network up. Nothing else to be held up. I know that I can do it by examining /sys/class/net/dormant, but fitting this in as a dependency is at best a klutz, as far as I can see, in the service managers that I have looked at.

There are lots of init replacements with service management, such as minit, ninit, cinit, myinit. Or, service management with optional init replacement, such as runit and s6. Then there are some that are service managers only, such as serel.

I got excited that Runit applets are in Busybox, until I realised how limited it is. Not the Busybox implementation, the fundamental concept of daemontools. Unless my understanding has not fully grasped the capabilities, which is possible.

Here is another...

OpenRC

OpenRC is developed for Gentoo, and adopted by some other distros. It is a service manager, though apparently can become an init-replacement. What is particularly interesting is that thought has gone into using it with Busybox and Busybox's init:

https://blog.flameeyes.eu/2012/03/using-busybox-with-openrc/

Download source:

https://github.com/OpenRC/openrc/releases

Docs;

https://www.calculate-linux.org/main/en/openrc_manuals

https://wiki.archlinux.org/index.php/OpenRC

https://wiki.gentoo.org/wiki/OpenRC

However, I was unable to compile it statically, so have taken it no further.

Will keep researching...

Tags: easy, quirky

Busybox 1.25.1 Runit applets

February 13, 2018 — BarryK

There is a problem with all the puplets, including Easy and Quirky. Well, unless a developer has tackled in it their puplet.

The problem is the order of execution of service scripts at startup, for example, those in /etc/init.d -- we execute those in a fixed order, however, that is barely adequate.

It may be that a certain service script should only run after there is a network connection, for example. Or sound is working, or X is running, and so on.

I have been looking at various solutions, and liked the look of Runit. Homepage:

http://smarden.org/runit/

Then, surprise, surprise, I found that the Runit utilities are in Busybox. They are:

chpst, setuidgid, envuidgid, envdir, softlimit, runsv, runsvdir, sv, svlogd

Using Landley's Aboriginal Linux, the older chrootable filesystem with uClibc (he has shifted to musl, unfortunately), I have compiled Busybox statically with those Runit applets enabled. I also enabled these network applets:

inetd, udpsvd, tcpsvd

Here is the PET:

http://distro.ibiblio.org/quirky/quirky6/amd64/packages/pet_packages-xerus/busybox-1.25.1-patched-2-static-amd64.pet

...however, don't install it, as it will replace all the "full" utilities with symlinks to busybox. For example, 'sort' and 'date'. If you really want to use it, open it up with the 'pet2dir' utility, then copy the 'busybox' executable, then the above applet symlinks.

I have also compiled 's6' statically, and there are PETs, however, it is a solution that will add megabytes to the build. So, at this stage I am favouring Busybox Runit. To actually use it, is next on the agenda.

Here is an interesting read:

http://www.mikeperham.com/2014/07/07/use-runit/ 

Tags: easy, quirky

woofQ uploaded, 2018-02-11

February 11, 2018 — BarryK

This is the latest woofQ, as used to build Quirky Xerus 8.4.

Uploaded as a tarball here:

http://distro.ibiblio.org/easyos/project/woofq/

I also updated the woofQ Bones repository. Read about Bones here:

http://bkhome.org/archive/bones/

And woofQ and Bones here:

http://bkhome.org/news/201501/woofq-hosted-by-bones.html 

Tags: easy, quirky

EasyShare file and printer sharing, v0.3

February 02, 2018 — BarryK

At first there was QuickSamba, then with EasyOS 0.7 the name changed to EasyShare.

Now, EasyShare has been almost completely rewritten, with a more dynamically-changing GUI and support for SSHFS.

There is a tutorial, which I have also had to rewrite:

http://bkhome.org/easyshare/easyshare-super-easy-network-file-and-printer-sharing.html

This is an early release, and there are some "peculiarities" and limitations. EasyShare 0.3 will be in the next release of EasyOS, 0.7.1 or later.

Tags: easy

Busybox 1.27.2 compiled statically

January 25, 2018 — BarryK

The last time that i compiled Busybox statically, it was version 1.25.1, using Landley's Aboriginal Linux, chrootable root-filesystem. That was late 2016, and i have been using that up until now.

I wanted to enable some more applets in Busybox, 'nsenter', 'dnsd' and 'inetd', so went through the exercise again.

I downloaded the latest stable source, version 2017.02.09, of Buildroot, from here:

https://buildroot.org/download.html

This defaults to using Busybox 1.27.2. I added my own 'busybox.config', and jamesbond's 'guess_fstype' applet patch.

Then ran:

# make menuconfig
# make uclibc-menuconfig
# make busybox-menuconfig
# make

I didn't change much. Chose target of x86_64 nocona CPU, and for uClibc turned on 'wchar' and 'locale' support.

There were a couple of fails. I had to replace /usr/include/limits.h in the host system (EasyOS Pyro64 0.7) with limits.h from Xerus64.

Then Busybox compile failed and I had to disable 'nfs' support for the 'mount' applet. I can live with that. It is probably something lacking in the uClibc configure options.

Anyway, got there. If any pup builders want the latest Busybox, statically compiled, with lots of applets, here is the PET (728KB):

http://distro.ibiblio.org/easyos/amd64/packages/pet/pet_packages-pyro/busybox-1.27.2-patched-static-br-amd64.pet

EDIT 2018-02-09:

I reverted to the 1.25.1 busybox PET. Testing the above, something seemed not quite right (got a hang at bootup). When building with "make menuconfig", using the .config from 1.25.1, an incredible number of configure options were flagged as no longer valid, and maybe something has changed to cause the problem. I might have to advance the version number more gradually, and maybe go back to Landley's Aboriginal Linux as a precaution.

Another thing. The motivation to upgrade was to enable some more applets, including 'dnsd' (domain name server), however, it (dnsd) doesn't work. Really weird, but I did a search with google and it seems that "nobody" actually uses it. I have successfully used 'bind' and 'dnsmasq', so I think that I know what I am doing, regarding dns servers. The busybox dnsd does not respond to the port.

Tags: easy, quirky