site  contact  subhomenews

Managing changed SFS layers at bootup

September 19, 2017 — BarryK

Easy OS, like Puppy, runs on a layered filesystem, with one or more read-only (ro) SFS files in lower layers, and a read-write (rw) layer at the top.

The interactions of the files and folders in these layers can cause trouble. Puppy has some quite complicated code, in the init script in the initrd, and in /etc/rc.d/rc.update, to examine the layers and make sure that nothing "important" underneath is blocked by a higher file/folder or whiteout file.

The decisions made are not perfect though.

Easy 0.4 and earlier do no processing of the layers, they are just layered as-is. That is not satisfactory either.

The layers can change either by adding or removing a SFS file, or at a version upgrade. The rw layer is the .session folder, below will be session.sfs, then maybe extra SFS files such as devx-*.sfs, then q.sfs at the bottom.

File session.sfs is the accumulated sessions from previous versions, compressed and frozen as read-only.

I thought of handling the whiteout files in a radically different way: just delete all of them. That is, delete all of them in the .session folder.

At a version upgrade, this ensures that no whiteout files get frozen into session.sfs, and in a changed SFS layer arrangement, nothing from below will be hidden or renamed.

At first, this might seem too radical. However, all changes in the rw layer will not be undone. For example, you install a DEB package, then later uninstall it. This happens entirely in the rw layer.

Say that you install a package into /opt in the rw layer, then later delete /opt. There will then be a whiteout file hiding any /opt folders in lower layers. By removing the whiteout, the lower /opt will spring back into existence on top. Which is good.

Deleting all the whiteout files in the .session folder is now done in the init script in the initrd, a one-liner:

find /mnt/${WKG_DEV}/${WKG_DIR}.session -type f -name '.wh.*' -delete 2>/dev/null

But, this is only done when the layers change or at a version upgrade.

I know that this seems one step too far, and feedback from other pup developers is welcome. The Easy OS forum thread is here:

http://murga-linux.com/puppy/viewtopic.php?t=109958&start=150

Actually, I think this will work OK because only the top layer is rw. In Puppy, where the next-down may also be rw, it gets more complicated.

Following the reasoning presented above, take one more step: why not do away with whiteout files entirely?

Later in the init script, the layered filesystem is setup, using aufs, then there is a switch_root to the working partition, then /etc/rc.d/rc.sysinit will run, which will call rc.update.

Next, I have to put some code into rc.update, to process the changed layers some more. For example, the JWM menu might need to be reconstructed.

Tags: easy

Linux kernel 4.13.2 compiled

September 18, 2017 — BarryK

Yay, back onto developing Easy OS!

I have compiled the 4.13.2 kernel. Easy 0.4 has 4.11.12. Essentially the same configuration, a few more drivers enabled. Also, these changes:

CONFIG_PATA_LEGACY
This was turned off in 4.11.12, as it caused very slow bootup on the Alpha Litebook
CONFIG_USB_PCI (new)
wasn't sure about this. It defaults to "yes" so left it at that
CONFIG_MMC_SDHCI_XENON (new)
A driver, and I chose "yes", builtin, rather than as a module

Regarding CONFIG_PATA_LEGACY, after I release Easy 0.5, if you experience slow bootup on your machine, let me know.

The kernel source, patches and build scripts, are here:

http://distro.ibiblio.org/easyos/source/kernel/4.13.2/

The PET is here (42.4MB):

http://distro.ibiblio.org/easyos/amd64/packages/pet/pet_packages-xerus/linux_kernel-4.13.2-aufs-xerus64.pet

The kernel source is available as a SFS file (152.9MB):

http://distro.ibiblio.org/easyos/amd64/packages/sfs/xerus/kernel_src-4.13.2-patched.sfs

...it is big because the SFS is only gzipped.

I started thinking about updating the kernel, as I was considering trying the overlay filesystem again, instead of aufs. However, after reading recent posts in various mail-lists, decided that overlay is still not working properly.  Overlay has some config options, CONFIG_OVERLAY_FS_REDIRECT_DIR and CONFIG_OVERLAY_FS_INDEX, that promise to fix some things, however, break others. Whereas aufs, on the otherhand, "just works".

So, this 4.13.2 build is staying with aufs.

Tags: easy

Easy docs moved to bkhome.org

September 18, 2017 — BarryK

I had registered a new domain, easyos.info, and put Easy OS documentation on it, and also started a blog.

However, reconsidered, and consolidated the blog into just one, where you are reading now. That was a couple of weeks ago. Now, have done the same thing to the Easy docs. They are now here:

http://bkhome.org/easy

I decided that it is simpler to keep everything under the one umbrella domain name. Besides, I don't really see Easy as becoming a mainstream Linux distribution -- but then, you never know, and if that does happen, maybe then Easy can have its own home.

The last few days, I have been refining shellCMS, and now my website is pretty much as I want it. So, the intention now is to move back to developing Easy.

Tags: easy

How Easy works, part 2

September 06, 2017 — BarryK

Part-1 is more for those interested in the underlying details, especially if you have a strong knowledge of shell scripting. Part-1 is here:

http://bkhome.org/easy/how-easy-works.html

Part-2 is more for the user, posted here:

http://bkhome.org/easy/how-easy-works-part-2.html

Tags: easy, linux

How Easy works, part 1

September 05, 2017 — BarryK

I have imported part of "How Easy works" into the bkhome.org documentation CMS:

http://bkhome.org/easy/how-easy-works.html

This is for anyone to read, but is of particular interest to someone who wants to understand in depth. It has the content of the init script, color-syntax highlighted.

The Geany text editor has a html export plugin, which produced the nice colored code, that I just pasted into the post.

For someone who is not so much interested in the code in init, the post is still very useful, just jump over the code parts.

The post is part-1, and deals with the bootup, until a switch_root occurs, and Easy-proper is running.

Part-2 will be after Easy is up and running, so is of much more interest to the user. It is expected to be posted soon.


Tags: easy

Easy OS 0.4 pre-alpha released

August 28, 2017 — BarryK

I have uploaded Easy OS version 0.4. I keep going off on a tangent -- the latest excursion is fun with bashblog. But, there are some guys keen to try the latest Easy, so this is for you.

Download:
http://distro.ibiblio.org/easyos/amd64/releases/xerus/0.4/

Unresolved issues:
That problem reported earlier, about SNS not automatically connecting at bootup.

The desktop icons layout has changed, and you will notice two of them have symlink symbols. Need to remove those.

The is now "apps" icon on desktop, launches PupApps, but this is not configured for the apps in Easy, such as Libreoffice.

The container mechanism needs more work. I did fix some bugs since 0.3, however, want to rethink some of it, such as improving startup time. Also, the security parts of it need a lot of work.

New features:
Another new desktop icon "sfsget", you can use this to download the "devx" sfs.

Unlike previous Easy 0.3, this one is built from Ubuntu Xenial Xerus 16.04.3 DEBs.

The build includes the markdown plugin, and some other plugins, for Geany text editor.

Installation:
USB flash drive image only. There is documentation on how to write it to a flash drive:
http://distro.ibiblio.org/quirky/quirky6/amd64/releases/xerus-8.3/howto-install.htm

Frugal installation is documented here:
http://barryk.org/easy/easy-frugal-install.htm

Docs:
To get some idea what Easy OS is all about:
http://barryk.org/easy/how-easy-works.htm

Forum:
http://murga-linux.com/puppy/viewtopic.php?p=965700#965700

Tags: easy, linux

New and upgraded apps for Easy 0.4

August 26, 2017 — BarryK

Easy 0.4 is due out "soon", although I keep finding more things to fix or improve.
Anyway, hopefully early next week.

I have upgraded some packages:

momanager 20170815 (L18L)
Pfind 6.3 (zigbert)
Pmusic 5.4.2 (zigbert)
YASSM 4.0 (rcrsn51)

There is also a new package, PupApps 2.2, by radky.
This app has become a "first class citizen" (borrowing the term from Ubuntu developers), with an icon labeled "apps" on the desktop.

Now radky has two apps featured on the desktop. There is also icon "setup", which launches PupControl.

Tags: easy

Translating help files in Easy

August 19, 2017 — BarryK
I posted recently about an overhaul of the help files in Easy:
http://bkhome.org/news/201708/easy-help-overhauled.html

Then I considered moving from html to markdown:
http://bkhome.org/news/201708/markdown-to-html.html

But no, the richness of html is far superior to markdown, and mdview only supports a basic set of tags.

Folder /usr/share/doc/easy has help.sh, home.sh and welcome.sh. When run, these generate help_lang.htm, home_lang.htm and welcome_lang.htm, where "lang" is the two-character code for the language, ex "de" for German.

Symlinks help.htm, home.htm and welcome.htm are also created.

I am using momanager to create .mo files for help.sh, home.sh and welcome.sh. Just a note: currently there is a bug with latest momanager, so I am using an older version. As a precaution, I will leave the older version in woofQ, at /usr/sbin/momanager0.

TEXTDOMAIN is "easyhelp", and so far I have done fr, de and da, using google online translator:
https://translate.google.com.au/

Google asked for $68 to translate an entire .po file, but it can be done for free, more slowly, by pasting the text into the left-hand English side. But, the easyhelp.po file has to be done in two parts.

Obviously, the google translations are going to be less than optimum. I will upload 'easyos.po' and invite anyone interested, to insert translations. Will post a comment when that is done.

Read more...