Little fixes happening daily

Almost everyday I am fixing, tweaking or improving something in OE or EasyOS, and it often doesn't get reported to this blog. Here are a few...

Today I changed the menu back to inline. The current release of EasyOS has a cascading menu, but I have been finding it annoying. When I want to run a particular app or utility, sometimes I have to think where is that in the sub-menus, and then hunt for it. The inline submenus is more convenient.

The CUPS web interface has lots of links to help pages, but they don't work. The daemon 'cupsd' has an inbuilt http server, so take the browser to http://localhost:631 to see the web interface. The "doc root" is /usr/share/doc/cups or /usr/share/cups/doc-root -- the latter in Easy Buster and Easy OE-dunfell. This has file 'index.html' that is what you see at localhost:631

The CUPS http server is especially configured to use CGI modules, located at /usr/lib/cups/cgi-bin. I discovered that /etc/cups/cups-files.conf has a parameter "ServerBin /usr/lib/cups" so that the CUPS daemon knows where the 'cgi-bin' folder is.

It is these CGI modules that are invoked via the 'index.html' file. One of them, 'admin.cgi', provides the administration web interface. Another, 'help.cgi', is broken, as the help files do not exist.

I have fixed the web interface so that links that invoke 'help.cgi' are removed from 'index.html', so no broken links in the web interface.

In OE, I have recompiled everything, with some changes. One thing, I noticed that there are some large 'vulkan' drivers in the build. Vulkan is a replacement for opengl, but I don't know anything about it. So I added DISTRO_FEATURES_remove = "vulkan" to the 'local.conf' file. Some other changes too. 

Tags: easy

Compiling SeaMonkey in OE Dunfell

I need to document this, as I had to jump through some hoops to get the compile underway -- watching it now, cross the fingers that it will get to the end!

I am running EasyOS Dunfell-series, built from packages compiled in OE.

The web browser supported by OE is Epiphany, also known as "Gnome Web". It is based on webkitgtk, and I noticed that Epiphany and all of its dependencies are quite big, maybe even bigger than SeaMonkey. Those deps include webkitgtk and gstreamer.

Of course, with SM, we don't get just a browser, we get a whole suite. For me personally, I use the Composer WYSIWYG HTML editor, just about every day. So, I have a personal interest in bundling SM with every release of EasyOS.

I know that using a WYSIWYG HTML editor is considered "old school", but it is a key component of my shellCMS, that manages my websites.

I know of others who use the SM Mail & News module, as well as the Addressbook module.

Anyway, jumping those hurdles...

SM needs Rust and Cargo to compile, and I discovered that it is pretty simple to install. Precompiled binaries can be installed just by doing this:

...oh bother, compile has just stopped. A video problem, something about "vp8". Anyway, back onto Rust:

https://www.rust-lang.org/tools/install

# curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

installs to /root/.cargo

In the terminal where compiling SM, need this:

# export PATH="/root/.cargo/bin:${PATH}"

Another hurdle is that python3 is no good, python2 is required. So, I downloaded version 2.7.18 source and compiled:

https://www.python.org/downloads/release/python-2718/

# ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var --build=x86_64-pc-linux-gnu --with-ensurepip=yes --with-system-ffi --with-system-expat --enable-unicode=ucs4
# make
# new2dir make install

Information can be found here:

http://www.linuxfromscratch.org/blfs/view/svn/general/python2.html

In the SM source:

# export SHELL=/bin/sh
# export CC=gcc
# export CXX=g++

Then a 'mozconfig' file is required in the source, then start the compile:

# make -f client.mk

Regarding that error reported above, I fixed it by "--disable-webrtc" in mozconfig. WebRTC is in all modern browsers, but do I need it? Hmmm, anyway, the compile is now progressing... and the compile has succeeded.

This is how to install:

# make -f client.mk install

Now to test. Will SM be stable? There have been issues in the past, with using system libraries, such as cairo, pixman and icu. Taking it for a spin, visiting some challenging websites, such as youtube.com... But, failure at startup, with this message:

The application has been updated, but the SQLite library wasn't updated properly and the application cannot run. Please try to launch the application again. If that should still fail, please try reinstalling it, or contact the support of where you got the application from.

Yes, sqlite is another system library that can cause trouble. OK, modifying 'mozconfig' with "--disable-system-sqlite". Note that sqlite has to be compiled with certain configure options, that I did do in OE, but this time it isn't enough to satisfy SM.

This is the 'mozconfig' file:

mk_add_options MOZ_CO_PROJECT=suite
ac_add_options --enable-application=suite
ac_add_options --enable-system-hunspell
ac_add_options --prefix=/usr
ac_add_options --host=x86_64-pc-linux-gnu
ac_add_options --disable-dbus
ac_add_options --disable-accessibility
ac_add_options --with-system-bz2
ac_add_options --disable-updater
ac_add_options --disable-parental-controls
ac_add_options --disable-system-sqlite
ac_add_options --enable-system-cairo
ac_add_options --enable-strip
ac_add_options --with-system-jpeg
ac_add_options --with-system-zlib
ac_add_options --disable-tests
ac_add_options --disable-crashreporter
ac_add_options --with-system-libvpx
ac_add_options --disable-necko-wifi
ac_add_options --without-system-nspr
ac_add_options --without-system-nss
ac_add_options --with-system-icu
ac_add_options --disable-pulseaudio
ac_add_options --enable-alsa
ac_add_options --enable-system-ffi
ac_add_options --with-pthreads
ac_add_options --enable-system-pixman
ac_add_options --disable-debug
ac_add_options --with-system-libevent
ac_add_options --enable-optimize='-O2'
ac_add_options --enable-ffmpeg
ac_add_options --disable-stylo
ac_add_options --disable-webrtc

Compiled again, tested, runs great. Starts remarkably fast. I am using the Composer module to post this blog report.  

Tags: easy

Celluloid compile fail mystery in OE

"Celluloid" is the new name for Gnome-MPV, the media player, frontend GUI for MPV, the latter being a CLI media player (with a primitive pseudo-GUI). Project home page:

https://celluloid-player.github.io/

I have a "meta-quirky" layer for the Dunfell release of OpenEmbedded, see two previous posts in the saga:

https://bkhome.org/news/202009/pango-version-144-is-a-disaster.html

https://bkhome.org/news/202009/openembeddedyocto-dunfell-meta-quirky-first-release.html

All 755 packages compile, except for one, Celluloid. Or rather, it sometimes succeeds, sometimes not. Early this morning, 1.24am,  I started a complete recompile, then went to bed. I had wanted to time the build, but when I got up at 7.30am, I discovered that the build had stopped on an error -- oh no, Celluloid again.

On previous occasions, I had repeated the build of Celluloid, and maybe after a couple of attempts, it would succeed. Like this:

# bitbake -c cleansstate celluloid
# bitbake -c build celluloid

I am getting different error messages each time. This time, it reported a missing termination of a comment, and I discovered that the source file is truncated. Huh!?

How can that even happen?

Here is the recipe, and I have made a comment about the failure, and put in a possible fix:

# Recipe created by recipetool
# recipetool create -o celluloid_0.18.bb https://github.com/celluloid-player/celluloid/archive/v0.18.tar.gz

LICENSE = "GPLv3"
LIC_FILES_CHKSUM = "file://COPYING;md5=8006d9c814277c1bfc4ca22af94b59ee"

SRC_URI = "https://github.com/celluloid-player/celluloid/archive/v${PV}.tar.gz"
SRC_URI[md5sum] = "3fde6852840798ad61b3e39f1513f8d4"
SRC_URI[sha1sum] = "17c7bcc5f34f003140cad407b6a3ecd4d87ce769"
SRC_URI[sha256sum] = "3ce6158097d94786a62de5f26ab2cb71301e9fd0841ede8381e1535489cf0de8"
SRC_URI[sha384sum] = "1f0eafd8b6542178abee06ce68c978ac3a6a9ebd2b479e85136e8db77e2614a4e70aca622ae4a44a4ff383762ec508a2"
SRC_URI[sha512sum] = "acb76bd216100afbe3b083919e122308def1431845955dd418e7b2d57f37fb1353209d1f77f41790d3e01603ee1137fe2c5a57c7bb0ca9b0b13388278cc291f1"

DEPENDS = "libepoxy mpv gtk+3 glib-2.0 appstream-glib glib-2.0-native"

inherit gettext pkgconfig autotools-brokensep

EXTRA_OECONF = ""

# 20200929 why does build sometimes fail?
# today, error "unterminated comment" line 680 in src/mpris/celluloid-mpris-gdbus.c
# and i see 680 is end of the file -- however, file is supposed to have 8355 lines!
# try this:
PARALLEL_MAKE = "-j 1"

HOMEPAGE = "https://celluloid-player.github.io/"
SUMMARY = "GUI frontend for MPV media player"

Tried again, this time success. Now the build of all packages is continuing.

All 755 packages compile successfully, consistently, except for Celluloid. Why just Celluloid? The source package looks like a pretty ordinary autotools-based setup, can't see anything odd in the makefiles.

I put in that PARALLEL_MAKE = "j 1", but I don't see how that would have anything to do with a source file getting truncated. Is it something to do with ext4? A kernel problem?

Bitbake, the program that runs the OE build, uses all available cores, in my case 4, and will be building 4 packages at once, swapping between them continuously. Actually, it will be building many more than 4, but only 4 at the same time, rotating through them, like a 4-package moving window. 

Tags: oe

Pango version 1.44+ is a disaster

A couple of days ago, I posted about some fonts rendering as rectangular boxes in EasyOS built from packages compiled in OpenEmbedded:

https://bkhome.org/news/202009/first-bootup-easyos-dunfell-series.html

After a bit more research, I discovered that the culprit is Pango, version 1.44 onwards. Support for bitmap and Type1 fonts has been dropped.

I could live with that, I think, however, that problem turned out to be the tip of the iceberg. I discovered this in ROX-Filer:

...what is wrong with that? Hyphens are inserted at line-breaks. For example, file "freememapplet_tray" is showing as "freemem-applet-tray". How misleading!!!!

It seems that individual applications can turn hyphenation off, so ROX-Filer would need to be patched. There does not seem to be global configuration setting to turn it off.

The problems don't end there. Pango 1.44+ has also stuffed up rendering of DejaVu fonts. With DejaVu Mono, a character with underscore will not show the underscore. In rxvt-unicode, the horizontal spacing between characters is huge.

Here are some relevant links:

https://github.com/linuxmint/nemo/issues/2214

https://gitlab.gnome.org/GNOME/pango/-/issues/386

https://gitlab.gnome.org/GNOME/pango/-/issues/392

https://blogs.gnome.org/mclasen/2019/07/27/more-text-rendering-updates/

The version of Pango in OE Dunfell is 1.44.7, in Debian Buster it is 1.42.4.

Looking at the releases of OE/Yocto, going back in time from left to right:

Dunfell Zeus Warrior Thud Sumo Rocko Pyro

Pyro was released May 2017. Warrior has Pango 1.42.4, so I might use that recipe and do a complete recompile.  I hate to be going backwards, but it won't be the first time that I have done this (with other packages). 

Tags: easy

OpenEmbedded/Yocto Dunfell meta-quirky first release

A few years ago, I created "oe-qky-src", a port of OpenEmbedded cross-compiler build system, with recipes to build nearly all the packages required for woofQ -- the latter is the build system for Quirky Linux or EasyOS. WoofQ is a derivative of Woof2, as is woof-CE that is used to build the latest official releases of Puppy Linux.

"oe-qky-src" is in a git repository:

https://github.com/bkauler/oe-qky-src

...that has my "meta-quirky" layer for OE. It is based on the "Pyro" release of OE/Yocto.

"oe-qky-src" is getting a bit "long in the tooth", so I decided to do a complete update, based on the latest release of OE/Yocto, named "Dunfell".

A couple of weeks of very intense work later, and we now have the first release. I have decided not to put this onto a git repository, instead it is just a tarball:

http://distro.ibiblio.org/easyos/project/oe/dunfell/dunfell-20200927.tar.gz

Download it, expand it, and type in a few commands, and several hours later you have compiled 755 packages! Detailed instructions here:

http://distro.ibiblio.org/easyos/project/oe/dunfell/readme.txt

...the readme is also inside the tarball.

One point: it does include 'pulseaudio'. With "oe-qky-src" there was only 'alsa', but this time I decided to include 'pulseaudio', despite reservations, as it is very difficult to implement bluetooth audio without 'pulseaudio'. I will build EasyOS from these packages and see how it goes.

If I change my mind, it is easy to remove 'pulseaudio' and recompile everything.

Those compiled packages can be imported into any woof* derivative. However, the '0pre-oe' and '0pre-oe-add' scripts are required to perform the import, and they are currently only in woofQ -- furthermore, I fixed some bugs in those scripts yesterday, and have not yet uploaded the latest woofQ tarball. Intend to do so soon.

I think, in theory, if you are a Puppy Linux developer using woof-CE, you should be able to use those scripts, but there are some differences from woofQ, so you would need to look through the scripts and make sure they are woof-CE-compatible.

Oh, another thing, Woof* has a directory 'packages-templates' and the one in woofQ has diverged somewhat from that in woof-CE. This might affect the functioning of some packages when a Puppy build is done, that is when run the '2createpackages' script.

Here is the package list:

http://distro.ibiblio.org/easyos/project/oe/dunfell/pkglist-20200927.txt

Anyway, lots of fun. Certainly a very niche audience, as although 755 packages seems like a lot, it is tiny compared with the package repositories of the major distributions such as Debian -- note though, that 755 is "whole" packages, not split-up ones -- Debian will typically split an original package into 2 - 4 DEBs, so that 755 would become >1500 packages! 

Tags: oe

First bootup EasyOS Dunfell-series

So far, have built 736 packages in OE. That's whole packages, not split up ones. Previous post about this project to compile everything from source in the latest release of OpenEmbedded/Yocto:

https://bkhome.org/news/202009/more-patches-for-an-old-package-in-oe.html

...actually, that "horrible hack" did solve the compile fail, but glibc has probably removed the infrastructure to actually make that code work at runtime. I probably should take a look at the latest version of xlockmore (says he, writing yet another reminder on all the bits of paper scattered around on his desk).

So, I did a complete build, and also imported the compiled packages into woofQ, and then built EasyOS ...and hey, it worked, got a desktop!

It seemed perfect, but then started to notice few things wrong...

Opened a text file in Geany text editor, and all characters in the file displayed as rectangular boxes only. Clicked on the "console" icon on the desktop to start the Sakura terminal emulator, same problem.

After considerable bafflement, I realised that what is failing is the monospace Type1 font (Nimbus Mono L). It does display OK in the urxvt terminal emulator, which is an X11 app. It is GTK apps that are the problem, both gtk2 and gtk3.

A very brief online search has suggested that the harfbuzz package is the cause of this problem. I won't try to track the cause down any further, rather, will remove the Type1 font.

Why am I still using a Type1 font anyway? This is an early Postscript font format, and, I don't know the precise details, but it is a very restrictive format compared with TrueType. That to-do has been added on one of those bits of paper.

The build has the Epiphany web browser. Seems OK. As that is the one officially supported by OE, might stay with that, and provide SeaMonkey as an SFS. Chromium as an SFS also. The OE build does not have LibreOffice, so might just have that as an SFS also.

Gotta say, this is an enjoyable exercise! 

Tags: easy

More patches for an old package in OE

I am continuing to compile old packages, that are still good, and included in releases of EasyOS. Some notes have been appended to the previous blog post:

https://bkhome.org/news/202009/progress-compiling-in-oe-dunfell-release.html

The package 'xlockmore' is an example of very "long in the tooth". It compiled OK in the Pyro-OE, but not in Thud-OE. Nor in the latest OE, Dunfell-OE. There have been patches in the past to get it to compile. This time, the error message is that "DOMAIN" is undefined.

It turns out that the problem is with glibc 2.27+, as explained here:

https://github.com/flang-compiler/flang/issues/475

...jamborm has done what he calls a "horrible hack". Yes, but it works. I appended his code onto the end of any include file, xlock/xlock.h, in the recipe for xlockmore, like this:

    echo '
#ifndef LIBM_ERRNO_AMD_H_INCLUDED
#define LIBM_ERRNO_AMD_H_INCLUDED 1
 
#ifndef DOMAIN
struct exception {
  int    type;      /* Exception type */
  char  *name;      /* Name of function causing exception */
  double arg1;      /* 1st argument to function */
  double arg2;      /* 2nd argument to function */
  double retval;    /* Function return value */
};

#define DOMAIN         1
#define SING           2
#define OVERFLOW       3
#define UNDERFLOW      4
#define TLOSS          5
#define PLOSS          6
#endif

#endif /* LIBM_ERRNO_AMD_H_INCLUDED */' >> xlock.h

Yay, xlockmore is still alive!

Actually, there is a later version of xlockmore that what I am using. I have 5.31, latest is 5.65. I continue to use the old one as it has many hacks to make it very small. Ideally, I should apply those "smallness" hacks to the latest version. Here is the xlockmore project page:

http://sillycycle.com/xlockmore.html   

Tags: oe

Progress compiling in OE Dunfell release

I posted about commencing this a few days ago:

https://bkhome.org/news/202009/buildroot-and-openembedded-revisited.html

hey, how time goes when we are having fun! Though, I don't know if "fun" is the right word. Endless hours, into the wee small hours of the morning, fixing packages that won't compile.

As of this morning, have compiled 675 source packages. There are some failures, that did compile in earlier OE releases (Pyro and Thud). This is them:

binutils-static busybox-initramfs-static coreutils-static dictd-client e2fsprogs-static gdk-pixbuf0 gdmap gettext0-static gftp glib1 glipper-lite gnome-mpv gnupg1-static gpart gtk1 gtkhash gtklp gwhere idump-static ktsuss leafpad libcap-static make-static mped-static ndiswrapper ndiswrapper-exe netpbm nnn-static noice-static notecase patchutils pekwm pup-tools rman rubix xarchive xdialog xine-ui xsane xsoldier

The "-static" ones are only meant for when linking with musl, which haven't tried yet. Currently linking with glibc.

Changes in the build infrastructure are behind these failures. Glibc is 2.31, gcc is 9.3.0. There is only python3, no python2, which might be an issue with some source packages, that have python scripts in the build configuration.

Anyway, 41 failures and 675 successes is quite good. Some of those failures don't matter, I will try to fix those that do.

'pup-tools' is one failure that currently has me stumped. I have reported the problem to the BaCon forum:

https://basic-converter.proboards.com/thread/1183/compile-fail-openembedded-dunfell

Some of those failures are due to "-Werror" gcc flag. OE is very sophisticated and complex, and sometimes trying to configure something is like looking for a needle in a haystack. I think perhaps I can override by "CFLAGS += "-Wno-error"" in the package build recipe.

EasyOS Buster series, although built with Debian Buster (10.5) DEBs, also uses many packages that were compiled in the Pyro series of OE. No problem with that, they work, but thinking it is time to update everything in the latest OE.

EDIT 20200922:
I discovered that OE imposes "-Werror=format-security" on all package compiles. This is a new feature of OE, not in the Pyro and Thud releases, and resulted in several packages failing. I turned it off in each recipe, with this:

  CFLAGS_append = " -Wno-error=format-security "

Another issue has been tighter QA (Quality Assessment) checking after compiling and installing. In each recipe, if I don't know how to fix the QA failure, or haven't invested the time to do so, the offending QA can be disabled in each recipe, for example:

ERROR_QA_remove = "file-rdeps"

...that changes it to a warning rather than a failure. Continuing to have "fun"! 

Tags: easy