site  contact  subhomenews

Extra patches applied to bash in OE

December 27, 2020 — BarryK

I reported about a nasty bug in bash, running in EasyOS Dunfell 0.101 and earlier:

https://bkhome.org/news/202012/easyos-dunfell-update-weird-error.html

I recompiled bash in running Easy Dunfell, with patches from Debian, and created a PET, then released Easy 0.102.

However, it needs to be fixed in OpenEmbedded. The bash recipe in OE applies patches 001 to 016, however, I see from here, that 017 and 018 are available:

https://ftp.gnu.org/gnu/bash/bash-5.0-patches/

And it looks like 017 fixes the bug. So, in my fork of OE, I have created meta-quirky/recipes-extended/bash/bash_5.0.bbappend:

# BK 20201227 bash compiled in OE has a serious bug:
# https://bkhome.org/news/202012/easyos-dunfell-update-weird-error.html
# it looks like patch #17 will fix it. see list of patches here:
# https://ftp.gnu.org/gnu/bash/bash-5.0-patches/

PR = "r1"

SRC_URI += " \
${GNU_MIRROR}/bash/bash-${PV}-patches/bash50-017;apply=yes;striplevel=0;name=patch017 \
${GNU_MIRROR}/bash/bash-${PV}-patches/bash50-018;apply=yes;striplevel=0;name=patch018"

SRC_URI[patch017.sha256sum] = "4cf3b9fafb8a66d411dd5fc9120032533a4012df1dc6ee024c7833373e2ddc31"
SRC_URI[patch018.sha256sum] = "7c314e375a105a6642e8ed44f3808b9def89d15f7492fe2029a21ba9c0de81d3"

I have recompiled bash in OE, and will use this in the next release of Easy Dunfell, instead of the PET. 

Tags: oe

Fixes for OpenEmbedded Dunfell build

December 19, 2020 — BarryK

I have a fork of OpenEmbedded (OE), Dunfell release, to build most of the packages required for EasyOS.

Note that EasyOS Buster-series, currently at version 2.5.3, is built with Debian Buster 10.7 DEBs, but also does use some PETs that were compiled in OpenEmbedded Pyro. There is also an experimental EasyOS Dunfell-series built entirely from the packages compiled in OpenEmbedded Dunfell.

OE Dunfell is not in a git repository, only uploaded as tarballs, here:

http://distro.ibiblio.org/easyos/project/oe/dunfell/

Jon (scsijon in the forum) has been testing the tarball, trying to do a complete compile on his computer, but has hit failures:

https://easyos.org/forum/showthread.php?tid=287

So, I tried a build, on a host system running EasyOS 2.5.4 (not yet released), and also hit failures. Then I tried EasyOS Pyro64 1.3, also failures. Hmmm, interesting, considering I had done the build before, many times, and it ran right through.

I don't know why OE is failing to compile or install some packages, that succeeded before. Very odd. Anyway, went through, fixed them, and now have a tarball that works for me, on both EasyOS Buster 2.5.4 and Pyro 1.3:

http://distro.ibiblio.org/easyos/project/oe/dunfell/dunfell-20201219.tar.gz

I have reported to the forum, and Jon will give it a go.

OE Dunfell is using the 5.4 kernel, however, I want to base EasyOS Dunfell-series on the 5.10 kernel, so have now attempted to bump the kernel, and now doing a compile of OE Dunfell.

OE is very complicated, hope that I have got the kernel-bump right. We shall see. Have applied the CAP_SYS_MOUNT patch also. 

Tags: oe

OE gcc target default commandline options

October 15, 2020 — BarryK

I uploaded Easy Dunfell 0.91 yesterday:

https://bkhome.org/news/202010/easyos-dunfell-series-091-alpha-release.html

Which was a disaster, as most people couldn't get it to boot. I had feedback from Rodney, Feodor and David, and on the forum, banned and average1. I received an email from Alfons confirming that the USB-stick booted on a PC with Intel i7 CPU, not earlier CPUs.

The problem is a bug in OE. I set the target as an Intel Core2 CPU, which is one of the earliest x86 64bit CPUs. That's fine, all of the applications were compiled for a Core2 CPU. However, after building EasyOS, and compiling the Linux kernel, SeaMonkey and Libreoffice in a running EasyOS, then building Easy 0.91 with those, that's when disaster struck.

The gcc compiler in a running EasyOS does not default to Core2, it compiles using extra instructions that are in my i3 CPU but not in earlier CPUs.

On the Internet I found a simple way to discover the default options that gcc uses, and did this in Easy 0.91:

# echo "" | gcc -v -E - 2>&1 | grep cc1
/usr/libexec/gcc/x86_64-poky-linux/9.3.0/cc1 -E -quiet -v - -march=core2 -mmmx -mno-3dnow \
-msse -msse2 -msse3 -mssse3 -mno-sse4a -mcx16 -msahf -mno-movbe -mno-aes -mno-sha -mno-pclmul \
-mno-popcnt -mno-abm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-sgx -mno-bmi2 -mno-pconfig \
-mno-wbnoinvd -mno-tbm -mno-avx -mno-avx2 -mno-sse4.2 -msse4.1 -mno-lzcnt -mno-rtm -mno-hle \
-mno-rdrnd -mno-f16c -mno-fsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mno-xsave \
-mno-xsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 \
-mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma \
-mno-avx512vbmi -mno-avx5124fmaps -mno-avx5124vnniw -mno-clwb -mno-mwaitx -mno-clzero -mno-pku \
-mno-rdpid -mno-gfni -mno-shstk -mno-avx512vbmi2 -mno-avx512vnni -mno-vaes -mno-vpclmulqdq \
-mno-avx512bitalg -mno-movdiri -mno-movdir64b -mno-waitpkg -mno-cldemote -mno-ptwrite -mtune=generic

 Wow! Complicated, but I would have thought it would be OK with that "-march=core2". Obviously not though.

OE's most generic x86_64 CPU setting is Core2, but back when I ported the Pyro release of OE, I modified it for a Nocona CPU, which is the earliest 64-bit CPU. I attempted to port those Nocona changes into my Dunfell OE derivative, and did a complete recompile today. It did seem to have compiled everything for a Nocona CPU, but now I have used those packages and built Easy 0.92, and applied the same test:

# echo "" | gcc -v -E - 2>&1 | grep cc1
/usr/libexec/gcc/x86_64-poky-linux/9.3.0/cc1 -E -quiet -v - -march=skylake -mmmx -mno-3dnow -msse \
-msse2 -msse3 -mssse3 -mno-sse4a -mcx16 -msahf -mmovbe -maes -mno-sha -mpclmul -mpopcnt -mabm \
-mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -msgx -mbmi2 -mno-pconfig -mno-wbnoinvd -mno-tbm -mavx \
-mavx2 -msse4.2 -msse4.1 -mlzcnt -mno-rtm -mno-hle -mrdrnd -mf16c -mfsgsbase -mrdseed -mprfchw \
-madx -mfxsr -mxsave -mxsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf \
-mno-prefetchwt1 -mclflushopt -mxsavec -mxsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl \
-mno-avx512ifma -mno-avx512vbmi -mno-avx5124fmaps -mno-avx5124vnniw -mno-clwb -mno-mwaitx \
-mno-clzero -mno-pku -mno-rdpid -mno-gfni -mno-shstk -mno-avx512vbmi2 -mno-avx512vnni -mno-vaes \
-mno-vpclmulqdq -mno-avx512bitalg -mno-movdiri -mno-movdir64b -mno-waitpkg -mno-cldemote \
-mno-ptwrite -mtune=generic

Oh dear, "-march=skylake" is not what I want!

So, I booted up Easy Pyro 1.3:

# echo "" | gcc -v -E - 2>&1 | grep cc1
/usr/libexec/gcc/x86_64-oe-linux/6.3.0/cc1 -E -quiet -v - -mtune=generic -march=x86-64

Nice and simple!

This is going to take me a bit longer to fix. I think what I want is "-march=nocona -mtune=nocona" and I have to study OE and find out how it has gone wrong, and how all that other stuff has got in.

Changing the subject...

Gtk3 scrollbar

If you have used a GTK-based app, such as SeaMonkey, you might have noticed how annoying the scrollbars have become. In gtk2 if you click above or below the slider, the window will scroll up or down by one window of text. Gtk3 on the other hand, there will be a jump way up or down in the text file, which is hopeless if you want to scroll through the text a window-height at a time.

Some gtk3 themes do restore the gtk2 behaviour, but it is not the default. However, I after a bit of online searching, I found the fix. Add this line under "[Settings]" in /root/.config/gtk-3.0/settings.ini:

gtk-primary-button-warps-slider = false

I have put this into woofQ. 

Tags: oe

OE gatesgarth-pre compile

October 13, 2020 — BarryK

I am doing a recompile in OE, based on the latest from git master branch. Previously I used the 'dunfell' branch. 'gatesgarth' release of OE is due out this month, but as it is not yet out, I got everything from master branch.

OE is a cross-compiler environment, that can build packages for a complete Linux distribution. The people who use it are mostly targeting embedded hardware, however, it can be used to create packages for a complete Linux distro such as as EasyOS.

OE is composed of layers, each one named 'meta-*', for example 'meta-gnome' and 'meta-networking', and each layer contains recipes for downloading, compiling and installing packages. I have created 'meta-quirky', and meta-quirky/recipes-quirky currently has recipes for 187 packages. These are not provided by OE, they are created by me, packages required for building EasyOS or Quirky Linux. Or any pup.

Usually, when I upgrade to the latest OE, many of my recipes in 'meta-quirky' get broken, and it takes a lot of work to fix them, with some that have to give up on. I decided to download OE "gatesgarth-pre" and see how my 187 recipes hold up.

Not the only reason though. I wanted to remove 'modemmanager' and 'pulseaudio'.

Also wanted to bump to the 5.8.x kernel, as it has enhancements to /proc that look like might improve security in containers.

I am watching it now, compiling on same computer that I am sending this blog post. Going OK.

gcc has bumped from 9.x to 10.x, and I was expecting that might cause failure of some of my recipes, but OK so far.

Note that although OE is very sophisticated and complex, and daunting at first, it can be pretty straightforward to use. I have provided a OE dunfell tarball, with simple instructions, and it is just a matter of typing a few lines and watch it compile 745+ packages.

I intend to provide a OE gatesgarth tarball also, though might do so after 'gatesgarth' is officially released. Coz, the developers are likely finding issues that need fixing in master.

Here is an overview of OE:

https://en.wikipedia.org/wiki/OpenEmbedded

Another overview:

https://elinux.org/Yocto_Project_Introduction

The "Yocto" project tends to get most attention, however, it is really just a layer in OpenEmbedded. 'meta-yocto' contains recipes for building for specific boards. I am using "genericx86-64", which is good for PCs.

EDIT 2020-10-14:
I have decided to abandon the gatesgarth build and stay with dunfell release of OE.

A couple of reasons. One is that lots of my recipes are failing in the gatesgarth build. They are all giving a similar error message when compiling, about symbol redefinition. It would seem that there is some fundamental structural change in OE that is causing this problem. It will probably be in the release-notes when gatesgarth is released, but I don't really need to go down another rabbit hole.

Another reason is that gatesgarth is apparently a "dev" release, whereas dunfell is "lts" -- long term supported. Dunfell obviously has had a lot of effort put into it, given how easy it was to get my recipes to work. Also, layers maintained by other people are likely to become compatible with dunfell sometime, if not already. Many layers are maintained by just one person or a few people, and often lag behind the official releases -- meaning that they might not work with the dunfell release. "meta-debian" is an example, currently only compatible with the warrior release of OE.   

Tags: oe

Celluloid compile fail mystery in OE

September 29, 2020 — BarryK

"Celluloid" is the new name for Gnome-MPV, the media player, frontend GUI for MPV, the latter being a CLI media player (with a primitive pseudo-GUI). Project home page:

https://celluloid-player.github.io/

I have a "meta-quirky" layer for the Dunfell release of OpenEmbedded, see two previous posts in the saga:

https://bkhome.org/news/202009/pango-version-144-is-a-disaster.html

https://bkhome.org/news/202009/openembeddedyocto-dunfell-meta-quirky-first-release.html

All 755 packages compile, except for one, Celluloid. Or rather, it sometimes succeeds, sometimes not. Early this morning, 1.24am,  I started a complete recompile, then went to bed. I had wanted to time the build, but when I got up at 7.30am, I discovered that the build had stopped on an error -- oh no, Celluloid again.

On previous occasions, I had repeated the build of Celluloid, and maybe after a couple of attempts, it would succeed. Like this:

# bitbake -c cleansstate celluloid
# bitbake -c build celluloid

I am getting different error messages each time. This time, it reported a missing termination of a comment, and I discovered that the source file is truncated. Huh!?

How can that even happen?

Here is the recipe, and I have made a comment about the failure, and put in a possible fix:

# Recipe created by recipetool
# recipetool create -o celluloid_0.18.bb https://github.com/celluloid-player/celluloid/archive/v0.18.tar.gz

LICENSE = "GPLv3"
LIC_FILES_CHKSUM = "file://COPYING;md5=8006d9c814277c1bfc4ca22af94b59ee"

SRC_URI = "https://github.com/celluloid-player/celluloid/archive/v${PV}.tar.gz"
SRC_URI[md5sum] = "3fde6852840798ad61b3e39f1513f8d4"
SRC_URI[sha1sum] = "17c7bcc5f34f003140cad407b6a3ecd4d87ce769"
SRC_URI[sha256sum] = "3ce6158097d94786a62de5f26ab2cb71301e9fd0841ede8381e1535489cf0de8"
SRC_URI[sha384sum] = "1f0eafd8b6542178abee06ce68c978ac3a6a9ebd2b479e85136e8db77e2614a4e70aca622ae4a44a4ff383762ec508a2"
SRC_URI[sha512sum] = "acb76bd216100afbe3b083919e122308def1431845955dd418e7b2d57f37fb1353209d1f77f41790d3e01603ee1137fe2c5a57c7bb0ca9b0b13388278cc291f1"

DEPENDS = "libepoxy mpv gtk+3 glib-2.0 appstream-glib glib-2.0-native"

inherit gettext pkgconfig autotools-brokensep

EXTRA_OECONF = ""

# 20200929 why does build sometimes fail?
# today, error "unterminated comment" line 680 in src/mpris/celluloid-mpris-gdbus.c
# and i see 680 is end of the file -- however, file is supposed to have 8355 lines!
# try this:
PARALLEL_MAKE = "-j 1"

HOMEPAGE = "https://celluloid-player.github.io/"
SUMMARY = "GUI frontend for MPV media player"

Tried again, this time success. Now the build of all packages is continuing.

All 755 packages compile successfully, consistently, except for Celluloid. Why just Celluloid? The source package looks like a pretty ordinary autotools-based setup, can't see anything odd in the makefiles.

I put in that PARALLEL_MAKE = "j 1", but I don't see how that would have anything to do with a source file getting truncated. Is it something to do with ext4? A kernel problem?

Bitbake, the program that runs the OE build, uses all available cores, in my case 4, and will be building 4 packages at once, swapping between them continuously. Actually, it will be building many more than 4, but only 4 at the same time, rotating through them, like a 4-package moving window. 

Tags: oe

OpenEmbedded/Yocto Dunfell meta-quirky first release

September 27, 2020 — BarryK

A few years ago, I created "oe-qky-src", a port of OpenEmbedded cross-compiler build system, with recipes to build nearly all the packages required for woofQ -- the latter is the build system for Quirky Linux or EasyOS. WoofQ is a derivative of Woof2, as is woof-CE that is used to build the latest official releases of Puppy Linux.

"oe-qky-src" is in a git repository:

https://github.com/bkauler/oe-qky-src

...that has my "meta-quirky" layer for OE. It is based on the "Pyro" release of OE/Yocto.

"oe-qky-src" is getting a bit "long in the tooth", so I decided to do a complete update, based on the latest release of OE/Yocto, named "Dunfell".

A couple of weeks of very intense work later, and we now have the first release. I have decided not to put this onto a git repository, instead it is just a tarball:

http://distro.ibiblio.org/easyos/project/oe/dunfell/dunfell-20200927.tar.gz

Download it, expand it, and type in a few commands, and several hours later you have compiled 755 packages! Detailed instructions here:

http://distro.ibiblio.org/easyos/project/oe/dunfell/readme.txt

...the readme is also inside the tarball.

One point: it does include 'pulseaudio'. With "oe-qky-src" there was only 'alsa', but this time I decided to include 'pulseaudio', despite reservations, as it is very difficult to implement bluetooth audio without 'pulseaudio'. I will build EasyOS from these packages and see how it goes.

If I change my mind, it is easy to remove 'pulseaudio' and recompile everything.

Those compiled packages can be imported into any woof* derivative. However, the '0pre-oe' and '0pre-oe-add' scripts are required to perform the import, and they are currently only in woofQ -- furthermore, I fixed some bugs in those scripts yesterday, and have not yet uploaded the latest woofQ tarball. Intend to do so soon.

I think, in theory, if you are a Puppy Linux developer using woof-CE, you should be able to use those scripts, but there are some differences from woofQ, so you would need to look through the scripts and make sure they are woof-CE-compatible.

Oh, another thing, Woof* has a directory 'packages-templates' and the one in woofQ has diverged somewhat from that in woof-CE. This might affect the functioning of some packages when a Puppy build is done, that is when run the '2createpackages' script.

Here is the package list:

http://distro.ibiblio.org/easyos/project/oe/dunfell/pkglist-20200927.txt

Anyway, lots of fun. Certainly a very niche audience, as although 755 packages seems like a lot, it is tiny compared with the package repositories of the major distributions such as Debian -- note though, that 755 is "whole" packages, not split-up ones -- Debian will typically split an original package into 2 - 4 DEBs, so that 755 would become >1500 packages! 

Tags: oe

More patches for an old package in OE

September 22, 2020 — BarryK

I am continuing to compile old packages, that are still good, and included in releases of EasyOS. Some notes have been appended to the previous blog post:

https://bkhome.org/news/202009/progress-compiling-in-oe-dunfell-release.html

The package 'xlockmore' is an example of very "long in the tooth". It compiled OK in the Pyro-OE, but not in Thud-OE. Nor in the latest OE, Dunfell-OE. There have been patches in the past to get it to compile. This time, the error message is that "DOMAIN" is undefined.

It turns out that the problem is with glibc 2.27+, as explained here:

https://github.com/flang-compiler/flang/issues/475

...jamborm has done what he calls a "horrible hack". Yes, but it works. I appended his code onto the end of any include file, xlock/xlock.h, in the recipe for xlockmore, like this:

    echo '
#ifndef LIBM_ERRNO_AMD_H_INCLUDED
#define LIBM_ERRNO_AMD_H_INCLUDED 1

#ifndef DOMAIN
struct exception {
int type; /* Exception type */
char *name; /* Name of function causing exception */
double arg1; /* 1st argument to function */
double arg2; /* 2nd argument to function */
double retval; /* Function return value */
};

#define DOMAIN 1
#define SING 2
#define OVERFLOW 3
#define UNDERFLOW 4
#define TLOSS 5
#define PLOSS 6
#endif

#endif /* LIBM_ERRNO_AMD_H_INCLUDED */' >> xlock.h

Yay, xlockmore is still alive!

Actually, there is a later version of xlockmore that what I am using. I have 5.31, latest is 5.65. I continue to use the old one as it has many hacks to make it very small. Ideally, I should apply those "smallness" hacks to the latest version. Here is the xlockmore project page:

http://sillycycle.com/xlockmore.html   

Tags: oe

Porting OpenEmbedded Thud release

March 21, 2019 — BarryK

The packages used in EasyOS 0.9.x - 1.0.x were compiled by "oe-qky-src", which is a port of the "Pyro" release of Yocto/OpenEmbedded:

https://github.com/bkauler/oe-qky-src

The Yocto/OpenEmbedded release schedule is here:

https://wiki.yoctoproject.org/wiki/Releases

...Pyro was released in May 2017, and EasyOS is showing its age a bit. Nothing really wrong with the older glibc, gcc, etc., but EasyOS is probably due for a freshen-up with later packages.

I did do a partial upgrade of the Xorg and some multimedia packages, from the "Sumo" release awhile back.

Anyway, now having a go at a complete upgrade, to the latest stable release, codenamed "Thud", version 2.6.1.

Intention is, there will be a new repository, probably at github.com, named "oe-easy-src".

It is doing a compile right now, with the Thud package recipes as-is, just to see if it will compile. If that succeeds, I will port some recipe modifications that were applied in oe-qky-src, and about a hundred extra packages that are not in Yocto/OpenEmbedded -- for which I created recipes. 

Tags: oe