Hit roadblock building EasyArch

Yes, I am building EasyOS based on Arch Linux binary packages.

I had not attempted this previously, partly because Arch is a rolling release. However, a few days ago I discovered that Arch keeps snapshots of all the packages, going back many years. For example, December 31, 2021:

https://archive.archlinux.org/repos/2021/12/31/

I knew that peebee in the Puppy Forum had created a 32-bit Arched-based pup, "arch32pup". I also discovered a few days ago that he has created "arch64pup" -- so I downloaded and booted it. Looks good.

Here is the forum thread on arch32pup:

https://forum.puppylinux.com/viewtopic.php?t=56

I decided to give it a go in woofQ, using that 2021/12/31 snapshot. Had to fix some scripts in woofQ, such as '0setup', but then ran into a serious problem, the filesystem hierarchy...

https://man.archlinux.org/man/file-hierarchy.7.en

I see that peebee has kept the traditional folder hierachy, and I began to see why...

I decided to embrace the Arch folder hierarchy, with, for example, /lib as a symlink to /usr/lib. Unfortunately, this opens a can of worms. It is OK for building EasyOS from Arch binary packages, however, Easy is also constructed with other packages. For example, there is the Linux kernel PET -- when woofQ tries to copy it:

cp: cannot overwrite non-directory 'rootfs-complete/lib' with directory 'linux_kernel-5.10.90-dunfell64/lib'

This is a fundamental problem. The rsync utility has a workaround for this, with the "--keep-dirlinks" option; however, 'cp' doesn't.

It is looking like I will have to keep the traditional folder hierarchy, like peebee has done. That will require a workaround, as non-Arch packages may, for example, have binary executables that expect /lib/libc.so.6 and /lib/ld-linux-x86-64.so.2, or /lib64/libc.so.6 and /lib64/ld-linux-x86-64.so.2

Have to think about it a bit more. Another very hot day here in Perth, Western Australia, hovering around 35 - 40°C for several days. A visit to the ocean is in order, then will look at this again in the evening.

EDIT:
Haven't been to the beach yet, the memory has returned; the Arch symlinks are not such a problem after all. Woof* was designed from scratch to build a distro from a variety of packages, and handles that "cannot overwrite non-directory" problem.

The error shown above is when '3buildeasydistro' script in woofQ copies the Linux kernel to rootfs-complete. However, packages are copied with extra code that handles that problem:

 cp -a --remove-destination packages-${DISTRO_FILE_PREFIX}/${ONEPKG}/* sandbox3/rootfs-complete/ 2> /tmp/3builddistro-cp-errlog
 sync
 if [ -s /tmp/3builddistro-cp-errlog ];then
  ...

...the code detects that "cannot overwrite non-directory" in the error file, and copies it correctly.

In the case of copying the kernel, I just did this:

 #20220121 arch-linux, /lib is a symlink...
 cp -a --remove-destination $KERNPKGNAMEONLY/* rootfs-complete/ 2> /tmp/3builddistro-cp-errlog
 if [ -s /tmp/3builddistro-cp-errlog ];then
  cp -a --remove-destination $KERNPKGNAMEONLY/lib/* rootfs-complete/usr/lib/
 fi

The PPM (Puppy Package Manager), also known as PETget in EasyOS, also has this special handling when installing packages -- see /usr/local/petget/installpkg.sh.  

Tags: easy

Attempting to compile Shotcut video editor

Here is the Shotcut homepage:

https://www.shotcut.org/

Compile fail, have raised two issues with them:

https://github.com/mltframework/shotcut/issues/1216

https://github.com/mltframework/shotcut/issues/1217

...hope they respond!

EDIT:
I managed to compile it, and it starts up, but is unusable. This is a show-stopper bug, already reported, I have posted here:

https://github.com/mltframework/shotcut/issues/1057

What is going on? This is the third video editor I have tried and they all have show-stopper bugs. Well, no, there was a fourth, Avidemux, that works.      

Tags: easy

Didiwiki personal wiki retired

DidiWiki has been in the "Personal" menu category for ...forever. EasyOS, Quirky, and the pups before, all had it. However, I just discovered that it is broken, cannot create new wiki pages.

EasyOS has didiwiki version 0.8, and a quick search revealed that it was patched by Amigo in 2009:

https://oldforum.puppylinux.com/viewtopic.php?t=47185

It must have worked back then! No one has ever complained about it, which shows how popular it is.

I looked around. There is a version 0.66 that looks like it has been well maintained:

https://github.com/pepa65/didiwiki

There is also ciwiki, that forked from didiwiki, still being maintained here:

https://github.com/escapecode/ciwiki

I tested ciwiki, yeah it works, and it does have more features than didiwiki.

However, don't think will use ciwiki. Even though can run as user 'spot' or any other user, probably not so good that it doesn't support https:// connections. It does allow hyperlinks, but only http://

So, didiwiki is now retired, and won't use ciwiki. Will have a look around and see if there is anything else.   

Tags: easy

Rollback exceptions extended

Worked on Osmo personal information manager yesterday:

https://bkhome.org/news/202201/osmo-pim-0214-compiled-in-oe.html

Which today has prompted extensions to the exemptions when rollback. This is script /usr/local/easy_version/erase-exceptions:

...added today are the "PIM" and "Figaros" checkboxes.

Currently only bringing up this erase-exceptions window when do a rollback to pristine first bootup, but perhaps should consider it for other rollback choices -- such as rollback to an earlier version of EasyOS or a snapshot.

Note, the "Web browser profile" checkbox will save all firefox, seamonkey, chrome and chromium profiles under /home, as well as all mozilla profiles running as root (/root/.mozilla).       

Tags: easy

Osmo PIM 0.2.14 compiled in OE

Very good news about Osmo. "PIM" means Personal Information Manager, and Osmo has been in the pups for a very long time. However, it is somewhat neglected and not as well integrated as could be.

The way this latest work with Osmo came about, is I have an elderly relative who has a mobile phone with prepaid account with a telco. I pay it, every 4 weeks. Don't want to do a direct-debit with a bank, so renewal requires me to remember when to pay... and I forget.

Besides, renewal is peculiar. I have to pay $15 for the renewal, then separately have to pay $5 international-calls addon. That second payment requires a manual online choice. I cannot see how to automate this with direct-debit or bpay. The telco is Optus.

I also forget birthdays!

Hence the thought of Osmo came up. It needs to default to run in the systray at every startup. It has great task management features, and I can set it to remind me every 4 weeks, by audio alarm and notify popup. It can also flash in the systray.

I have modified the package, it is no longer in the menu. It now starts every time in the systray, minimized until you click on it.

EasyOS and the pups use Osmo version 0.2.10, released in March 2010, as that is the last that supports 'libgtkhtml' for text rendering. After that, Osmo requires webkitgtk, which is enormous, and is a gross overkill.

So we stayed with 0.2.10. wdlkmpx has maintained it for woof-CE, with several patches:

https://github.com/wdlkmpx/osmo/

Today I wondered if it might be possible to revert the removal of libgtkhml on a later version of osmo. Version 0.2.14 was released in 2015:

https://sourceforge.net/projects/osmo-pim/files/osmo-pim/

I found the commit where libgtkhtml was removed:

https://sourceforge.net/p/osmo-pim/osmo/ci/f7bb3bbb97bee8d851b7d49d71261016b7d9e59b/

I then downloaded version 0.2.12 and attempted to revert that commit:

# git clone -b RELEASE_0_2_12 https://git.code.sf.net/p/osmo-pim/osmo osmo-0.2.12
# cd osmo-0.2.12
# git revert f7bb3b

...interesting, succeeded except for src/task.c that it badly screwed up. By reference to the original commit, I fixed src/task.c

It took awhile, but the end result is have patches to restore libgtkhtml for osmo 0.2.12 and 0.2.14:

https://distro.ibiblio.org/easyos/source/alphabetical/o/

...the "autoreconf" patches are also required, to update the 'configure' script. The "-p2" tarballs have the patches applied.

I have compiled 0.2.14 in OpenEmbedded. Only needed the restore-libgtkhtml patch, as OE runs autoreconf to rebuild the 'configure' script.

Here is the Osmo homepage:

https://clayo.org/osmo/

Regarding libgtkhtml, that is a dead project; however, we have patched it a few times. James did some work on it awhile back, and we have version 2.12:

https://bkhome.org/news/202002/helpsurfer-crash-fixed.html

I use that source in OpenEmbedded, but apply another patch to fix working with libxml2. If you want that patch, you will have to open up the OE tarball:

https://distro.ibiblio.org/easyos/project/oe/dunfell/

Note that libgtkhtml is also used by 'notecase' notes manager and 'surfer' local web page viewer.

EDIT 2022-01-18:
We have discussed this on the Puppy Forum:

https://forum.puppylinux.com/viewtopic.php?t=4959

It turns out there is a bug with the patch for 0.2.14. However, I found that the libgtkhtml patch for 0.2.12 works. Something has changed between 0.2.12 and 0.2.14 that causes the bug.

So will compile 0.2.12 in OE. 

Tags: easy

Opencv compiled in OpenEmbedded

Opencv is an optional dependency for mlt, lives, flowblade and other video editor packages. I have compiled it in OE, and it brings in these new dependencies:

ade       0.1.1f
gflags    2.2.2
glog      0.3.5
libeigen  3.3.7
libunwind 1.3.1
opencv    4.5.2
tbb       1_2020.2

The recipes already existed in OE, thank goodness as the recipe for opencv is complicated.

The opencv xz'ed package is 48MB, gigantic. I don't want to use it unless really have to. Certainly not for a package that will be builtin to Easy.

Here is the opencv project page:

https://github.com/opencv/opencv   

Tags: easy

Frei0r compiled in OpenEmbedded

Frei0r is a library of video effects. It is used in many video editors, including LiVES and Flowblade. It is also an optional dependancy of MLT. MLT is a dependency of Flowblade and some other video editors (Kdenlive, Openshot, Shotcut). Here is the frei0r home page:

https://frei0r.dyne.org/

I have now compiled frei0r in OpenEmbedded. The source supports building with either cmake or autotools, and OE defaulted to using cmake, which was a headache -- sometimes cmake is not cross-compiler-friendly. I changed the recipe to use autotools, and succeeded. Here is the recipe, file meta-quirky/recipes-quirky/frei0r/frei0r_1.7.0.bb:

# Recipe created by recipetool
# recipetool create -o frei0r_1.7.0.bb https://github.com/dyne/frei0r/archive/refs/tags/v1.7.0.tar.gz
# note: package supports both cmake and autotools.

LICENSE = "GPLv2"
LIC_FILES_CHKSUM = "file://COPYING;md5=59530bdf33659b29e73d4adb9f9f6552"

SRC_URI = "https://github.com/dyne/frei0r/archive/refs/tags/v${PV}.tar.gz \
           file://1001_fix_metainformation.patch \
           file://1002_spelling.patch"

SRC_URI[md5sum] = "d46006b92c3382a2f3260804249dcc11"
SRC_URI[sha256sum] = "6f7cf95ea2257687cc31db0ed9c9bc0eec152e953d515f346eabec048ed2b29d"

DEPENDS = "cairo zlib libxxf86vm libxrender libxfixes libxext libxdmcp libxdamage \
           libxcb libxau libx11 util-linux libpng pixman mesa freetype fontconfig \
           expat libdrm libjpeg-turbo"

inherit pkgconfig autotools

EXTRA_OECONF = "--disable-cpuflags --without-opencv"

#| ../../frei0r-1.7.0/src/filter/ndvi/ndvi.cpp:21:10: fatal error: frei0r.hpp: No such file or directory
#|    21 | #include "frei0r.hpp"
# ...i got it from debian. ***NO*** hey, it is already in the source...
do_compile_prepend() {
 cp -f ${S}/include/frei0r.hpp ${S}/src/filter/ndvi/
 cp -f ${S}/include/frei0r.h ${S}/src/filter/ndvi/
 cp -f ${S}/include/frei0r_cairo.h ${S}/src/filter/ndvi/
 cp -f ${S}/include/frei0r_cfc.h ${S}/src/filter/ndvi/
 cp -f ${S}/include/frei0r_colorspace.h ${S}/src/filter/ndvi/
 cp -f ${S}/include/frei0r_math.h ${S}/src/filter/ndvi/
 cp -f ${S}/include/blur.h ${S}/src/filter/ndvi/
 
 #see error msg below. fix...
 sed -i -e 's%^docs_DATA =.*%docs_DATA = %' ${B}/Makefile
}

#| make[1]: Entering directory '/mnt/sda1/nvme/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/frei0r/1.7.0-r6/build'
#| make[1]: *** No rule to make target 'README.txt', needed by 'all-am'.  Stop.
#| make[1]: Leaving directory '/mnt/sda1/nvme/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/frei0r/1.7.0-r6/build'

#install this as well, like debian does...
do_install_append() {
 install ${S}/include/frei0r.hpp ${D}/usr/include
}

HOMEPAGE = "https://frei0r.dyne.org/"
SUMMARY = "A minimalistic plugin API for video effects" 

I then added frei0r as a dependency in the mlt recipe, here is file meta-quirky/recipes-support/mlt/mlt_7.4.0.bb:

# Recipe created by recipetool
# recipetool create -o mlt_7.4.0.bb https://github.com/mltframework/mlt/archive/refs/tags/v7.4.0.tar.gz

#20220115 bump r6 to r7...
PR = "r${@int(PR_NUM) + 1}"

LICENSE = "LGPLv2.1 & GPLv2 & GPLv3 & MIT"
LIC_FILES_CHKSUM = "file://COPYING;md5=4b54a1fd55a448865a0b32d41598759d \
                    file://GPL;md5=b234ee4d69f5fce4486a80fdaf4a4263 \
                    file://GPLv3;md5=d32239bcb673463ab874e80d47fae504 \
                    file://src/modules/plus/ebur128/COPYING;md5=b7edcc6cb01ace25ebd2555cf15473dc"

SRC_URI = "https://github.com/mltframework/mlt/archive/refs/tags/v${PV}.tar.gz"
SRC_URI[md5sum] = "4bc74ec681e67310340bafc235178a81"
SRC_URI[sha256sum] = "17c19843ffdbca66777aaadf39acb11829fd930eaded92f768cbcb8ae59a5f37"

#removed:  tcl-native ruby-native
# 20220115 add: frei0r  removed:  python3-native
DEPENDS = "libx11 swig-native qtbase virtual/libgl \
     libsdl libsdl2 ffmpeg zlib libxxf86vm libxv xvidcore libxtst libxshmfence \
     libxrender libxrandr libxml2 libxi libxfixes libxext libxdmcp libxdamage \
     libxcb libxau x265 x264 libx11 libwebp wavpack libvpx libvorbis libvdpau \
     libva v4l-utils util-linux libusb1 eudev libtheora openssl speex speexdsp \
     sox libsndfile1 libsndfile1-native libsm libsamplerate0 libraw1394 \
     qtdeclarative qtscript qtsvg qttools qtx11extras qtxmlpatterns \
     pulseaudio libpthread-stubs libpthread-stubs-native libpng libpcre libpcre2 \
     pango opencore-amr libogg lame libmad xz libjpeg-turbo libid3tag icu \
     libice harfbuzz graphite2 mesa glib-2.0 glib-2.0-native gdk-pixbuf fribidi \
     freetype fontconfig flac fftw libffi expat libexif libdrm libdc1394 \
     libcdio-paranoia libcdio libcap bzip2 libbluray ffmpeg libao alsa-lib \
     swig lua-native perl-native frei0r"

# 20220115 just guessing, add python*...
inherit cmake cmake_qt5 pkgconfig python3-dir python3native

#removed:  -DCPU_SSE2=OFF
# 20220115 frei0r now on...
EXTRA_OECMAKE = "-DMOD_DECKLINK=OFF -DMOD_JACKRACK=OFF -DMOD_XINE=OFF \
     -DMOD_FREI0R=ON -DMOD_KDENLIVE=OFF -DMOD_RTAUDIO=OFF -DMOD_RUBBERBAND=OFF \
     -DSWIG_PYTHON=ON -DSWIG_LUA=ON -DSWIG_PERL=OFF"

#| ninja: error: build.ninja:4240: multiple rules generate out/lib/mlt.so [-w dupbuild=err]
#fix, ref: https://www.mail-archive.com/openembedded-devel@lists.openembedded.org/msg63379.html
EXTRA_OECMAKE_BUILD += "-w dupbuild=warn"

#/mlt/7.4.0-r6/image/usr/lib/perl5/vendor_perl/5.30.1/x86_64-linux/auto/mlt/mlt.so
#| CMake Error at src/swig/perl/cmake_install.cmake:50 (file):
#|   file RPATH_CHANGE could not write new RPATH:
# have changed -DSWIG_PERL to OFF.

SUMMARY = "MLT Multimedia Framework"
HOMEPAGE = "https://www.mltframework.org"

When the author of lives fixes the bugs, I intend to update the recipe in OE, adding frei0r dependency.

Testing flowblade in Easy 3.2.1, try to do a transition effect between two clips and flowblade crashes. I am hoping that adding frei0r will fix that.   

Tags: easy