site  contact  subhomenews

Kernel 4.14.87 compiled

December 10, 2018 — BarryK

I was attempting to do a frugal install on the Mele PCG35 Apo mini-PC, with the boot-partition in the internal drive, and the working-partition on a plugin SD-card. However, the kernel does not recognise the Realtek card-reader at early bootup, as the drivers for it are modules. So, compiling the latest 4.14.*, with those drivers builtin. These are the changes since 4.14.81:

Device Drivers
Multifunction device drivers
<*> Realtek USB card reader CONFIG_MFD_RTXS_USB
<*> MMC/SD/SDIO card support
<*> Realtek USB SD/MMC card interface driver CONFIG_MMC_REALTEK_USB

This was enabled, disabled it:

Processor type and features
< > Avoid speculative indirect branches in kernel CONFIG_RETPOLINE

...that last one, of course, is controversial. However, I have never configured a kernel with hyperthreading enabled, except that it now is. That is, I always had CONFIG_SMP enabled, but not CONFIG_SCHED_SMT -- however, after compiling this kernel, I discovered that CONFIG_SCHED_SMT has become enabled -- huh, that is sneaky, I always use the .config file from the previous compile, so somewhere recently, the kernel developers have chosen to over-rule me.

Looked, CONFIG_SCHED_SMT was not enabled in the 4.14.81 compile. So they have done it in a more recent kernel. Cr*p, now will have to recompile it. 

Note, way back in the early days when I was first compiling the kernel for multi-core CPUs, SMT (hyperthreading), which simulates more than one core for each actual core, was causing file corruption. So, disabled it, and never trusted it after that. 

EDIT 2018-12-10
OK, I found the retpoline patch was introduced in 4.14.82, and using the .config from 4.14.81, was able to configure with retpoline disabled, and also SMT disabled. Fine, so using that kernel now.

Took another look at the 4.14.87 kernel, wondering why SMT was forced enabled. I found out why:

Processor type and features
[ ] SMT (hyperthreading) scheduler support

...is missing!!! Yep, not there. Has it been moved? Hunted around, couldn't find it. So, for now, staying with 4.14.82. 

EDIT 2018-12-16
Kernel 4.14.88 came out, and I discovered CONFIG_SCHED_SMT is still forced on, so looked back through the kernel changelog, and found that it is deliberate. I have sent an email to the guy to contributed the patch, and cc'ed all of those who signed-off on it. This is the email:

Mr Gleixner,
I was upset when I compiled 4.14.87 found that SCHED_SMT had been forced on. At the time, I just reported it to my blog, and posted a question about it to a couple of forums, and stayed with an earlier kernel.

However, when 4.14.88 came out, and still the same situation, alarm bells went off, and I looked through the kernel changelog. Found it, 4.14.86:

"x86/Kconfig: Select SCHED_SMT if SMP enabled"

Then:

"CONFIG_SCHED_SMT is enabled by all distros, so there is not a real point to
    have it configurable. ..."

...that is a lie. It would be correct to state that is true of the distros you use, and presumably also for all of you guys who signed off on it.

Puppy Linux is an example of a distro that has mostly not had SCHED_SMT enabled. Ditto for most of the forks of Puppy. Two distros that I currently maintain, Quirky and EasyOS (easyos.org) have SMP enabled but not SCHED_SMT.

The difference between them is important, they should remain independently settable. I am so surprised that all of you guys went along with forcing it on.

For the record, my blog post:

http://bkhome.org/news/201812/kernel-41487-compiled.html

Regards,
Barry Kauler

The kernel changelog is here:

https://mirrors.edge.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.14.86  

If you would like to provide some input, there is a forum thread here:

http://murga-linux.com/puppy/viewtopic.php?t=114921 

EDIT 2018-12-17
I had another feedback, one of the kernel developers has informed me that the retpoline security threat is not just when CONFIG_SCHED_SMT is enabled, it may also be a threat when disabled.

That changes things. It seems that my understanding was too simplistic.

So, might just give in, and go along with what the kernel developers have decided. Don't have a choice anyway.

It is disconcerting though, when a kernel configure option suddenly disappears, and the explanation doesn't definitively justify it's removal.

I still have suspicions about enabling SCHED_SMT, from way back in the "early days" when I experienced file corruption, but it seems whatever that problem was, it has long ago been fixed.

Thus ends my little session of annoying the kernel developers! 

Tags: easy

CUPS+Gutenprint printing fixed

December 09, 2018 — BarryK

I reported yesterday that printing to my Canon MX310 printer does not work, using the CUPS+Gutenprint driver (which is what the CUPS web interface automatically selects):

http://bkhome.org/news/201812/canon-mx310-mfp-rescued-from-road-verge.html

The error-log showed "gs: no such file or directory", which I thought that I had fixed by creating a symlink, but no, the error is still in the log. But, ghostscript is installed, and 'gs' is there, at /usr/bin/gs.

Then a thought occurred: 'cups-filters' is compiled in 'oe-qky-src', my fork of OpenEmbedded. The 'ghostscript' package in oe-qky-src does not install 'gs', it installs 'gsc' and it is in Woof where I create a symlink, 'gs' to 'gsc'.
It could be that during configuration of the source, cups-filters cannot determine where 'gs' is and this is causing the failure to find it in the running EasyOS.

Anyway, I compiled the latest 'cups-filters', version 1.21.5, which also required 'qpdf' to be bumped to 8.2.1, and yay, printing works!

I have made both of those into PETs, which will be used in the next release, 0.9.11, but soon will have to go back to oe-qky-src and sort out what has gone wrong with configuring cups-filters.

So, good news. I am very tempted to buy ink cartridges, just so that I can see something printing! Officeworks has them:

https://www.officeworks.com.au/shop/officeworks/p/canon-pg-37-ink-cartridge-black-capg37bk

https://www.officeworks.com.au/shop/officeworks/p/canon-pg-40-ink-cartridge-black-capg40bk

...they are black. Duo, black and tri-colour cartridges:

https://www.officeworks.com.au/shop/officeworks/p/canon-pg-40-and-cl-41-ink-combo-pack-ca4041com

Some inkjet printers won't work with only a black cartridge, which was the situation with mine 10 years go -- a nasty trick by the manufacture. Hopefully the Canon will print with just the black cartridge, so I would have to spend AU$35 ...that would be an indulgence, as I hardly ever need to print these days. 

EDIT 20181210:
Have updated and recompiled 'ghostscript', 'poppler', 'qpdf' and 'cups-filters' in oe-qky-src:

https://github.com/bkauler/oe-qky-src/commits/master

...and built a pre-release of Easy 0.9.11, printing to the Canon MX310 works.  So, no need to use the PETs. 

Tags: easy

EasyOS version 0.9.10 released

December 06, 2018 — BarryK

A lovely new version of EasyOS, for computers with 64-bit CPU. This is what has been named the "Pyro" series and has now reached version 0.9.10.

Lots of things have happened since 0.9.9, and I recommend browse through the blog posts.

Download from here, kindly hosted by ibiblio.org;

http://distro.ibiblio.org/easyos/amd64/releases/pyro/0.9.10/

There is a choice, either a USB-image file for writing to a USB-stick, or an ISO live-CD file. Yes, for the first time, Easy is now provided as a live-CD, see announcements here:

http://bkhome.org/news/201812/easyos-live-cd-debut.html

http://bkhome.org/news/201812/permanent-saving-for-easyos-live-cd.html

There is plenty of online documentation on how to write an ISO file to CD or DVD media, if you need it. If you need help with writing the USB-image to a USB-stick, see here:

http://bkhome.org/easy/how-to-write-easyos-to-a-flash-drive.html

Feedback is welcome, either to the EasyOS forum, or there is a thread on the Puppy Forum:

https://easyos.org/forum/

http://murga-linux.com/puppy/viewtopic.php?p=1012192#1012192 

Note, I have updated the "How and why EasyOS is different" page:

https://easyos.org/about/how-and-why-easyos-is-different.html

And there is a new tutorial for frugal installation:

https://easyos.org/install/easy-frugal-installation.html

Have fun! 

Tags: easy

New tutorial for frugal installation

December 06, 2018 — BarryK

Here it is, a brand new tutorial, frugal installation of EasyOS, mostly targeting older traditional-BIOS computers:

https://easyos.org/install/easy-frugal-installation.html

The old one, at bkhome.org/easy, will be removed soon, or rather, will redirect to this new one.

Still haven't formally announced Easy 0.9.10. The tutorial took all morning to write, need to go for lunch and some shopping. Hope to do the announcement this evening. 

Tags: easy

Permanent saving for EasyOS live-CD

December 05, 2018 — BarryK

I posted yesterday about creating a live-CD of EasyOS, that runs totally in RAM, without any ability to save the session, that is, a pristine first-bootup every time:

http://bkhome.org/news/201812/easyos-live-cd-debut.html

However, I have reconsidered, for the sake of those who have an older computer that will not boot from USB. Now, the CD will automatically discover a plugged-in USB-stick or SD-card, and use that as the working-partition.

In the above link, I wrote about the Disk Identifier. One thing that I did not mention, the 4-byte (8 hex digits) code is for dos partitions. A Guid partition table (gpt) has a longer Disk Identifier, for example:

# fdisk -l /dev/sdc
Disk /dev/sdc: 3.7 GiB, 4005560320 bytes, 7823360 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: B082636C-2E48-4D76-93EC-CAD9A98383EE

What I have done with the live-CD, is it will look for a drive with disk id of 0x12345678, if found, it will use that as the working-partition. If not found, it will bootup in RAM only. This is great, it means that users who have to bootup from a CD, now have permanent storage, no need even for internal hard drive.

To create such a USB-stick is very easy. Find a spare stick, plug it in, run Gparted, and create a msdos partition table, via the "Device -> Create partition table..." menu. In my case:

image1

As my USB-stick already had a guid partition table (gpt), this gets rid of it. Regardless of what is on the stick, this gets rid of it and you have a clean slate. Then, create a partition with ext4 filesystem, to fill the drive.

Confirming, now have a dos partition table:

# fdisk -l /dev/sdc
Disk /dev/sdc: 3.7 GiB, 4005560320 bytes, 7823360 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xf8d6e744

However, what we want is to change the disk identifier to 0x12345678, and this can be done with fdisk. You can run fdisk by doing this:

# fdisk /dev/sdc

...then chose "x" (expert mode), then "i" to change the disk-id, then "r" to return to main menu, then "w" to write change and exit. Or, you can do it in one line:

# echo -e "x\ni\n0x12345678\nr\nw" | fdisk /dev/sdc
# sync
...please be careful that you do this to the entire drive, in my case sdc, not a partition.

Reboot the live-CD and the USB-stick is automatically discovered and used. I have stuck a label on my USB-stick, with masking tape, this can now be used for all future live-CD usage:

image2

Interested? If so, the live-CD is here:

http://distro.ibiblio.org/easyos/amd64/releases/pyro/0.9.10/

I intend to announce the release of EasyOS version 0.9.10 tomorrow. First, I want to rewrite the frugal-install tutorial, there have been so many changes. Should get that done tomorrow sometime. 

Tags: easy

EasyOS live-CD debut

December 04, 2018 — BarryK

A few people have contacted me recently to let me know that they have computers that will not boot from USB. In that situation, the person could do a direct frugal installation to hard drive.

That would mean opening up the downloaded USB-stick image file, then copying 'vmlinuz', 'easy.sfs' and 'initrd' to somewhere on the hard drive, then opening up 'initrd' and fix file 'BOOT_SPECS', then make an entry in the boot manager (such as GRUB).

Those steps are fairly easy, but most easy if already running EasyOS ...which is a "Catch 22" situation if cannot boot from USB.

So, although I said that I wasn't going to do it, have done so, created an EasyOS live-CD. However, one important thing about it, it runs totally in RAM and there is no facility to save sessions. It is thus a pristine bootup everytime.

The intention is that this live-CD is only for those who cannot boot from USB, or who just want to do a quick evaluation of EasyOS. It is intended as an easy means to do a frugal installation to hard drive.

However, theoretically, the live-CD could be enhanced to save sessions, similar to how it is done with the Quirky live-CD. I don't plan to do that though. If someone skilled with shell scripting wants to do it, you are welcome, and if you do so, I will consider incorporating it into the official release.

This CD ISO file will be available with the next release, version 0.9.10. 

An interesting technical detail:

Disk identifier

While I was working out how to create the live-CD, I wanted to be able to create an ISO file with a known "disk identifier", that I could put into the "BOOT_DISKID" parameter in 'BOOT_SPECS'. This identifier is what you see when you run "fdisk -l /dev/<drive>", for example:

# fdisk -l /dev/sdc
Disk /dev/sdc: 7.5 GiB, 8019509248 bytes, 15663104 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x69522b3b

Device Boot Start End Sectors Size Id Type
/dev/sdc1 * 2048 1310719 1308672 639M ef EFI (FAT-12/16/32)
/dev/sdc2 1310720 15663103 14352384 6.9G 83 Linux

The disk identifier is eight hex characters (4 bytes), a random number generated when the partition table is created.

In Woof, I have a new script 'create-live-cd', which generates a random hex number, and that is also interesting, I found this neat way to do it:

# openssl rand -hex 4
d67b249a

My script assigns this to 'BOOT_DISKID' variable in file 'BOOT_SPECS' inside the 'initrd'.

When the ISO file is created, it will have a random disk-identifier, however, I can change it to the value that I generated. This web page explains how:

https://www.linuxquestions.org/questions/linux-general-1/what-is-disk-identifier-740408/

My script uses 'fdisk' to do it. At bootup of the live-CD, the 'init' script is easily able to find the boot-drive (the CD iso9660 filesystem, usually /dev/sr0) by reading the disk-identifiers of the drivers. 

Tags: easy

Fix rpm extraction in Xarchive

December 03, 2018 — BarryK

EasyOS has the Xarchive archive management GUI utility. This has wrapper shell scripts in /usr/lib/xarchive/wrappers.

It was reported that rpm extraction does not work. I traced this problem to the Busybox rpm applet, that is missing a required commandline option. There is a full rpm package in the package repository, but we would really like extraction to work without having to install that.

I found an old fix for Busybox rpm, created by Puppy Forum member 'mikeb':

http://www.murga-linux.com/puppy/viewtopic.php?t=26792

...yes, the GUI now displays the files inside the rpm, but nothing gets extracted.

EasyOS also has another utility, 'exploderpm', for examining and extracting from an rpm file.

I have limited time to work on this, devised a /usr/lib/xarchive/wrappers/rpm-wrap.sh that works, but with a temporary workaround:

#110808 rerwin: Replace all redirections to stderr to be appends, to protect error log.
#181203 mikeb: fix for busybox rpm. ref: http://www.murga-linux.com/puppy/viewtopic.php?t=26792
#181203 temporary hack for extract using busybox rpm and exploderpm.

# set up exit status variables
E_UNSUPPORTED=65

# Supported file extentions for tar
EXTS="rpm"

# Setup awk program
AWK_PROGS="mawk gawk awk"
AWK_PROG=""
for awkprog in $AWK_PROGS; do
if [ "$(which $awkprog)" ]; then
AWK_PROG="$awkprog"
break
fi
done

# Programs to wrap. 181203
XRPMflg="$(which exploderpm)"
if [ "$XRPMflg" ];then
XRPM_PROG="exploderpm"
[ ! -f /usr/lib/xarchive/wrappers/exploderpm-wrap.sh ] && cp -a -f /usr/lib/xarchive/wrappers/rpm-wrap.sh /usr/lib/xarchive/wrappers/exploderpm-wrap.sh
fi
BBRPMflg="$(rpm --help 2>&1 | grep -o '^BusyBox')"
RPM_PROG="rpm"
RPM2CPIO_PROG="rpm2cpio"
CPIO_PROG="cpio"

# setup variables opt and archive.
# the shifting will leave the files passed as
# all the remaining args "$@"
opt="$1"
shift 1
archive="$1"
shift 1

# the option switches
case "$opt" in
-i) # info: output supported extentions for progs that exist
if [ ! "$AWK_PROG" ]; then
echo none of the awk programs $AWK_PROGS found >> /dev/stderr
echo extentions $EXTS ignored >> /dev/stderr
elif [ ! "$(which $RPM2CPIO_PROG)" ]; then
echo $RPM2CPIO_PROG required by $0, but not found >> /dev/stderr
echo extentions $EXTS ignored >> /dev/stderr
elif [ ! "$(which $CPIO_PROG)" ]; then
echo $CPIO_PROG required by $0, but not found >> /dev/stderr
echo extentions $EXTS ignored >> /dev/stderr
elif [ "$(which $RPM_PROG)" ]; then
for ext in $EXTS; do
printf "%s;" $ext
done
else
echo program $RPM_PROG not found >> /dev/stderr
echo extentions $EXTS ignored >> /dev/stderr
fi
printf "\n"
exit
;;

-o) # open: mangle output of rpm cmd for xarchive
# format of output:
# usr grp attr size time(epoch) name
# 1 2 3 4 5 6
#181203 use best rpm...
if [ "$BBRPMflg" == "" ];then #have full rpm
$RPM_PROG -q --qf '[%{FILEUSERNAME} %{FILEGROUPNAME} %{FILEMODES:perms} %{FILESIZES} %{FILEMTIMES} %{FILENAMES}\n]' -p "$archive" | $AWK_PROG '
{
uid=$1
gid=$2
attr=$3
size=$4
time_cmd="date -d \"1970-01-01 UTC " $5 " seconds\" +\"%H:%M\""
time_cmd | getline time
close(time_cmd)
date_cmd="date -d \"1970-01-01 UTC " $5 " seconds\" +\"%Y-%m-%d\""
date_cmd | getline date
close(date_cmd)

split($0, linesplit, $5 " ")
name=linesplit[2]
link="-"
printf "%s;%s;%s;%s;%s;%s;%s;%s\n",name,size,attr,uid,gid,date,time,link
}'
elif [ "$XRPM_PROG" ];then #have exploderpm
$XRPM_PROG -lv "$archive" | $AWK_PROG '
{
uid=$3
gid=$4
attr=$1
size=$5
time="-"
name=$9
link="-"
date=$7$6$8
printf "%s;%s;%s;%s;%s;%s;%s;%s\n",name,size,attr,uid,gid,date,time,link
}'
else #busybox rpm
$RPM_PROG -qlp "$archive" | $AWK_PROG '
{
uid="-"
gid="-"
attr="-"
size="-"
time="-"
name=$1
link="-"
date="-"
printf "%s;%s;%s;%s;%s;%s;%s;%s\n",name,size,attr,uid,gid,date,time,link
}'
fi
exit
;;

-a) # adding to archive unsupported
# use appropriate rpm tools to build rpms
exit $E_UNSUPPORTED
;;

-n) # create new archive unsupported
# use appropriate rpm tools to build rpms
exit $E_UNSUPPORTED
;;

-r) # removing from archive unsupported
# use appropriate rpm tools to modify rpms
exit $E_UNSUPPORTED
;;

-e) # extract: from archive passed files
# convert rpm to a temporary cpio file
tmpcpio="$(mktemp -t cpiotmp.XXXXXX)"
$RPM2CPIO_PROG "$archive" > "$tmpcpio"
# extract files from the temporary cpio
#181203 ***BROKEN*** there is no $1 passed in here. maybe there is with full rpm?...
if [ "$BBRPMflg" == "" ];then #have full rpm
while [ "$1" ]; do
$CPIO_PROG --no-absolute-filenames -idmv ".$1" < "$tmpcpio"
shift 1
done
else #hack for now, extract everything...
$CPIO_PROG --no-absolute-filenames -idmv < "$tmpcpio"
fi
wrapper_status=$?
# remove temporary cpio
rm "$tmpcpio"
exit $wrapper_status
;;

*) echo "error, option $opt not supported"
echo "use one of these:"
echo "-i #info"
echo "-o archive #open"
echo "-a archive files #add"
echo "-n archive file #new"
echo "-r archive files #remove"
echo "-e archive files #extract"
exit
esac

If you are reading this and you have some prior experience with fixing/configuring Xarchive, would appreciate if you can fix the "***BROKEN***" part!

Or, I could migrate to UExtract, an alternative created by 'SFR':

http://murga-linux.com/puppy/viewtopic.php?t=87864

Note, there is another, older, alternative, Xarchiver, for which there are PETs around somewhere. 

Tags: easy

Grub4Dos PET for EasyOS

December 01, 2018 — BarryK

I posted yesterday about a still-active fork of Grub4Dos, version 0.4.6a, that apparently has fixed the "64bit ext4" problem:

http://bkhome.org/news/201811/compiling-grub4dos-version-046a.html

Well, I tested it, merged it into shinobar's Grub4dos PET, and when attempted to boot the installation (testing on my "new" Compaq Presario), got this:

Processing the preset menu ...
grub>

It seems to have read 'menu.lst', but then dropped to the grub prompt. Don't know what to do, so staying with Grub4dos 0.4.4.

It turns out that the "64bit ext4" problem is not likely to be encountered, at least not on older computers for which Grub4dos will be used.

Furthermore, I discovered that Gparted defaults to creating 32bit ext4 filesystems, as it runs:

mke2fs ... -O ^64bit ...

...the "^" means "do not enable what follows". That's great avoids hassles, so staying with shinobar's PET.

However, the PET has a script, /usr/sbin/grub4dosconfig, that does not recognise frugal installations of EasyOS. I also found that the v1.9.3 PET did not recognise a frugal installation of Racy Puppy 5.5, whereas the older v1.8.0 did.

Fixed both of those problems, and created a new PET, uploaded here (187KB):

http://distro.ibiblio.org/easyos/noarch/packages/pet/pet_packages-noarch/grub4dos-0.4.4.v1.9.3-easy1.pet

...strictly, it is not really "noarch", as it does have a DOS binary executable.

This PET will be in the next release of Easy, version 0.9.10, coming soon. If you are running 0.9.9 or earlier, don't bother trying this PET, as 0.9.10 has some changes, for example, 'initrd.q' is now named 'initrd' -- which the Grub4dos PET now expects when looking for installed EasyOS. So, please wait until 0.9.10!

I hope that shinobar does not cringe too much if he reads some of the hacks I did in his script! :)  

Tags: easy