easy.sfs recompress with LZ4_HC

Thanks to forum member rufwoof, who got me thinking about this. File 'easy.sfs' has the entire EasyOS filesystem -- we don't split up into smaller SFS files as is done in Puppy. Except, still have the separate 'devx' SFS.

To keep the download file small, easy.sfs is xz-compressed. This is the smallest file-size; however, decompression when in use is also slower than other compression methods. With a fast CPU, you probably won't notice it, but with a somewhat older CPU there may be noticeable sluggishness, perhaps in app startup.

The first time that you bootup Easy, the QuickSetup window appears. This can also be run at any time, via the "Setup" category in the menu. Here it is, and notice the arrow I have drawn:

That checkbox gives you the opportunity to recompress easy.sfs with a faster compression method. Up until now it has been "gzip" compression. What I have done now is change that to "lz4", actually, "LZ4_HC", which has higher compression than plain lz4.

Let's see the effect on the filesize for each compression method. Taking 'easy'sfs' from the latest Easy Dunfell, 3.4.7, which is xz-compressed. The method to recompress is to first mount easy.sfs:

# mkdir mntpt
# mount -t squashfs easy.sfs mntpt
# mksquashfs mntpt easy-gz.sfs -comp gzip
# mksquashfs mntpt easy-lz4.sfs -comp lz4
# mksquashfs mntpt easy-lz4-hc.sfs -comp lz4 -Xhc
# sync
# umount mntpt

Look at the sizes:

xz	gzip -9	lz4	lz4 -9
602M	720M	1009M	850M

The last column is LZ4_HC. Tables of compression and decompression speed are here:

https://www.privex.io/articles/which-compression-algorithm-tool/

Fast compression is great of course, but my main interest is decompression speed. Table from that link, for maximum "-9" compression for all of them:

NAME        DECOMP TIME        MB DECOMP PER SEC
lbzip2:     2.776 seconds      222.62 MB/S
lz4:        1.484 seconds      416.44 MB/S
gzip:       7.666 seconds      80.61 MB/S
xz:         15.532 seconds     39.78 MB/S

...wow, look at LZ4_HC! This has convinced, me LZ4_HC is the best choice.

I posted about recompiling the kernel to support squashfs lz4:

https://bkhome.org/news/202205/kernel-51539-compiled-in-easy-bookworm.html

...next up, will do the same for Easy Dunfell series.

Then, updated 'mksquashfs' in the initrd to support lz4:

https://bkhome.org/news/202205/mksquashfs-in-initrd-updated-with-lz4-support.html

Finally, have edited the 'quicksetup' script, and 'init' script in the initrd, changed recompression from gzip to lz4 -9.

Technical details worth knowing:

easy.sfs in the boot-partition is not resized, as there isn't room for it. The boot partition was 639M, and I have increased it to 767M, so not big enough.

I recently explained that at first bootup, 'easy.sfs' is copied to the working-partition, to support rollback. This copy in the working-partition is what gets resized. At bootup, the copy is the one that gets loaded in the aufs layered filesystem.

Another useful point to know, is that if there is sufficient RAM that copy of 'easy.sfs' gets copied into RAM, then mounted in the aufs layered filesystem. The 'init' script considers various factors before deciding to do this, such as is there sufficient RAM. There is a tradeoff here, as 'easy.sfs' is going to sitting in RAM, reducing what is available. You would need at least 4GB RAM for 'easy.sfs' to be copied into RAM.   

Tags: easy