Busybox experiment

I compiled version 1.16.0 and enabled almost everything. The 'busybox' executable is now 628KB, compared with 427KB in recent puppies.

Over the years I have incrementally replaced Busybox applets with the "full" versions, whenever I found the former to be inadequate.

For Quirky 007 I have decided to experiment with rewinding back to using more of the Busybox applets. For the record, these are the packages that I have changed:

Cut down
coreutils_cut, bc_cut

Removed
bzip2, cpio, diffutils, ed, eject, findutils, grep, gzip, hdparm, less, mktemp, net-tools, psmisc, sed, tar, time, unzip

Left alone
dosfstools, dpkg-deb, ifplugd, ifplugd_old, lzma, module-init-tools, ncurses, pciutils, procps, util-linux, wget.

The packages that I have "left alone", that is, left in Quirky, are those that I know or am pretty certain Busybox cannot adequately replace. Note though, I have left the Busybox applets enabled and they can be executed by 'busybox <applet>' for comparison purposes.

I created 'coreutils_cut' PET package which is very cut down, with only three utilities (install, paste, pr).

I know that we are going to find problems with this and will have to bring back some full utilities, but I'm hoping that it won't be many.

Then there's Bash... I know that there are some scripts written specifically for Bash.
The 'bash' executable is 498KB. Should I be a devil and take that out too? Create a script /bin/bash that just runs /bin/ash?

...so tempting. But I know that doing all of this, especially the Bash thing, is going to cause trouble with compiling some packages. But then, this is Quirky...


Posted on 30 Jan 2010, 8:47


Comments:

Posted on 30 Jan 2010, 12:56 by clarf
Bash Cures Cancer
Hi BarryK,

IMHO Bash is an updated Burn Shell that should keep as is. This is a nice link about Bash:

http://bashcurescancer.com/

clarf


Posted on 30 Jan 2010, 13:09 by R/T
bash vs ash
If you're thinking of replacing bash with ash, lots of scripts may behave unpredictably, in particular, any script that does string manipulation. Expressions like ${var%pattern} don't work in ash.


Posted on 30 Jan 2010, 13:30 by technosaurus
noexec/nofork
if you remove bash AND leave unused busybox utils compiled in, the problems will go up tenfold because ash will try to use the busybox builtin -- how would you do it anyways? an alias I guess since a symlink for bash to ash would give "applet not found"

it would be an interesting experiment though and could help speed up some scripts with minor rewrites

dir2pet, tgz2pet...
stat --format=%s >>> stat -c %s


Posted on 30 Jan 2010, 18:17 by Iguleder
Keep Bash
BusyBox's ash is extremely weak, it is unable to run even simple scripts with just one loop.

I experimented it on my distro and it proved to be too minimal. Also, make sure Bash is the default shell, because many scripts assume that /bin/sh points to /bin/bash and not anything else.


Posted on 31 Jan 2010, 23:08 by Dougal
Bashisms
About a year ago I changed /bin/sh to point to Busybox and... it broke ldd.
I was able to get past it by just changing the shebang in ldd to point to /bin/bash, but you won't be able to do that...
The odd thing is that the exit code was not any of those used in the script, so I have a feeling that the script simply broke when it tried to run "set -o pipefail".

BTW, the example R/T gave is incorrect: only ${VAR/A/B} will break.


Posted on 1 Feb 2010, 17:16 by BarryK
Bash Rollback
Hmmm, it seemed like a good initiative, but I've been comparing more Busybox applets with the full versions, and have rolled many more back to the full versions:

Cut down:
bc_cut

Removed:
bzip2, diffutils, ed, eject, gzip, hdparm, less, mktemp, net-tools, psmisc, time, unzip

Left alone:
coreutils, cpio, dosfstools, dpkg-deb, findutils, grep, ifplugd, ifplugd_old, lzma, module-init-tools, ncurses, pciutils, procps, sed, tar, util-linux, wget.

...and I won't be surprised if even more have to be reverted.



Posted on 2 Feb 2010, 5:20 by Dougal
Internal Commands
Note that the "internal commands" mentioned in the Busybox release notes are NOT the busybox applets -- it refers to the _shell's_ builtins...
http://tldp.org/LDP/abs/html/internal.html

Also, the fact that busybox is already in memory should give no advantage over the standalone versions of the coreutils etc., as once those are used (and mapped into memory), they'll likely stay in memory for as long as you use the computer -- unless you suffer from serious memory pressure (or specifically request the kernel to drop caches).



Posted on 2 Feb 2010, 10:44 by technosaurus
nofork/noexec
@Dougal

while not ALL applets are not nofork/noexec, it is now significantly more than just shell builtins

See the busybox mailing list for "nofork" "noexec"

or documentation from the latest tarball
/busybox-master/docs/nofork_noexec.txt

(from include/applets.h)
NOEXECs: AWK, CHGRP, CHMOD, CHOWN, CP, CUT, DD, ENV, FIND, HD, HEXDUMP, LN, LS, SORT, TAC, XARGS

NOFORKs: BASENAME, CAT, DIRNAME, ECHO, FALSE, FSYNC, HOSTID, LENGTH, LOGNAME, MKDIR, PRINTF, PWD, RM, RMDIR, SEQ, SLEEP, SYNC, TEST, TOUCH, TRUE, USLEEP, WHOAMI, YES


Posted on 3 Feb 2010, 23:07 by Dougal
Fork/Exec
I was refering to the part quoted from the release notes, where it said shell scripts will run faster "especially builtin commands". Everything in the lists you posted is applets, obviously.

Fork/exec is something different, relating to how the busybox binary handles its different threads, if it spins them off as separate processes or not. (I haven't read the Busybox docs on it, but as far as I know it can have an overhead since the program's memory has to be duplicated when doing so -- which might have an effect on CPU cache eviction, but that has nothing to do with the kernel caches...)




Posted on 16 Feb 2010, 19:04 by BarryK
Busybox rollback
The rollback continues. Busybox gunzip does not support the '-r' (recursive) option.

Cut down:
bc_cut

Removed:
bzip2, diffutils, ed, eject, hdparm, less, mktemp, net-tools, psmisc, time, unzip

Left alone:
coreutils, cpio, dosfstools, dpkg-deb, findutils, grep, gzip, ifplugd, ifplugd_old, lzma, module-init-tools, ncurses, pciutils, procps, sed, tar, util-linux, wget.



Posted on 17 Feb 2010, 8:05 by BarryK
Busybox rollback
Xarchive requires the full 'unzip' package.
I have examined the options in the Busybox 'diff' applet and think they are inadequate.

Cut down:
bc_cut

Removed:
bzip2, ed, eject, hdparm, less, mktemp, net-tools, psmisc, time

Left alone:
coreutils, cpio, diffutils, dosfstools, dpkg-deb, findutils, grep, gzip, ifplugd, ifplugd_old, lzma, module-init-tools, ncurses, pciutils, procps, sed, tar, unzip, util-linux, wget.



Posted on 17 Feb 2010, 8:31 by BarryK
Busybox rollback
I have been comparing more utilities. Rolled back more to the full versions:

Cut down:
bc_cut

Removed:
bzip2, ed, hdparm, less, time

Left alone:
coreutils, cpio, diffutils, dosfstools, dpkg-deb, eject, findutils, grep, gzip, ifplugd, ifplugd_old, lzma, mktemp, module-init-tools, ncurses, net-tools, pciutils, procps, psmisc, sed, tar, unzip, util-linux, wget.



Posted on 17 Feb 2010, 8:41 by BarryK
More Busybox rollback
Sigh, bzip2 also...

Cut down:
bc_cut

Removed:
ed, hdparm, less, time

Left alone:
bzip2, coreutils, cpio, diffutils, dosfstools, dpkg-deb, eject, findutils, grep, gzip, ifplugd, ifplugd_old, lzma, mktemp, module-init-tools, ncurses, net-tools, pciutils, procps, psmisc, sed, tar, unzip, util-linux, wget.



Posted on 2 Mar 2010, 8:16 by technosaurus
busybox installer check
Just wanted to follow up on this one last time... I recently compiled busybox from git (incorporating 9 patches to 1.16) and added a script to check for the installed programs before making any symlinks (in case they are installed to a non-standard location - where busybox --install -s won't work) - The commands would still be accessible via busybox <command> or by using the ash shell.

allyesconfig actually disabled some features - so I restored those (dpkg -i, modutils...) and only removed PAM and SELinux related stuff
bbconfig will show the .config options

http://murga-linux.com/puppy/viewtopic.php?mode=attach&id=26850

in woof it would probably need to be named zzbusybox... so that it installs after the full versions are installed?


Posted on 32 Mar 2010, 8:56 by BarryK
patch applet
My brave experiment with Busybox has hit another problem. I had enabled the 'patch' applet, however I have just found that it breaks the T2 build system -- the usual problem, a missing commandline option. So I have gone back to the full 'patch' package.



Posted on 21 Jul 2012, 22:45 by linuxcbon
bash needed
I removed bash and replaced it with busybox, but console won't work anymore and it says "applet not found".


Posted on 21 Jul 2012, 23:07 by linuxcbon
compiling busybox
I read that bash can be replaced by busybox, if busybox is compiled with CONFIG_FEATURE_BASH_IS_ASH enabled.

See http://blog.gmane.org/gmane.linux.busybox/month=20120301



Posted on 21 Jul 2012, 23:59 by Iguleder1
Bash wrapper
You can also use a wrapper, so you don't have to set this feature. Also, it's good because /bin/bash is no longer a symlink :)

#!/bin/busybox ash

exec /bin/busybox ash $@



Posted on 21 Jul 2012, 24:15 by linuxcbon
script works
I read that solution too in the mailing list. I tried it, it works (after the chmod +x). Time to get rid of big bad bash, if it doesn't break scripts.


Posted on 22 Jul 2012, 12:03 by technosaurus
replacing bash
It _will_ break scripts to replace bash with busybox ash. Mostly bash arrays and variables like $RANDOM $PWD... There is a script out there that checks for bashisms (oddly it is in perl iirc), but ash does implement a lot of them, and hush will do some things that ash won't like {,} expanding. I still keep bash 3.2-xx for an in between that supports the majority of useful bashisms. If ash had arrays and you add PWD=$(pwd) to the env via etc/profile (or other rc file) it would get ~90% of them ... Note that $(command) is a bashism too, but backticks are hard to read, but more importantly, not in my Droid's keyboard. If you are still not deterred, see my bashbox thread - its old, but I ported a lot of puppy into a multicall ash script.