Locale setting is to blame

Well, well, well, this is amazing. I have discovered why scripts are slow in Intrepid.

Firstly, about Debian Lenny. I built the live-CD, and it worked. ROX-Filer did not start as a library is missing, but that is a small detail. It is a testimony to the Woof Unleashed concept that I was so easily able to switch from Ubuntu Intrepid to Debian Lenny.

Having booted Woof-Lenny, the first thing I did was run the 'speed-test' script. Oh dear, 143 seconds. Then I tried something that was in the back of my mind, something that I had noticed in the System Profiler -- it did seem that libc was very busy handling locale lookups. I did this:

# export LANG=C
# ./speed-test

Oh wow, I got 28 seconds!

So, I booted Woof-Intrepid, did the same LANG=C thing and also got 28 seconds.

Then I booted Puppy 4.1.2, which runs the script at 33 seconds, but with LANG=C I got just 22 seconds!

What this means is this:

Locale Time
en_US.utf8 appr. 150
en_US 33
C 22

So, what to do? This affects all our scripts. I moved to utf8 as that is the standard that all major distros have adopted, but it slows our scripts right down. How do we fix this, so that our scripts still run fast? Changing to LANG=C is bad for internationalisation. Perhaps we should stay with non-utf8?

I think that I'll stay with Lenny anyway, as it has Xorg 7.3. It was a bit strong to say the Ubuntu developers have lost the plot -- I would blame the release model, really they should release when the product is ready for release as Debian does, not to a timetable.

Posted on 31 Jan 2009, 22:04


Posted on 31 Jan 2009, 22:20 by John Biles
I Love Debian over ubuntu
Hello Barry K,
While you'll lose some Apps that are in Ubuntu and not Debian due to Debian's GPL Policy, I've found that Debian Packages are far more easy to get working than Ubuntu and I love the idea of Woof now being based around Debian and not Ubuntu.

Now I'm really looking forward to the next test release of Woof

Posted on 31 Jan 2009, 24:00 by John Logan
Another Debian Lover
Hello Barry,
It was puppy which got me really interested in linux a few months ago after being a Windows user pretty much since getting my first pc (i am pushing 40). For the last couple of months I have tried a number of distros before finally settling on Debian as my day to day one. I love puppy and if it could be based on Debian that would be like Xbox games in choclate packaging. Your work astounds me.

Posted on 32 Jan 2009, 2:13 by eprv
speed test
On my Intel D945GCLF Atom based mini itx board
the script is running 63 seconds in puppy 4.1.2
and 81 seconds in xubuntu 8.10
My locale is en_US
It uses the Intel video chip but I had the flicker problem very rarely in Puppy and never In Xubuntu.
Anyway The ubuntu people did bring Linux to the masses so they must be credited for that.

Posted on 32 Jan 2009, 3:12 by labrador
I had libc locale problem before
I found that libc in Puppy 4.x has some bugs with certain locale definitions. It crashes programs which try to fetch gettext strings. After replacing it with libc 2.7 in ubuntu repositories, everything works fine now. I also find another solution -- use the utf8 locale definition in libc 2.3 source code. This sounds absurd, but interestingly it works.

Hope this could help.

Posted on 1 Feb 2009, 16:01 by magerlab
UTF-8 need
it does not sound good for me as i speak russian and
unicode is the only one good solution to have all the characters to be displayed well in every application and translated scripts

Posted on 1 Feb 2009, 17:29 by 01micko
big brother?
Chris will know what I am talking about, maybe some others. I like it Barry.

Posted on 1 Feb 2009, 18:47 by Dougal
Barry, since your scripts are not internationalized, you don't need anything but C for them -- the fancy locales are only used for the user apps. So you can just set "LANG=C" (or "LOCALE=C" or whatever) at the beginning of each of your scripts... if you don't export it it shouldn't affect anything else.

Posted on 10 Feb 2009, 9:44 by VVoodstock69
Speed test
Barry my test results are as follows on a Compaq nx6320 laptop:

LANG=en_AU.UTF-8 (35 sec)

LANG=C (15 sec)

I'm using Linux Mint 5 KDE CE