VFAT codepage fixed

There has been some discussion recently on this blog and feedback on the Woof alphas on the forum, about the on-going problem with non-English characters and the vfat filesystem.

The advice given to me, that has been applied to some other puplets, to change the default codepage from 437 to 850 and the iocharset from iso8859-1 to utf8, seems to be not such a good idea -- see my recent post explaining why.

So, I configured the 2.6.29.3 kernel with 437 and iso8859-1, as per all previous official puppies.

Instead, it is better to modify the codepage and iocharset when mounting the vfat partition. This can be done by appropriate parameters to the 'mount' command. Here is explanation from the 'mount' manpage:

codepage=value
Sets the codepage for converting to shortname characters on FAT and VFAT filesystems. By default, codepage 437 is used.
iocharset=value
Character set to use for converting between 8 bit characters and 16 bit Unicode characters. The default is iso8859-1. Long filenames are stored on disk in Unicode format.


What I have done is put some code into /usr/sbin/pmount and /usr/local/bin/drive_all that provides the codepage parameter. I should really have placed this common code elsewhere, but for now it is duplicated in both scripts:

  vfat)

#w476 modify codepage and iocharset...
#note, could also set iocharset=iso8859-2 (default: codepage=437, iocharset=iso8859-1)
CODEPAGE_PARAM=''
if [ -f /etc/keymap ];then #set in /etc/rc.d/rc.country
KEYMAP="`cat /etc/keymap | cut -f 1 -d '.'`"
case $KEYMAP in
de)
CODEPAGE_PARAM=',codepage=850'
;;
esac
fi
mount -t vfat -o shortname=mixed,quiet${CODEPAGE_PARAM} $DODEV /mnt/$DEVNAME #v411 added quiet.
RETVAL1=$?
;;


It is not necessary to create an entry in /etc/fstab, the above code should take care of the situation automatically.

However, I need help. As you can see, it only tests for "de", which is fine for MU. For anyone interested in this topic, could you look at the Keyboard Wizard in the Setup menu and figure out what codepages are appropriate for the other keyboard layouts?

The default is 437, which is fine for USA/Canada/Australia. I think 850 is fine for Western European countries? Eastern European keyboard layout need a different codepage?

If I can get some help on this, I can flesh out that case statement. For example, if Denmark and France keyboards are to also have codepage 850:

case $KEYMAP in
de|dk|fr)
CODEPAGE_PARAM=',codepage=850'
;;



Posted on 21 May 2009, 9:30


Comments:

Posted on 22 May 2009, 8:30 by Leon
Problem mounting vfat partition with upupsave.2fs file
I modified /usr/sbin/pmount and /usr/local/bin/drive_all and this code works for sl_SI locale:

case $KEYMAP in
slovene)
CODEPAGE_PARAM=',codepage=852,iocharset=iso8859-2'


It works in Puppy 4.1.2 and upup-473.

Problem mounting vfat partition with upupsave.2fs
My upupsave.2fs is located on vfat partition and I modified the init script and initrd.gz file to properly mount it.
I changed the line 687

from:
mntfunc -t $PUPSAVEFS -o noatime /dev/$PUPSAVEDEV /mnt/dev_save

to:
mntfunc -t $PUPSAVEFS -o noatime,shortname=mixed,codepage=852,iocharset=iso8859-2 /dev/$PUPSAVEDEV /mnt/dev_save

When rebooted the boot process ended with syncing error and kernel panic after

Loading the upup_473.sfs main file...

boot information message was displayed.

Information that you search for

National language keyboard types and SBCS code pages
See the last column: IBM Personal Computer SBCS code page
http://publib.boulder.ibm.com/iseries/v5r1/ic2924/index.htm?info/nls/rbagsnatlangkeybrdtype.htm

ISO/IEC 8859
See: The Parts of ISO/IEC 8859
http://en.wikipedia.org/wiki/ISO_8859



Posted on 22 May 2009, 13:01 by BarryK
iso8859-2 iocharset
Leon,
Thanks for your feedback.

I think the problem at bootup is because the nls_iso8859-2.ko module is not there. I'll see if I can get that fixed for alpha9.



Posted on 22 May 2009, 15:25 by pakt
Good 'ol OS/2 handbook for keyboards and code pages
This little booklet for IBM's OS/2 Warp (remember that ) sums up international keyboards and code pages. It may not be up-to-date though
http://www.borgendale.com/keyboard.pdf


Posted on 22 May 2009, 19:02 by BarryK
init and vfat
Leon,
I have just done a quick thing for alpha9, as I don't have much time. The 'init' script will recognise the 'pkeys' kernel boot parameter, so if you put "pkeys=slovene" then vfat partition will mount with the correct codepage and iocharset.

It needs further attention though, that I will look at later. For example, 'pkeys' was only intended for the "old 2-letter" keyboard layout system, which has less choices. Perhaps the time has come to dump that old system.

For alpha9, if you use "pkeys=slovene", which is not one of the old 2-letter codes, then it doesn't work later on in the rc.country script.



Posted on 22 May 2009, 20:57 by Leon
'pkeys' kernel boot parameter
Barry,
Thanks for fixing the 'init' script.

Boot parameter seems to be a good choice to keep the filenames display settings independent of other locale settings. Someone that not use locale specific character in filenames might want to keep default kernel settings.

Properly displayed filenames are important for me because I share my data files among Windows 98 SE, Windows Vista and Puppy.

The final solution is near and it's worth the efforts.