site  contact  subhomenews

Scanning USB drives in the initrd

November 24, 2015 — BarryK
Years ago, this was an area of difficulty as we tried to shave seconds off the boot time.

In the 'init' script in the initrd, I was parsing the kernel log (using dmesg) to ascertain when all plugged-in drives were available. The relevant information got removed from the kernel, so I restored it, with my "usb-storage" patch, that I still use with latest kernels.

I decided to take a fresh look at this. Booting Quirky on a USB stick, with kernel 4.2.6, I logged devices available in /sys/block and usb-storage information from the kernel log, in a 0.25 second loop:
LOOP CNT: 1

sys/block: sda sr0
dmesg:
[ 3.391499] usbcore: registered new interface driver usb-storage
TAIL: [ 3.413954] Freeing unused kernel memory: 820K (ffffffff816eb000 - ffffffff817b8000)

LOOP CNT: 2
sys/block: sda sr0
dmesg:
[ 3.391499] usbcore: registered new interface driver usb-storage
TAIL: [ 3.428840] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0

LOOP CNT: 3
sys/block: sda sr0
dmesg:
[ 3.391499] usbcore: registered new interface driver usb-storage
TAIL: [ 3.822649] hub 2-1:1.0: 8 ports detected

LOOP CNT: 4
sys/block: sda sr0
dmesg:
[ 3.391499] usbcore: registered new interface driver usb-storage
TAIL: [ 3.822649] hub 2-1:1.0: 8 ports detected

LOOP CNT: 5
sys/block: sda sr0
dmesg:
[ 3.391499] usbcore: registered new interface driver usb-storage
TAIL: [ 4.251601] usb 2-1.2: new high-speed USB device number 4 using ehci-pci

LOOP CNT: 6
sys/block: sda sr0
dmesg:
[ 3.391499] usbcore: registered new interface driver usb-storage
[ 4.347139] usb-storage 2-1.2:1.0: USB Mass Storage device detected
[ 4.347240] usb-storage: device found at 4
[ 4.347257] scsi host6: usb-storage 2-1.2:1.0
[ 4.347308] usb-storage: waiting for device to settle before scanning
TAIL: [ 4.446489] psmouse serio1: synaptics: queried max coordinates: x [..5772], y [..5086]

LOOP CNT: 7
sys/block: sda sr0
dmesg:
[ 3.391499] usbcore: registered new interface driver usb-storage
[ 4.347139] usb-storage 2-1.2:1.0: USB Mass Storage device detected
[ 4.347240] usb-storage: device found at 4
[ 4.347257] scsi host6: usb-storage 2-1.2:1.0
[ 4.347308] usb-storage: waiting for device to settle before scanning
[ 4.511473] usb-storage 2-1.3:1.0: USB Mass Storage device detected
[ 4.511550] usb-storage: device found at 5
[ 4.511576] scsi host7: usb-storage 2-1.3:1.0
[ 4.511697] usb-storage: waiting for device to settle before scanning
TAIL: [ 4.634617] input: SynPS/2 Synaptics TouchPad as /devices/platform/i8042/serio1/input/input2

LOOP CNT: 8
sys/block: sda sr0
dmesg:
[ 3.391499] usbcore: registered new interface driver usb-storage
[ 4.347139] usb-storage 2-1.2:1.0: USB Mass Storage device detected
[ 4.347240] usb-storage: device found at 4
[ 4.347257] scsi host6: usb-storage 2-1.2:1.0
[ 4.347308] usb-storage: waiting for device to settle before scanning
[ 4.511473] usb-storage 2-1.3:1.0: USB Mass Storage device detected
[ 4.511550] usb-storage: device found at 5
[ 4.511576] scsi host7: usb-storage 2-1.3:1.0
[ 4.511697] usb-storage: waiting for device to settle before scanning
TAIL: [ 4.634617] input: SynPS/2 Synaptics TouchPad as /devices/platform/i8042/serio1/input/input2

LOOP CNT: 9
sys/block: sda sr0
dmesg:
[ 3.391499] usbcore: registered new interface driver usb-storage
[ 4.347139] usb-storage 2-1.2:1.0: USB Mass Storage device detected
[ 4.347240] usb-storage: device found at 4
[ 4.347257] scsi host6: usb-storage 2-1.2:1.0
[ 4.347308] usb-storage: waiting for device to settle before scanning
[ 4.511473] usb-storage 2-1.3:1.0: USB Mass Storage device detected
[ 4.511550] usb-storage: device found at 5
[ 4.511576] scsi host7: usb-storage 2-1.3:1.0
[ 4.511697] usb-storage: waiting for device to settle before scanning
TAIL: [ 4.634617] input: SynPS/2 Synaptics TouchPad as /devices/platform/i8042/serio1/input/input2

LOOP CNT: 10
sys/block: sda sr0
dmesg:
[ 3.391499] usbcore: registered new interface driver usb-storage
[ 4.347139] usb-storage 2-1.2:1.0: USB Mass Storage device detected
[ 4.347240] usb-storage: device found at 4
[ 4.347257] scsi host6: usb-storage 2-1.2:1.0
[ 4.347308] usb-storage: waiting for device to settle before scanning
[ 4.511473] usb-storage 2-1.3:1.0: USB Mass Storage device detected
[ 4.511550] usb-storage: device found at 5
[ 4.511576] scsi host7: usb-storage 2-1.3:1.0
[ 4.511697] usb-storage: waiting for device to settle before scanning
TAIL: [ 5.348271] scsi 6:0:0:0: CD-ROM TSSTcorp CDDVDW SE-208DB TS01 PQ: 0 ANSI: 0

LOOP CNT: 11
sys/block: sda sr0 sr1
dmesg:
[ 3.391499] usbcore: registered new interface driver usb-storage
[ 4.347139] usb-storage 2-1.2:1.0: USB Mass Storage device detected
[ 4.347240] usb-storage: device found at 4
[ 4.347257] scsi host6: usb-storage 2-1.2:1.0
[ 4.347308] usb-storage: waiting for device to settle before scanning
[ 4.511473] usb-storage 2-1.3:1.0: USB Mass Storage device detected
[ 4.511550] usb-storage: device found at 5
[ 4.511576] scsi host7: usb-storage 2-1.3:1.0
[ 4.511697] usb-storage: waiting for device to settle before scanning
[ 5.355526] usb-storage: device scan complete
TAIL: [ 5.355526] usb-storage: device scan complete

LOOP CNT: 12
sys/block: sda sr0 sr1
dmesg:
[ 3.391499] usbcore: registered new interface driver usb-storage
[ 4.347139] usb-storage 2-1.2:1.0: USB Mass Storage device detected
[ 4.347240] usb-storage: device found at 4
[ 4.347257] scsi host6: usb-storage 2-1.2:1.0
[ 4.347308] usb-storage: waiting for device to settle before scanning
[ 4.511473] usb-storage 2-1.3:1.0: USB Mass Storage device detected
[ 4.511550] usb-storage: device found at 5
[ 4.511576] scsi host7: usb-storage 2-1.3:1.0
[ 4.511697] usb-storage: waiting for device to settle before scanning
[ 5.355526] usb-storage: device scan complete
TAIL: [ 5.355526] usb-storage: device scan complete

LOOP CNT: 13
sys/block: sda sdb sr0 sr1
dmesg:
[ 3.391499] usbcore: registered new interface driver usb-storage
[ 4.347139] usb-storage 2-1.2:1.0: USB Mass Storage device detected
[ 4.347240] usb-storage: device found at 4
[ 4.347257] scsi host6: usb-storage 2-1.2:1.0
[ 4.347308] usb-storage: waiting for device to settle before scanning
[ 4.511473] usb-storage 2-1.3:1.0: USB Mass Storage device detected
[ 4.511550] usb-storage: device found at 5
[ 4.511576] scsi host7: usb-storage 2-1.3:1.0
[ 4.511697] usb-storage: waiting for device to settle before scanning
[ 5.355526] usb-storage: device scan complete
[ 5.838795] usb-storage: device scan complete
TAIL: [ 5.859375] sd 7:0:0:0: [sdb] Attached SCSI removable disk
<

...the dmesg entries with "usb-storage:" are due to my patch.

I had a USB2 Flash drive plugged in, and a USB2 DVD drive. It took a tad over two seconds for them to become available.

I still don't have an efficient alternative to parsing of the kernel log (and my usb-storage patch). Parsing /sys/devices is very inefficient. There is the possibility of monitoring kernel uevents, with a daemon, however the simple initrd in Quirky does not have that.

This is how I have now coded the 'init' script in the initrd:
if [ -f /BOOT_SPECS ];then

. /BOOT_SPECS
mkdir /partition
if [ "${BOOT_DEV:0:4}" == "UUID" ];then #151122. 151124 scan-loop
echo 'Scanning USB drives...'
sleep 1
while [ 1 ];do #151124
sleep 0.25
DMESG="$(dmesg)"
CNTfound=$(echo "$DMESG" | grep 'usb-storage: device found at ' | wc -l)
CNTscanned=$(echo "$DMESG" | grep 'usb-storage: device scan complete' | wc -l)
[ $CNTscanned -ge $CNTfound ] && break
done
xBOOT_DEV="$(blkid | grep "\"${BOOT_DEV:5}\"" | cut -f 1 -d ':' | cut -f 3 -d '/')"
else
xBOOT_DEV="$BOOT_DEV"
fi
mount -o rw -t $BOOT_FS /dev/$xBOOT_DEV /partition
QPATH="/partition/${BOOT_FOLDER}"
fi
<

...this is just a section of 'init', so difficult to grasp the big picture. But, BOOT_DEV is a parameter that can specify a partition, for example "sdb2", or be of the format:
BOOT_DEV=UUID=1234-4567-5678-0921
...which is the same syntax that the kernel uses.

Of course, that waiting loop could just be replaced with a sleep command, "sleep 3" should be enough.

Comments

I found an old thread where I discussed this problem with kernel developers:
http://marc.info/?l=linux-usb&m=132995909227440&w=2

They weren't understanding my problem, so at the time I just politely bowed out. Fairly politely anyway.

Back then, what Greg was saying, about waiting for the required device node to appear, is not appropriate, as we don't know what USB devices are plugged in --back in the Puppy days, we had to wait for them all to become available then mount each one to find the Puppy files, maybe the save-file, where we might have booted off a live-CD.

So, Greg wasn't getting it, and that other fellow, Alan, was just (in my opinion) a <deleted>.

But yeah, the idea of using udev is probably the most proper solution, or at least the little "hotplug" utility that we used to use for monitoring kernel uevents.

Another point, despite their feedback that relying on "scan complete" in the kernel log is not reliable, in every single test that I have done, it always has been OK.

Another thing to bare in mind, is that even when a device node appears, it does not necessarily mean the drive is ready to be mounted. That was something we found years ago with USB optical drives, it needed a few more seconds before 'mount' worked.

It probably can be parsed to detect USB drives, and wait for all to be available, much the same as my code above does.

My Quirky situation though, where I want a particular UUID, this is not provided in a uevent. So I would have to use 'blkid' still. Can't see how to find UUID in /sys.
I don't have /dev/disk/by-uuid, don't know how to get that, must be a kernel config option.

In this particular Quirky case, I don't have to wait for all USB drives to become available, only the one matching the UUID.
However, the situation with Puppy is that we do have to wait for all drives. I am also thinking of some situations in Quirky where I will have to wait for all drives.

Tags: linux