Scanning USB drives in the initrd

Years ago, this was an area of difficulty as we tried to shave seconds off the boot time.

In the 'init' script in the initrd, I was parsing the kernel log (using dmesg) to ascertain when all plugged-in drives were available. The relevant information got removed from the kernel, so I restored it, with my "usb-storage" patch, that I still use with latest kernels.

I decided to take a fresh look at this. Booting Quirky on a USB stick, with kernel 4.2.6, I logged devices available in /sys/block and usb-storage information from the kernel log, in a 0.25 second loop:

LOOP CNT: 1

sys/block: sda sr0 

dmesg: 

[    3.391499] usbcore: registered new interface driver usb-storage

TAIL: [    3.413954] Freeing unused kernel memory: 820K (ffffffff816eb000 - ffffffff817b8000)



LOOP CNT: 2

sys/block: sda sr0 

dmesg: 

[    3.391499] usbcore: registered new interface driver usb-storage

TAIL: [    3.428840] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0



LOOP CNT: 3

sys/block: sda sr0 

dmesg: 

[    3.391499] usbcore: registered new interface driver usb-storage

TAIL: [    3.822649] hub 2-1:1.0: 8 ports detected



LOOP CNT: 4

sys/block: sda sr0 

dmesg: 

[    3.391499] usbcore: registered new interface driver usb-storage

TAIL: [    3.822649] hub 2-1:1.0: 8 ports detected



LOOP CNT: 5

sys/block: sda sr0 

dmesg: 

[    3.391499] usbcore: registered new interface driver usb-storage

TAIL: [    4.251601] usb 2-1.2: new high-speed USB device number 4 using ehci-pci



LOOP CNT: 6

sys/block: sda sr0 

dmesg: 

[    3.391499] usbcore: registered new interface driver usb-storage

[    4.347139] usb-storage 2-1.2:1.0: USB Mass Storage device detected

[    4.347240] usb-storage: device found at 4

[    4.347257] scsi host6: usb-storage 2-1.2:1.0

[    4.347308] usb-storage: waiting for device to settle before scanning

TAIL: [    4.446489] psmouse serio1: synaptics: queried max coordinates: x [..5772], y [..5086]



LOOP CNT: 7

sys/block: sda sr0 

dmesg: 

[    3.391499] usbcore: registered new interface driver usb-storage

[    4.347139] usb-storage 2-1.2:1.0: USB Mass Storage device detected

[    4.347240] usb-storage: device found at 4

[    4.347257] scsi host6: usb-storage 2-1.2:1.0

[    4.347308] usb-storage: waiting for device to settle before scanning

[    4.511473] usb-storage 2-1.3:1.0: USB Mass Storage device detected

[    4.511550] usb-storage: device found at 5

[    4.511576] scsi host7: usb-storage 2-1.3:1.0

[    4.511697] usb-storage: waiting for device to settle before scanning

TAIL: [    4.634617] input: SynPS/2 Synaptics TouchPad as /devices/platform/i8042/serio1/input/input2



LOOP CNT: 8

sys/block: sda sr0 

dmesg: 

[    3.391499] usbcore: registered new interface driver usb-storage

[    4.347139] usb-storage 2-1.2:1.0: USB Mass Storage device detected

[    4.347240] usb-storage: device found at 4

[    4.347257] scsi host6: usb-storage 2-1.2:1.0

[    4.347308] usb-storage: waiting for device to settle before scanning

[    4.511473] usb-storage 2-1.3:1.0: USB Mass Storage device detected

[    4.511550] usb-storage: device found at 5

[    4.511576] scsi host7: usb-storage 2-1.3:1.0

[    4.511697] usb-storage: waiting for device to settle before scanning

TAIL: [    4.634617] input: SynPS/2 Synaptics TouchPad as /devices/platform/i8042/serio1/input/input2



LOOP CNT: 9

sys/block: sda sr0 

dmesg: 

[    3.391499] usbcore: registered new interface driver usb-storage

[    4.347139] usb-storage 2-1.2:1.0: USB Mass Storage device detected

[    4.347240] usb-storage: device found at 4

[    4.347257] scsi host6: usb-storage 2-1.2:1.0

[    4.347308] usb-storage: waiting for device to settle before scanning

[    4.511473] usb-storage 2-1.3:1.0: USB Mass Storage device detected

[    4.511550] usb-storage: device found at 5

[    4.511576] scsi host7: usb-storage 2-1.3:1.0

[    4.511697] usb-storage: waiting for device to settle before scanning

TAIL: [    4.634617] input: SynPS/2 Synaptics TouchPad as /devices/platform/i8042/serio1/input/input2



LOOP CNT: 10

sys/block: sda sr0 

dmesg: 

[    3.391499] usbcore: registered new interface driver usb-storage

[    4.347139] usb-storage 2-1.2:1.0: USB Mass Storage device detected

[    4.347240] usb-storage: device found at 4

[    4.347257] scsi host6: usb-storage 2-1.2:1.0

[    4.347308] usb-storage: waiting for device to settle before scanning

[    4.511473] usb-storage 2-1.3:1.0: USB Mass Storage device detected

[    4.511550] usb-storage: device found at 5

[    4.511576] scsi host7: usb-storage 2-1.3:1.0

[    4.511697] usb-storage: waiting for device to settle before scanning

TAIL: [    5.348271] scsi 6:0:0:0: CD-ROM            TSSTcorp CDDVDW SE-208DB  TS01 PQ: 0 ANSI: 0



LOOP CNT: 11

sys/block: sda sr0 sr1 

dmesg: 

[    3.391499] usbcore: registered new interface driver usb-storage

[    4.347139] usb-storage 2-1.2:1.0: USB Mass Storage device detected

[    4.347240] usb-storage: device found at 4

[    4.347257] scsi host6: usb-storage 2-1.2:1.0

[    4.347308] usb-storage: waiting for device to settle before scanning

[    4.511473] usb-storage 2-1.3:1.0: USB Mass Storage device detected

[    4.511550] usb-storage: device found at 5

[    4.511576] scsi host7: usb-storage 2-1.3:1.0

[    4.511697] usb-storage: waiting for device to settle before scanning

[    5.355526] usb-storage: device scan complete

TAIL: [    5.355526] usb-storage: device scan complete



LOOP CNT: 12

sys/block: sda sr0 sr1 

dmesg: 

[    3.391499] usbcore: registered new interface driver usb-storage

[    4.347139] usb-storage 2-1.2:1.0: USB Mass Storage device detected

[    4.347240] usb-storage: device found at 4

[    4.347257] scsi host6: usb-storage 2-1.2:1.0

[    4.347308] usb-storage: waiting for device to settle before scanning

[    4.511473] usb-storage 2-1.3:1.0: USB Mass Storage device detected

[    4.511550] usb-storage: device found at 5

[    4.511576] scsi host7: usb-storage 2-1.3:1.0

[    4.511697] usb-storage: waiting for device to settle before scanning

[    5.355526] usb-storage: device scan complete

TAIL: [    5.355526] usb-storage: device scan complete



LOOP CNT: 13

sys/block: sda sdb sr0 sr1 

dmesg: 

[    3.391499] usbcore: registered new interface driver usb-storage

[    4.347139] usb-storage 2-1.2:1.0: USB Mass Storage device detected

[    4.347240] usb-storage: device found at 4

[    4.347257] scsi host6: usb-storage 2-1.2:1.0

[    4.347308] usb-storage: waiting for device to settle before scanning

[    4.511473] usb-storage 2-1.3:1.0: USB Mass Storage device detected

[    4.511550] usb-storage: device found at 5

[    4.511576] scsi host7: usb-storage 2-1.3:1.0

[    4.511697] usb-storage: waiting for device to settle before scanning

[    5.355526] usb-storage: device scan complete

[    5.838795] usb-storage: device scan complete

TAIL: [    5.859375] sd 7:0:0:0: [sdb] Attached SCSI removable disk

<

...the dmesg entries with "usb-storage:" are due to my patch.

I had a USB2 Flash drive plugged in, and a USB2 DVD drive. It took a tad over two seconds for them to become available.

I still don't have an efficient alternative to parsing of the kernel log (and my usb-storage patch). Parsing /sys/devices is very inefficient. There is the possibility of monitoring kernel uevents, with a daemon, however the simple initrd in Quirky does not have that.

This is how I have now coded the 'init' script in the initrd:

if [ -f /BOOT_SPECS ];then

. /BOOT_SPECS

 mkdir /partition

 if [ "${BOOT_DEV:0:4}" == "UUID" ];then #151122. 151124 scan-loop

  echo 'Scanning USB drives...'

  sleep 1

  while [ 1 ];do #151124

   sleep 0.25

   DMESG="$(dmesg)"

   CNTfound=$(echo "$DMESG" | grep 'usb-storage: device found at ' | wc -l)

   CNTscanned=$(echo "$DMESG" | grep 'usb-storage: device scan complete' | wc -l)

   [ $CNTscanned -ge $CNTfound ] && break

  done

  xBOOT_DEV="$(blkid | grep "\"${BOOT_DEV:5}\"" | cut -f 1 -d ':' | cut -f 3 -d '/')"

 else

  xBOOT_DEV="$BOOT_DEV"

 fi

 mount -o rw -t $BOOT_FS /dev/$xBOOT_DEV /partition

 QPATH="/partition/${BOOT_FOLDER}"

fi

<

...this is just a section of 'init', so difficult to grasp the big picture. But, BOOT_DEV is a parameter that can specify a partition, for example "sdb2", or be of the format:
BOOT_DEV=UUID=1234-4567-5678-0921
...which is the same syntax that the kernel uses.

Of course, that waiting loop could just be replaced with a sleep command, "sleep 3" should be enough.

---

Comments

I found an old thread where I discussed this problem with kernel developers:
http://marc.info/?l=linux-usb&m=132995909227440&w=2

They weren't understanding my problem, so at the time I just politely bowed out. Fairly politely anyway.

Back then, what Greg was saying, about waiting for the required device node to appear, is not appropriate, as we don't know what USB devices are plugged in --back in the Puppy days, we had to wait for them all to become available then mount each one to find the Puppy files, maybe the save-file, where we might have booted off a live-CD.

So, Greg wasn't getting it, and that other fellow, Alan, was just (in my opinion) a <deleted>.

But yeah, the idea of using udev is probably the most proper solution, or at least the little "hotplug" utility that we used to use for monitoring kernel uevents.

Another point, despite their feedback that relying on "scan complete" in the kernel log is not reliable, in every single test that I have done, it always has been OK.

Another thing to bare in mind, is that even when a device node appears, it does not necessarily mean the drive is ready to be mounted. That was something we found years ago with USB optical drives, it needed a few more seconds before 'mount' worked.

It probably can be parsed to detect USB drives, and wait for all to be available, much the same as my code above does.

My Quirky situation though, where I want a particular UUID, this is not provided in a uevent. So I would have to use 'blkid' still. Can't see how to find UUID in /sys.
I don't have /dev/disk/by-uuid, don't know how to get that, must be a kernel config option.

In this particular Quirky case, I don't have to wait for all USB drives to become available, only the one matching the UUID.
However, the situation with Puppy is that we do have to wait for all drives. I am also thinking of some situations in Quirky where I will have to wait for all drives.

Tags: linux