# Delay during gptzfsboot/mirror on USB sticks



## aspoon (May 8, 2012)

After reading the RootOnZFS/GPTZFSBoot/Mirror wiki page, I installed the 9.0-RELEASE-amd64 on a pair of USB sticks with swap inside the ZFS and my own set of ZFS filesystems.  (This was done on an Intel Atom N330 motherboard, if it matters)

However, every time it boots, it takes around ten+ minutes to get the kernel loaded.  It would be stuck at the spinning prompt after BTX Loader (with the Drives C:, D:, ... shown), with one USB stick being constantly accessed.  Around five minutes later, the other USB stick would be constantly accessed for another five minutes or so.  After the whole ordeal, the kernel would be loaded, and the rest of the boot up process would continue without a fuss.

My original impression was that it may have something to do with the -b 34 option when I created the freebsd-boot partition with gpart, (somehow misaligned) since I searched around the 'net and some pages would put the -b 34 in, while some would simply omit that option.

Has anyone experienced this before?  Is there a way to eliminate the delay?

Thanks!


----------



## HarryE (May 10, 2012)

*Try different computers*

It seems to be a BIOS USB access problem. (USB 1.1 booting may be very slow). Try to boot the same stick on different computers. Post the results.


----------



## aspoon (Sep 14, 2012)

Just a _very late_ update... I tried turning off USB legacy in BIOS, and the USB drives are not seen by BIOS (not bootable at all)

Apparently I need USB legacy enabled to boot a USB 2.0 flash drive...


----------



## wblock@ (Sep 14, 2012)

-b 34 is not required.  On a GPT-partitioned drive, the first 34 blocks are used for the partition table.  gpart(8) will use the next available block when you create a partition, so there really isn't much point to using -b 34.

Similarly, it's not an alignment problem.  I recall reading that USB boot was slow because it reads single blocks, which USB doesn't do quickly.  With ZFS, there is more to load before it can boot and start reading big chunks at a time.


----------



## aspoon (Sep 15, 2012)

@HarryE, I tried booting on a different machine, it's still slow.

@wblock, thanks.  I'll remember that next time I create another one.


----------



## HarryE (Sep 16, 2012)

How fast the stock 9.0-RELEASE-amd64 usb install image boots on the same computer, from the same stick? 

I get, on the same machine, with different generations of Kingston sticks, different boot times (btx spinning cursor) up to 20 seconds, but never such long boots. 
I'd try different USB stick manufacturer.
Some Atom BIOSes (incl. my D525TUD) have a hard time booting from GPT, try also the MBR partition scheme.


----------



## Sebulon (Sep 19, 2012)

IÂ´m thinking, this:
ZFS partition probing causing long delay at BTX loader

The GPT ZFS-loader was recently changed from probing 4 partitions to 128 for booting, even if the disk only has two partitions... Imagine what this did to storage systems with 50 disks(or more) attached; thatÂ´s 50*128 of the delay it takes to probe at BTX. So, to change back this behavior:


```
Phase 1.
[CMD="#"]sed -i '' 's/128/4/' /sys/boot/zfs/zfs.c[/CMD]

Phase 2.
Rebuild kernel/world

Phase 3.
Profit!
```

And booting is back to normal. This is something I always have to do with newly installed storage servers with FreeBSD-9.0-RELEASE to get back to a normal reboot time.

/Sebulon


----------



## wblock@ (Sep 19, 2012)

It doesn't seem right to check partitions that don't even exist.  Please enter a PR for this.


----------



## Sebulon (Sep 20, 2012)

Done!

http://www.freebsd.org/cgi/query-pr.cgi?pr=171805

When you click the link, it cannot find any PR matching my ID, but IÂ´m guessing it just takes a while for it to show up.

/Sebulon

EDIT: Yepp, now I see it


----------



## Sebulon (Sep 20, 2012)

OK,

so IÂ´ve been in contact with a developer that says that the problem lies rather in "zfsloader" instead of "gptzfsboot". He has made changes that has made booting much faster in 10-HEAD that he hopes to be included in 8.4- and 9.2-RELEASE. So weÂ´ll have to keep doing the sed-edit until then, or possibly installing the 10-HEAD, or shortly 8- and 9-STABLE version of "zfsloader" over 8.>4 or 9.>2, or just track STABLE all together.

/Sebulon


----------



## aspoon (Mar 22, 2014)

Just a very late update... installed 10.0-RELEASE via stock installer with mirrored ZFS over MBR on a pair of USB sticks.  Works like a charm!  Boot time is almost comparable to the 10.0-RELEASE installer memory stick image it was installed from.

Alvin


----------

