# How to install to boot from USB with root on disk?



## ffarn (Jun 4, 2010)

How do I set up a USB stick to boot FreeBSD and then mount the hard disk as root?  So basically the USB stick is used just to boot the system.  The system then runs as if it had booted from the internal disk.


----------



## vrachil (Jun 7, 2010)

I think that you have to keep only /boot.
Just remember to set the "vfs.root.mountfrom" variable to your loader.conf file.


----------



## ffarn (Jun 8, 2010)

*Closer, but two problems remain.*

Thanks for the tip.  Here is what I have done so far.  I formatted the usb stick and installed the boot code on it: (I'm working mostly from memory on this part)

```
# bsdlabel -B da1
# newfs /dev/da1a
# mkdir /mnt/usb
# mount /dev/da1a /mnt/usb
# mkdir -p /mnt/usb/boot/kernel
# cp /dist/boot/kernel/kernel /mnt/usb/boot/kernel
# cp /dist/boot/kernel/opensolaris.ko /mnt/usb/boot/kernel
# cp /dist/boot/kernel/zfs.ko /mnt/usb/boot/kernel
```

Then I copied several files from the dist boot directory to my usb stick:

```
# cp /dist/boot/loader /mnt/usb/boot
# cp /dist/boot/loader.* /mnt/usb/boot
# cp /dist/boot/*.4th /mnt/usb/boot
# cat >/mnt/usb/loader.conf
zfs_load="YES"
vfs.root.mountfrom="zfs:zroot"
^D
#
```

Finally, I copied the zfs information to the boot drive:

```
# mkdir /mnt/usb/boot/zfs
# cp /boot/zfs/zpool.cache /mnt/usb/boot/zfs
```

When I reboot, everything seems to work just fine.  The kernel loads, but then it gets to the end and says it can't mount a root filesystem:

```
Loader variables:
vfs.root.mountfrom=
vfs.root.mountfrom.options=

Manual root filesystem specification:
  <fstype>:<device>  Mount <device> using filesystem <fstype>
                       eg. ufs:/dev/da0s1a
                       eg. cd9660:/dev/acd0
                       This is equivalent to: mount -t cd9660 /dev/acd0 /

  ?                  List valid disk boot devices
  <empty line>       Abort manual input
da0 at umass-sim0 bus 0 target 0 lun 0
da0: <SanDisk Cruzer Mini 0.2> Removable Direct Access SCSI-2 device
da0: 40.000MB/s transfers
da0: 244MB (501759 512 byte sectors: 64H 32S/T 244C)

mountroot>
```

The loader variables aren't set up correctly and the zfs module never loaded.  The configuration file isn't getting loaded.  I figured out how to correct this by stopping the loader manually after the kernel loads and loading the configuration file twice:

```
OK read-conf /boot/loader.conf
Warning: syntax error on file /boot/loader.conf
x
 ^
OK read-conf /boot/loader.conf
OK load zfs
/boot/kernel/zfs.ko size 0x18fb40 at 0xe18000
loading required module 'opensolaris'
/boot/kernel/opensolaris.ko size 0x3868 at 0xfa8000
OK boot
```

Again the kernel loads just fine.  It loads ZFS, but then it still gets to the end and says it can't find the root filesystem:

```
...
ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is present;
            to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf.
ZFS filesystem version 13
ZFS storage pool version 13
...
Loader variables:
vfs.root.mountfrom=zfs:zroot
da0 at umass-sim0 bus 0 target 0 lun 0
da0: <SanDisk Cruzer Mini 0.2> Removable Direct Access SCSI-2 device
da0: 40.000MB/s transfers
da0: 244MB (501759 512 byte sectors: 64H 32S/T 244C)
vfs.root.mountfrom.options=

Manual root filesystem specification:
  <fstype>:<device>  Mount <device> using filesystem <fstype>
                       eg. ufs:/dev/da0s1a
                       eg. cd9660:/dev/acd0
                       This is equivalent to: mount -t cd9660 /dev/acd0 /

  ?                  List valid disk boot devices
  <empty line>       Abort manual input

mountroot> zfs:zroot
Trying to mount root from zfs:zroot
ROOT MOUNT ERROR:
If you have invalid mount options, reboot, and first try the following from
the loader prompt:

     set vfs.root.mountfrom.options=rw

and the remove invalid mount options from /etc/fstab.

Loader variables:
...

mountroot>
```

Why does it take two tries to read in the configuration file loader.conf?  Even when I load the zfs module by hand and point it to the root filesystem by hand, why can't it mount root??


----------



## ffarn (Jun 8, 2010)

*Just needed defaults*

Found the problem.  I needed to copy over /boot/defaults/loader.conf

Now the OS boots.

The only problem is that it hangs when the kernel recognizes the USB stick.  I have to remove the stick and then the boot continues:


```
Trying to mount root from zfs:zroot
da0 at umass-sim0 bus 0 target 0 lun 0
da0: <SanDisk Cruzer Mini 0.2> Removable Direct Access SCSI-2 device
da0: 40.000MB/s transfers
da0: 244MB (501759 512 byte sectors: 64H 32S/T 244C
```

The boot then hangs until I pull the USB stick out.  The boot then proceeds:


```
ugen2.2: <SanDisk Corporation> at usbus2 (disconnected)
umass0: at uhub2, port 7, addr 2 (disconnected)
(da0:umass-sim0:0:0:0): lost device
(da0:umass-sim0:0:0:0): Invalidating pack
(da0:umass-sim0:0:0:0): Synchronize cache failed, status == 0xa, scsi status == 0x0
(da0:umass-sim0:0:0:0): removing device entry
Setting hostuuid: ...
```

Is there some sort of flag I'm supposed to set to tell the OS this is a USB stick so it won't do whatever it's trying to do when it hangs?


----------



## vrachil (Jun 8, 2010)

Hmmm. I haven't really seen something reported concearning a usb stick during boot.
What happens if you let the stick in the port for a loooong time?


----------



## PhenomII (Jun 8, 2010)

Greetings,
 I did something along these lines some time ago. When I needed to perform some installs on a PIII that couldn't boot to the usb DVD/RW. So I "baked" the install DVD to a MEM stick.
Forgive me, as this has been awhile, so all the details aren't keen in my mind as I write this. That said; I have the suspicion that you're /etc/fstab isn't consistent with your actual/current setup. If I read your post correctly, you probably have "root overlap" - at least to the extent that you have dao*/boot && da1*/boot. So when fbsd finally mounts your "original" root slice (da1*) "double vision" ensues. This is all purely supposition; as I'm not privy to all your details - your /etc/fstab, and the fact that you posted many of the details "by memory". But odds are, that's the trouble.

HTH
Best wishes.


----------



## ffarn (Jun 8, 2010)

*fstab is empty*

/etc/fstab is empty on both the real root filesystem (ZFS on harddisk) and on the boot-only (USB memstick) filesystem.  If the files are empty does that create some kind of implied conflict?

Also, when I boot from the installation USB memstick, if I put my boot memstick in the other USB socket the boot hangs at exactly the same point, and is fixed in exactly the same way.  Pulling the boot usb out and then reinstalling it once the computer has booted past that part resolves the problem.

Thanks for the help.

I did find that USB filesystems should be created with "soft updates" enabled.  Don't know if that will help.


----------

