# Dual boot multiple versions of FreeBSD



## avner (Oct 13, 2022)

I am currently running 13.1-RELEASE on my server with UFS file system and would like to install 13-stable in a free GPT partition with the idea that I could dual boot by selecting one or the other OS through the UEFI boot menu. During initially install I copied /boot/loader.efi to /EFI/BOOT/BOOTX64.efi on eps partition. I have read the efibootmgr man page but I don't quite get how to tell BOOTX64.efi to use a particular partition as root. I think I can update /etc/fstab to use the new partition and stick with same loader.efi. Another option would be setting currdev= and rootdev=  in loader.conf; would appreciate clarification of listing device name "/dev/mirror/root0" vs "root0". Wonder if there is a better solution and would appreciate some guidance in this regard. Is it recommended to use grub or rEFInd to manage the boot process? 

More info from efibootmgr if it helps:

```
root@daria:/boot # efibootmgr -v
BootCurrent: 0000
Timeout    : 0 seconds
BootOrder  : 0004, 0001, 0003, 0000, 0002, 0005
 Boot0004* FreeBSD HD(1,GPT,f5de00f2-0ca9-11ed-85fc-782bcb02e2bc,0x28,0x82000)/File(\efi\freebsd\loader.efi)
 Boot0001* PLDS DVD+/-RW DS-8A5SH                   PcieRoot(0x0)/Pci(0x1f,0x2)/Ata(Primary,Master,0x0)
 Boot0003  Windows Boot Manager HD(3,GPT,66822183-2af8-484a-ae79-f0ca805e8ea7,0xa800800,0x32000)/File(\EFI\Microsoft\Boot\bootmgfw.efi)
+Boot0000* EFI Fixed Disk Boot Device 1 PcieRoot(0x0)/Pci(0x4,0x0)/Pci(0x0,0x0)/SAS(0x5000c50026029eb5,0x0,0x1,SAS,External,Direct,0,0x0)/HD(13,GPT,5b694d58-217c-11ed-9295-782bcb02e2bc,0x3000028,0x104000)
 Boot0002* EFI Fixed Disk Boot Device 2 PcieRoot(0x0)/Pci(0x4,0x0)/Pci(0x0,0x0)/SAS(0x5000c50026353ced,0x0,0x1,SAS,External,Direct,0,0x0)/HD(13,GPT,5adc7cec-217c-11ed-9295-782bcb02e2bc,0x3000028,0x104000)
 Boot0005* EFI Fixed Disk Boot Device 3 PcieRoot(0x0)/Pci(0x4,0x0)/Pci(0x0,0x0)/SAS(0x5000c50026398b99,0x0,0x1,SAS,External,Direct,0,0x0)/HD(13,GPT,5aa830bc-217c-11ed-9295-782bcb02e2bc,0x3000028,0x104000)
```


----------



## SirDice (Oct 14, 2022)

You know you can use boot environments to boot different versions of FreeBSD? See bectl(8) or beadm(1).


----------



## Crivens (Oct 14, 2022)

Caution, good Sir. These only work with ZFS. OK, that is the default for some time now, but be says he is running UFS.


----------



## rootbert (Oct 14, 2022)

vermadens blog might be useful: https://vermaden.wordpress.com/2021/04/02/ufs-boot-environments/


----------



## Alain De Vos (Oct 14, 2022)

Is your O.S. installed on ZFS or UFS would be a first question.


----------



## Crivens (Oct 14, 2022)

Alain De Vos


avner said:


> I am currently running 13.1-RELEASE on my server with UFS file system


So UFS it seems.


----------



## SirDice (Oct 14, 2022)

Check vermaden 's website on the link rootbert posted. He has found a way to get boot environments with UFS. Don't know if those BEs would show up in the loader menu though but it should certainly be possible to "dual-boot" different versions of FreeBSD that way.


----------



## Crivens (Oct 14, 2022)

Wow, nice. You learn something new any day.


----------



## vermaden (Oct 14, 2022)

SirDice said:


> Check vermaden 's website on the link rootbert posted. He has found a way to get boot environments with UFS. Don't know if those BEs would show up in the loader menu though but it should certainly be possible to "dual-boot" different versions of FreeBSD that way.


There is not loader integration (and its not planned I think) - but at least you can have several systems/roots to play with - priceless with upgrades and major changes.


----------



## T-Daemon (Oct 14, 2022)

vermaden, ufsbe.sh seems broken with UEFI on my test system VM's (VirtualBox), I've checked twice. ufsbe.sh doesn't work with UEFI. It works fine with BIOS.

EDIT: It can't work with UEFI, loader.efi(8) doesn't manage "bootme" attribute as gptboot(8) can. /EDIT

Apparently the activated boot environments loader `currdev=` variable for loader.efi(8) is not exported, it remains the original partitions (disk0p3, which boots the primary root partition).

Also when activating a empty boot environment it gets activated:


----------



## avner (Oct 14, 2022)

Thanks for the advice. Looks like I may have to experiment some  with efibootmgr or maybe use legacy bios booting mode. Will wait until I’m physically closer to server though in case I get locked out.


----------



## T-Daemon (Oct 14, 2022)

avner said:


> I have read the efibootmgr man page but I don't quite get how to tell BOOTX64.efi to use a particular partition as root.


This can be done by creating /boot/efi/efi/freebsd/loader.env and set, i.e `currdev=disk0p3:`



avner said:


> Another option would be setting currdev= and rootdev= in loader.conf


currdev= is sufficient. See loader(8) for difference between the two variables.



avner said:


> would appreciate clarification of listing device name "/dev/mirror/root0" vs "root0".


What do you mean by listing device name?



avner said:


> Wonder if there is a better solution and would appreciate some guidance in this regard. Is it recommended to use grub or rEFInd to manage the boot process?


Grub2 works pretty good with minimal configuration, i.e.:

```
menuentry 'FreeBSD' {
    set root='hd0,gpt3'
    chainloader /boot/loader.efi
}
```

In ports is sysutils/grub2-pcbsd and sysutils/grub2-efi. It's also possible to create a dedicated grub partition, boot a Linux live CD and install there.

Tried rEFInd, but couldn't figure out how to set the root partition to boot from.

One can also escape to loader prompt at the boot menu and set currdev= manually.


----------



## tingo (Oct 14, 2022)

Bug 229770 is relevant here.


----------



## avner (Oct 14, 2022)

T-Daemon said:


> This can be done by creating /boot/efi/efi/freebsd/loader.env and set, i.e `currdev=disk0p3:`
> 
> 
> currdev= is sufficient. See loader(8) for difference between the two variables.
> ...


Thanks for those tips. Particularly regarding loader.env which I was not aware of.


----------



## cy@ (Oct 14, 2022)

Crivens said:


> Caution, good Sir. These only work with ZFS. OK, that is the default for some time now, but be says he is running UFS.


Sadly it does not. In Solaris back in the day one could use bectl on UFS withing Solstice Disksuite.

One can do similar by hand installing the new system to an alternate partition and booting off it next reboot. Any common filesystems would either be ZFS or use UFS labels.


----------



## Phishfry (Oct 15, 2022)

T-Daemon said:


> This can be done by creating /boot/efi/efi/freebsd/loader.env and set, i.e `currdev=disk0p3:`


Don't you have an extra /efi/ directory in there?






						rS346879
					






					reviews.freebsd.org
				








						rG6d162b90b54b
					






					reviews.freebsd.org


----------



## T-Daemon (Oct 15, 2022)

Phishfry said:


> Don't you have an extra /efi/ directory in there?


No, it's correct. The second efi/ is the ESP root directory under the  /boot/efi/ mountpoint.

Partition containing the ESP is mounted on /boot/efi from /etc/fstab. The entry was created by the FreeBSD installer:

```
/dev/nvd0p1        /boot/efi    msdosfs    rw        2    2
```


```
% ls -R /boot/efi/
efi

/boot/efi/efi:
boot    freebsd

/boot/efi/efi/boot:
bootx64.efi

/boot/efi/efi/freebsd:
loader.efi
```
All directories created and files copied by FreeBSD automatic Root-on-ZFS installer.


----------



## avner (Oct 17, 2022)

T-Daemon said:


> This can be done by creating /boot/efi/efi/freebsd/loader.env and set, i.e `currdev=disk0p3:`
> 
> currdev= is sufficient. See loader(8) for difference between the two variables.


As this is a remote headless server I think the loader.env currdev= variable will be the most beneficial for me.

In my case "loader.efi" was renamed to "BOOTX64.efi" and is located in /dev/da0p13:/EFI/BOOT/ ( unmounted ) in other words the full unmounted path to active loader.efi is, "/dev/da0p13:/EFI/BOOT/BOOTX64.efi". Based on the path to the loader.efi the full path to loader.env should be "/dev/da0p13:/EFI/BOOT/efi/freebsd/loader.env" or rather "/dev/da0p13:/EFI/freebsd/loader.env"? Seems like it should be the latter but trying to make sure as it will be a little bit of a drive if I need to reboot server with a USB drive. 

And, if I understand your links regarding the code reviews, this only works in 13-STABLE not 13.1-RELEASE, correct? So I should use loader.efi from 13-STABLE not from 13.-RELEASE?


----------



## T-Daemon (Oct 17, 2022)

avner said:


> As this is a remote headless server I think the loader.env currdev= variable will be the most beneficial for me.


You can set it also in /boot/loader.conf, it makes little difference. Both files are read by the loader, the environments set, before the kernel is booted.


avner said:


> Based on the path to the loader.efi the full path to loader.env should be "/dev/da0p13:/EFI/BOOT/efi/freebsd/loader.env" or rather "/dev/da0p13:/EFI/freebsd/loader.env"?


/dev/da0p13:/EFI/freebsd/loader.env is the correct path.



avner said:


> Seems like it should be the latter but trying to make sure as it will be a little bit of a drive if I need to reboot server with a USB drive.


You can always override currdev specified in a configuration file from the loader prompt (at the boot menu escape to loader prompt, for UFS `set currdev=diskXpX:`



avner said:


> And, if I understand your links regarding the code reviews, this only works in 13-STABLE not 13.1-RELEASE, correct? So I should use loader.efi from 13-STABLE not from 13.-RELEASE?


No, the bug report PR 229770 tingo linked is related to rEFInd. When rootdev= (or currdev=) is set as option in the rEFInd menuentry, the variable is not exported to the loader environment.

The 13.1-RELEASE loader.efi will boot the specified partition just fine.


----------



## avner (Oct 19, 2022)

I did succeed at creating a new boot method using efibootmgr from an esp on a different disk from the one on which the kernel resides. This required setting `rootdev=disk4p16` in /EFI/freebsd/loader.env. UEFI disk4p16 apparently corresponds to /dev/da2p16 as understood by FreeBSD after boot on my machine. The UEFI boot process I think believes this to be the disk in enclosure 4 although on my machine this corresponds to the slot labeled, I think, 1 or 2. I was able to figure out the correct disk number to use by watching the console during the boot process. When UEFI does not find the correct disk at first it cycles through each disk and partition until it finds the correct one and then loads the kernel. By watching the console closely you can catch the last `currdev` tested before boot process continues and that is the correct value to set for `rootdev=`. Also, it may not be good to enclose the value to the right of the = in quotes.


----------

