# Why it's important to upgrade /efi/boot/BOOTx64.efi



## Emrion (Mar 24, 2021)

I noticed last week on a VM (FreeBSD as guest on VirtualBox, efi boot) that the boot environment didn't work when you select it on the beasty menu. The kernel runs until it comes to mount root. Then, you read "internal error: failed to initialize ZFS library". After that, it seems that the ZFS root filesystem remains read-only. You can imagine easily that it crashes soon after because there is no device in /dev for example.

But, if you activate a BE with beadm (for instance) and reboot, all is working well.
I even experimented no boot at all with efi but cannot tell the exact circonstances.

I eventually understood where the problem lies. To tell the story briefly, I came from FreeBSD 12.0 (or before) and never updated this essential efi file, I mean /efi/boot/BOOTx64.efi.

My previous BOOTx64. efi: 393216 bytes.
The 12.1 loader.efi: 460800 bytes (no problem with 12.2).
The 12.2 loader.efi: 488960 bytes.

So, at each OS upgrade, even minor, do something like that:
`mount -t msdosfs /dev/drive-name-partition /mnt
cp /boot/loader.efi /mnt/efi/boot/BOOTx64.efi`

_drive-name-partition_ is often ada0p1 but you have to look where the efi partition lies with `gpart show`.


----------



## scottro (Mar 25, 2021)

In a VM, I did a quick 12.2 install. I then upgraded with freebsd-update. (Note that this was just a single ZFS virtual drive, not a mirror or other, more sophisticated setup.)  It was able to boot. I also ran zpool upgrade -a as 13 is using ZFS 2.  I had wondered, due to various posts I've seen here as well as some information in release notes, if this would work without me doing anything to the efi loader. 








						FreeBSD 13.0-RELEASE Release Notes
					

FreeBSD is an operating system used to power modern servers, desktops, and embedded platforms.




					www.freebsd.org
				




So far, on one VM with EFI boot (in the VM--VirtualBox to be specific, running on CentOS 8.3 stream, version 6.1.18, and another test, on bare metal, but again a single drive laptop, upgrading from 12.2 to 13.0 and then upgrading the zpool has given me no problems. These are very simple setups, just for testing, but note that 13.0 release notes also mention the boot loader on efi. (See link above).


----------



## Emrion (Mar 25, 2021)

The problem I was talking about is very vicious. I think I kept the efi loader of the 12.0-RELEASE which worked perfectly on 12.1 but not on 12.2. And you see the problem only if you try to activate a BE with the beasty menu (thing that most people don't do often).

I tried to understand the trouble with the installation of a 12.1-RELEASE in a VM. Updated it and then, upgraded to 12.2 (without changing the efi loader). I made several BEs before upgrading zpool and after. All was working, no problem (including starting from the beasty menu)... Why? Because the efi loader of the 12.1-RELEASE works on the 12.2.

So, in this case, the efi loader is compatible with the version+1 but not with version+2. That means if you don't update this loader at each upgrade, something will bite you later.

The problem is also due to that `freebsd-update` tells you to update the freebsd-boot partition even if your boot method is UEFI and even if you don't have one. But, in no way, it talks about the update of BOOTx64.efi.


----------



## Argentum (Mar 25, 2021)

Emrion said:


> The problem I was talking about is very vicious. I think I kept the efi loader of the 12.0-RELEASE which worked perfectly on 12.1 but not on 12.2. And you see the problem only if you try to activate a BE with the beasty menu (thing that most people don't do often).


And you can prevent this failure by upgrading the loader first. This desktop, I am writing this message on, has 12.2, but I have already *upgraded the EFI loader to 13.0-RC3*. This is backward compatible. And the file size is *896000*. Almost twice as big and needs a bigger partition if you still have an old 800K EFI partition.

It works very well with 12.2 and one day I decide to upgrade this machine, I am not struggling with boot issues.

BTW, the loader can be built independently of base and kernel by:

```
cd /usr/src/stand/
make
```


----------



## Emrion (Mar 25, 2021)

Argentum said:


> This is backward compatible.


Is this guarantee? I mean, is it documented somewhere or just the result of your experiments?
Will the 14.0 efi loader work on a 13.x system?

I said that because I had a problem with the gptzfsboot of a 12.0-RELEASE which was incompatible with the 11.2-RELEASE: https://forums.freebsd.org/threads/cant-load-kernel-when-back-to-be-11-2-from-12-0-release.68795/

Normally, you just have to update the efi loader when freebsd-update advise you to update the freebsd-boot partition (which you should do as well if you have the said partition type).


----------



## SirDice (Mar 25, 2021)

Emrion said:


> I said that because I had a problem with the gptzfsboot of a 12.0-RELEASE which was incompatible with the 11.2-RELEASE


11.2 is newer than 12.0. Counter-intuitive I know but a result of the way new versions are released.


----------



## Emrion (Mar 25, 2021)

SirDice said:


> 11.2 is newer than 12.0. Counter-intuitive I know but a result of the way new versions are released.



It seems that this is the case of 11.3−RELEASE but not 11.2. Am I wrong?


> *12.0* (December, 2018) _Announcement: Release Notes: Installation Instructions: Hardware Notes: Readme: Errata:Checksums_
> *11.3* (July, 2019) _Announcement: Release Notes: Installation Instructions: Hardware Notes: Readme: Errata:Checksums_
> *11.2* (June, 2018) _Announcement: Release Notes: Installation Instructions: Hardware Notes: Readme: Errata:Checksums_


----------



## Argentum (Mar 25, 2021)

Emrion said:


> Is this guarantee? I mean, is it documented somewhere or just the result of your experiments?
> Will the 14.0 efi loader work on a 13.x system?


A good question. _Making predictions is a hard job, especially about future_ 

But, as long as *ZFS* is in scope, I think it will be backward compatible. I dug a bit inside the 12.2 loader, rebuilt it with my ownd debug messages,  to find out why it did not boot the upgraded *OpeZFS pool*. And the reason is that loader has a simple stand-alone implementation of ZFS and old implementation does not recognize new pool version. The general policy of ZFS development has been backward compatibility. So, it was not a surprise that new (13.0) loader can boot old system, but old loader in general may not boot new system.


----------



## ct85711 (Mar 25, 2021)

I do recall it's been said numerous times, that you should only upgrade the zfs pool after you know for certain that you are not going to use it on a older system.  It ends up being, it does keep backwards compatibility, there is nothing saying it will keep forward compatability of which it doesn't know anything of (future changes).


----------



## zirias@ (Mar 25, 2021)

Looks like you _can_ recover from a zpool upgrade now by setting a "checkpoint" before you do it, as the loader is able to rewind it?








						Bringing zpool checkpoints to a FreeBSD bootloader
					

Almost two years ago I wrote a blog post about checkpoints in ZFS. I didn’t hide that I was a big fan of them. That said, after those two years, I still feel that there are underappreciated features in the ZFS world, so I decided to do something about that.




					www.oshogbo.vexillium.org
				




Of course, it's still wise not to do it early, rewinding will erase anything done to the pool _after_ the checkpoint…


----------



## Argentum (Mar 26, 2021)

ct85711 said:


> I do recall it's been said numerous times, that you should only upgrade the zfs pool after you know for certain that you are not going to use it on a older system.  It ends up being, it does keep backwards compatibility, there is nothing saying it will keep forward compatability of which it doesn't know anything of (future changes).


This was my *test system,* and I was trying to get the OpenZFS pool booting before 13.0 was out. That was the intention of the experiment.

To get the updated pool back, the easiest and *foolproof* way is  to create an empty lower version pool and using `zfs send` and `zfs receive` move the upgraded pool to the empty lower version pool. Set it bootable and everything is OK. That is what I did with my test system.


----------



## Emrion (Mar 26, 2021)

I always make a checkpoint before upgrading a pool, by principle. But a checkpoint is something you can't keep for too long because it prohibits some zfs functionalities.

Anyway, as I said previously, I experimented a case where zfs backward compatibility wasn't, at least for BIOS booting: 11.2 --> 12.0. So, i won't count on any backward or forward compatibility and, except for testing purposes, no one should.


----------



## grahamperrin@ (Jan 9, 2022)

byrnejb said:


> FreeBSD-12.3 internal error failed to initialize ZFS library





Emrion said:


> Check this: https://forums.freebsd.org/threads/why-its-important-to-upgrade-efi-boot-bootx64-efi.79503/





byrnejb said:


> `FreeBSD vhost05.hamilton.harte-lyne.ca 12.2-RELEASE-p7 FreeBSD 12.2-RELEASE-p7 GENERIC amd64 1202000 1202000`



FreeBSD bug 255318 – handbook: Document how to update the bootloader


----------

