# Failed to boot after upgrade from 12.2 to 13.0



## dtoch (May 15, 2021)

I have a ASUS RS500-E6-PS4 server.

On the first boot after normal installation 13.0-RELEASE hangs at "FreeBSD/x86 bootstrap loader, Revision 1.1".

After few tries I decided to install 12.2-RELEASE and upgrade to 13.0-RELEASE manually. 12.2 boots fine without any suspicious messages.

After "freebsd-update -R 13.0-RELEASE upgrade" I was asked to reboot and system hangs again.

Root FS is on raidz1.

Any ideas?


----------



## dtoch (Jun 2, 2021)

Ok. Here is additional info.
Did the same but this time upgraded via sources as written at https://docs.freebsd.org/en/books/handbook/cutting-edge/#makeworld
Reboot after 'make buildword' and 'make kernel' is fine. 'uname' shows 13.0-p1 and freebsd-version shows 12.2-p7 but after 'make installworld' and reboot system hangs again.


----------



## covacat (Jun 2, 2021)

try to update gptzfsboot (or the efi boot code)


----------



## mer (Jun 2, 2021)

covacat said:


> try to update gptzfsboot (or the efi boot code)


This is a key bit right here.  13.0-RELEASE is using OpenZFS for the ZFS bits, which is different from 12.2-RELEASE.  
If you have 13 in a different boot environment from your 12.2 you can boot into 12.2, mount the 13.0 BE (bectl or beadm) and then grab gptzfsboot or the efi boot code from the mounted 13.0 BE.


----------



## zirias@ (Jun 2, 2021)

13 (with OpenZFS) works well on older pools. So, as always, installing new bootcode is only required once you upgrade your ZFS pool (which would affect a 12.2 BE as well).


----------



## dtoch (Jun 2, 2021)

Thanks for advice. But...
It's not booting even after clean install on this server.  All this fun with updates is an attempt to figure out problem source.
Anyway. I boot to livecd and run 'gpart bootcode -p /boot/gptzfsboot -i 1 ada0'(ada1,ada2) but OS still stop booting in the same place.

After that booted to livecd again and take a look on my pools. Looks like some problem with my root - no root now.


----------



## covacat (Jun 2, 2021)

try to replace /boot/loader with 12.2 /boot/loader
probably also loader.rc and loader.4th with older versions


----------



## SirDice (Jun 2, 2021)

After an import you'll need to actually mount the filesystems. That doesn't happen automatically. Use `zfs mount <dataset>` to mount specific datasets or just `zfs mount -a` to blindly mount all of them.


----------



## dtoch (Jun 2, 2021)

covacat said:


> try to replace /boot/loader with 12.2 /boot/loader
> probably also loader.rc and loader.4th with older versions


Unfortunately still no luck.


----------



## covacat (Jun 2, 2021)

if you rollback bootcode and loader with all its deps it has to work
you already said it booted with old bootenv and new kernel


----------



## dtoch (Jun 3, 2021)

covacat said:


> if you rollback bootcode and loader with all its deps it has to work
> you already said it booted with old bootenv and new kernel


Sorry. Forgot to update bootcode. Replacing loader, loader.rc, loader.4th and then run 'gpart ...' helps to boot. Uname shows 13.0 and freebsd-version shows 13.0. Any suggestion about further steps?


----------



## covacat (Jun 3, 2021)

don't upgrade your pool and / or turn on new features on because old bootcode wont be able to boot
file PR and / or try to debug loader yourself
try to boot from 13-RELEASE usb stick and if it works you can create a usb boot device and still mount your zfs root
alternatively if you still have some unused space on you internal storage create an ufs boot partition (that holds just /boot) and try to boot from it


----------



## dtoch (Jun 8, 2021)

Here we are again. 
Ok. I got about "don't touch you pool" but...
I reinstalled 13.0-RELEASE on this server and it's failed to boot obviously. But at the boot stage I pressed ESC and got screenshot #1. Then I enter "/boot/loader" to prompt as shown on screenshot #2 and OS boots normal. 
Sorry for bothering but this is my first "rescue operation" of FreeBSD.


----------



## SirDice (Jun 8, 2021)

If you reinstalled you got the new bootloader with the installation.


----------



## dtoch (Jun 8, 2021)

Yes, I know. But it works incorrect too. And I don't understand the difference between /boot/zfsloader and /boot/loader. According to md5 sum they are the same. So difference somewhere in boot configuration or environment, I think.


----------



## SirDice (Jun 8, 2021)

Is this a CSM (traditional BIOS) boot or EFI? GPT or MBR? All these boot slightly different.

This is the "traditional" boot with MBR:








						Chapter 14. The FreeBSD Booting Process
					

An introduction to the FreeBSD Booting Process, demonstrates how to customize the FreeBSD boot process, including everything that happens until the FreeBSD kernel has started, probed for devices, and started init




					docs.freebsd.org
				




For GPT it's slightly different, here the boot sector loads the code from the freebsd-boot partition, this can be gptboot(8) or gptzfsboot(8). That in turn loads the loader(8) from the first UFS or ZFS partition. The loader(8) loads the kernel and various modules if required.

With EFI the system boots by executing the EFI/boot/bootx64.efi file that's on the efi partition. That bootx64.efi is basically an EFI version of the loader(8).


----------



## grahamperrin@ (Jun 11, 2021)

dtoch said:


> … ASUS RS500-E6-PS4 server. …



I'm having trouble visualising the story in this topic (sorry), partly because I see a menu bar of sorts in the images at <https://forums.FreeBSD.org/threads/80435/post-515358>. Is something virtualised?


----------



## dtoch (Jun 15, 2021)

grahamperrin said:


> I'm having trouble visualising the story in this topic (sorry), partly because I see a menu bar of sorts in the images at <https://forums.FreeBSD.org/threads/80435/post-515358>. Is something virtualised?


Nope. It's real server and screenshots is from virtual KVM. I have access to it through web-console.

Fast digging in sources make me think that problem some where in boot environment or boot sequence of server itself. Looks like server configured in bios to boot from network first and boot loader detects wrong boot device or something. But this is just a guess that I cannot verify yet.


----------

