# upgrade 12.2-13 and mirrored zpool with geli



## o.tsiparinda (Sep 1, 2021)

Hi,
I recently upgraded from 12.2 to FreeBsd-13.0. After reboot boot halted with message "module kernel exists but with wrong version". Next time I boot in old kernel successfully, but had not working userland programs (for example. vi didn't run). Decided boot from livecd to edit loader.conf. The original problem was caused by the module nvidia-modeset as it turned out later. But now I have problem with zpool: LiveCD sees the mirrored zpool (zpool import) as two separate zpools with state "FAUTED corrupted data" and loading from disks stops at the prompt "mountpoint>".
But! When i boot in the usual way from "corrupted" zpool, I abort the download in the third step (Loader) and can see the disk structure (ls) and even the contents of the files (more...). And command lszfs workins also! Zpool sees as one pool!


----------



## SirDice (Sep 1, 2021)

Boot to single user mode with the old kernel. Then remove the drivers from /boot/loader.conf. You probably have a Virtualbox, DRM or Nvidia driver loaded there, that needs to be updated for the new kernel before you can load it again. So for now just disable it, then finish the upgrade to 13.0. When the OS upgrade is finished make sure to reinstall _everything_ ports/packages. Then you can enable those drivers again.


----------



## o.tsiparinda (Sep 1, 2021)

I would do this if I could boot. loading stops at the prompt mountpoint>


----------



## SirDice (Sep 1, 2021)

Try entering `zfs:system/ROOT/default`. Judging from the picture it looks like your boot pool is called system. There should be a default dataset under the ROOT name. Unless you changed that for a  boot environment (beadm(1)/bectl(8)).


----------



## o.tsiparinda (Sep 1, 2021)

I tried like this - to no avail. I don't understand what happened: run LiveCD (entering the password geli at the beginning of the download), zpool import -f -R /mnt system, vi /boot/loader.conf... And the mirror disintegrated, pool failed. And one question: why does the bootloader see the pool? Can I recover data?


----------



## SirDice (Sep 1, 2021)

Did you do anything else besides the `zpool import ...` from the LiveCD? If you didn't issue any other commands, like removing drives or upgrade pools, then the mirror should theoretically still be intact. Can you still boot from the LiveCD? Make sure that's based on FreeBSD 12.2 or 13.0. Can you boot from that and, without importing anything yet, post the output from `gpart show`? We need to get an idea of how your disks have been set up.


----------



## T-Daemon (Sep 1, 2021)

Please post also the output of
`geli status`
`zpool import`
executed from the LiveCD.


----------



## o.tsiparinda (Sep 1, 2021)

disks: ada0, ada1
mirrored pool name was "system" (system0 and system1 now)
set up disks likehere: https://gist.github.com/forquare/b4e12938b1240238ef64e3d6ba5d9669
two years ago, but without swap partition (swap as dataset)


----------



## T-Daemon (Sep 2, 2021)

What you see in LiveCD as pools 'system0' and 'system1' are not the 'system' pool. `geli status` returns nothing, that means the geli(8) encrypted mirrored 'system' pool's partitions are not attached, `zpool import` shows other (previously used, undestroyed?) pools. I'm sure pool 'system' is intact.

I suggest you backup (recover) the data first as you are asking for in post #5. This brings up the question: Don't you have any backups of the system and didn't you made a backup before upgrading? If that's the case, next time upgrading do a backup before. Also during normal usage you should have a backup strategy.

To recover the data boot the LiveCD, drop to 'Shell', execute:

```
geli attach ada0p2    (enter password)
geli attach ada1p2

Check geli(8) provider:
geli status

Check pool 'system':
zpool import
zpool import -R /mnt system
zfs mount system/ROOT/default

Copy data.

zpool export system
```

To the unbootable system, I suspect here the FreeBSD efi boot loader. Try following from 13.0 LiveCD:

```
mount_msdosfs /dev/ada0p1 /mnt

cp /boot/loader.efi /mnt/efi/boot/bootx64.efi
cp /boot/loader.efi /mnt/efi/freebsd/loader.efi
Eventually you need to create /mnt/efi/freebsd/ directory.

umount /mnt
```
Procede the same with ada1p1.

Try booting the upgraded systems 13.0 kernel. If it boots continue with freebsd-update(8).


----------



## o.tsiparinda (Sep 2, 2021)

Thank you! 
All your advice is working, very helpful!
I boot from LiveCD, geli both partitions, import zpool, change loader.conf to original condition (and disable nvidia-modeset), AND run zpool export (I suspect this was my mistake when I first edited the file loader.conf).


----------

