# Boot failed with error 2 unknown file system after ugrade 10.3 to 11.2



## gerald (Dec 25, 2018)

Hi,

I've tried to upgrade a 10.3 to 11.2 (that is a dedicated remote server without direct access), the boot to 11.2 failed.
I got a message from my provider that the server was blocked with this message :
`mounting from zfs:root failed with error 6`

After restarting the server in rescue mode (have two rescue mode a 10.3 and 11.0) i've made a rollback after mounting the zfs pool.
Booting failed again, then with this error:
`Mounting from zfs:zroot failed with error 2: unknown file system`

I've rebooted again in rescue mode and made a `nextboot -k kernel.old`.

So system is now up with the old system and kernel 10.3-RELEASE. (p24 for kernel, p28 for userland).

How can I fix that for upgrade to 11.2 ?


```
$ zpool get bootfs zroot
NAME   PROPERTY  VALUE   SOURCE
zroot  bootfs    zroot   local
```


```
$ gpart show
=>        34  3907029101  ada0  GPT  (1.8T)
          34           6        - free -  (3.0K)
          40        1024     1  freebsd-boot  (512K)
        1064    16777216     2  freebsd-swap  (8.0G)
    16778280  3890250848     3  freebsd-zfs  (1.8T)
  3907029128           7        - free -  (3.5K)

=>        34  3907029101  ada1  GPT  (1.8T)
          34           6        - free -  (3.0K)
          40        1024     1  freebsd-boot  (512K)
        1064    16777216     2  freebsd-swap  (8.0G)
    16778280  3890250848     3  freebsd-zfs  (1.8T)
  3907029128           7        - free -  (3.5K)
```


```
$ zpool list
NAME    SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
zroot  3,59T   196G  3,40T         -     8%     5%  1.00x  ONLINE  -
```


```
$ zpool status
  pool: zroot
state: ONLINE
  scan: none requested
config:

    NAME           STATE     READ WRITE CKSUM
    zroot          ONLINE       0     0     0
      raidz1-0     ONLINE       0     0     0
        gpt/disk0  ONLINE       0     0     0
        gpt/disk1  ONLINE       0     0     0

errors: No known data errors
```


```
$ cat /boot/loader.conf 
autoboot_delay="5" 
vfs.root.mountfrom="zfs:zroot"
vfs.zfs.prefetch_disable=0
zfs_load=YES
```

I've added the zfs.* lines after the failed boot, but that did not resolve the problem.

Thanks in advance.

For informations, I've folowed this guides to create the pool and install the system.
Don't remember wich version of FreeBSD was the provider rescue system at this time, not sure, but perhaps since in 10.3?

https://blog.imirhil.fr/2016/07/19/ovh-freebsd-root-zfs-raidz.html (Fench)

Then my own memo :
https://blog.gegeweb.org/freebsd-9-2-en-full-zfs-sur-un-kimsufi-ps-4g.html (French)


----------



## gerald (Dec 26, 2018)

Another upgrade attempt tonight.
After the first freebsd-update, before restarting I tried to update the bootloader:


```
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1
```

Same problem (I can't confirm, no more KVM access than I had for 24 hours), the system didn't start.
I'm back in 10.3 again, as a precaution I updated the bootloader with the 10.3 version of the rescue system before rebooting on the previous snapshot, just before the `freebsd-update install`.


----------



## gerald (Dec 26, 2018)

Found this *Bug 221075* - geom_flashmap(4) exposes race rendering / on ZFS unbootable
In this thread (about the first error 6) : https://groups.google.com/forum/#!topic/muc.lists.freebsd.stable/PywE42FLBjw

Could it be that?

If yes, the question is how to get out of it?…


----------



## gerald (Dec 28, 2018)

Attempt to update to 12.0, after upgrading to 10.4.
The system does not start either.
Since I no longer have KVM, I can only assume that it is the same problem.

So… go back to 10.4.

Is there a way to get out of this bad situation and update?


----------



## Khaine (Dec 29, 2018)

I had a similar issue upgrading a system from 11.3-RELEASE to 12.0-RELEASE.  

I solved it by manually installing freebsd 12.0-RELEASE.  I booted from the live cd and mounted my zfs volume to mnt moved all the data to a temporary folder and then manually installed freebsd.  
`# cd /tmp/zfs
# tar xvJf /usr/freebsd-dist/base.txz 
[FONT=inherit]# tar xvJf /usr/freebsd-dist/kernel.txz[/FONT]`

For good measure I copied my boot loader settings across to the new installation.

Then I had to re-install the boot loader, recreate all the ports and move all my configuration back.

[FONT=monospace]gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1[/FONT]

I'm not sure if there is an easier way.


----------



## gerald (Dec 29, 2018)

Khaine said:


> I had a similar issue upgrading a system from 11.3-RELEASE to 12.0-RELEASE.
> 
> I solved it by manually installing freebsd 12.0-RELEASE.  I booted from the live cd and mounted my zfs volume to mnt moved all the data to a temporary folder and then manually installed freebsd.
> 
> I'm not sure if there is an easier way.



Thank you for this solution. If i can't find another, i'll do that.
Unfortunatly not a physical access to the computer, only over SSH and a FreeBSD rescue system.
Could have a KVM, but not for free! So, if I need it, i need to be sure to solve my problem !

What do you mean about a temporary folder for the data, on the same filesystem?

However, there is something strange about my problem!
I just installed a 10.3 with exactly the same setup as this machine in a VM, using geom and labels, creating the raidz pool using `/dev/gtp/disk*`.
Upgrade to 11.2-REALEASE and the system boots well on the 11.2-RELEASE-p7 kernel.

Another strange thing, I had (that I'm sure) not `vfs.root.mountfrom="zfs:zroot"` in `/boot/loader.conf`, only `zfs_load="YES"`, and `zpool set bootfs=zroot zroot` at setup, 10.3 shouldn't have been able to start, should it?
Edit: the FreeBSD 11.2 upgraded from 10.3 in the VM start well without.

Error 2 was caused by a misconfiguration: `vfs.root.mountfrom="zfs:zroot"`  before `zfs_load="YES"`.
But the 10.3 started well with this error.
Trying to boot on a 11.2 or 12.0 kernel after fixing this, and system doesn't start, i suppose with the first error 6 (no KVM/console, only ssh access, so I'm not sure. After a few minute without echo reply I do a hard reboot to restart in rescue and reboot on the old kernel and rollback to the previous snapshot).

I don't remember if we was already on 10.* at installation, or on 9.*.
Edit: I've noticed this in dmesg (with symlinks in `/dev`:

```
ada0: Previously was known as ad4
ada1: Previously was known as ad6
```
So I was probably on 9, perhaps on 8. Don't know where the name of device file was modified in FreeBSD.

Should I try a `zpool set cachefile=/mnt/boot/zfs/zpool.cache zroot` before trying to restart on 11.2?
But if something went always wrong, could I restart on 10.4?
I'm now on 10.4, but lot of packages out of date and some with security issues. so, that's why even If it works, I want to fix It!


----------

