# Mounting fails after upgrading 9.3R to 10.1R



## arapaima (Oct 8, 2015)

I did a binary upgrade with freebsd-update(8).

It went something like this:

`# freebsd-update -r 10.1-RELEASE upgrade
# freebsd-update install`

```
Installing updates...chflags: ///var/empty: Read-only file system
```
`# shutdown -r now`

I didn't really mind the error and just rebooted the system when `freebsd-update` finished.

I adjusted the zfs readonly property for /var/empty, and I guess it just resumed the previous installation? (since it didn't keep on with installing userland updates)

`# zfs set readonly=off tank/var/empty
# freebsd-update install`

```
Installing updates...
Kernel updates have been installed. Please reboot and run
"/usr/sbin/freebsd-update install" again to finish installing updates.
```
`# shutdown -r now`

Now FreeBSD is unable to mount the system.


```
Mounting failed from zfs:tank with error 2.
```

I'm not sure where to go from here and I have no clue on what the problem might be, so I would need some pointers.

I booted from a FreeBSD 10.1-RELEASE image and imported the pool by

`# mkdir /tmp/mnt && zpool import -R /tmp/mnt -f tank`

```
cannot share 'tank/user/local/storage': share (1M) failed
```

This didn't mount datasets with legacy mount points. I needed to mount these separately.
`mount -t zfs tank /tmp/[...]` .

My /boot/loader.conf (part of, ZFS related)

```
zfs_load="YES"
vfs.root.mountfrom="zfs:tank"
(...)
```

Is there a way of mounting all datasets under one single 'altroot'? (even those with legacy mount points).
This would give me an option to `chroot <path>` in to the filesystem and try `freebsd-update rollback` to restart the upgrade from the beginning.

Thanks!


----------



## arapaima (Oct 13, 2015)

Now I'm able to mount all datasets under one single 'altroot' making it possible for me to `chroot` into the system.
`zpool import` couldn't handle legacy mount points, so I had to set one by `zfs set mountpoint=/ tank`.

I was also thinking maybe some features were missing on my pool making FreeBSD 10.1R unable to mount zfs:tank, so I upgraded it by `zfs upgrade` and `zpool upgrade` (from a FreeBSD 10.1R image).
This broked the boot process since the installed boot loader complained about missing feature flags.

I booted the FreeBSD 10.1-RELEASE image again to upgrade the bootloader
`gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada2`

Maybe I'm dealing with a partial upgrade. I'm thinking that `freebsd-upgrade` should've upgraded the bootloader if needed?

Above just left me in the beginning again ("Mounting failed from zfs:tank with error 2"), so I've `zfs rollback [...]` all individual datasets from the snapshots (a state where the system was FreeBSD 9.3R).
Now I'm not able to boot at all:


```
ZFS: Unspoorted feature: com.delphix:embedded_data
ZFS: Unspoorted feature: com.delphix:embedded_data
ZFS: Unspoorted feature: com.delphix:embedded_data
(...)
ZFS: can't find pool by guid
ZFS: can't find pool by guid

can't load 'kernel'
```

I thought this was because of the 10.1R boot loader I installed earlier (the system should be FreeBSD 9.3R since I `zfs rollback` to a state before `freebsd-upgrade`.
Or maybe because of the kernel/ZFS implementation not supporting the 'embedded_data' feature.
I booted from a 9.3R image and replaced the bootloaders again. It didn't help. I'm also unable to import my pool from a 9.3R disk image (I need a 10.1R image for that)

So it seems like I'm in a FreeBSD 9.3R system were the kernel doesn't support a ZFS feature which is written to my ZFS pool. As far as I know there are no ways of downgrading zfs pools.

What options do I got, can I make a freebsd-upgrade on the pool and try to repair it?

As I see it now my only option is to connect an additional hard drive, copy the files and make a clean reinstallation.


----------

