# Upgrading from 11.2 to 12.0-Release - boot directory issues



## zgee (Dec 12, 2018)

Greetings. I was hoping you could help me figure out an issue I've encountered. I have a working system at the moment, but I'm worried the solution I ended up with is (at best) bad practice, or potentially ends up breaking my system down the road.
Please excuse my lengthy summary. Because I'm posting this after trying a couple of different fixes (most proved unsuccessful), and because I'm very much an amateur, I was unsure what information to leave out. So I decided to include most of it.

During my upgrade from FreeBSD 11.2 to 12.0-Release, I ran into a problem with gaining access to the /boot directory.
I was working my way through the upgrade process (according to section 2.2), but when I attempted to run `freebsd-update install` for the final time, before the final reboot, I received the following output:

```
Cannot identify running kernel
```
This left me stumped, but I decided to reboot and attempt to run the command again. Same result.
I tried running `freebsd-version` and it returned

```
12.0-RELEASE
```
as expected.
I then started searching the FreeBSD forums for posts regarding a similar issue, but most of the relevant posts I found were dated to around 2017 and earlier (I suspected it was an issue with my 12.0-Release upgrade). Nevertheless, I tried a few things after reading through all of them.
I tried running `zpool status` and got

```
pool: zroot
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
```
and

```
pool: bootpool
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
```
I ran both `zpool upgrade zroot` and `zpool upgrade bootpool`, and tried running `zpool status` again. I now had

```
pool: bootpool
state: ONLINE
  scan: none requested
config:

    NAME        STATE     READ WRITE CKSUM
    bootpool    ONLINE       0     0     0
      ada0s1a   ONLINE       0     0     0

errors: No known data errors

  pool: zroot
state: ONLINE
  scan: none requested
config:

    NAME           STATE     READ WRITE CKSUM
    zroot          ONLINE       0     0     0
      ada0s1d.eli  ONLINE       0     0     0

errors: No known data errors
```
I read about potential problems with the canmount value in some zfs filesystem configurations, so I ran `zfs get canmount` and got

```
NAME                PROPERTY  VALUE     SOURCE
bootpool            canmount  on        default
zroot               canmount  on        default
zroot/ROOT          canmount  on        default
zroot/ROOT/default  canmount  noauto    local
zroot/tmp           canmount  on        default
zroot/usr           canmount  off       local
zroot/usr/home      canmount  on        default
zroot/usr/ports     canmount  on        default
zroot/usr/src       canmount  on        default
zroot/var           canmount  off       local
zroot/var/audit     canmount  on        default
zroot/var/crash     canmount  on        default
zroot/var/log       canmount  on        default
zroot/var/mail      canmount  on        default
zroot/var/tmp       canmount  on        default
```
I changed zroot/ROOT/default's value to on with `zfs set canmount=on zroot/ROOT/default` and ran `zfs get canmount` again:

```
NAME                PROPERTY  VALUE     SOURCE
bootpool            canmount  on        default
zroot               canmount  on        default
zroot/ROOT          canmount  on        default
zroot/ROOT/default  canmount  on        local
zroot/tmp           canmount  on        default
zroot/usr           canmount  off       local
zroot/usr/home      canmount  on        default
zroot/usr/ports     canmount  on        default
zroot/usr/src       canmount  on        default
zroot/var           canmount  off       local
zroot/var/audit     canmount  on        default
zroot/var/crash     canmount  on        default
zroot/var/log       canmount  on        default
zroot/var/mail      canmount  on        default
zroot/var/tmp       canmount  on        default
```
After most of the above steps (and with reboots in-between) I would check to see if `freebsd-update install` was able to run, but it still returned

```
Cannot identify running kernel
```
I then tried updating/upgrading the bootcode on my drive's MBR (`gpart show` output posted below for reference), with the following command:
`gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0`
but it gave me an error along the lines of "error: file unavailable" (I'm sorry, I don't have the exact output, but needless to say the command was unable to run).
(`gpart show` output for refence)

```
=>       63  976773105  ada0  MBR  (466G)
         63          1        - free -  (512B)
         64  976773096     1  freebsd  [active]  (466G)
  976773160          8        - free -  (4.0K)

=>        0  976773096  ada0s1  BSD  (466G)
          0    4194304       1  freebsd-zfs  (2.0G)
    4194304   16777216       2  freebsd-swap  (8.0G)
   20971520  955801568       4  freebsd-zfs  (456G)
  976773088          8          - free -  (4.0K)

=>        40  1953525088  ada1  GPT  (932G)
          40        2008        - free -  (1.0M)
        2048  1953521664     1  freebsd-ufs  (932G)
  1953523712        1416        - free -  (708K)
```
It took me this long, and all of the above steps, to finally realize that /boot was completely inaccessible.
I tried running `ls /boot/` and got the output

```
entropy modules
```
This confused me quite a bit, because I had what appeared to be a somewhat working system that could boot, so I started poking around for loader.conf with `find / -name 'loader.conf' 2>/dev/null` and got the result:

```
/bootpool/boot/loader.conf
```
I inspected /bootpool/boot/ more closely and found all my familiar kernel modules (virtualbox and nvidia) in the/bootpool/boot/modules/ directory.
So I decided to rename /boot/ to /boot-11.2-12.0R-upgrade/ and create a symbolic link from /boot/ to /bootpool/boot/ using `ln -s /bootpool/boot/ /boot/`.
Checking with `ls -l / | grep boot` gave me:

```
lrwxr-xr-x   1 root  wheel    14 Dec 12 01:51 boot -> /bootpool/boot
drwxr-xr-x   3 root  wheel     4 Dec 12 01:16 boot-11.2-12.0R-upgrade
drwxr-xr-x   3 root  wheel     3 Nov  9  2016 bootpool
```
And I was now finally able to run `freebsd-update install` and let the upgrade complete without any further errors.
I'm just left wondering if my "fix" is bad practice or if I can carry on from here, and consider the upgrade successful.
Apart from one other minor issue with `openssl` (which I'm guessing is unrelated), my system seems to be running perfectly fine.
Am I good to go? Thank you if you made it this far!


----------



## vendion (Dec 13, 2018)

I ran into this issue as well, in my case though my bootpool is mirrored across multiple drives

`[/u/h/vendion]─> zpool status
  pool: bootpool
 state: ONLINE
  scan: scrub repaired 0 in 0 days 00:00:00 with 0 errors on Sun Dec  2 00:00:00 2018
config:

    NAME        STATE     READ WRITE CKSUM
    bootpool    ONLINE       0     0     0
      mirror-0  ONLINE       0     0     0
        ada0p3  ONLINE       0     0     0
        ada1p3  ONLINE       0     0     0
        ada2p3  ONLINE       0     0     0
        ada3p3  ONLINE       0     0     0

errors: No known data errors

  pool: tank
 state: ONLINE
  scan: scrub repaired 0 in 0 days 00:25:09 with 0 errors on Sun Dec  2 00:25:09 2018
config:

    NAME            STATE     READ WRITE CKSUM
    tank            ONLINE       0     0     0
      mirror-0      ONLINE       0     0     0
        ada4p1.eli  ONLINE       0     0     0
        ada5p1.eli  ONLINE       0     0     0

errors: No known data errors

  pool: zroot
 state: ONLINE
  scan: scrub repaired 0 in 0 days 00:06:18 with 0 errors on Sun Dec  2 00:06:18 2018
config:

    NAME            STATE     READ WRITE CKSUM
    zroot           ONLINE       0     0     0
      raidz2-0      ONLINE       0     0     0
        ada0p5.eli  ONLINE       0     0     0
        ada1p5.eli  ONLINE       0     0     0
        ada2p5.eli  ONLINE       0     0     0
        ada3p5.eli  ONLINE       0     0     0

errors: No known data errors`

Which makes installing the bootcode a little more confusing in my case as i want the system to still be able to boot off any of the 4 drives. Regardless symlinking /bootpool/boot to /boot worked and allowed me to finish the update with `freebsd-update install`. I'm curious if this is due to changes that was made as a result of the addition of `bectl` and handling of multiple zpools.


----------

