# BEADM information sought



## byrnejb (Feb 16, 2021)

I have a simple question:

When using `beadm` to perform an update, if the update succeeds then does one  simply stay on that boot environment and never return to the original?   What happens with the original?

Given

```
# beadm list
BE               Active Mountpoint  Space Created
default          NR     /          172.8G 2020-05-20 11:25
12.2-RELEASE-p13a -      -          756.7K 2021-02-16 16:43
```

If I activate and reboot into `12.2-RELEASE-p13a`; and assuming that the update succeeds; do I just leave things as they are and continue to boot into `12.2-RELEASE-p13a`?  If so, then what happens to `default`?  Is it left for all time, destroyed, or is something else supposed to be carried out to finish the update?


----------



## scottro (Feb 16, 2021)

When I first messed with it, I was also slightly confused. This Michael Lucas page explained it pretty well in my opinion.


			FreeBSD and beadm – Michael W Lucas
		


When you're sure you're done with the old one, you can destroy it.


----------



## SirDice (Feb 16, 2021)

The easiest way for me was to view the BE you create with `beadm create` as a restore point. So the 12.2-RELEASE-p13a is your point in time to return to if the update/upgrade gets botched. For all intents and purposes it is a snapshot of the current state of the machine.


----------



## Mjölnir (Feb 17, 2021)

latest 12.*1* is _-p13_, right now 12.2 is_ @ _patch level _-p_*3*...
byrnejb: you can keep the previous version for a week or two, then delete it once you're shure that the new, upgraded release works sufficiently well.  beadm(1) (or bectl(8)) will then do all necessary tasks behind the scene (promote clone & delete unused snapshots).


----------



## xtaz (Feb 17, 2021)

Although one thing I noticed when switching from beadm to bectl is doing destroy seems to work slightly differently between both. With bectl you also have to provide the -o switch or else it leaves behind the old snapshot. Beadm seems to destroy the snapshot automatically.

On second thoughts, maybe it doesn't any longer. Looking at the source I think they've changed the behaviour.


----------



## scottro (Feb 17, 2021)

There was a bectl bug,  where at times it wasn't properly destroying snapshots.  A few people, including myself, ran into it, but fixed it by using beadm instead. I don't know if that was fixed or not.  I don't do major upgrades often, so haven't used it all that much.


----------



## byrnejb (Feb 17, 2021)

I was going through the process of creating a new BE, documenting it step by step, when I ran `ps -auwx` so see what was going on behind the scenes.  This is what I discovered:


```
root        62914    0.0  0.0    11892   3208  4  I+   09:00       0:00.01 /bin/sh -e /usr/local/sbin/beadm activate 12.2-RELEASE-p
root        62968    0.0  0.0    11892   3204  4  I+   09:00       0:00.00 /bin/sh -e /usr/local/sbin/beadm activate 12.2-RELEASE-p
root        62973    0.0  0.0    12952   3996  4  D+   09:00       0:00.25 zfs promote zroot/ROOT/12.2-RELEASE-p13rp
```

I am curious as to why there are two processes evidently carrying out the same instruction to activate the BE.


----------



## byrnejb (Feb 17, 2021)

I have a couple of other questions as well:

1.  How long should activation take?  It has been running for about an hour so far but, as the default BE is 187G in size, I do not know if this is expected behaviour or not.

2.  I use `zfsnap` to take frequent snapshots of this system.  These are only kept for 96 hours but they are taken every two minutes.  Would this adversely impact`beadm` activation?


----------



## Mjölnir (Feb 17, 2021)

byrnejb
Look at the process tree: `ps -adjx`
Usually the activation is more or less instant, as it basically (only) involves setting the zpool(8)'s _bootfs_ & promoting the clone, i.e. changing the parent-child relation between two filesystems and the originating snapshot.
I don't know, but I can imagine that taking a snapshot that often could interfere.  Just try it out by setting the frequency to let's say 5 minutes, or pause the snapshots before you do the activation and resume afterwards.


----------



## byrnejb (Feb 17, 2021)

I took a look at


Mjölnir said:


> byrnejb: Look at the process tree: `ps -adjx`


I did ps -v and saw this:

```
62973 D+    0:00.25 127 127      0   12952  3996   -   52  0.0  0.0 zfs promote zroot
```

So, D+ says that the zfs promote process is stuck in an uninterruptible wait state. fstat for pis 62973 shows this:


```
root     zfs        62973 text /            67 -r-xr-xr-x  109696  r
root     zfs        62973 ctty /dev        176 crw--w----   pts/4 rw
root     zfs        62973   wd /            34 drwxr-xr-x     353  r
root     zfs        62973 root /             4 drwxr-xr-x      27  r
root     zfs        62973    0* pipe fffff800124d2be0 <-> fffff800124d2d48      0 rw
root     zfs        62973    1 /dev        176 crw--w----   pts/4 rw
root     zfs        62973    2 /dev        176 crw--w----   pts/4 rw
root     zfs        62973    3 /dev         88 crw-rw-rw-     zfs rw
root     zfs        62973    4 /dev         17 crw-rw-rw-    zero  r
root     zfs        62973    5 /         32770 -rw-r--r--       0  r
root     zfs        62973    6 /dev         88 crw-rw-rw-     zfs rw
root     zfs        62973    7 /dev         17 crw-rw-rw-    zero  r
```


----------



## byrnejb (Feb 17, 2021)

This is the tree for the zfs promote pid.  It is being run as root inside a mate-terminal session.

```
byrnejb_hll 53603  2194 53603 53603    0 Is    4      0:00.20 | |-- -bash (bash)
root        53699 53603 53699 53603    1 I     4      0:00.01 | | `-- sudo su -m
root        53700 53699 53699 53603    1 I     4      0:00.00 | |   `-- su -m
root        53701 53700 53701 53603    1 I     4      0:00.03 | |     `-- _su -m (csh
root        53704 53701 53704 53603    1 I     4      0:05.34 | |       `-- bash -l -
root        62914 53704 62914 53603    1 I+    4      0:00.01 | |         `-- /bin/sh
root        62968 62914 62914 53603    1 I+    4      0:00.00 | |           `-- /bin/
root        62973 62968 62914 53603    1 D+    4      0:00.25 | |             `-- zfs
```
[/CMD][/CMD]


----------



## Mjölnir (Feb 17, 2021)

My intend was to see if the 2nd shell is a child of the 1st.  Concerning the lockup, you may want to file in a bug report & include wether you're using OpenZFS (from ports): @top of this page, select _Support_ -> _Bug Reports_.  OTOH, please try to pause the snapshotting task while doing the activation...


----------



## byrnejb (Feb 17, 2021)

This is exhibiting behaviour very similar to what happened to me on Sunday.  The only way I could get my system back was to reboot.


----------



## byrnejb (Feb 17, 2021)

So, I guess that is the only way out of this again.  Sigh.


----------



## Mjölnir (Feb 17, 2021)

byrnejb said:


> [...], as the default BE is 187G in size, I do not know if this is expected behaviour or not. [...]


This is a hint that your ZFS filesystem layout is suboptimal.  You can split off all the data that can be shared by all BEs to their own ZFS filesystems.  E.g. mine looks like this:


Spoiler: /etc/fstab





```
# Device                Mountpoint              FStype          Options                    Dump     Pass#
#----------------------------------------------------------------------------------------------------------------------------
t450s/ROOT/12.2-RELEASE /                       zfs             rw,noatime                 00
/dev/gpt/DUMP           none                    swap            sw                         00
/dev/zvol/t450s/SWAP    none                    swap            sw,late                    00
SYS                     /proc                   procfs          rw                         00
SYS                     /dev/fd                 fdescfs         rw,late                    00
RAM                     /tmp                    tmpfs           rw,nosuid,size=6G,mode=177700
RAM                     /var/run                tmpfs           rw,noexec,size=2M          00
linproc                 /compat/linux/proc      linprocfs       rw                         00
linsys                  /compat/linux/sys       linsysfs        rw                         00
SYS                     /compat/linux/dev/fd    fdescfs         rw,late                    00
RAM                     /compat/linux/dev/shm   tmpfs           rw,late,nosuid,size=4G,mode=1777            0       0
/ports/quarterly        /usr/ports              unionfs         rw,late,noatime,below,copymode=transparent  0       0
/src/12.2-REL           /usr/src                unionfs         rw,late,noatime,below,copymode=transparent  0       0
/dist/12.2-REL          /usr/freebsd-dist       unionfs         rw,late,noatime,below,copymode=transparent  0       0
# TODO make these autofs(5)
/src/12.1-REL   /home/paul/Projects/FreeBSD/src/12.1-REL unionfs rw,noauto,noatime,below,copymode=transparent       0       0
/src/12.2-REL   /home/paul/Projects/FreeBSD/src/12.2-REL unionfs rw,noauto,noatime,below,copymode=transparent       0       0
/src/12-STABLE  /home/paul/Projects/FreeBSD/src/12-STABLE unionfs rw,noauto,noatime,below,copymode=transparent      0       0
/src/13-STABLE  /home/paul/Projects/FreeBSD/src/13-CUR unionfs rw,noauto,noatime,below,copymode=transparent 0       0
/src/14-CUR     /home/paul/Projects/FreeBSD/src/13-CUR unionfs rw,noauto,noatime,below,copymode=transparent 0       0
#
# NOTE DB directories common to all OS versions that run on this machine _NON_CONCURRENTLY_,that can be shared among them,
# have been moved to ZFS pool/var/db, each in it's own dataset.  I.e., pool/var/db has the property: canmount=off
# This is just like /var/log is on it's own dataset by default.  E.g.
# /var/db/vnstat: network traffic accounting;  shared, no traffic is lost whichever OS runs;
# /var/db/clamav: virus definition DB;  do updates only once, whichever OS runs;
# /var/db/cve...: CVE vulnerability DB;  - " -  etc.pp.
```



I do mount /usr/src & /usr/ports explicitely instead of having symlinks to not forget to update that on new BEs.  EDIT But I do have these symlinks, as these are costant for all BEs: /usr/home -> ../home, /ports/{latest,quarterly}/distfiles -> /var/cache/distfiles, and by default /sys -> usr/src/sys /EDIT EDIT2 and /usr/obj -> ../var/obj /EDIT2


Spoiler: zfs list





```
NAME                            USED  AVAIL  REFER  MOUNTPOINT
t450s                           210G  26,8G    25K  /
t450s/ROOT                     38,0G  26,8G    23K  /ROOT
t450s/ROOT/12.1-202012            4K  26,8G  22,0G  legacy
t450s/ROOT/12.1-RELEASE        6,25M  26,8G  21,7G  legacy
t450s/ROOT/12.2-RELEASE        38,0G  26,8G  24,0G  legacy
t450s/SWAP                     21,3G  26,8G  21,3G  -
t450s/dist                      537M  26,8G    23K  /dist
t450s/dist/12-STABLE             23K  26,8G    23K  /dist/12-STABLE
t450s/dist/12.1-REL             535M  26,8G   535M  /dist/12.1-REL
t450s/dist/12.2-REL            1,39M  26,8G  1,39M  /dist/12.2-REL
t450s/home                     89,8G  26,8G  88,0G  /home
t450s/ports                    15,5G  26,8G  1,36M  /ports
t450s/ports/latest             15,0G  26,8G   688M  /ports/latest
t450s/ports/quarterly           569M  26,8G   563M  /ports/quarterly
t450s/src                      5,30G  26,8G    23K  /src
t450s/src/12-STABLE            1,66G  26,8G   757M  /src/12-STABLE
t450s/src/12.1-REL              512M  26,8G   512M  /src/12.1-REL
t450s/src/12.2-REL              577M  26,8G   779M  /src/12.2-REL
t450s/src/13-STABLE            1,78G  26,8G   773M  /src/13-STABLE
t450s/src/14-CUR                815M  26,8G   815M  /src/14-CUR
t450s/usr                        23K  26,8G    23K  /usr
t450s/var                      37,2G  26,8G    23K  /var
t450s/var/account               452M  26,8G   396M  /var/account
t450s/var/audit                5,34M  26,8G  2,68M  /var/audit
t450s/var/cache                29,2G  26,8G  14,0G  /var/cache
t450s/var/cache/distfiles      15,2G  26,8G  15,2G  /var/cache/distfiles
t450s/var/crash                2,99G  26,8G  2,98G  /var/crash
t450s/var/db                   1,72G  26,8G  1,38M  /var/db
t450s/var/db/AccountsService   2,65M  26,8G  1,36M  /var/db/AccountsService
t450s/var/db/GeoIP             57,9M  26,8G  51,0M  /var/db/GeoIP
t450s/var/db/clamav             681M  26,8G   301M  /var/db/clamav
t450s/var/db/cvechecker        2,72M  26,8G  1,36M  /var/db/cvechecker
t450s/var/db/cvechecker/cache  1,36M  26,8G  1,36M  /var/db/cvechecker/cache
t450s/var/db/ntopng            26,9M  26,8G  14,6M  /var/db/ntopng
t450s/var/db/portsnap           342M  26,8G   199M  /var/db/portsnap
t450s/var/db/redis             4,77M  26,8G  1,76M  /var/db/redis
t450s/var/db/rrdtool           1,36M  26,8G  1,36M  /var/db/rrdtool
t450s/var/db/sonic             2,64M  26,8G  1,36M  /var/db/sonic
t450s/var/db/squidGuard        1,36M  26,8G  1,36M  /var/db/squidGuard
t450s/var/db/sshguard          1,36M  26,8G  1,36M  /var/db/sshguard
t450s/var/db/sudo              1,36M  26,8G  1,36M  /var/db/sudo
t450s/var/db/tor               2,64M  26,8G  1,36M  /var/db/tor
t450s/var/db/tripwire          2,72M  26,8G  1,36M  /var/db/tripwire
t450s/var/db/tripwire/report   1,36M  26,8G  1,36M  /var/db/tripwire/report
t450s/var/db/vnstat            4,74M  26,8G  1,63M  /var/db/vnstat
t450s/var/db/vuls               621M  26,8G   621M  /var/db/vuls
t450s/var/db/zfsd              1,36M  26,8G  1,36M  /var/db/zfsd
t450s/var/log                   255M  26,8G   238M  /var/log
t450s/var/mail                 5,13M  26,8G  1,97M  /var/mail
t450s/var/obj                  2,55G  26,8G  2,55G  /var/obj
t450s/var/tmp                  6,66M  16,0G  3,65M  /var/tmp
```






Spoiler: zfs get -rt filesystem canmount





```
NAME                           PROPERTY  VALUE     SOURCE
t450s                          canmount  off       local
t450s/ROOT                     canmount  noauto    local
t450s/ROOT/12.1-202012         canmount  noauto    local
t450s/ROOT/12.1-RELEASE        canmount  noauto    local
t450s/ROOT/12.2-RELEASE        canmount  noauto    local
t450s/dist                     canmount  off       local
t450s/dist/12-STABLE           canmount  on        default
t450s/dist/12.1-REL            canmount  on        default
t450s/dist/12.2-REL            canmount  on        default
t450s/home                     canmount  on        default
t450s/ports                    canmount  off       local
t450s/ports/latest             canmount  on        default
t450s/ports/quarterly          canmount  on        default
t450s/src                      canmount  off       local
t450s/src/12-STABLE            canmount  on        local
t450s/src/12.1-REL             canmount  on        default
t450s/src/12.2-REL             canmount  on        default
t450s/src/13-STABLE            canmount  on        default
t450s/src/14-CUR               canmount  on        default
t450s/usr                      canmount  off       local
t450s/var                      canmount  off       local
t450s/var/account              canmount  on        default
t450s/var/audit                canmount  on        default
t450s/var/cache                canmount  on        default
t450s/var/cache/distfiles      canmount  on        default
t450s/var/crash                canmount  on        default
t450s/var/db                   canmount  off       local
t450s/var/db/AccountsService   canmount  on        default
t450s/var/db/GeoIP             canmount  on        default
t450s/var/db/clamav            canmount  on        default
t450s/var/db/cvechecker        canmount  off       local
t450s/var/db/cvechecker/cache  canmount  on        default
t450s/var/db/ntopng            canmount  on        default
t450s/var/db/portsnap          canmount  on        default
t450s/var/db/redis             canmount  on        default
t450s/var/db/rrdtool           canmount  on        default
t450s/var/db/sonic             canmount  on        default
t450s/var/db/squidGuard        canmount  on        default
t450s/var/db/sshguard          canmount  on        default
t450s/var/db/sudo              canmount  on        default
t450s/var/db/tor               canmount  on        default
t450s/var/db/tripwire          canmount  off       local
t450s/var/db/tripwire/report   canmount  on        default
t450s/var/db/vnstat            canmount  on        default
t450s/var/db/vuls              canmount  on        default
t450s/var/db/zfsd              canmount  on        default
t450s/var/log                  canmount  on        default
t450s/var/mail                 canmount  on        default
t450s/var/obj                  canmount  on        default
t450s/var/tmp                  canmount  on        default
```






Spoiler: zfs get -rt filesystem atime





```
NAME                           PROPERTY  VALUE  SOURCE
t450s                          atime     off    local
t450s/ROOT                     atime     off    inherited from t450s
t450s/ROOT/12.1-202012         atime     off    inherited from t450s
t450s/ROOT/12.1-RELEASE        atime     off    inherited from t450s
t450s/ROOT/12.2-RELEASE        atime     off    inherited from t450s
t450s/dist                     atime     off    inherited from t450s
t450s/dist/12-STABLE           atime     off    inherited from t450s
t450s/dist/12.1-REL            atime     off    inherited from t450s
t450s/dist/12.2-REL            atime     off    inherited from t450s
t450s/home                     atime     off    inherited from t450s
t450s/ports                    atime     off    inherited from t450s
t450s/ports/latest             atime     off    inherited from t450s
t450s/ports/quarterly          atime     off    inherited from t450s
t450s/src                      atime     off    inherited from t450s
t450s/src/12-STABLE            atime     off    inherited from t450s
t450s/src/12.1-REL             atime     off    inherited from t450s
t450s/src/12.2-REL             atime     off    inherited from t450s
t450s/src/13-STABLE            atime     off    inherited from t450s
t450s/src/14-CUR               atime     off    inherited from t450s
t450s/usr                      atime     off    inherited from t450s
t450s/var                      atime     off    inherited from t450s
t450s/var/account              atime     off    inherited from t450s
t450s/var/audit                atime     off    inherited from t450s
t450s/var/cache                atime     off    inherited from t450s
t450s/var/cache/distfiles      atime     off    inherited from t450s
t450s/var/crash                atime     off    inherited from t450s
t450s/var/db                   atime     off    inherited from t450s
t450s/var/db/AccountsService   atime     off    inherited from t450s
t450s/var/db/GeoIP             atime     off    inherited from t450s
t450s/var/db/clamav            atime     off    inherited from t450s
t450s/var/db/cvechecker        atime     off    inherited from t450s
t450s/var/db/cvechecker/cache  atime     off    inherited from t450s
t450s/var/db/ntopng            atime     off    inherited from t450s
t450s/var/db/portsnap          atime     off    inherited from t450s
t450s/var/db/redis             atime     off    inherited from t450s
t450s/var/db/rrdtool           atime     off    inherited from t450s
t450s/var/db/sonic             atime     off    inherited from t450s
t450s/var/db/squidGuard        atime     off    inherited from t450s
t450s/var/db/sshguard          atime     off    inherited from t450s
t450s/var/db/sudo              atime     off    inherited from t450s
t450s/var/db/tor               atime     off    inherited from t450s
t450s/var/db/tripwire          atime     off    inherited from t450s
t450s/var/db/tripwire/report   atime     off    inherited from t450s
t450s/var/db/vnstat            atime     off    inherited from t450s
t450s/var/db/vuls              atime     off    inherited from t450s
t450s/var/db/zfsd              atime     off    inherited from t450s
t450s/var/log                  atime     off    inherited from t450s
t450s/var/mail                 atime     on     local
t450s/var/obj                  atime     off    inherited from t450s
t450s/var/tmp                  atime     off    inherited from t450s
```


----------



## sko (Feb 17, 2021)

SirDice said:


> The easiest way for me was to view the BE you create with `beadm create` as a restore point. So the 12.2-RELEASE-p13a is your point in time to return to if the update/upgrade gets botched. For all intents and purposes it is a snapshot of the current state of the machine.


I circumvent that confusion about naming/reverting by creating a BE with the name of the version I'm going to update to, mount that BE and start a jail with it. Then I perform the update within the jail environment, unmount & activate it, reboot and finish the update process.
For major upgrades I also snapshot the whole zroot pool with a meaningfully named snapshot (e.g. '@11.4-12.2_upgrade) 'just in case'... (I never actually needed one of those snapshots)

This way one is using BEs the exact same way as snapshots and the system always boots from the "latest" BE, named after the actual running version. Deleting old BEs is just like removing old snapshots.


----------



## byrnejb (Feb 17, 2021)

Spoiler: /etc/fstab



cat /etc/fstab
# Device        Mountpoint    FStype    Options        Dump    Pass#
/dev/ada0p2        none    swap    sw        0    0
/dev/ada1p2        none    swap    sw        0    0
/dev/ada2p2        none    swap    sw        0    0
fdesc        /dev/fd            fdescfs    rw        0    0
proc            /proc        procfs    rw        0    0
# These for Jasper (linux_base-c7)
linprocfs   /compat/linux/proc    linprocfs    rw    0    0
linsysfs    /compat/linux/sys    linsysfs    rw    0    0
tmpfs    /compat/linux/dev/shm    tmpfs    rw,mode=1777    0    0
#end jasper (linux_base-c7)





Spoiler: zfs list



zfs list
NAME                                             USED  AVAIL  REFER  MOUNTPOINT
zroot                                            552G  4.70T   117K  /zroot
zroot/ROOT                                       186G  4.70T   117K  none
zroot/ROOT/12.2-RELEASE-p13rp                    714K  4.70T   155G  /
zroot/ROOT/default                               607K  4.70T   155G  /
zroot/ROOT/hang-test                             186G  4.70T   155G  /
zroot/iocage                                     206G  4.70T   139K  /zroot/iocage
zroot/iocage/download                            371M  4.70T   117K  /zroot/iocage/download
zroot/iocage/download/12.1-RELEASE               371M  4.70T   371M  /zroot/iocage/download/12.1-RELEASE
zroot/iocage/images                              117K  4.70T   117K  /zroot/iocage/images
zroot/iocage/jails                               204G  4.70T   128K  /zroot/iocage/jails
zroot/iocage/jails/accounting-2                 17.7G  4.70T   123K  /zroot/iocage/jails/accounting-2
zroot/iocage/jails/accounting-2/root            17.7G  4.70T  11.2G  /zroot/iocage/jails/accounting-2/root
zroot/iocage/jails/bkuprcvy3                    14.8G  4.70T   123K  /zroot/iocage/jails/bkuprcvy3
zroot/iocage/jails/bkuprcvy3/root               14.8G  4.70T  14.8G  /zroot/iocage/jails/bkuprcvy3/root
zroot/iocage/jails/pgsql-dbms                   3.53G  4.70T   123K  /zroot/iocage/jails/pgsql-dbms
zroot/iocage/jails/pgsql-dbms/root              3.53G  4.70T  3.52G  /zroot/iocage/jails/pgsql-dbms/root
zroot/iocage/jails/smb4-1                       54.7G  4.70T   123K  /zroot/iocage/jails/smb4-1
zroot/iocage/jails/smb4-1/root                  54.7G  4.70T  54.6G  /zroot/iocage/jails/smb4-1/root
zroot/iocage/jails/smb4-2                       63.4G  4.70T   123K  /zroot/iocage/jails/smb4-2
zroot/iocage/jails/smb4-2/root                  63.4G  4.70T  47.0G  /zroot/iocage/jails/smb4-2/root
zroot/iocage/jails/sshpipe3                     44.4G  4.70T   123K  /zroot/iocage/jails/sshpipe3
zroot/iocage/jails/sshpipe3/root                44.4G  4.70T  11.6G  /zroot/iocage/jails/sshpipe3/root
zroot/iocage/jails/theheart-do_not_delete       5.25G  4.70T   123K  /zroot/iocage/jails/theheart-do_not_delete
zroot/iocage/jails/theheart-do_not_delete/root  5.25G  4.70T  5.24G  /zroot/iocage/jails/theheart-do_not_delete/root
zroot/iocage/log                                1.49M  4.70T   170K  /zroot/iocage/log
zroot/iocage/releases                           1.61G  4.70T   117K  /zroot/iocage/releases
zroot/iocage/releases/12.1-RELEASE              1.61G  4.70T   117K  /zroot/iocage/releases/12.1-RELEASE
zroot/iocage/releases/12.1-RELEASE/root         1.61G  4.70T  1.61G  /zroot/iocage/releases/12.1-RELEASE/root
zroot/iocage/templates                           117K  4.70T   117K  /zroot/iocage/templates
zroot/poudriere                                 6.96G  4.70T  2.23G  /usr/local/poudriere
zroot/poudriere/jails                           1.61G  4.70T   117K  /usr/local/poudriere/jails
zroot/poudriere/jails/freebsd_12-1x64           1.61G  4.70T  1.60G  /usr/local/poudriere/jails/freebsd_12-1x64
zroot/poudriere/ports                           2.25G  4.70T   117K  /usr/local/poudriere/ports
zroot/poudriere/ports/PORTS-12-1                2.25G  4.70T  1.17G  /usr/local/poudriere/ports/PORTS-12-1
zroot/tmp                                        672M  4.70T  2.83M  /tmp
zroot/usr                                        118G  4.70T   117K  /usr
zroot/usr/home                                   102G  4.70T  77.5G  /usr/home
zroot/usr/ports                                 15.9G  4.70T  14.8G  /usr/ports
zroot/usr/src                                    117K  4.70T   117K  /usr/src
zroot/var                                        448M  4.70T   117K  /var
zroot/var/audit                                  117K  4.70T   117K  /var/audit
zroot/var/crash                                  117K  4.70T   117K  /var/crash
zroot/var/log                                   34.3M  4.70T  3.01M  /var/log
zroot/var/mail                                  1.04M  4.70T   160K  /var/mail
zroot/var/tmp                                    412M  4.70T   408M  /var/tmp
zroot/vm                                        20.1G  4.70T  17.0G  /zroot/vm
zroot/vm/smb4-a                                 2.45G  4.70T  1.95G  /zroot/vm/smb4-a
zroot/vm/smb4-b                                  709M  4.70T   696M  /zroot/vm/smb4-b





Spoiler: zfs get -rt filesystem canmount



zfs get -rt filesystem canmount
NAME                                            PROPERTY  VALUE     SOURCE
zroot                                           canmount  on        default
zroot/ROOT                                      canmount  on        default
zroot/ROOT/12.2-RELEASE-p13rp                   canmount  noauto    local
zroot/ROOT/default                              canmount  noauto    local
zroot/ROOT/hang-test                            canmount  noauto    local
zroot/iocage                                    canmount  on        default
zroot/iocage/download                           canmount  on        default
zroot/iocage/download/12.1-RELEASE              canmount  on        default
zroot/iocage/images                             canmount  on        default
zroot/iocage/jails                              canmount  on        default
zroot/iocage/jails/accounting-2                 canmount  on        default
zroot/iocage/jails/accounting-2/root            canmount  on        default
zroot/iocage/jails/bkuprcvy3                    canmount  on        default
zroot/iocage/jails/bkuprcvy3/root               canmount  on        default
zroot/iocage/jails/pgsql-dbms                   canmount  on        default
zroot/iocage/jails/pgsql-dbms/root              canmount  on        default
zroot/iocage/jails/smb4-1                       canmount  on        default
zroot/iocage/jails/smb4-1/root                  canmount  on        default
zroot/iocage/jails/smb4-2                       canmount  on        default
zroot/iocage/jails/smb4-2/root                  canmount  on        default
zroot/iocage/jails/sshpipe3                     canmount  on        default
zroot/iocage/jails/sshpipe3/root                canmount  on        default
zroot/iocage/jails/theheart-do_not_delete       canmount  on        default
zroot/iocage/jails/theheart-do_not_delete/root  canmount  on        default
zroot/iocage/log                                canmount  on        default
zroot/iocage/releases                           canmount  on        default
zroot/iocage/releases/12.1-RELEASE              canmount  on        default
zroot/iocage/releases/12.1-RELEASE/root         canmount  on        default
zroot/iocage/templates                          canmount  on        default
zroot/poudriere                                 canmount  on        default
zroot/poudriere/jails                           canmount  on        default
zroot/poudriere/jails/freebsd_12-1x64           canmount  on        default
zroot/poudriere/ports                           canmount  on        default
zroot/poudriere/ports/PORTS-12-1                canmount  on        default
zroot/tmp                                       canmount  on        default
zroot/usr                                       canmount  off       local
zroot/usr/home                                  canmount  on        default
zroot/usr/ports                                 canmount  on        default
zroot/usr/src                                   canmount  on        default
zroot/var                                       canmount  off       local
zroot/var/audit                                 canmount  on        default
zroot/var/crash                                 canmount  on        default
zroot/var/log                                   canmount  on        default
zroot/var/mail                                  canmount  on        default
zroot/var/tmp                                   canmount  on        default
zroot/vm                                        canmount  on        default
zroot/vm/smb4-a                                 canmount  on        default
zroot/vm/smb4-b                                 canmount  on        default





Spoiler: zfs get -rt filesystem atime



zfs get -rt filesystem atime  
NAME                                            PROPERTY  VALUE  SOURCE
zroot                                           atime     off    local
zroot/ROOT                                      atime     off    inherited from zroot
zroot/ROOT/12.2-RELEASE-p13rp                   atime     off    inherited from zroot
zroot/ROOT/default                              atime     off    inherited from zroot
zroot/ROOT/hang-test                            atime     off    inherited from zroot
zroot/iocage                                    atime     off    inherited from zroot
zroot/iocage/download                           atime     off    inherited from zroot
zroot/iocage/download/12.1-RELEASE              atime     off    inherited from zroot
zroot/iocage/images                             atime     off    inherited from zroot
zroot/iocage/jails                              atime     off    inherited from zroot
zroot/iocage/jails/accounting-2                 atime     off    inherited from zroot
zroot/iocage/jails/accounting-2/root            atime     off    inherited from zroot
zroot/iocage/jails/bkuprcvy3                    atime     off    inherited from zroot
zroot/iocage/jails/bkuprcvy3/root               atime     off    inherited from zroot
zroot/iocage/jails/pgsql-dbms                   atime     off    inherited from zroot
zroot/iocage/jails/pgsql-dbms/root              atime     off    inherited from zroot
zroot/iocage/jails/smb4-1                       atime     off    inherited from zroot
zroot/iocage/jails/smb4-1/root                  atime     off    inherited from zroot
zroot/iocage/jails/smb4-2                       atime     off    inherited from zroot
zroot/iocage/jails/smb4-2/root                  atime     off    inherited from zroot
zroot/iocage/jails/sshpipe3                     atime     off    inherited from zroot
zroot/iocage/jails/sshpipe3/root                atime     off    inherited from zroot
zroot/iocage/jails/theheart-do_not_delete       atime     off    inherited from zroot
zroot/iocage/jails/theheart-do_not_delete/root  atime     off    inherited from zroot
zroot/iocage/log                                atime     off    inherited from zroot
zroot/iocage/releases                           atime     off    inherited from zroot
zroot/iocage/releases/12.1-RELEASE              atime     off    inherited from zroot
zroot/iocage/releases/12.1-RELEASE/root         atime     off    inherited from zroot
zroot/iocage/templates                          atime     off    inherited from zroot
zroot/poudriere                                 atime     off    local
zroot/poudriere/jails                           atime     off    inherited from zroot/poudriere
zroot/poudriere/jails/freebsd_12-1x64           atime     off    local
zroot/poudriere/ports                           atime     off    inherited from zroot/poudriere
zroot/poudriere/ports/PORTS-12-1                atime     off    local
zroot/tmp                                       atime     off    inherited from zroot
zroot/usr                                       atime     off    inherited from zroot
zroot/usr/home                                  atime     off    inherited from zroot
zroot/usr/ports                                 atime     off    inherited from zroot
zroot/usr/src                                   atime     off    inherited from zroot
zroot/var                                       atime     off    inherited from zroot
zroot/var/audit                                 atime     off    inherited from zroot
zroot/var/crash                                 atime     off    inherited from zroot
zroot/var/log                                   atime     off    inherited from zroot
zroot/var/mail                                  atime     on     local
zroot/var/tmp                                   atime     off    inherited from zroot
zroot/vm                                        atime     off    local
zroot/vm/smb4-a                                 atime     off    inherited from zroot/vm
zroot/vm/smb4-b                                 atime     off    inherited from zroot/vm



Other than for `iocage` I have not done very much with zfs on this system.

In any case, with `zsnap` background snapshoting turned off, beadm now seems to work:


```
[root@vhost01 ~ (master)]# beadm create hang-test
Created successfully

[root@vhost01 ~ (master)]# beadm activate hang-test
Activated successfully
[root@vhost01 ~ (master)]# zfs list | grep hang
zroot/ROOT/hang-test                             186G  4.70T   155G  /

[root@vhost01 ~ (master)]# zfs get origin zroot/ROOT/hang-test
NAME                  PROPERTY  VALUE   SOURCE
zroot/ROOT/hang-test  origin    -       -

[root@vhost01 ~ (master)]# beadm list
BE                 Active Mountpoint  Space Created
default            N      /          852.0K 2020-05-20 11:25
12.2-RELEASE-p13rp -      -          964.0K 2021-02-17 08:52
hang-test          R      -          186.0G 2021-02-17 11:23

[root@vhost01 ~ (master)]# zfs get origin zroot/ROOT/12.2-RELEASE-p13rp
NAME                           PROPERTY  VALUE                                     SOURCE
zroot/ROOT/12.2-RELEASE-p13rp  origin    zroot/ROOT/hang-test@2021-02-17-08:52:21  -

[root@vhost01 ~ (master)]# beadm activate 12.2-RELEASE-p13rp
Activated successfully

[root@vhost01 ~ (master)]# beadm list
BE                 Active Mountpoint  Space Created
default            N      /          852.0K 2020-05-20 11:25
12.2-RELEASE-p13rp R      -          186.0G 2021-02-17 08:52
hang-test          -      -            7.1M 2021-02-17 11:23

[root@vhost01 ~ (master)]# zfs get origin zroot/ROOT/12.2-RELEASE-p13rp
NAME                           PROPERTY  VALUE   SOURCE
zroot/ROOT/12.2-RELEASE-p13rp  origin    -       -

[root@vhost01 ~ (master)]# zfs get origin zroot/ROOT/default        
NAME                PROPERTY  VALUE                                     SOURCE
zroot/ROOT/default  origin    zroot/ROOT/hang-test@2021-02-17-11:23:00  -

[root@vhost01 ~ (master)]# zfs get origin zroot/ROOT/hang-test
NAME                  PROPERTY  VALUE                                              SOURCE
zroot/ROOT/hang-test  origin    zroot/ROOT/12.2-RELEASE-p13rp@2021-02-17-08:52:21  -

[root@vhost01 ~ (master)]# beadm activate default          
Activated successfully
[root@vhost01 ~ (master)]# beadm list
BE                 Active Mountpoint  Space Created
default            NR     /          186.0G 2020-05-20 11:25
12.2-RELEASE-p13rp -      -          975.0K 2021-02-17 08:52
hang-test          -      -            1.2M 2021-02-17 11:23

[root@vhost01 ~ (master)]# zfs get origin zroot/ROOT/default 
NAME                PROPERTY  VALUE   SOURCE
zroot/ROOT/default  origin    -       -

[root@vhost01 ~ (master)]# zfs get origin zroot/ROOT/hang-test
NAME                  PROPERTY  VALUE                                   SOURCE
zroot/ROOT/hang-test  origin    zroot/ROOT/default@2021-02-17-11:23:00  -

[root@vhost01 ~ (master)]# zfs get origin zroot/ROOT/12.2-RELEASE-p13rp
NAME                           PROPERTY  VALUE                                   SOURCE
zroot/ROOT/12.2-RELEASE-p13rp  origin    zroot/ROOT/default@2021-02-17-08:52:21  -
```

So, the background snapshoting may indeed be the culprit.


----------



## byrnejb (Feb 17, 2021)

> sko said:
> 
> 
> > I circumvent that confusion about naming/reverting by creating a BE with the name of the version I'm going to update to, mount that BE and start a jail with it. Then I perform the update within the jail environment, unmount & activate it, reboot and finish the update process.
> ...


----------



## SirDice (Feb 17, 2021)

sko said:


> I never actually needed one of those snapshots


I rarely use BEs but figured I might give it a shot when I upgraded one of my machines from 12-STABLE to 13-STABLE. Normally I just go for it, I can usually fix any botched upgrade/update I did but with the ZFS changes I was a little more hesitant. I remembered how I completely hosed my system with the UFS1->UFS2 transition with FreeBSD 5.0 by not reading /usr/src/UPDATING _before_ doing the upgrade


----------



## byrnejb (Feb 17, 2021)

Well, I am going to give this a whirl once more.


----------



## Mjölnir (Feb 17, 2021)

byrnejb, you have a fairly large on-disk /tmp.  If you have enough RAM, you may want to make that a tmpfs(5) instead, like in my fstab(5) above.  When there's memory pressure, it uses swap space, so on modern systems with RAM >=4 GB it is usually beneficial to have that, especially one can give it a _size_ option.  OTT, I'm wondering why your root filesystem is so large.  Like I wrote above, you can split off all commonly used data, e.g. /usr/obj, /usr/ports (or split that to _latest_ & _quarterly_ like I showed above) and /var/cache.  If you do not intend to switch between BEs and can tolerate some time gaps in data series when you try out new BEs, you can skip the next hint: many entries under /var/db can be shared among BEs.  Outline: `zfs create -o canmount=off zroot/var/db`, `cd /var/db`, then for each entry (service, e.g. net/ntopng, ...)

`zfs create -p zroot/x/entry`
`service enty stop`
`cd entry && find . -print0|cpio -p0dmv /x/entry`
`cd .. && rm -fr enty`
`zfs rename zroot/x/entry zroot/var/db/entry`
service entry start
Beware, some services have common data & data that is private for each BE, see above for e.g. security/tripwire (yes, oldschool) and security/cvechecker.


----------

