# How to properly update a system



## tOsYZYny (Jan 25, 2022)

Previously, I hosed my system through an automatic update.  Fortunately though, I was able to recover.

That said, I was perusing:


			https://freebsdfoundation.org/wp-content/uploads/2021/07/Seven-Ways-to-Increase-Security-in-a-New-FreeBSD-Installation.pdf
		


And, I am following all of the steps except:

```
pkg upgrade
```

It didn't say anything about any options we should use or avoid.

I think I was previously advised I shouldn't take all package updates because there could be random ones that could hose my system and that has happened to me at least once.

So, to make the process a bit more robust, I do:


```
1. check for updates
  freebsd-update fetch updatesready
  pkg upgrade -n (return code == 2)
  pkg audit -F (non-zero return code)

2. if updates are available
    a. determine latest patch sequence and increment it
    b. create new Boot Environment (including the patch)

3. instruct myself to reboot into the new BE
4. upon reboot (@reboot cron job)
     run freebsd-update fetch install
     run pkg upgrade -v -y (only install vulnerability fixes)
     check for further updates and notify the user

5. use the system as normal, if any problems are detected, revert to previous BE
```

I am wondering, but perhaps what I should do is once this is done, check if there are other updates available and create a separate BE for that so if the non-vulnerable updates broke the system, I can at least keep the vulnerability fixes.  I'm confirming that it is a bit risky to just take all pkg updates regardless.

I would also like to have "the latest and greatest" to the extent that it is stable and reliable.


----------



## Geezer (Jan 25, 2022)




----------



## jmos (Jan 25, 2022)

tOsYZYny said:


> I think I was previously advised I shouldn't take all package updates


I think that was a really bad advice.


----------



## tOsYZYny (Jan 25, 2022)

Why?


----------



## SirDice (Jan 25, 2022)

tOsYZYny said:


> I hosed my system through an automatic update.


Blindly running updates is never a good idea. 


tOsYZYny said:


> I think I was previously advised I shouldn't take all package updates because there could be random ones that could hose my system and that has happened to me at least once.


I think you misunderstood or you are taking advice for a specific issue out of context.


----------



## tOsYZYny (Jan 25, 2022)

SirDice said:


> I think you misunderstood or you are taking advice for a specific issue out of context.


That is entirely possible.

So, that leads me back to my original question, the PDF from the foundation didn't specify any arguments, so it would be updating everything; however, that doesn't sound like the best (wisest / safest) approach.

Would the best approach then be just to do:

```
pkg upgrade -v
```

So that only the vulnerabilities are patched and then if I feel lucky (after creating and booting into a new BE), do the remaining, 
	
	



```
pkg upgrade
```
?


----------



## SirDice (Jan 25, 2022)

Best way is to run `pkg upgrade` and let it update/upgrade everything it wants too. Dependencies are set at build time when the package is created. If you upgrade PackageA but not PackageB you can get discrepancies with this dependency chain. The newer version of PackageA might be linked to a newer version of PackageB. If you only updated PackageA then it can fail because PackageB is still at the old version.

Treat all the installed software as a whole, not as individual packages. And keep an eye on /usr/ports/UPDATING before updating anything.


----------



## mer (Jan 25, 2022)

About the only option I ever use on pkg upgrade is "-n"  Why?  So I can see what it wants to update and think on it.  If I currently have firefox open doing something for paying work, I don't want to pkg upgrade firefox.  It also lets me have a chance to see what depends on what, a chance to decide "do I need that".  Otherwise, I just let it run.
I'll sometimes create a new BE to upgrade into but not always, because BEs can accumulate like old newspapers.


----------



## tOsYZYny (Jan 25, 2022)

SirDice said:


> Best way is to run `pkg upgrade` and let it update/upgrade everything it wants too. Dependencies are set at build time when the package is created. If you upgrade PackageA but not PackageB you can get discrepancies with this dependency chain. The newer version of PackageA might be linked to a newer version of PackageB. If you only updated PackageA then it can fail because PackageB is still at the old version.



Okay, that makes sense.  It would be good if that were explained somewhere.  I would think that since the pkg command has the -v (vulnerabilities) flag that it would imply it is smart enough not to hose your system and that if pkg A depends on pkc C, that it will also pull in pkg C.

In regards to BE's, yes, that is something I'm looking to script with a cron job or find an existing pkg or tool that does it.  I want to periodically remove stale BE's whilst leaving me an escape hatch.  I really like how zap works, so I am thinking that I will use that as a reference and write some shell scripts with cron to achieve a similar thing with BE's.


----------



## jmos (Jan 25, 2022)

mer said:


> About the only option I ever use on pkg upgrade is "-n"  Why?  So I can see what it wants to update and think on it.


pkg gives you by default first a summary and asks you to proceed. So I don't use this option.


----------



## mer (Jan 25, 2022)

jmos said:


> pkg gives you by default first a summary and asks you to proceed. So I don't use this option.


Agreed, but it prevents me from doing anything stupid.


----------



## SirDice (Jan 25, 2022)

tOsYZYny said:


> Okay, that makes sense. It would be good if that were explained somewhere. I would think that since the pkg command has the -v (vulnerabilities) flag that it would imply it is smart enough not to hose your system and that if pkg A depends on pkc C, that it will also pull in pkg C.


During the three month period of a quarterly branch you can probably safely do `pkg upgrade -v`. During that three month period updates will be far and few in between. And will only have security and/or major break fixes. So there can't be any significant version differences. Just after a change-over (from Q1 to Q2 for example) you need to be more careful though. You will get _all_ the changes and updates that happened in that three month period on latest all at once. This can include potentially breaking updates, like a change from PostgreSQL 12 to 13. Or the next default version of Ruby, Perl, Python, LUA, etc.

The less often you update the bigger the jumps are going to get and therefor the bigger the risk of breaking changes. So my mantra is really, update regularly. If you track quarterly (-RELEASE versions do by default) at the very least once every three months.


----------



## grahamperrin@ (Jan 25, 2022)

tOsYZYny said:


> ```
> …
> 
> 2. if updates are available
> ...



If the installation of updates at step 2a creates an environment that is problematic with problems not _immediately_ obvious, then what *earlier* boot environment do you have?


----------



## tOsYZYny (Jan 26, 2022)

grahamperrin said:


> If the installation of updates at step 2a creates an environment that is problematic with problems not _immediately_ obvious, then what *earlier* boot environment do you have?



The script / cron I plan to create is that I would expire old BE's after a period of 30 days or some sort of timeout.  Once the BE is both not current and not scheduled to be active for the next reboot and is > 30 days since it was created, then I will destroy it.

I expect to catch problems when the patches are applied +- a day should let me touch all of the stuff I use on a day-to-day basis.  Anything beyond that would fall into the bucket of general triage / break / fix.


----------



## grahamperrin@ (Jan 26, 2022)

I mean, it seems that you install updates _before_ creating a new boot environment. To me this seems risky, because you might subsequently find problems with the updated *pre-new* environment.


----------



## tOsYZYny (Jan 26, 2022)

I don't believe so, maybe I misstated something.

1. check if updates are available, if yes, create new BE, and prompt user to reboot into BE
2. upon booting into new BE, utilizing cron via @reboot schedule, apply updates, repeat #1, if the user has not booted into the new BE (I create a patch file indicating the BE and what updates are to be applied), then a message will be logged and no updates applied.
3. attempt to prune old BE's daily


I tested out 1 and 2 today and it works, time will tell what I need to fix.  I will test the pruning process tomorrow.  Something like this needs to exist just like the beadm utility.


----------



## Deleted member 30996 (Jan 26, 2022)

Don't believe everything you hear.
Not everyone who acts like they know what they're talking about actually does.



tOsYZYny said:


> I would also like to have "the latest and greatest" to the extent that it is stable and reliable.


Me, too. That's never involved booting into a new BE in 16 years.

What it does involve is updating the ports tree snapshot, checking for new vulnerabilities and running freebsd-update on a daily basis:


```
portsnap fetch update
pkg audit -F
freebsd-update fetch
```

When there's a vulnerability in a port I upgrade as soon as it's available in the ports tree using ports-mgmt/portmaster. 
Whether or not I used pkg to install it, but that's me.

When there is an update to freebsd-update I run it as soon as it's available and then reboot:


```
freebsd-update iinstall
shutdown -r now
```

I run `portmaster -a` when I want to update my 3rd party rograms because ports are updated more frequently than pkg.


----------



## grahamperrin@ (Jan 26, 2022)

Thanks, now I can visualise that part.



tOsYZYny said:


> … Something like this needs to exist just like the beadm utility.



vermaden can you think of anything? 

I don't have a link handy, but I think first of code that might be archived in GitHub for PC-BSD.


On the *GhostBSD* side, FYI Better use of boot environments with Update Station - GhostBSD, the key points are probably: 

currently, Update Station creates a new boot environment as a backup
grahamperrin could you make a Feature Request about this?
… most of the major OSs upgrade on boot and not on the desktop. … will be needed for 13 upgrades to 14.









						GitHub - ghostbsd/update-station: GhostBSD update manager
					

GhostBSD update manager. Contribute to ghostbsd/update-station development by creating an account on GitHub.




					github.com
				




Also, less directly relevant: <https://github.com/ghostbsd/software-station>


On the *FreeBSD* side, re: the Foundation's Technology Roadmap, I have done nothing recent with PkgBase. Awaiting feedback on a bug report. 









						PkgBase
					

… how to safely update the system (regardless of how far out of date) reliably. …   Let's assume that PkgBase is the way forward.   and so on.  https://lists.freebsd.org/mailman/listinfo/freebsd-pkgbase  In addition to the list, there's sometimes discussion of PkgBase in IRC for FreeBSD...




					forums.freebsd.org


----------



## vermaden (Jan 26, 2022)

grahamperrin said:


> vermaden can you think of anything?



I do not think its that easy.

When less downtime is needed then I prefer this way of upgrade:
https://vermaden.wordpress.com/2021/02/23/upgrade-freebsd-with-zfs-boot-environments/

I do not think its possible to safely automate *pkg upgrade* or *freebsd-update install* parts. Boot Environments will just create a 'safe' BE that you can get back to safely.

You can not automatically 'just do' *pkg upgrade -y *because in the process of SAT solver (or any other automatic solver) you may end up with some packages removed. That means that for example *net/samba *will be removed as a conflicted dependency and you suddenly do not serve your SMB shares now ...

You can not automatically 'just do' *freebsd-update install *because sometimes config files need to be merged. You can end up with just plain 'skel files' that are part of fresh FreeBSD installation or with your config but without new group or new service user that would break some part of the FreeBSD Base System.

For the record I do not think that _PKG Base_ would solve that problem either - for the same reasons I gave for *pkg upgrade* command.

I may be wrong of course (and often am) but with current way how the *pkg(8)* and *freebsd-update(8)* work I do not see the point in such automatic updates. Of course the created BE would keep you safe and you can reboot back to working BE that you created before that automatic update process.

Also its not always needed to update. For example new security hole was found in Bluetooth stack*. *If you do not use Bluetooth at all you should just read the SA to check if its really harmless to you. Another example may be a bug in IPSec Tunnel implementation. If you do not use it then read SA and make sure upgrade is not needed. Same with packages. IMHO that takes less time (and less downtime) then having to *reboot(8)* bunch of servers into working BE created before automatic update - then disable the *auto-update-script.sh* thing temporary - then fix it (or make workarounds if not possible to fix) - then keep fingers crossed that this will not break again.

My $0.02.

Regards.


----------



## vermaden (Jan 26, 2022)

Another thing ... if the kernel is not involved in the update why reboot?

Create new BE.

Update currently running BE.

Then use *sysutils/checkrestart* to see what needs to be restarted.

Restart it.

Continue running without reboot.

Example output from *sysutils/checkrestart* shown below (from my laptop/desktop system).


```
# checkrestart
  PID   JID USER         COMMAND      WHY ARGUMENTS
81541     0 vermaden     firefox      bin /usr/local/lib/firefox/firefox
93327     0 vermaden     geany        .so /usr/local/bin/geany
56719     0 vermaden     telegram-des .so /usr/local/bin/telegram-desktop
81939     0 vermaden     firefox      bin /usr/local/lib/firefox/firefox
49428     0 vermaden     gvfs-udisks2 .so /usr/local/libexec/gvfs-udisks2-volume-monitor
56988     0 vermaden     firefox      bin /usr/local/lib/firefox/firefox
81820     0 vermaden     firefox      bin /usr/local/lib/firefox/firefox
74021     0 vermaden     firefox      bin firefox
31141     0 vermaden     openbox      .so /usr/local/bin/openbox
48933     0 vermaden     dzen2        .so /usr/local/bin/dzen2
78890     0 root         urxvt        .so /usr/local/bin/urxvt
65326     0 vermaden     deadbeef     .so /usr/local/bin/deadbeef
41393     0 vermaden     mkvtoolnix-g .so /usr/local/bin/mkvtoolnix-gui
78391     0 vermaden     urxvt        .so /usr/local/bin/urxvt
85055     0 vermaden     soffice.bin  .so /usr/local/lib/libreoffice/program/soffice.bin
75328     0 vermaden     firefox      bin /usr/local/lib/firefox/firefox
82244     0 vermaden     firefox      bin /usr/local/lib/firefox/firefox
41422     0 vermaden     tint2        .so /usr/local/bin/tint2
76115     0 root         Xorg         .so /usr/local/bin/Xorg
75228     0 vermaden     dbus-daemon  .so /usr/local/bin/dbus-daemon
66143     0 vermaden     caja         .so /usr/local/bin/caja
 7014     0 vermaden     pulseaudio   .so /usr/local/bin/pulseaudio
81274     0 vermaden     firefox      bin /usr/local/lib/firefox/firefox
32637     0 vermaden     transmission .so /usr/local/bin/transmission-gtk
```


----------



## tOsYZYny (Jan 26, 2022)

Yes, these are all very good points.

Food for thought, should I take the red pill or the blue pill .


----------



## Deleted member 30996 (Jan 26, 2022)

tOsYZYny said:


> Food for thought, should I take the red pill or the blue pill .


That's a choice you have to make:


----------



## zsolt (Jan 26, 2022)

What if someone take both pills?


----------



## Friend Of Jolly Devil (Jan 26, 2022)

S/he will be disappointed. Due to the laws of thermodynamics, people can't be a power source, it requires more energy to keep them alive than they can produce. Matrix is just a variation on perpetuum mobile, completely impossible


----------



## tOsYZYny (Jan 26, 2022)

That wasn't the blue pill I was going to take.



vermaden said:


> Another thing ... if the kernel is not involved in the update why reboot?



It is my understanding that a new BE requires a reboot.  If the update is problematic and I don't create a new BE just because it doesn't have kernel updates, then some user application will be impacted.




vermaden said:


> Example output from *sysutils/checkrestart* shown below (from my laptop/desktop system).



Another good point, once done, I should record that and notify the members of the wheel group.

I think the bottom line is that these updates cannot be 100% automated because they might involve configuration changes .  In that case, it'd still be nice to automate as much as possible and guide the user through the process just like the freebsd installer does at install time.


----------



## tOsYZYny (Feb 24, 2022)

EDIT:

Since I have a custom kernel, then my /etc/freebsd-update.conf should look like:

```
Components src world
```


Then, I should also do a git pull on the kernel source, and if there are any updates, run make buildkernel and installkernel?


----------

