# HOWTO: keeping FreeBSD's base system and packages up-to-date



## vermaden (Sep 5, 2011)

Today, I would like to 'touch' an ungrateful topic of keeping both FreeBSD's base system and installed packages up-to-date.

After I started using FreeBSD at 5.4 times (2005) I have tried various methods of keeping my FreeBSD installations up-to-date, many of them terribly failed, but some recent ones seem to do the job as advertised. Even not so recently ago I thought, lets stick to RELEASE and do not compile newer versions of packages as there are available packages at FTP ... but there is a big problem with such attitude. First, once the RELEASE is completed, there are only security fixes for the base system, but there are no bug fixes for the RELEASE. Its even worse with packages for RELEASE since once they are built they are never later updated, even if they have security issues, not even mentioning bugs. So that is definitely not the right way.

The sollution seems to be tracking STABLE tree for the base system along with packages that are built every 2 weeks for the STABLE tree and compiling only when there are security issues in some of the installed packages, but there are for example 10 more days before their rebuilt versions would show up on the STABLE tree FTP. Below I would try to describe all that process of keeping FreeBSD up-to-date as simple as possible. In the first part I would focus on the base system and the second one will cover keeping packages up-to-date.

Some important information about keeping Your system this way. You would not rebuild the base system every day, not even every week, just when needed. Now what does it mean 'when needed' ... For example when there is a security issue, You would just follow the instructions in the SA (security advisory) to fix that issue, there is no need to rebuild whole world. The only reasons to rebuild the base system are that there has been found and fixed a bug in STABLE that affects You or that You need new features that has been merged into the STABLE branch (from CURRENT for example) like newer ZFS version or whatever.

As for the installation, You can install the RELEASE version and update to STABLE or install the daily STABLE snapshot so You would not have to build entire base system from source, the daily ISO images are available at http://pub.allbsd.org/FreeBSD-snapshots/ server.

*EDIT:*

Article also available as PDF here: http://bsdmag.org/magazine/1784-freebsd-get-up-to-date

If You want to have it separated from other articles, here's the way:
[CMD=""]# pkg_add -r pdftk
% pdftk 'BSDmag 2012-01.pdf' cat 28-36 output 'BSDmag 2012-01 up-do-date.pdf'[/CMD]

... or as plain HTML at the bottom of this post.


----------



## vermaden (Sep 5, 2011)

*PART I. Keeping the FreeBSD base system up-to-date*

*Some facts about FreeBSD's base system:*
-- once RELEASE is completed, there are only security fixes, there are no bug fixes
-- bugs in STABLE tree are fixed
-- security issues are also fixed in STABLE
-- the RELEASE branch allows to use binary updates via freebsd-update tool for security fixes
-- the STABLE branch requires compiling of the FreeBSD base system

We need to clone the current cource tree if we want to build up to date STABLE branch FreeBSD's base system, we will also need to update our sources to the current state so its quite handy to find fastest server for Your location, it can be easily done by using fastest_cvsup package.


```
# [color="Blue"]pkg_add -r fastest_cvsup[/color]
# [color="blue"]fastest_cvsup -c all[/color]
(...)

>>  Speed Daemons:
    - 1st: cvsup.pl.freebsd.org
    - 2nd: cvsup11.ua.freebsd.org
    - 3rd: cvsup5.de.freebsd.org
```

For my location it is *cvsup.pl.freebsd.org* which in most cases will be different ther for Your location, so remember to put Your's fastest in the next steps.

Create simple *supfile* that will be used by csup to keep FreeBSD's base system sources up-to-date. There are useful examples under /usr/share/examples/cvsup/ if you want to 'dig more'.


```
# [color="blue"]cat > /root/stable-supfile << EOF
*default host=[b]cvsup.pl.freebsd.org[/b]
*default base=/var/db
*default prefix=/usr
*default release=cvs tag=RELENG_8
*default delete use-rel-suffix
*default compress
src-all
EOF[/color]
```

Now lets get/update our sources to the current state, the list of edited/checked files will be quite different on Your box since I already have quite up-to-date sources, this will take more time if You do not have the sources on the disk.


```
# [color="blue"]csup /root/stable-supfile[/color]
Connected to 188.125.237.138
Updating collection src-all/cvs
 Edit src/bin/ed/buf.c
 Edit src/sbin/fsck_ffs/main.c
 Edit src/sbin/mdconfig/mdconfig.8
 Edit src/sbin/mdconfig/mdconfig.c
 Edit src/share/man/man4/ath.4
 Edit src/share/man/man4/ath_hal.4
 Edit src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c
 Edit src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c
 Edit src/sys/dev/ahci/ahci.c
 Edit src/sys/fs/msdosfs/msdosfs_vnops.c
 Edit src/sys/fs/nfsclient/nfs_clbio.c
 Edit src/sys/fs/nfsserver/nfs_nfsdserv.c
 Edit src/sys/fs/nwfs/nwfs_io.c
 Edit src/sys/fs/smbfs/smbfs_io.c
 Edit src/sys/fs/tmpfs/tmpfs_vnops.c
 Edit src/sys/gnu/fs/xfs/FreeBSD/xfs_vnops.c
 Edit src/sys/kern/uipc_usrreq.c
 Edit src/sys/kern/vfs_vnops.c
 Edit src/sys/nfsclient/nfs_bio.c
 Edit src/sys/sparc64/sbus/sbus.c
 Edit src/sys/sys/vnode.h
 Edit src/sys/ufs/ffs/ffs_inode.c
 Edit src/sys/ufs/ffs/ffs_vnops.c
Finished successfully
```

Alternatively, You can grab the sources by SVN protocol, but You will need devel/subversion16 port/package for that purpose. Its generally a lot faster/easier to 'setup' then csup but the _'csup way'_ has one important advantage, its in the FreeBSD's base system, so its always available, anywhere. With SVN, You will have to add a package first which sometimes may be cumbersome. But as the FreeBSD source tree is kept under SVN it is possible that SVN will be part of the FreeBSD's base system one day.

Its also important to mention, that sources downloaded by subversion are not compatible with the sources grabbed by csup, so once You will decide which method to use, stick with it, unless You want to download the whole FreeBSD's source tree again. Below is the line needed to update the FreeBSD sources to 9-STABLE latest state.


```
# [color="blue"]svn checkout svn://svn.freebsd.org/base/stable/9 /usr/src[/color]
```

Its the same no matter if You download the whole tree or just doing an update from yesterday. If svn will complain about anything, just delete the /usr/src and type the command again.

Now as we have the sources we can continue to building the FreeBSD's base system from source. As for editing the kernel config, You do not even have to bother about it, just use GENERIC, this guide is not about stripping the base system and kernel components, its about keeping everything up-to-date. Of course if You want to, then use Your tweaked kernel config, it will not interfere with the rest of this guide. You may want to put nice -n 20 in front of make buildworld ... line to make that build process less 'amusing' for your system. As instructions are completed, Your system will reboot.


```
# [color="blue"]cd /usr/src[/color]
# [color="blue"]rm -r -f /usr/obj[/color]
# [color="blue"]make buildworld kernel KODIR=/boot/testing[/color]
# [color="blue"]nextboot -k testing[/color]
# [color="blue"]shutdown -r now[/color]
```

We are now proceeding to the second phase of the upgrade process, after normal boot (single user mode not required and definitely prohibited while doing upgrade over the network) stop all unneeded services (remember to keep sshd daemon alive if you are doing upgrade via network). If your system booted up properly, then You can make the new testing kernel the default one, at least there should not be any problems with the GENERIC kernel config 


```
# [color="blue"]cd /boot[/color]
# [color="blue"]rm -r -f OLD[/color]
# [color="blue"]mv kernel OLD[/color]
# [color="blue"]mv testing kernel[/color]
```


```
# [color="Blue"]killall dhclient moused[/color]
# [color="blue"]/etc/rc.d/cron stop[/color]
# [color="blue"]/etc/rc.d/devd stop[/color]
# [color="blue"]/etc/rc.d/sshd stop[/color]
# [color="blue"]/etc/rc.d/powerd stop[/color]
# [color="blue"]/etc/rc.d/syslogd stop[/color]
```

The list of started processes will look something like that including (or not) sshd for keeping up the network connection.


```
# [color="blue"]top -b[/color]
last pid: 64835;  load averages:  0.00,  0.00,  0.00  up 0+03:11:51    10:24:37
119 processes: 2 running, 117 sleeping

Mem: 960M Active, 355M Inact, 4014M Wired, 6096K Cache, 8368K Buf, 2527M Free
Swap:

  PID USERNAME      THR PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
 2178 root            1  54    0 10304K  2748K ppwait  0   0:00  0.00% csh
 2174 root            1  45    0 21696K  1992K wait    0   0:00  0.00% login
 2177 root            1  76    0  6912K  1284K ttyin   0   0:00  0.00% getty
 2176 root            1  76    0  6912K  1284K ttyin   1   0:00  0.00% getty
 2175 root            1  76    0  6912K  1284K ttyin   0   0:00  0.00% getty
  114 root            1  76    0  2764K  1056K pause   0   0:00  0.00% adjkerntz
```

Now we can continue to type rest of needed instructions to finish the update, the mergemaster will ask You for the differences in startup scripts that You have modified and configuration files, type 'I' to install the new/default config and/or script and select 'D' to leave the version that you have in the system, remember that You can also add these changes later, it may be not appreciate to install default firewall config or customized OpenSSH config while doing the network upgrade.


```
# [color="blue"]cd /usr/src[/color]
# [color="blue"]mergemaster -p[/color]
# [color="blue"]make installworld[/color]
# [color="blue"]mergemaster -iU[/color]
# [color="blue"]make delete-old[/color]
# [color="blue"]shutdown -r now[/color]
```

After that second reboot You should have updates to STABLE branch FreeBSD's base system, I wrote 'should' because sometimes things do not go the way we want them to go, especially if you are doing it the first time as once _Aerosmith_ sing _"I know it's everybody's sin, You got to lose to know how to win"_. It would be best to do these instructions as exercise under virtual machine like VirtualBox or QEMU.

Also, if you do not feel that STABLE is 'production enought', then You may want to use STABLE packages along with RELEASE base system, You will need to define environment variable PACKAGESITE that will point to ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8-stable/Latest/ at least for FreeBSD 8.x system.


----------



## vermaden (Sep 5, 2011)

*PART II. Keeping the FreeBSD packages up-to-date*

Keeping packages up to date is little more tricky, we will also need the STABLE branch for them as these in RELEASE are not updated. Lets assume that You installed the FreeBSD STABLE snapshot a month ago, along with packages that were built by then, now there will be quite a lot of new versions for many packages which is not that important, but some of them can (and probably have) security issues and definitely should be updated. You can of course compile them from Ports using portmaster but why waste time for compiling, when You can use built every 2 weeks packages from the STABLE branch? The pkg_upgrade script from the bsdadminscripts package will be quite helpful here. It will fetch latest available packages from the STABLE FTP and there is a chance that the security issues will be solved by the newer versions, if not, we are forced to rebuild those packages from source using portmaster, but its a lot better and faster to recompile 1-2 packages instead of 30 or more.

As for updating the packages, I generally check them daily, mostly for security issues that would be reported with portaudit, there are often new versions reported, sometimes even quite lot, but as long as there are 0 problem(s) in your installed packages found. I do not bother. From time to time I fire up pkg_upgrade -a -C to fetch the latest packages from the STABLE branch FTP.

Some of You would certainly ask why use pkg_upgrade instead of updating with portmaster? Well, for example You have package *z-1.0* installed in Your system, latest package available on the FTP is *z-1.1* (newer) but version in Ports is *z-1.2*, so portmaster will omit that *z-1.1* package no matter if its newer or not and will force You to compile the *z-1.2* package from the Ports system.

*Keeping FreeBSD packages up-to-date in short:*
-- use packages from STABLE that are built every 2 weeks
-- use pkg_upgrade to update packages
-- use portmaster to rebuild packages that have security issues

*Some facts about being up-to-date with FreeBSD's packages:*
-- with every RELEASE packages are built and then they are never updated, even if they have security issues
-- for the STABLE tree packages are rebuilt every 2 weeks

First, we need to install tools that we will use to keep FreeBSD packages up-to-date.


```
# [color="blue"]pkg_add -r bsdadminscripts portmaster portaudit[/color]
```

Optionally, we can allow users in group *wheel* to perform these task using sudo (You will have to add *sudo* package with pkg_add -r sudo commend) as they are already allowed to login on the *root* account, we can of course create separate group like *maintainers* that will be allowed to perform upgrades. You will need this line below in /usr/local/etc/sudoers file.


```
%wheel ALL=NOPASSWD: /usr/sbin/portsnap, /usr/local/sbin/pkg_upgrade, /usr/local/sbin/portmaster, /usr/local/sbin/portaudit
```

Here is the most important part, the commands put together into functions that will allow us easy checking for newer versions of the packages, security issues and updating them to newer/fixed versions. The *ports-check* function fetches latest Ports tree, then shows what new packages are available comparing to those installed on the system, next the security issues are checked with portaudit and last, the /usr/ports/UPDATING file is checked for various messages that can affect us. The *ports-check* does not rebuild or update any packages, only, as the name says, checks.


```
[color="blue"]function ports-check {
  # FETCH LATEST PORTS TREE
  sudo portsnap fetch update

  # CHECK WHAT NEW VERSIONS EXIST
  sudo portmaster -L --index-only | awk '/ [Nn]ew / { print substr($0,9,9999) }'

  # CHECK SECURITY ISSUES
  sudo portaudit -Fda

  # CHECK /usr/ports/UPDATING MESSAGES
  pkg_updating -d $( ls -ltr -D '%Y%m%d' /var/db/pkg | awk 'END{print $6}' )
}[/color]
```

The second function, *ports-update* is for updating the packages using the STABLE branch, it uses pkg_upgrade from bsdadminscripts, but it will not compile from Ports.


```
[color="blue"]function ports-update {
  # UPDATE PACKAGES USING STABLE PATH ON FTP
  sudo pkg_upgrade -a -C 2>&1 | grep --color=none --line-buffered -E "^(=+>|/usr/ports|/var/db)"

  # FIX DEPENDENCIES AS NEEDED
  sudo portmaster --check-depends
}[/color]
```

The last one named *ports-build* rebuilds the specified package or all of them if You do not specify one, used mostly to rebuild packages with security issues.


```
[color="blue"]function ports-build {
  # REBUILD SINGLE, SEVERAL OR ALL PORTS
  case ${#} in
    (0) sudo portmaster -y --no-confirm --packages-if-newer -m 'BATCH=yes' -d -a ;;
    (*) sudo portmaster -y --no-confirm --packages-if-newer -m 'BATCH=yes' -d $@ ;;
  esac

  # FIX DEPENDENCIES AS NEEDED
  sudo portmaster --check-depends
}[/color]
```

... and thats it generally, I would show some example of these functions usage below.

You will have to put these functions into Your shells startup files, it will be /etc/profile for *sh* shell and *bash*, /etc/zshrc for *zsh*. It will not work for C-shells like *csh* since they do not support functions and are retarded in many other ways: http://www.grymoire.com/Unix/CshTop10.txt

*Drawbacks*

Using this way of keeping the installed packages up-to-date You have to remember two things.

Customized packages. If You built some package with non-default options by compiling it, after upgrade it will 'revert' do the default options and You will have to build it again.

Kernel modules. Any package that comes with kernel modules can and probably will break at some point because the STABLE source tree is a 'moving target', that is one of the good reasons to update the base system and then update to latest packages. The packages that have kernel modules are for example emulators/virtualbox-ose (VirtualBox), sysutils/fusefs-kmod (FUSE implementation) and most notably x11/nvidia-driver (binary nVidia graphics driver).


----------



## vermaden (Sep 5, 2011)

*EXAMPLE II.1. Typical output about new/updated ports and new versions available.*


```
% [color="blue"]ports-check[/color]
Looking up portsnap.FreeBSD.org mirrors... 5 mirrors found.
Fetching snapshot tag from portsnap5.freebsd.org... done.
Fetching snapshot metadata... done.
Updating from Mon Sep  5 07:11:28 CEST 2011 to Mon Sep  5 08:51:01 CEST 2011.
Fetching 3 metadata patches.. done.
Applying metadata patches... done.
Fetching 0 metadata files... done.
Fetching 10 patches.....10 done.
Applying patches... done.
Fetching 1 new ports or files... done.
Removing old files and directories... done.
Extracting new files:
/usr/ports/chinese/c2t/
/usr/ports/chinese/hc/
/usr/ports/devel/Makefile
/usr/ports/devel/p5-System-Command/
/usr/ports/german/mythes/
/usr/ports/math/p5-Statistics-R/
/usr/ports/polish/hunspell/
/usr/ports/textproc/es-mythes/
/usr/ports/textproc/nl-mythes/
/usr/ports/textproc/sk-mythes/
/usr/ports/textproc/sl-mythes/
Building new INDEX files... done.
New version available: arc-5.21p
New version available: ca_root_nss-3.12.11_1
New version available: expat-2.0.1_2
New version available: tinyxml-2.6.2
New version available: bash-4.1.11
New version available: gstreamer-plugins-0.10.35_1,3
New version available: gtk-2.24.6
New version available: gtk-update-icon-cache-2.24.6
New version available: libsamplerate-0.1.8_1
New version available: nas-1.9.3
New version available: nettle-2.4
New version available: p5-Date-Manip-6.25
New version available: p5-Mail-IMAPClient-3.29
New version available: p5-XML-Parser-2.41
New version available: xterm-273
New version available: filezilla-3.5.1
New version available: firefox-6.0.1,1
New version available: gtk-oxygen-engine-1.1.2
New version available: nginx-1.0.6,1
New version available: qemu-0.11.1_10
20 have new versions available
New database installed.
Database created: Thu Sep  1 21:20:00 CEST 2011
0 problem(s) in your installed packages found.
```

*EXAMPLE II.2. A report that also shows some security issues.*


```
% [color="blue"]ports-check[/color]
Looking up portsnap.FreeBSD.org mirrors... 5 mirrors found.
Fetching snapshot tag from portsnap1.freebsd.org... done.
Fetching snapshot metadata... done.
Updating from Mon Sep  5 10:28:51 CEST 2011 to Mon Sep  5 12:07:23 CEST 2011.
Fetching 3 metadata patches.. done.
Applying metadata patches... done.
Fetching 0 metadata files... done.
Fetching 20 patches.....10....20 done.
Applying patches... done.
Fetching 1 new ports or files... done.
Removing old files and directories... done.
Extracting new files:
/usr/ports/MOVED
/usr/ports/Mk/bsd.sites.mk
/usr/ports/cad/Makefile
/usr/ports/devel/Makefile
/usr/ports/devel/p5-Bread-Board-Declare/
/usr/ports/devel/p5-Curses-UI/
/usr/ports/devel/p5-Data-Peek/
/usr/ports/devel/p5-Scope-Upper/
/usr/ports/dns/pear-Net_DNS2/
/usr/ports/lang/p5-Try-Tiny/
/usr/ports/mail/p5-Email-Valid/
/usr/ports/math/p5-Math-BigInt/
/usr/ports/math/p5-Statistics-Descriptive/
/usr/ports/net/pear-Net_SMTP/
/usr/ports/ports-mgmt/portaudit-db/
/usr/ports/sysutils/py-supervisor/
/usr/ports/sysutils/zfsnap/
/usr/ports/www/nginx-devel/
/usr/ports/www/nginx/
/usr/ports/www/rubygem-passenger/
/usr/ports/x11/Makefile
Building new INDEX files... done.
New version available: arc-5.21p
New version available: ca_root_nss-3.12.11_1
New version available: expat-2.0.1_2
New version available: tinyxml-2.6.2
New version available: bash-4.1.11
New version available: gstreamer-plugins-0.10.35_1,3
New version available: gtk-2.24.6
New version available: gtk-update-icon-cache-2.24.6
New version available: libsamplerate-0.1.8_1
New version available: nas-1.9.3
New version available: nettle-2.4
New version available: p5-Date-Manip-6.25
New version available: p5-Mail-IMAPClient-3.29
New version available: p5-XML-Parser-2.41
New version available: xterm-273
New version available: filezilla-3.5.1
New version available: firefox-6.0.1,1
New version available: gtk-oxygen-engine-1.1.2
New version available: nginx-1.0.6,1
New version available: qemu-0.11.1_10
20 have new versions available
auditfile.tbz                                 100% of   69 kB   54 kBps
New database installed.
Database created: Mon Sep  5 12:35:01 CEST 2011
Affected package: ca_root_nss-3.12.9
Type of problem: ca_root_nss -- Extraction of unsafe certificates into trust bundle..
Reference: [url]http://portaudit.FreeBSD.org/1b27af46-d6f6-11e0-89a6-080027ef73ec.html[/url]

Affected package: ca_root_nss-3.12.9
Type of problem: nss/ca_root_nss -- Fraudulent Certificates issued by DigiNotar.nl.
Reference: [url]http://portaudit.FreeBSD.org/aa5bc971-d635-11e0-b3cf-080027ef73ec.html[/url]

2 problem(s) in your installed packages found.

You are advised to update or deinstall the affected package(s) immediately.
```



*EXAMPLE II.3. Solving the security issues by rebuilding the problematic package.*


```
% [color="blue"]ports-build ca_root_nss[/color]

===>>> Currently installed version: ca_root_nss-3.12.9
===>>> Port directory: /usr/ports/security/ca_root_nss

(...)

===>>> Updating dependency entry for ca_root_nss-3.12.11_1 in each dependent port
===>>> Upgrade of ca_root_nss-3.12.9 to ca_root_nss-3.12.11_1 complete

% [color="blue"]ports-check[/color]
Looking up portsnap.FreeBSD.org mirrors... 5 mirrors found.
Fetching snapshot tag from portsnap2.freebsd.org... done.
Fetching snapshot metadata... done.
Updating from Mon Sep  5 12:07:23 CEST 2011 to Mon Sep  5 12:25:09 CEST 2011.
Fetching 3 metadata patches.. done.
Applying metadata patches... done.
Fetching 0 metadata files... done.
Fetching 2 patches.. done.
Applying patches... done.
Fetching 0 new ports or files... done.
Removing old files and directories... done.
Extracting new files:
/usr/ports/devel/p5-File-NFSLock/
/usr/ports/devel/p5-MooseX-Aliases/
Building new INDEX files... done.
New version available: arc-5.21p
New version available: expat-2.0.1_2
New version available: tinyxml-2.6.2
New version available: bash-4.1.11
New version available: gstreamer-plugins-0.10.35_1,3
New version available: gtk-2.24.6
New version available: gtk-update-icon-cache-2.24.6
New version available: libsamplerate-0.1.8_1
New version available: nas-1.9.3
New version available: nettle-2.4
New version available: p5-Date-Manip-6.25
New version available: p5-Mail-IMAPClient-3.29
New version available: p5-XML-Parser-2.41
New version available: xterm-273
New version available: filezilla-3.5.1
New version available: firefox-6.0.1,1
New version available: gtk-oxygen-engine-1.1.2
New version available: nginx-1.0.6,1
New version available: qemu-0.11.1_10
19 have new versions available
auditfile.tbz                                 100% of   69 kB   54 kBps
New database installed.
Database created: Mon Sep  5 12:40:01 CEST 2011
0 problem(s) in your installed packages found.
```



*EXAMPLE II.4. Updating the installed packages using STABLE branch.*


```
% [color="Blue"]ports-update[/color]
/var/db/uma/FTPINDEX                          100% of   21 MB 1139 kBps 00m00s
/usr/ports/packages/All/nettle-2.4.tbz        100% of 1082 kB  332 kBps
/usr/ports/packages/All/gtk-2.24.6.tbz        100% of 9822 kB  253 kBps 00m00s
/usr/ports/packages/All/gstreamer-plugins-0.10100% of 4091 kB  942 kBps
/usr/ports/packages/All/gtk-oxygen-engine-1.1.100% of  509 kB  339 kBps
/usr/ports/packages/All/filezilla-3.5.1.tbz   100% of 3301 kB  232 kBps 00m00s
/usr/ports/packages/All/nas-1.9.3.tbz         100% of  487 kB  494 kBps
/usr/ports/packages/All/expat-2.0.1_2.tbz     100% of  130 kB  129 kBps
/usr/ports/packages/All/xterm-273.tbz         100% of  262 kB  104 kBps
/usr/ports/packages/All/p5-XML-Parser-2.41.tbz100% of  184 kB  381 kBps
/usr/ports/packages/All/nginx-1.0.6,1.tbz     100% of  225 kB  206 kBps
/usr/ports/packages/All/qemu-0.11.1_10.tbz    100% of   12 MB  359 kBps 00m00s
/usr/ports/packages/All/tinyxml-2.6.2.tbz     100% of  170 kB  121 kBps
/usr/ports/packages/All/p5-Date-Manip-6.25.tbz100% of 1301 kB  681 kBps
===> Update <tinyxml-2.6.1_1> to <tinyxml-2.6.2> (textproc/tinyxml)
=> Update <tinyxml-2.6.1_1> to <tinyxml-2.6.2> (textproc/tinyxml) succeeded
===> Update <p5-Date-Manip-6.24> to <p5-Date-Manip-6.25> (devel/p5-Date-Manip)
=> Update <p5-Date-Manip-6.24> to <p5-Date-Manip-6.25> (devel/p5-Date-Manip) succeeded
===> Update <nginx-1.0.5,1> to <nginx-1.0.6,1> (www/nginx)
=> Update <nginx-1.0.5,1> to <nginx-1.0.6,1> (www/nginx) succeeded
===> Update <nettle-2.2> to <nettle-2.4> (security/nettle)
=> Update <nas-1.9.2> to <nas-1.9.3> (audio/nas) succeeded
===> Update <expat-2.0.1_1> to <expat-2.0.1_2> (textproc/expat2)
=> Update <expat-2.0.1_1> to <expat-2.0.1_2> (textproc/expat2) succeeded
===> Update <xterm-271> to <xterm-273> (x11/xterm)
=> Update <xterm-271> to <xterm-273> (x11/xterm) succeeded
===> Update <qemu-0.11.1_9> to <qemu-0.11.1_10> (emulators/qemu)
=> Update <qemu-0.11.1_9> to <qemu-0.11.1_10> (emulators/qemu) succeeded
===> Update <gtk-2.24.5_1> to <gtk-2.24.6> (x11-toolkits/gtk20)
=> Update <gtk-2.24.5_1> to <gtk-2.24.6> (x11-toolkits/gtk20) succeeded
===> Update <firefox-6.0_1,1> to <firefox-6.0.1,1> (www/firefox)
=> Update <firefox-6.0_1,1> to <firefox-6.0.1,1> (www/firefox) succeeded
===> Update <filezilla-3.5.0_1> to <filezilla-3.5.1> (ftp/filezilla)
=> Update <filezilla-3.5.0_1> to <filezilla-3.5.1> (ftp/filezilla) succeeded
===> Update <bash-4.1.10> to <bash-4.1.11> (shells/bash)
=> Update <bash-4.1.10> to <bash-4.1.11> (shells/bash) succeeded
===> Update <arc-5.21o_1> to <arc-5.21p> (archivers/arc)
=> Update <arc-5.21o_1> to <arc-5.21p> (archivers/arc) succeeded
===>>> Checking 2bsd-vi-050325_1
===>>> Checking ImageMagick-6.7.1.10
===>>> Checking ORBit2-2.14.19
===>>> Checking OpenEXR-1.6.1_3
===>>> Checking Thunar-1.2.2_2
(...)
```


----------



## vermaden (Sep 5, 2011)

Feel free to sent Your opinions/questions and Your ways of keeping both FreeBSD's base system and installed packages.


----------



## Kiiski (Sep 5, 2011)

Hi

 I use ports myself, but about this:



> Some of You would certainly ask why use pkg_upgrade instead of updating with portmaster? Well, for example You have package z-1.0 installed in Your system, latest package available on the FTP is z-1.1 (newer) but version in Ports is z-1.2, so portmaster will omit that z-1.1 package no matter if its newer or not and will force You to compile the z-1.2 package from the Ports system.



From 
`man portmaster`


```
--packages-if-newer
         use package if newer than installed even if the package is not the
         latest according to the ports tree
```

Shouldn't that take care of situation you described?


----------



## vermaden (Sep 6, 2011)

@Kiiski

Maybe it has changed recently, portmaster always used the Ports tree version on my box, I also tried the -PP option and if the first package to update does not exist on the FTP, portmaster will end its work with an error instead of trying to get update for the next package.


----------



## vermaden (Sep 8, 2011)

Kiiski said:
			
		

> Shouldn't that take care of situation you described?



I have just checked how portmaster -a -PP again behaves and that is the result:


```
[color="green"]===>>> Checking package repository for latest available version[/color]

[color="Red"]===>>> The newest available package (sudo-1.8.1_5)
       is older than the version in ports (sudo-1.8.2)[/color]

===>>> Try --packages-if-newer, or do not use -PP/--packages-only
===>>> Aborting update

===>>> Update for security/sudo failed
[color="Red"]===>>> Aborting update[/color]
```

The --packages-if-newer option is useless here since it will try to REBUILD/COMPILE all packages and we want to omit that.


----------



## alie (Sep 19, 2011)

Awesome good job! this thread enters Distrowatch weekly issue

http://distrowatch.com/weekly.php?issue=20110919#news


----------



## Yampress (Sep 19, 2011)

very good work Ver.  ;D


----------



## nORKy (Sep 19, 2011)

Hi, I have 1 question : What is the diffrence between portmaster and pkg_* ?

EDIT : and an other : where do you track STABLE bugfix/securityfix ??


----------



## wblock@ (Sep 19, 2011)

pors-mgmt/portmaster is an add-on tool for managing ports and packages.  The various pkg_* programs are base system programs that only deal with packages.


----------



## Symbiosis (Sep 19, 2011)

Hi,

First of all thanks for your useful post.

I want to comment something about this:



> Also, if you do not feel that STABLE is 'production enought', then You may want to use STABLE packages along with RELEASE base system, You will need to define environment variable PACKAGESITE that will point to ftp://ftp.freebsd.org/pub/FreeBSD/po...stable/Latest/ at least for FreeBSD 8.x system.



I think that if you are on -RELEASE branch and you change to -STABLE packages you have to modify the environment variable 'BRANCH' for uma. If I don't add BRANCH=8-stable to my /usr/local/etc/uma.conf then the script ports-update doesn't work.

Best regards


----------



## vermaden (Sep 19, 2011)

@alie;@Yampress;@Symbiosis  

Thanks mates 



			
				nORKy said:
			
		

> Hi, I have 1 question : What is the diffrence between portmaster and pkg_* ?



Beyond the difference that *wblock *already mentioned, *pkg_** tools allows You to install/manage/delete binary packages from FTP and/or other source and *portmaster* is a tool tat automates (re)building from source using the _FreeBSD's Ports _tree.



			
				nORKy said:
			
		

> EDIT : and an other : where do you track STABLE bugfix/securityfix ??



I check SA at the usual place, its the same for STABLE and RELEASE: http://www.freebsd.org/security/notices.html

... about BUGs, well, I browse from time to time this one: http://svnweb.freebsd.org/base/

Also reading FreeBSD's Mailing Lists sometimes helps where 'that and that has been merged from current' or 'fixed in stable'.



			
				Symbiosis said:
			
		

> I think that if you are on -RELEASE branch and you change to -STABLE packages you have to modify the environment variable 'BRANCH' for uma. If I don't add BRANCH=8-stable to my /usr/local/etc/uma.conf then the script ports-update doesn't work.



Interesting, I havent heard before about UMA, must read more about this, but it seams that its not applicable here since UMA is for using RELASE packages on STABLE and CURRENT systems while this guide is for the opposite, using STABLE packages on RELEASE system


----------



## mgp (Sep 20, 2011)

You may want to try portrac to keep track of the port updates. It's a simple GUI tool that checks periodically if there are updates to the ports that are installed. I'd also love to get some feedback about the tool.


----------



## vermaden (Sep 20, 2011)

@mgp

I have just tried it, not impressed, have that functionality already and even with port versions


----------



## nORKy (Sep 20, 2011)

This how-to is very interesing but, I don't think I can track STABLE on my productions servers. I don't want to stop my sever too many times when there is a bug in the base system.
I think, the best for me, it's to have a RELEASE base system  and packages compiled and up to date.

I'm building a local server ports for my servers and a configuration mangament server (with puppet). I think it's better for me. And, in the futur, I think about a PXE boot, and automatic installations.


----------



## vermaden (Sep 20, 2011)

nORKy said:
			
		

> This how-to is very interesting but, I don't think I can track STABLE on my productions servers.


Currently I am using this 'way' on my laptop and home NAS, but I agree that for servers that have compiled ports it would require to rebuild the needed ports every time there was packages available, which may also be a little pain, on servers its also often 3-5 ports with little dependencies that need to be rebuild, so it also does not take ages like with the whole desktop/x11/multimedia and so.



> I don't want to stop my sever too many times when there is a bug in the base system.


Who forces You to do so? 

As I written in the HOWTO, its done very rarely, for most of the time its not needed to rebuild the base system, but on STABLE You have that possibility, on RELEASE, You are 'dead' 



> I'm building a local server ports for my servers and a configuration mangament server (with puppet). I think it's better for me. And, in the futur, I think about a PXE boot, and automatic installations.



That is the best attitude, even to have the own *freebsd-update *server, so You could fix these potential bugs from STABLE using the *freebsd-update *utility.



			
				vermaden said:
			
		

> I check SA at the usual place, its the same for STABLE and RELEASE: http://www.freebsd.org/security/notices.html
> 
> ... about BUGs, well, I browse from time to time this one: http://svnweb.freebsd.org/base/




I also forgot about that one: http://freshbsd.org/search?branch=RELENG_8&project=freebsd


----------



## graudeejs (Sep 20, 2011)

mgp said:
			
		

> You may wanna try portrac to keep track of the port updates. It's a simple GUI tool that checks periodically if there are updates to the ports that are installed. I'd also love to get some feedback about the tool.



Or you may try ports-mgmt/bxpkg for package updates (development is stalled, until FreeBSD-9)


----------



## alie (Sep 20, 2011)

Anything can do these ?

1. Periodically refresh package databases and prompt for updates.
2. Install packages from tarballs.
3. Search for packages by name, description, category or file.
4. Show package dependencies, files and reverse dependencies. 
5. System tray notifiers


----------



## vermaden (Sep 20, 2011)

> 1. Periodically refresh package databases and prompt for updates.


*ports-check *+ *ports-update* 



> 2. Install packages from tarballs.


The _Ports_ tree automates that.



> 3. Search for packages by name, description, category or file.




```
# cd /usr/ports
# make search name=NAME
```

... or use http://freshports.org



> 4. Show package dependencies, files and reverse dependencies.




```
# pkg_info -L package
# pkg_info -r package
# pkg_info -R package
```

also *BXPKG *is able to do that graphically.



> 5. System tray notifiers


Doesn't PC-BSD have that? (haven't tried PC-BSD for quite long time)


----------



## alie (Sep 20, 2011)

@Vermaden: Thanks for the reply. What I mean is a GUI Frontends coded with KDE/Qt/Gtk/X11 :e

Yes PC-BSD does that but not sure if the GUI available for FreeBSD?


----------



## nORKy (Sep 20, 2011)

vermaden said:
			
		

> That is the best attitude, even to have the own *freebsd-update *server, so You could fix these potential bugs from STABLE using the *freebsd-update *utility.



hum.. freebsd-update can work on STABLE?


----------



## vermaden (Sep 20, 2011)

No, I mean HAVE YOUR OWN update server, check this out 
http://freebsd.org/doc/en/articles/freebsd-update-server/index.html

That way You will be building ONCE and then using freebsd-update directly to your update server to binary updates for STABLE.


----------



## alie (Sep 21, 2011)

We should donate to build freebsd-update server for STABLE


----------



## alie (Sep 21, 2011)

vermaden said:
			
		

> PART I. Keeping the FreeBSD base system up-to-date



Do we need to run all these points for every updates?


----------



## vermaden (Sep 21, 2011)

alie said:
			
		

> Do we need to run all these points for every updates ?



After the first time only steps from the [font="Courier New"]# csup /root/stable-supfile[/font] are needed (including the [font="Courier New"]csup[/font] step of course).


----------



## alie (Oct 6, 2011)

What happens if I change the tag to RELENG_9? Will it compiles with llvm/clang instead of gcc?


----------



## vermaden (Oct 6, 2011)

@alie

You probably mean building the world/kernel, dunno mate, I haven't tried to buildworld on 9.x yet.


----------



## alie (Oct 6, 2011)

Yes, building the world. Then how to make sure it builds with llvm/clang?


----------



## vermaden (Oct 6, 2011)

alie said:
			
		

> Yes, building the world. Then how to make sure it builds with llvm/clang ?



If the commands to build/install world/kernel are the same, then what is the difference?


----------



## alie (Oct 6, 2011)

hmmm...

Another question. How to turn off debug mode for FreeBSD9-STABLE? Is it on by default?


----------



## vermaden (Oct 6, 2011)

alie said:
			
		

> hmmm...
> 
> Another question. How to turn off debug mode for FreeBSD9-STABLE ? Is it on by default ?



When You download ISO images that are BETA, then Yes, debug is enabled in the kernel (WITNESS for example), when 9.0 will reach RC state, the additional debug is removed from the kernel default config (GENERIC), so its all about editing the kernel config.


----------



## alie (Oct 6, 2011)

I am new with kernel config. Is there any tutorial how to edit the kernel config? What is the config file location and name anyway?


----------



## vermaden (Oct 6, 2011)

alie said:
			
		

> I am new with kernel config. Is there any tutorial how to edit the kernel config ?



Sure, whole chapter at _FreeBSD Handbook_: http://freebsd.org/handbook/kernelconfig.html



			
				alie said:
			
		

> What is the config file location and name anyway ?


Location is /usr/src/sys/${ARCH}/conf/GENERIC, for example /usr/src/sys/amd64/conf/GENERIC.

Every time You update Your local source tree with csup/svn it gets overwritten, so its better to:

[CMD=""]# cd /usr/src/sys/${ARCH}/conf
# cp GENERIC /root/MINE
# ln -s MINE[/CMD]

And then use KERNCONF=MINE at the make buildkernel/installkernel/kernel option.


----------



## Carpetsmoker (Oct 6, 2011)

alie said:
			
		

> What happens if I change the tag to RELENG_9? Will it compiles with llvm/clang instead of gcc?



llvm/clang will be installed, but GCC is still the default compiler. Progress is being made, but it will probably remain the default for quite some time. There are 23,000+ ports that are typically *only* tested with GCC by the developers.


----------



## alie (Oct 7, 2011)

Means 9.0-RELEASE will be compiled with gcc instead of llvm/clang ?


----------



## Carpetsmoker (Oct 7, 2011)

I believe that is what I said...

You can use clang if you want, but it's not the default.


----------



## vermaden (Oct 7, 2011)

Carpetsmoker said:
			
		

> I believe that is what I said...
> 
> You can use clang if you want, but it's not the default.



Isn't that the base system will be compiled by llvm/clang and gcc is kept for building ports?


----------



## alie (Oct 7, 2011)

This is something really not clear... I couldnt find any information about this anywhere on the FreeBSD wiki, forum or handbook... we should put this information somewhere that visible and easy to find. And there's no document about how to compile with llvm instead of gcc or otherwise...


----------



## Carpetsmoker (Oct 7, 2011)

vermaden said:
			
		

> Isn't that the base system will be compiled by llvm/clang and gcc is kept for building ports?



IIRC I picked up something like that too once.

I don't know what future plans are. All I know is that FreeBSD 9 BETA3 ships with Both llvm/Clang and GCC4.2 and uses FreeBSD 4.2 as the default.

If you want to use Clang, simply set CC to point to Clang instead of GCC ... I didn't try it, but I don't think it's more complicated then that


----------



## kpa (Oct 7, 2011)

vermaden said:
			
		

> Sure, whole chapter at _FreeBSD Handbook_: http://freebsd.org/handbook/kernelconfig.html
> 
> 
> Location is /usr/src/sys/${ARCH}/conf/GENERIC, for example /usr/src/sys/amd64/conf/GENERIC.
> ...



Instead of making a full copy of GENERIC you can write your kernel config like this:


```
ident MINE

include GENERIC

nooption         KDB 
nooption         DDB 
nooption         GDB
nooption         DEADLKRES
nooption         INVARIANTS
nooption         INVARIANT_SUPPORT 
nooption         WITNESS
nooption         WITNESS_SKIPSPIN 
nooption         MALLOC_DEBUG_MAXZONES
```

Big timesaver when GENERIC eventually changes. You don't have to manually track the changes between the new GENERIC  and your own configuration.


----------



## Caliante (Oct 8, 2011)

Thank you for this post, Vermaden


----------



## vermaden (Oct 9, 2011)

@Caliante

Welcome mate


----------



## alie (Oct 10, 2011)

Hi Guys, I have 1 query regarding kernel config.

What do i need to set for cpu option on kernel config if i have this from dmesg for 64 bit profile (amd64):


```
CPU: Intel(R) Core(TM) i5 CPU         750  @ 2.67GHz (2673.35-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x106e5  Family = 6  Model = 1e  Stepping = 5
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,
  Features2=0x98e3fd<SSE3,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,
  AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
  TSC: P-state invariant, performance statistics
```

Is it cpu I686_CPU ?


----------



## grigorovl (Oct 10, 2011)

I686 for 32bit, HAMMER for 64bit.

More here:
http://forums.freebsd.org/showthread.php?t=2403


----------



## alie (Oct 10, 2011)

From the handbook:


> for a custom kernel it is best to specify only the CPU you have



So is it the best already to use HAMMER for my CPU ?


----------



## wblock@ (Oct 10, 2011)

I experimented with this earlier this summer: http://lists.freebsd.org/pipermail/freebsd-questions/2011-June/230491.html
For me, turning off i486 but leaving i586 and i686 resulted in a slight (6%) speed increase in buildworld.  That was on a C2D E8400, FreeBSD 8-STABLE, i386.


----------



## Yampress (Oct 10, 2011)

I  up-to-date base system with a newer version of the cd freebsd stable. In sysinstall  screen choose *upgrade*....


----------



## YZMSQ (Oct 11, 2011)

Yampress said:
			
		

> I  up-to-date base system with a newer version of the cd freebsd stable. In sysinstall  screen choose *upgrade*....


Does bsdinstall, the latest installer in FreeBSD9, offer this "upgrade" option?


----------



## alie (Oct 11, 2011)

YZMSQ said:
			
		

> Does bsdinstall, the latest installer in FreeBSD9, offer this "upgrade" option?



Nope, AFAIK


----------



## Yampress (Oct 11, 2011)

but now it is sysinstall still in stable 8 ...


----------



## vermaden (Oct 11, 2011)

Yampress said:
			
		

> but now it is sysinstall still in stable 8 ...



But NOT in STABLE/9 ...


----------



## tankist02 (Oct 19, 2011)

On 9 RC1 amd64 I got:


```
pkg_upgrade -a -C
fetch: ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9-stable/INDEX: File unavailable (e.g., file not found, no access)
```

The correct URL is:


```
ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9-current/
```

How can I fix the URL in the script?


----------



## alie (Oct 19, 2011)

`# setenv PACKAGESITE [url]ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9-current//Latest/[/url]`


----------



## tankist02 (Oct 19, 2011)

Thanks a lot, works great!


----------



## bbzz (Oct 22, 2011)

Vermaden, great guide. I used your zfs one in the past as well. It's always nice to have condensed information you can quickly check.

I upgraded to FreeBSD 9-RC1. I had to add a line in /usr/local/etc/uma.conf

```
BRANCH=9-current
```
otherwise ports-update fails; it checks 9-stable for packages, which doesn't exist.

Now couple of questions. When portmaster is run in ports-build, it checks for packages on 9.0-release depository, for example:

```
===>>> Checking package repository for latest available version
===>>> Package and/or archive not found at:
        http://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-9.0-release/audio/
```

This happens even after I export PACKAGESITE to "ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-9-current/Latest/".

More importantly, if I'm already using portmaster to build from ports, does it really have to check for packages (I thought pkg_update will do that)?

Thanks again for your help.


----------



## jrm@ (Oct 22, 2011)

vermaden said:
			
		

> The only reasons to rebuild the base system are that there has been found and fixed a bug in STABLE that affects You or that You need new features that has been merged into the STABLE branch (from CURRENT for example) like newer ZFS version or whatever.



This is an important point.  From the handbook:


> ...this means that at any given time, the sources for FreeBSD-STABLE may or may not be suitable for any particular purpose. It is simply another engineering development track, not a resource for end-users.


----------



## oliverh (Oct 22, 2011)

The handbook is not the bible, it's just a bunch of advices. There is even a lot of 'nonsense' in it at least nowadays. I'm using stable since 5.0 without any problems, on the other hand you could have severe problems while using releases and your hardware is somewhat picky. It solely depends on your environment and your experience. Don't make the mistake and compare terms like beta, RC, stable etc. pp. with those used in the Linux world. 15+ RCs for the Linux kernel are hardly comparable to the modest use of those in BSD world. The same is true for terms like stable, try to compare it to Debians testing or sid and you'll fail.

Vermadens advices are experience-driven, so are mine. 'It just works' is a completely different world, there is a RC dangerous and betas are almost deadly ;-)

Stick to Vermadens advices and remember: FreeBSD is about quality, not quantity. This is even true for the things in development.


----------



## jrm@ (Oct 23, 2011)

oliverh said:
			
		

> Stick to Vermadens advices...


  Sure, like when he says, 





> The only reasons to rebuild the base system are that there has been found and fixed a bug in STABLE that affects You or that You need new features that has been merged into the STABLE branch.


  There have been times when STABLE is, well, stable and there have been times when I really wished I had spent more time reading the stable/current lists before upgrading.  To each his own.  I just posted to share my experience in case it helps others.  In '97 I needed to share a dsl connection with my roommates, so I tried IP masquerading with linux.  A friend suggested NAT on FreeBSD might work better.  We had a 286 running 2.something (I think) and it ran without problems for the entire year.  I don't think I've installed a linux distribution since.   So, my only advice is to do your research and don't blindly upgrade if there isn't a good reason.


----------



## vermaden (Oct 29, 2011)

Sorry for late response, I have been pretty offline lately ...



			
				bbzz said:
			
		

> Vermaden, great guide. I used your zfs one in the past as well. It's always nice to have condensed information you can quickly check.


Welcome mate.



			
				bbzz said:
			
		

> I upgraded to FreeBSD 9-RC1. I had to add a line in /usr/local/etc/uma.conf
> 
> ```
> BRANCH=9-current
> ...



I havent tried that 'guide' on 9.x yet as I use 8.2-* series, havent got time to play more with 9.x series.



			
				bbzz said:
			
		

> More importantly, if I'm already using portmaster to build from ports, does it really have to check for packages (I thought pkg_update will do that)?


Good point, if pkg_upgrade hasn't found anything, then why would portmaster? 

Remove the --packages-if-newer from the ports-build function, I will modify the first post some time when I have more time (about a week from now).


----------



## vermaden (Dec 13, 2011)

Added updating FreeBSD source by SVN.


----------



## purgatori (Dec 28, 2011)

Keeping FreeBSD up to date is a nightmare; such a nightmare, that I have abandoned the idea of attempting it anymore. The last time I attempted a binary upgrade from 8.1 to 8.2, it borked my system so bad that I couldn't even perform a clean install without using DBAN to completely erase the hard drive.

I appreciate the effort you've gone to here, but the multitude of steps involved don't really fill me with confidence. I was actually going to abandon FreeBSD altogether after what happened, but my attempts to go back to Linux were bug-filled experiences -- ahh, good ole Linux. Instead, what I'm going to do is not upgrade at all, until a major release comes along; at which point I will backup my home directory to an external HDD, and perform a clean install.

Even now, I'm looking at having to remove and reinstall all the packages I'm using, because I've been installing them from the STABLE branch, when I should have gone RELEASE. Since my clean install, I've been quickly trying to install every package I need, because if I install anything later, chances are it's going to want different versions of packages I already have installed -- such is the incredibly annoying nature of STABLE. 

In every other way, I think FreeBSD beats the 'competition,' but upgrades/package management is definitely not one of them.


----------



## vand777 (Dec 28, 2011)

purgatori said:
			
		

> Keeping FreeBSD up to date is a nightmare; such a nightmare, that I have abandoned the idea of attempting it anymore.



I would disagree with you. My servers are always up-to-date. Both FreeBSD and ports. I've been upgrading between major releases many times without having any problems. 

There are few simple rules. If you follow them then all upgrades will be very smooth and nothing will be broken.


----------



## vermaden (Dec 28, 2011)

purgatori said:
			
		

> Keeping FreeBSD up to date is a nightmare; such a nightmare, that I have abandoned the idea of attempting it anymore.


This will be solved by PKGNG project, take a look here: http://youtu.be/IRa6wFBLU28



			
				purgatori said:
			
		

> The last time I attempted a binary upgrade from 8.1 to 8.2, it borked my system so bad that I couldn't even perform a clean install without using DBAN to completely erase the hard drive.


I do not remember any 'mine' problems with binary FreeBSD update, but I 'walked' that road: _5.4 -> 6.0 -> 6.1 -> 6.2 -> 7.0 -> 7.1_ without any problems by doing source upgrades (make buildworld ...)



			
				purgatori said:
			
		

> I appreciate the effort you've gone to here, but the multitude of steps involved don't really fill me with confidence.


Its probably the longest HOWTO I have ever written, but maybe because I wanted to explain all the steps and the 'why that way' behind them, seems pretty simple after several tries, try it under clean machine in VirtualBox first to catch some confidence.



			
				purgatori said:
			
		

> Instead, what I'm going to do is not upgrade at all, until a major release comes along; at which point I will backup my home directory to an external HDD, and perform a clean install.


Its definitely not my problem, but I would at least suggest keeping open eye on FreeBSD Security Advisories and patching when needed, along with portaudit from _Ports_.



			
				purgatori said:
			
		

> Even now, I'm looking at having to remove and reinstall all the packages I'm using, because I've been installing them from the STABLE branch, when I should have gone RELEASE.


Thats very easy to accomplish, make a list of Your installed packages, then remove all installed packages, then install them again from the list You have just made, like that:


```
# pkg_info -qoa > LIST
# pkg_delete -a
# rm -r -f /var/db/pkg /usr/local
# while read I; do pkg_add -r $( basename ${I} ); done < LIST
```

In case You are using a CSH shell, type sh before while loop, like that:


```
# pkg_info -qoa > LIST
# pkg_delete -a
# rm -r -f /var/db/pkg /usr/local
# sh
# while read I; do pkg_add -r $( basename ${I} ); done < LIST
```



			
				purgatori said:
			
		

> Since my clean install, I've been quickly trying to install every package I need, because if I install anything later, chances are it's going to want different versions of packages I already have installed -- such is the incredibly annoying nature of STABLE.


It may sound up like an advertising, but using methods described in this HOWTO solves that problem, packages are built every 2 weeks, and if newer version appears, its installed.



			
				purgatori said:
			
		

> In every other way, I think FreeBSD beats the 'competition,' but upgrades/package management is definitely not one of them.


FreeBSD lacks a lot in serious clustering, like 3+ nodes sharing many resource groups with their services, a shared clustering filesystem, drivers for various Fibre Channel controllers etc, but that is a topic for long discussion on another thread 

Regards,
vermaden


----------



## gkontos (Dec 29, 2011)

vermaden said:
			
		

> This will be solved by PKGNG project, take a look here: http://youtu.be/IRa6wFBLU28



I saw the video and read the wiki. I wasn't aware of the project. Looks very promising and it is certainly something that would give a competitive advantage to FreeBSD.

Do you have more info regarding this project?


----------



## vermaden (Dec 29, 2011)

@gkontos

You can follow its development here:
http://freshbsd.org/search?project=pkgngc


----------



## purgatori (Dec 30, 2011)

First off, thanks for your thorough reply vermaden 



			
				vermaden said:
			
		

> This will be solved by PKGNG project, take a look here: http://youtu.be/IRa6wFBLU28



Oh wow, it's great to know that something is in the pipeline. I had no knowledge of this project, and was despairing that the only solution anyone had even conceived of was the 'PBI' + jails system that PCBSD uses -- in my opinion, this is a solution that ends up being worst than the problem ]I do not remember any 'mine' problems with binary FreeBSD update, but I 'walked' that road: 5.4 -> 6.0 -> 6.1 -> 6.2 -> 7.0 -> 7.1 without any problems by doing source upgrades (make buildworld ...)[/quote]

Its probably a mistake I made along the way somewhere. I'm not a computer, let-alone FreeBSD, expert. FreeBSD is a tool I use in aid of my profession (writing/editing), and although I have done my best to learn its ins-and-outs, I don't have the time it would take to achieve your level of knowledge.

Despite its reputation, I don't think FreeBSD is terribly difficult to learn and use, even if you're not a computing boffin; that is, with the exception of package management. It would be nice if *that* was a little more idiot-proof, so to speak 



			
				vermaden said:
			
		

> Its probably the longest HOWTO I have ever written, but maybe because I wanted to explain all the steps and the 'why that way' behind them, seems pretty simple after several tries, try it under clean machine in VirtualBox first to catch some confidence.



I was in no way taking issue with your HOWTO. On the contrary, I think your HOWTO was very thorough, and helfpful. It's just the amount of steps that need to be taken in order to 'do it right' are kinda silly compared to:


```
apt-get update
apt-get upgrade
```

That's a fault with FreeBSD's package management system, not your HOWTO. I'm not saying Debian/Ubuntu's system is perfect, of course, but it is a lot more straightforward, and worked... the majority of the time, at least.



			
				vermaden said:
			
		

> Thats very easy to accomplish, make a list of Your installed packages, then remove all installed packages, then install them again from the list You have just made, like that:



Ahhh, very handy, thankyou 



			
				vermaden said:
			
		

> FreeBSD lacks a lot in serious clustering, like 3+ nodes sharing many resource groups with their services, a shared clustering filesystem, drivers for various Fibre Channel controllers etc, but that is a topic for long discussion on another thread



Waaaaaay outta my depth, I'm afraid  I suppose I'm evaluating things more from a desktop-users perspective. My activities on FreeBSD basically consist of: editing/proofreading documents in Emacs (which is as much my operating system as FreeBSD is  ), writing LaTeX documents, running statistics & generating graphics in R, browsing the web/gopher, and watching videos/listening to music. For all of these activities, I find FreeBSD to perform admirably and with rock-solid stability -- and all on rather ancient hardware. But, as you rightly say, that's the topic of another discussion.


----------



## vand777 (Dec 31, 2011)

purgatori said:
			
		

> Waaaaaay outta my depth, I'm afraid  I suppose I'm evaluating things more from a desktop-users perspective. My activities on FreeBSD basically consist of: editing/proofreading documents in Emacs (which is as much my operating system as FreeBSD is  ), writing LaTeX documents, running statistics & generating graphics in R, browsing the web/gopher, and watching videos/listening to music. For all of these activities, I find FreeBSD to perform admirably and with rock-solid stability -- and all on rather ancient hardware. But, as you rightly say, that's the topic of another discussion.



In this case, PCBSD might be a more convenient choice for you: http://www.pcbsd.org/.


----------



## SNK (Dec 31, 2011)

Or try GhostBSD (http://ghostbsd.org/)

purgatori, FreeBSD is indeed an excellent choice for the activities you mention (math/R, LaTeX, etc., I use these on a daily basis as well.) If you have a day or two available then just experiment with source upgrades (tracking -STABLE) and port maintenance. The ports that get both (i) regular updates and (ii) need an hour or so to compile (because you may not want to wait for the package) will probably only be your web browser and only if you use e.g. www/chromium or www/firefox. The investment (including headaches) will pay off.


----------



## vermaden (Jan 1, 2012)

purgatori said:
			
		

> First off, thanks for your thorough reply vermaden


Welcome.



			
				purgatori said:
			
		

> Despite its reputation, I don't think FreeBSD is terribly difficult to learn and use, even if you're not a computing boffin; that is, with the exception of package management. It would be nice if *that* was a little more idiot-proof, so to speak


The good thing about FreeBSD, is when You learn something, that knowledge/experience very rarely outdates, with Linux You need to 'update Your knowledge' every release and as someone stated here, its often a lot different 'procedure' to achieve the same thing in, for example, RHEL 4.x, 5.x and 6.x, so You need to learn 3 different procedures, in FreeBSD, its just one.



			
				purgatori said:
			
		

> I was in no way taking issue with your HOWTO. On the contrary, I think your HOWTO was very thorough, and helfpful. It's just the amount of steps that need to be taken in order to 'do it right' are kinda silly compared to:
> 
> 
> ```
> ...



Its the difference between Linux and FreeBSD world, at Linux, everything is a regular package, even the kernel, or init, or crontab, so the command *apt *just updates them all.

In FreeBSD lan You have 'most important core', the FreeBSD's _base system_ and optional packages, because these two 'worlds' can be tracked and updated without 'touching' the other one there is separate procedure for updating each of them.

Also, its not typical to update the _base system_ as long as everything is working ok and there are no new security vulnerabilities, for the most part You just update the installed optional packages, which means using only these commands:


```
# ports-check
# ports-update
```



			
				purgatori said:
			
		

> Waaaaaay outta my depth, I'm afraid  I suppose I'm evaluating things more from a desktop-users perspective. My activities on FreeBSD basically consist of: editing/proofreading documents in Emacs (which is as much my operating system as FreeBSD is  ), writing LaTeX documents, running statistics & generating graphics in R, browsing the web/gopher, and watching videos/listening to music. For all of these activities, I find FreeBSD to perform admirably and with rock-solid stability -- and all on rather ancient hardware. But, as you rightly say, that's the topic of another discussion.



In 'that part' we are very similar, I also use FreeBSD as my NAS and as my laptop/workstation: http://browse.deviantart.com/?q=by:vermaden+freebsd


----------



## purgatori (Jan 4, 2012)

vand777 said:
			
		

> In this case, PCBSD might be a more convenient choice for you: http://www.pcbsd.org/.



I actually used PCBSD this time to install FreeBSD, because it sets up networking and a few other little things for me (for some reason, I couldn't get auto dhcp to discover my settings through Sysintall), but I don't like the PBI system at all. 

It's kinda ok if the stuff you want is in the PBI library, but if it's not, then you have to install it through packages or ports anyway. When you do so, you run the risk of borking the system due to version conflicts or whatever else -- in one instance, I lost audio because PCBSD uses some sort of tweaked/driver system to initialize your sound hardware, and an updated package I installed overwrote it -- unless you use jails. Jails are, in my opinion, too inconvenient to bother with.

@*SNK*: I'm not a Gnome user (I favor Ratpoison, Stumpwm and the like),but GhostBSD looks very cool. I was kinda wondering why someone hadn't made a 'desktop edition' of FreeBSD, other than PCBSD and its problematic PBIs.

As for ports, I have definitely 'experimented' with them. The first two times I installed and operated FreeBSD, I used ports rather than packages, but I ran into similar problems with version conflicts, and no easy way to upgrade installed ports/packages. At least with packages, things don't take too long to install if something goes awry.



			
				vermaden said:
			
		

> The good thing about FreeBSD, is when You learn something, that knowledge/experience very rarely outdates, with Linux You need to 'update Your knowledge' every release and as someone stated here, its often a lot different 'procedure' to achieve the same thing in, for example, RHEL 4.x, 5.x and 6.x, so You need to learn 3 different procedures, in FreeBSD, its just one.



Tragically true. I remember moving between releases back when I was using Ubuntu, and how there was always some new back-end or protocol succeeding another. Not only would you have to learn how to use it, but it often provided no benefit, and more often than not just broke everything: PulseAudio being a great example. System configuration also seemed to be increasingly opaque with each release]In 'that part' we are very similar, I also use FreeBSD as my NAS and as my laptop/workstation: http://browse.deviantart.com/?q=by:vermaden+freebsd[/quote]

I notice we're also both fans of Fallout/Bethesda games, and have similar views on religion


----------



## vermaden (Jan 4, 2012)

purgatori said:
			
		

> Tragically true. I remember moving between releases back when I was using Ubuntu, and how there was always some new back-end or protocol succeeding another. Not only would you have to learn how to use it, but it often provided no benefit, and more often than not just broke everything: PulseAudio being a great example. System configuration also seemed to be increasingly opaque with each release; there were less human readable configuration files, and more arcane, semi-functional GUI-mediated rituals


I forgot about _'Ubuntu problems'_ as I havent used that distribution since quite long time, 2008 maybe, but Ubuntu changes seem to be much frequent then UpstreamVendor/CentOS ones, it must be a horror to keep up with all thos changes 



			
				purgatori said:
			
		

> That said, I have CentOS on my laptop. It pretty much 'just works,' but I think that's only because I don't mess with it much.


Besides the need to disable SELinux and that its still Linux and the need to always add all these various additional repositories that provide me the packages that I have at FreeBSD from the start, its not that bad, at least comparing to other Linux distributions 



			
				purgatori said:
			
		

> I notice we're also both fans of Fallout/Bethesda games, and have similar views on religion


Good to know that there are more people like me, I am a big fan of Fallout series, but definitely pre-Bethesda ones, I would even say pre-Tactics ones 

Regards,
ver


----------



## hadrons123 (Jan 7, 2012)

vermaden said:
			
		

> *PART II. Keeping the FreeBSD packages up-to-date*



I have FreeBSD 9.0-RC1. I can't use the ports-check after following the above guide. I tried both bash and sh and it doesn't make any difference. Ports-check doesn't work if it's a root or a user account. What am I doing wrong? Do I need to edit anything else considering I am using 9.0-RC1?


```
[root@BSD ~]# ports-check
bash: ports-check: command not found
[root@BSD ~]# ports-audit
bash: ports-audit: command not found
```


----------



## vermaden (Jan 7, 2012)

You need to have these functions KNOWN by Your shell, for example they need to be at ~/.zshrc for the *ZSH *shell and You can always 'add' them manually by typing source ~/.zshrc in terminal.

I also do not know where You got the ports-audit, its not part of my HOWTO


----------



## vermaden (Jan 17, 2012)

Added PDF and HTML formats for download (first post).


----------



## purgatori (Jan 18, 2012)

vermaden said:
			
		

> Added PDF and HTML formats for download (first post).



That's awesome. Definitely going to keep that PDF on hand as a reference =D


----------



## vermaden (Feb 2, 2012)

Welcome.


----------



## ryu (Mar 24, 2012)

I've just installed FreeBSD 9.0-RELEASE and would like to update to FreeBSD 9.0-STABLE and have my system up-to-date. But it looks like this guide is for FreeBSD 8.2 or some older version?



			
				vermaden said:
			
		

> cat > /root/stable-supfile << EOF
> *default host=cvsup.pl.freebsd.org
> *default base=/var/db
> *default prefix=/usr
> ...



Should this not be *RELENG_9* ? Is it even possible to update with this guide from RELEASE to STABLE? Sorry for the noobish questions, but i don't want to destroy my fresh installed system.


----------



## bbzz (Mar 24, 2012)

First of all, why do you want to go from -RELEASE to -STABLE? Are you sure you know what you are doing?

Remember, you can use all the packages from -STABLE without needing to move base system from -RELEASE.


----------



## ryu (Mar 24, 2012)

bbzz said:
			
		

> First of all, why do you want to go from -RELEASE to -STABLE? Are you sure you know what you are doing?



Well, *I* guess FreeBSD-STABLE is something like Debian Testing or Debian Sid. If I'm right in this case, I think I know what I'm doing. Would just like to have a fully up-to-date system.



			
				bbzz said:
			
		

> Remember, you can use all the packages from -STABLE without needing to move base system from -RELEASE.



All right. Didn't know that. I've read in the handbook about freebsd-update, but it seems like it is not the best choice. For example, I want to update Opera and Pidgin to the newest STABLE version, how would I do this without also updating the RELEASE base system?


----------



## wblock@ (Mar 25, 2012)

The operating system and applications aren't tied together.  There is no -stable version of ports, the ports tree always has the latest version that's been ported.  The reason to run FreeBSD-STABLE is to get the latest features and bug fixes in the operating system.


----------



## Yampress (Mar 25, 2012)

STABLE -> Probably because of the binary package, he can update to newer version ...


----------



## fonz (Mar 25, 2012)

wblock@ said:
			
		

> The reason to run FreeBSD-STABLE is to get the latest features and bug fixes in the operating system.


FWIW: As well as the new bugs. -STABLE doesn't break often, but it does happen occasionally.

Fonz


----------



## srivo (May 29, 2012)

I was using that script in my /etc/profile and when I installed gnome2 with GDM, GDM doesn't want to start with this script! I put it in .profile and the same thing happens when I try to log with my user.

*I*s there any other place I can put it?


----------



## stefanlasiewski (Jul 3, 2012)

vermaden said:
			
		

> Thats very easy to accomplish, make a list of Your installed packages, then remove all installed packages, then install them again from the list You have just made, like that:
> 
> 
> ```
> ...



I have a FreeBSD 8.3-RELEASE system which I just upgraded to FreeBSD 9.0-RELEASE. When I try the above command, most packages install cleanly. But I get a variety of errors, like this:


```
# while read I; do pkg_add -r $( basename ${I} ); done < LIST
```
Error: Unable to get ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9.0-release/Latest/ruby-bdb.tbz: File unavailable (e.g., file not found, no access)
Error: Unable to get ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9.0-release/Latest/ncftp3.tbz: File unavailable (e.g., file not found, no access)
Error: Unable to get ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9.0-release/Latest/perl5.12.tbz: File unavailable (e.g., file not found, no access)
[/CODE]

From what I can tell, according to `pkg_info -qoa`, my system had a package named databases/ruby-bdb . However, there is no package named ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9.0-release/Latest/ruby-bdb.tbz . 

What is happening here? Is does ruby-bdb have a different name in ports then as a package?

I can go ahead and install the packages by hand on this one system, but is there a way to cleanly do this automatically with some shell code?


----------



## jb_fvwm2 (Jul 3, 2012)

> I can go ahead and install the packages by hand on this one system, but is there a way to cleanly do this automatically with some shell code?


For next time maybe, you may find the small howto I wrote yesterday useful... Unsure how
it would work for all ports, but I just completed a reinstall seamlessly.
Every so often I'd have to stop/restart it, thereupon I'd check the latest packages saved
to /usr/ports/packages/All and delete the subset (ports already 
reinstalled) from the temporary file (package)
list (see the writeup in the How_To section...) with "edit" or "nano -w", to restart the reinstall again.


----------



## stefanlasiewski (Jul 3, 2012)

jb_fvwm2 said:
			
		

> For next time maybe, you may find the small howto I wrote yesterday useful...



Thank you.

I assume you mean this tutorial: Unexpectedly easy reinstall with portmaster of some ports (mostly Perl)

I will check it out!


----------



## vermaden (Jul 9, 2012)

stefanlasiewski said:
			
		

> What is happening here? Is does ruby-bdb have a different name in ports then as a package?



Yes, its sometimes that way, that package has different name then a port.



			
				stefanlasiewski said:
			
		

> I can go ahead and install the packages by hand on this one system, but is there a way to cleanly do this automatically with some shell code?


Probably yes, it will require scanning all 23 000 Makefiles in the Ports tree to get create the mapping of port name and a package name, and according to that mapping translate port name to package name and vive versa.


----------



## JackOps (Sep 29, 2012)

*A*wesome, great, this is what *I* was looking for. *I*'m new with *F*reeBSD but on my environtment at least there are three machine*s*, and *I* update it manually. *T*hanks.


----------



## emmex (Mar 29, 2013)

I have successfully upgrade*d* my laptop O.S. to 9-STABLE from PC-BSD (not FreeBSD).

But I noticed that the binary packages in ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9-stable/ are, at least, 5 months old. Why? Can I use an alternative package site repository?


----------



## priyadarshan (Jul 13, 2013)

Thank you so much to @vermaden for this, and all the other wonderful tutorials. Since the HOWTO was written in September 2011 (last edited January 2012), would there be any changes now, for one wishing to track FreeBSD-stable? For example, the HOWTO lists:

`# pkg_add -r bsdadminscripts portmaster portaudit`

Would there be any change if one would want to use PKGNG?

Thank you so much, once again.


----------



## vermaden (Jul 14, 2013)

@@priyadarshan

Welcome.

I used this 'way' for updating quite a long time, but I haven't used it lately. I moved to PKGNG, but I think that PKGNG is not (yet) better then classic pkg_* tools and packages. It WOULD be better if it will add package 'variants' like on OpenBSD when you type `pkg_add -i ...` and you can make a choice in an interactive menu if you want Conky with TTF built in or without it. On FreeBSD with PKGNG you can only add packages that are built with default WITH/WITHOUT flags, so no TTF in Conky for example. The other issue with PKGNG is that it REINSTALLS the packages that you built from Ports, no matter of newer or older, *so after* pkg upgrade *you will have to rebuild your customized ports, which is pretty annoying*.

I currently use PKGNG as 'packages database format' because its faster, even for simple package info, but I very rarely use PKGNG to install/update packages, I just use portmaster which RESPECTS my WITH/WITHOUT flags and previously selected options for Ports.

I have tried the PC-BSD repository for PKGNG, but as its still PKGNG it has the same issues I described above.

There were times, that I would not compile big ones like editors/libreoffice, but now I just do it overnight.


----------



## priyadarshan (Jul 14, 2013)

Thank you, this is all very clear, and it is quite revealing in respect of shortcomongs of PKGNG. I shall try the pure, ports-mgmt/portmaster way, then. Thank you once again, @vermaden! Very much appreciated.


----------



## kpa (Jul 14, 2013)

If you build all PKGNG packages yourself in a jail(8) with for example poudriere you won't have problems with PKGNG packages conflicting with local modifications because `pkg install` and `pkg upgrade` will be the only sources of installed ports/packages and you get very good control over the options for built packages.


----------



## vermaden (Jul 15, 2013)

kpa said:
			
		

> If you build all PKGNG packages yourself in a jail(8) with for example poudriere you won't have problems with PKGNG packages conflicting with local modifications because `pkg install` and `pkg upgrade` will be the only sources of installed ports/packages and you get very good control over the options for built packages.



Yes, that is one of the 'solutions', but if I build the packages anyway with portmaster, why then waste time (and space) for poudriere setup and maintenance when I have to build these packages anyway? 

I only find that useful (setting up poudriere) with more then 2 systems, along with having personal freebsd-update server from 'STABLE'.


----------



## kpa (Jul 15, 2013)

Building times are roughly the same once you have the system set up, the jail cloning at the start up takes some extra time so if you build only a few ports then the difference may be noticeable. You also get the benefit of getting packages that are guaranteed to have been built in a clean environment where only the required dependencies are installed for building. There's never a possibility that there are leftover files from previous versions of the ports that you're building. Or some unrelated port (like the KDE4 build problems when KDE3 libraries are installed at the same time) interferes with the build. You don't get such guarantees when building ports unjailed with ports-mgmt/portmaster on the same host where you install them.


----------

