# freebsd-update and whatis



## pennello (Sep 13, 2015)

For the past three weeks, freebsd-update(8) has repeatedly needed to update /usr/share/man/whatis.


```
Looking up update.FreeBSD.org mirrors... 5 mirrors found.
Fetching metadata signature for 10.1-RELEASE from update6.freebsd.org... done.
Fetching metadata index... done.
Inspecting system... done.
Preparing to download files... done.

The following files will be updated as part of updating to 10.1-RELEASE-p19:
/usr/share/man/whatis
```

Despite doing these updates, it still needs to do it yet again the following run (the next week).  I found this thread, and confirmed that indeed, it's only after regenerating the whatis database that freebsd-update(8) comes to believe it needs updating again.  However, none of the suggestions panned out for me.

Has anyone else run into this problem?  Any thoughts on how to address it?

_Update: fixed missing link._


----------



## tingo (Sep 13, 2015)

You could ignore it. The whatis(1) database is updated by a periodic(8) job (weekly IIRC), see makewhatis(1) for more info.


----------



## pennello (Sep 13, 2015)

True, I _could_.  Here, though, I'm looking for a way so as to ensure that the product of makewhatis(1) is essentially the same as what's on the freebsd-update(8) server.  In the thread I linked to, the thought was that there was some difference in base.  I'm not sure what that difference might be.

Indeed, it's the case that if I do one run of `freebsd-update install` which updates /usr/share/man/whatis, and then another run of `freebsd-update install`, it believes that no further updates are needed.  But if I run the periodic(8) job in-between (/etc/periodic/weekly/320.whatis), then the second `freebsd-update install` run believes that an update to /usr/share/man/whatis is required.  This indicates a discrepancy.


----------



## tingo (Sep 14, 2015)

pennello said:


> True, I _could_.  Here, though, I'm looking for a way so as to ensure that the product of makewhatis(1) is essentially the same as what's on the freebsd-update(8) server.


In short: you can't. Why? Because the makewhatis database is a local database, updated by the periodic script, and each time influenced by what else has been installed on the local machine (read: ports, packages) since the last time the periodic script ran.
A good question to ask would be why freebsd-update(8) is distributing this database in the first place.


----------



## pennello (Sep 15, 2015)

tingo said:


> In short: you can't. Why? Because the makewhatis database is a local database, updated by the periodic script, and each time influenced by what else has been installed on the local machine (read: ports, packages) since the last time the periodic script ran.


Interesting.  Given that the database is actually a text file, I opted to diff the one generated locally, by the periodic script, and the one brought in remotely, by freebsd-update(8).  I have a number of ports and packages installed, but it doesn't look like the contents are influenced by them, as you stated.

```
--- whatis.remote    2015-09-14 05:43:38.000000000 -0700
+++ whatis.local    2015-09-14 05:43:57.000000000 -0700
@@ -363,6 +363,7 @@
basename(3)              - extract the base portion of a pathname
baudrate(3), erasechar(3), erasewchar(3), has_ic(3), has_il(3), killchar(3), killwchar(3), longname(3), term_attrs(3), termattrs(3), termname(3) - curses environment query routines
bc(1)                    - arbitrary-precision arithmetic language and calculator
+bcd(6), ppt(6)           - reformat input as punch cards or paper tape
bce(4)                   - QLogic NetXtreme II (BCM5706/5708/5709/5716) PCI/PCIe Gigabit Ethernet adapter driver
bcmfw(8)                 - firmware download utility for Broadcom BCM2033 chip based Bluetooth USB devices
bcmp(3)                  - compare byte string
@@ -469,6 +470,7 @@
c89(1)                   - POSIX.2 C language compiler
c99(1)                   - standard C language compiler
cacos(3), cacosf(3), cacosh(3), cacoshf(3), casin(3), casinf casinh(3), casinhf catan(3), catanf catanh(3), catanhf(3) - complex arc trigonometric and hyperbolic functions
+caesar(6), rot13(6)      - decrypt caesar ciphers
cal(1), ncal(1)          - displays a calendar and the date of Easter
calendar(1)              - reminder service
call_once(3), cnd_broadcast(3), cnd_destroy(3), cnd_init(3), cnd_signal(3), cnd_timedwait(3), cnd_wait(3), mtx_destroy(3), mtx_init(3), mtx_lock(3), mtx_timedlock(3), mtx_trylock(3), mtx_unlock(3), thrd_create(3), thrd_current(3), thrd_detach(3), thrd_equal(3), thrd_exit(3), thrd_join(3), thrd_sleep(3), thrd_yield(3), tss_create(3), tss_delete(3), tss_get(3), tss_set(3) - C11 threads interface
@@ -846,6 +848,7 @@
extattr_namespace_to_string(3), extattr_string_to_namespace(3) - convert an extended attribute namespace identifier to a string and vice versa
extattrctl(8)            - manage UFS1 extended attributes
fabs(3), fabsf(3), fabsl(3) - floating-point absolute value functions
+factor(6), primes(6)     - factor a number, generate primes
fail(9), KFAIL_POINT_CODE(9), KFAIL_POINT_RETURN(9), KFAIL_POINT_RETURN_VOID(9), KFAIL_POINT_ERROR(9), KFAIL_POINT_GOTO(9), fail_point(9), DEBUG_FP(9) - fail points
faith(4)                 - IPv6-to-IPv4 TCP relay capturing interface
faithd(8)                - FAITH IPv6/v4 translator daemon
@@ -940,6 +943,7 @@
form_requestname(3)      - handle printable form request names
form_userptr(3)          - associate application data with a form item
form_win(3)              - make and break form window and subwindow associations
+fortune(6)               - print a random, hopefully interesting, adage
forward(5)               - mail forwarding instructions
fpa(4), fea(4)           - device drivers for DEC FDDI controllers
fparseln(3)              - return the next logical line from a stream
@@ -1141,6 +1145,7 @@
graid(8)                 - control utility for software RAID devices
graid3(8)                - control utility for RAID3 devices
grantpt(3), ptsname(3), unlockpt(3) - pseudo-terminal access functions
+grdc(6)                  - grand digital clock (curses)
gre(4)                   - encapsulating network device
grep(1), egrep(1), fgrep(1), zgrep(1), zegrep(1), zfgrep(1) - file pattern searcher
grep(1), egrep(1), fgrep(1), zgrep(1), zegrep(1), zfgrep(1), bzgrep(1), bzegrep(1), bzfgrep(1) - print lines matching a pattern
@@ -1884,6 +1889,7 @@
module(9)                - structure describing a kernel module
moduli(5)                - Diffie-Hellman moduli
moncontrol(3), monstartup(3) - control execution profile
+morse(6)                 - reformat input as morse code
mos(4)                   - Moschip MCS7730/MCS7830/MCS7832 USB Ethernet driver
motd(5)                  - file containing message(s) of the day
mount(2), nmount(2), unmount(2) - mount or dismount a file system
@@ -2091,6 +2097,7 @@
ntptime(8)               - read kernel time variables
null(4)                  - the null device
nullfs(5)                - null file system
+number(6)                - convert Arabic numerals to English
nvd(4)                   - NVM Express disk driver
nve(4)                   - NVIDIA nForce MCP Networking Adapter device driver
nvme(4)                  - NVM Express core driver
@@ -2377,6 +2384,7 @@
pmcstat(8)               - performance measurement with performance monitoring hardware
poll(2)                  - synchronous I/O multiplexing
polling(4)               - device polling support
+pom(6)                   - display the phase of the moon
popen(3), pclose(3)      - process I/O
ports(7)                 - contributed applications
portsnap(8)              - fetch and extract compressed snapshots of the ports tree
@@ -2534,6 +2542,7 @@
rand(3), srand(3), sranddev(3), rand_r(3) - bad random number generator
random(3), srandom(3), srandomdev(3), initstate(3), setstate(3) - better random number generator; routines for changing generators
random(4)                - the entropy device
+random(6)                - random lines from a file or random numbers
random_harvest(9)        - gather entropy from the kernel for the entropy device
rarpd(8)                 - reverse ARP daemon
rbootd(8)                - HP remote boot server
@@ -2954,6 +2963,7 @@
strcoll(3)               - compare strings according to current collation
strcspn(3)               - span the complement of a string
strdup(3), strndup(3)    - save a copy of a string
+strfile(8), unstr(8)     - create a random access file for storing strings
strfmon(3)               - convert monetary value to string
strftime(3)              - format date and time
string2key(8)            - map a password into a key
```
Given this difference, it looks more like my local copy includes entries from the games base package, which I have installed, whereas those entries are missing from the remote copy.
One thing to point out is that this is _new_.  For months before, with the same base install on my system, freebsd-update(8) has not exhibited this behavior where it continually complains about /usr/share/man/whatis.


tingo said:


> A good question to ask would be why freebsd-update(8) is distributing this database in the first place.


Absolutely.  Now that we've asked, who can answer?


----------



## pvoigt (Sep 27, 2015)

I do not have a real solution or explanation for the described behavior, but I can confirm this issue on my 10.1-RELEASE machine: On a weekly basis freebsd-update(8) repeatedly needed to update /usr/share/man/whatis. I have searched the forum and asked on IRC with no solution.

When 10.2-RELEASE came out I decided to install my server from scratch, because I wanted a ZFS only system and I wanted to drop various unneeded ports. And now the good news: The whatis issue is gone. The bad thing is that I cannot conclude, if it is due to the reduced number of ports or due to the change of release number.

Peter


----------



## pennello (Oct 3, 2015)

And just like that, after a month or so of ostensibly spurious /usr/share/man/whatis updates, freebsd-update(8) announces them to me no longer...

```
Looking up update.FreeBSD.org mirrors... 5 mirrors found.
Fetching metadata signature for 10.1-RELEASE from update6.freebsd.org... done.
Fetching metadata index... done.
Fetching 2 metadata patches.. done.
Applying metadata patches... done.
Inspecting system... done.
Preparing to download files... done.
Fetching 4 patches... done.
Applying patches... done.

The following files will be updated as part of updating to 10.1-RELEASE-p22:
/bin/freebsd-version
/usr/sbin/rpcbind
/usr/src/sys/conf/newvers.sh
/usr/src/usr.sbin/rpcbind/rpcb_svc_com.c
```
¯\_(ツ)_/¯  The end?


----------



## derekschrock (Oct 9, 2015)

With whatis the issue is as previously mentioned this might not be the best thing for `freebsd-update` servers to distribute however I think we can deal with it.

Second, in /usr/src/ if you have it, there's ObsoleteFiles.inc, these are old files that might be left behind from previous versions.  If these files get left behind it's possible, if it's a man page for instance, that `whatis` will read it in and generate a differing database.

You can view obsolete files by running in /usr/src/ `make check-old`.  This will remove all old libs, dirs, and files.  There's also `check-old-{dirs,files,libs}`.  Be careful when doing this because it's possible you might break third-party software that depends on old libs.  See /usr/src/Makefile and search for check-old and /usr/src/ObsoleteFiles.inc  for more information.


----------



## pennello (Oct 11, 2015)

D'oh!  It's back.


```
Looking up update.FreeBSD.org mirrors... 5 mirrors found.
Fetching metadata signature for 10.1-RELEASE from update6.freebsd.org... done.
Fetching metadata index... done.
Inspecting system... done.
Preparing to download files... done.

The following files will be updated as part of updating to 10.1-RELEASE-p22:
/usr/share/man/whatis
```



derekschrock said:


> You can view obsolete files by running in /usr/src/ `make check-old`.  This will remove all old libs, dirs, and files.  There's also `check-old-{dirs,files,libs}`.  Be careful when doing this because it's possible you might break third-party software that depends on old libs.  See /usr/src/Makefile and search for check-old and /usr/src/ObsoleteFiles.inc  for more information.


Ah, interesting; thanks for the suggestion.  Unfortunately, it's a no-go.

```
spark:/usr/src % make check-old
>>> Checking for old files
>>> Checking for old libraries
>>> Checking for old directories
To remove old files and directories run 'make delete-old'.
To remove old libraries run 'make delete-old-libs'.
```
Nothing!
Check out my earlier post.  It looks like the difference between what's generated locally and what's pulled in remotely is the contents of some base package.  Maybe games?


----------



## derekschrock (Oct 12, 2015)

Ok I'm starting to recall my issue.  So I had other files (non-games) that could have been cleaned up `make delete-old` then I found out about the above games man pages.
I just deleted the games man pages for games and that fixed the issue.  However, looking back on it I guess this is really a case of the `freebsd-update` servers don't have the game dist installed or if it's being installed they don't update the whatis database before creating the binary updates? 

If they did then people without games installed would have the same problem.

Ok so maybe the freebsd-update servers shouldn't distribute the whatis database.


----------



## pennello (Oct 12, 2015)

Interesting.  One thing I mentioned earlier is that this is _new_.  For months before, with the same base install on my system, including games, freebsd-update(8) has not exhibited this behavior where it continually complains about /usr/share/man/whatis.  I can definitely follow the logic about why freebsd-update(8) shouldn't distribute the whatis(1) database, but I wonder why the behavior changed here.


----------



## pennello (Oct 25, 2015)

Well, this continues to happen.  Is the best next step to file a bug?


----------



## trev (Oct 27, 2015)

Have you considered excluding the file in freebsd-update.conf(5) ?


----------



## pennello (Oct 27, 2015)

trev said:


> Have you considered excluding the file in freebsd-update.conf(5) ?


Sure; I will give that a shot.  Although it feels like treating the symptom; honestly speaking, shouldn't this just not be happening at all in the first place?


----------



## pennello (Oct 27, 2015)

trev said:


> Have you considered excluding the file in freebsd-update.conf(5) ?


Interestingly enough, by default, /usr/share/man/whatis was already specified to be ignored by `freebsd-update IDS`:

```
# Paths which start with anything matching an entry in an IDSIgnorePaths
# statement will be ignored by "freebsd-update IDS".
IDSIgnorePaths /usr/share/man/cat
IDSIgnorePaths /usr/share/man/whatis
IDSIgnorePaths /var/db/locate.database
IDSIgnorePaths /var/log
```


----------



## pennello (Nov 15, 2015)

trev said:


> Have you considered excluding the file in freebsd-update.conf(5) ?


Well, it's been a couple weeks, and what you suggested, trev, has worked--no more spurious updates to /usr/share/man/whatis.
I hesitate to call this solved, as I fear all we've done is treat a symptom.  It's not like it's reasonable to expect _everyone_ to add this path to freebsd-update.conf.


----------



## trev (Dec 4, 2015)

It certainly solved it for me; glad it's working for you too.


----------

