# pkg upgrade -n --why-would-you-want-to-do-that



## nxg (Aug 16, 2022)

I have a specific question, which I think leads into a more generic one about understanding `pkg` logic.

I'm in the midst of upgrading a system from 13.0 to 13.1, and `pkg upgrade` tells me


```
The following 17 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
    libpsl: 0.21.1_4
    openldap24-client: 2.4.59_4

Installed packages to be UPGRADED:
    bash: 5.1.12 -> 5.1.16
    ca_root_nss: 3.71 -> 3.78
    curl: 7.80.0 -> 7.84.0
    gettext-tools: 0.21 -> 0.21_1
    icu: 70.1_1,1 -> 71.1,1
    libltdl: 2.4.6 -> 2.4.7
    lmdb: 0.9.29,1 -> 0.9.29_1,1
    mercurial: 6.0 -> 6.1.4
    nss-pam-ldapd-sasl: 0.9.11_1 -> 0.9.12_1
    portmaster: 3.20 -> 3.21
    py38-setuptools: 57.0.0 -> 62.1.0_1
    python38: 3.8.12_1 -> 3.8.13_2
    sudo: 1.9.8p2 -> 1.9.11p3
    zsh: 5.8 -> 5.9

Installed packages to be REINSTALLED:
    dialog4ports-0.1.6_1 (option removed: ACCESSLOG)
```

I'm puzzled, because I don't understand why pkg wants to install openldap24-client.  Checking with `pkg info -d`, none of the currently installed packages have a dependency on it.  In the absence of a `--why-on-earth-would-you-want-to-do-that` option for pkg, I'm scratching my head.

This isn't out of the blue, because the machine does have openldap26-client  and -server (ie, later versions) installed from ports:


```
% pkg lock -l
Currently locked packages:
cyrus-sasl-2.1.27_2
cyrus-sasl-saslauthd-2.1.27_2
openldap26-client-2.6.1
openldap26-server-2.6.1_1
```

But I don't see the logic that means that the presence of those portmaster-installed ports leads `pkg update` to try to downgrade them.  Is pkg perhaps spotting the presence of files installed by openldap26-{client,server} and jumping to the conclusion that they were put there by openldap24, which it's therefore repairing? That seems... unlikely.

I would have thought that `pkg info -d <packagename>` would have represented all of the information that pkg uses for its upgrade decisions, but that's clearly not the case.

I am aware that the handbook does say ‘it is generally ill-advised’ to use pkg and ports together (quarterly vs HEAD, etc), but I'm also aware, from these forums and elsewhere, that it's OK in practice more often than not, and I do seem to have got away with this (convenient) configuration in the past.

So the questions are:

  * Why is pkg trying to install openldap24-client?
  * Is this in fact indicating that my pkg-vs-ports luck has run out?  If so, what is pkg saying that makes that explicit, and how do I work out what the precise nature of that out-of-luck is? (that last question is really ‘am I really going to have to have another go with poudriere?’)
  * How do I get pkg to be a little more forthcoming about the logic of its conclusions?

Thanks for any pointers,

Norman


----------



## mer (Aug 16, 2022)

My understanding:
pkg uses the list of currently installed packages (roughly pkg prime-list) to figure out "what do I need to upgrade" but I think it uses info from the repo on what the dependencies are.
I have a pkg alias I call "reverse-depends" to find out "what packages depend on this package"

reverse-depends      'query %rn-%rv'
basically pkg reverse-depends openldap24-client would tell me what installed packages have openldap24-client as a dependency.

I think if you change the alias a little you can do that on the repo.  basically change the query to "rquery" in the alias.  Take that list and compare  to what you have installed.


----------



## SirDice (Aug 16, 2022)

Have you ever run pkg-autoremove(8)? Because to me it sounds like you still have some old (now orphaned) dependencies installed. A `pkg upgrade` will just upgrade anything and everything that's currently installed. 

The "default" OpenLDAP client was recently changed from 2.4 to 2.6. You probably did some upgrades after that change (which pulled in openldap26) but openldap24 might still be there if  it was never removed (pkg-autoremove(8)).






						ports - FreeBSD ports tree
					






					cgit.freebsd.org
				




But I'm wondering how you managed to install net/openldap26-client while keeping net/openldap24-client, because those ports/packages would conflict and the system will refuse to install both.


```
.if defined(CLIENT_ONLY)
CONFLICTS_INSTALL=	${PORTNAME}2[0-57-9]-client ${PORTNAME}-client
.else
CONFLICTS_INSTALL=	${PORTNAME}2[0-57-9]-server ${PORTNAME}-server
.endif
```


----------



## Erichans (Aug 16, 2022)

To satisfy your curiosity: if you want to check _afterwards_, let the install proceed as proposed and then try 
`pkg required-depends openldap24-client`
(`required-depends` is an alias, see `pkg alias | grep depend` for more predefined pkg aliases.) 
Then you'll have your answer and can decide what packages you want to remove.

You can dry-run pkg-autoremove(8) with `pkg autoremove -n`

P.S. 


nxg said:


> [...] I would have thought that `pkg info -d <packagename>` would have represented all of the information that pkg uses for its upgrade decisions, but that's clearly not the case.
> 
> I am aware that the handbook does say ‘it is generally ill-advised’ to use pkg and ports together (quarterly vs HEAD, etc), but I'm also aware, from these forums and elsewhere, that it's OK in practice more often than not, and I do seem to have got away with this (convenient) configuration in the past.


Note that where pkg-query(8) searches the _local_ pkg repository/database, both pkg-rquery(8) and pkg-search(8) apply to the _remote _repository. For example:
`pkg search -d wireguard` is the same as `pkg search --depends-on wireguard` or `pkg search -Q depends-on wireguard` and relate to the remote repository. Generally speaking, that means that the proposed installation of an additional package could also be triggered by the addition of an updated (new) version of a current (locally installed) package. In your case that is rather/very unlikely because it concerns another older version altogether (i.e. openldap24-client) that is being is proposed.


----------



## monwarez (Aug 16, 2022)

Did you try to unlock all locked packages ? They can optionally depends on some version of openldap, depending on when they were installed.
My guess is that after unlock all locked package, they can be updated and there should no longer be an openldap24 dependency.
Then you can update your ports tree and rebuild your packages with openldap support (by the ways your version of cyrus-sasl is subject to CVE-2019-19906)


----------



## nxg (Aug 16, 2022)

Greetings, all.

Thank you, all, for your thoughtful and insightful remarks.

I think I have some understanding (though incomplete) of what's
happened here.  Skip to 'Resolution', below, if you're not interested
in the flailing around in the middle.

(Context: This machine is an LDAP server, which since 11.x has
required careful selection of things from the ports tree, in order to
get the up-to-date openldap2X-server with needed options, whilst
avoiding packages which depend on an old/pkg
`openldap2X-{client,server}` -- as SirDice notes, the
different versions install into conflicting places.  I'm pretty sure
the same was true of 13.0, but I'm going to double-check whether 13.1
still forces me to do this.)

Investigations:

I had already run `pkg autoremove -n`, and it produced no
indication of a link to openldap24-client.  I had done the local
reverse-depends thing the hard way (list all of the packages, then
list their dependencies one by one): `pkg query %rn-%rv
openldap24-client` is a lot more straightforward -- thanks mer!  It produces
nothing.  _But_ one of its rquery-discovered reverse dependencies is
`nss-pam-ldapd-sasl 0.9.12_1`, and that is the only one
of those reverse dependencies which I do have installed (and it is one
of those returned by `pkg prime-list`).

So that would appear to be the culprit.  Except...


```
% pkg info -d nss-pam-ldapd-sasl
nss-pam-ldapd-sasl-0.9.11_1:
	openldap26-client-2.6.1
```

The locally installed `nss-pam-ldapd-sasl` seems to think
that it depends on openldap26-client.  It's further surprising that
the local package database seems to think this depends on
openldap26-client, but a _later_ version of the package
(0.9.12) apparently depends on openldap24-client.

But (_aha...._) this seems to link to Erichans' remark that


> Generally speaking, that means that the proposed installation of an
> additional package could also be triggered by the addition of an
> updated (new) version of a current (locally installed) package. In
> your case that is rather/very unlikely because it concerns another
> ...


and the observation that `pkg upgrade` is proposing to
upgrade `nss-pam-ldapd-sasl: 0.9.11_1 -> 0.9.12_1`.  But

```
% pkg rquery '%n-%v depends on %dn-%dv' nss-pam-ldapd-sasl
nss-pam-ldapd-sasl-0.9.12_1 depends on openldap24-client-2.4.59_4
```
Thus it appears that the newer version of nss-pam-ldapd-sasl is quoted
as depending on the older version of openldap.

But if I look at
<https://www.freebsd.org/cgi/ports.cgi?query=nss-pam-ldapd-sasl&stype=all>,
then I'm told that

```
nss-pam-ldapd-sasl-0.9.12_1
  Advanced fork of nss_ldap with sasl support
  Description : Changes
  Maintained by: zi@FreeBSD.org
  Requires: cyrus-sasl-2.1.28, openldap26-client-2.6.3
```

So what _appears_ to be the case is that (a) my
system wants to update nss-pam-ldapd-sasl to 0.9.12 (which seems
right), but (b) also thinks, for some reason, that that updated
version depends on openldap24-client-2.4.59_4 which appears to be
incorrect.  I wonder if I have some confusion in a local cache of a
remote dependency database (I think this is right -- see below).

Resolution:

What I've done is to (unlock and) remove the packages I installed from
ports, including nss-pam-ldapd-sasl.  That allows me to `pkg
upgrade`, and complete the upgrade to 13.1 (phew).  At this point (ie,
after removing `nss-pam-ldapd-sasl`), if I ask to install
`nss-pam-ldapd-sasl` pkg still reports that it wants to
install `openldap24-client: 2.4.59_4`, and so does
`portmaster --show-work`.  Hmm... portmaster(8)?... hmm...

And _then_ I update the ports tree with portsnap(8).

And _now_ `portmaster --show-work` reports that it
will install `net/openldap26-client`.

So I think I can blame an outdated ports tree for my problem.  But
that also fails to completely explain things, because I
_thought_ that the pkg system was independent of the local
ports tree.  When eg `portmaster` does its work, it installs
packages where pkg can see them, but I didn't think that
`pkg` could be thrown off-track by the ports tree.

Whatever the explanation for that, I think my next step is to discover
whether the current pkg version of openldap26 now has the defaults I
need, or at least to take more explicit note of just why I need the
ports version.

And yes, thanks to the prompt from Monwarez, I have now found `pkg
audit`.

Summary: So the things I've learned are:

  * yes, it is indeed just the network of dependencies that `pkg
    upgrade` works with, but
  * this comprises the local _and remote_ dependencies,
  * and my ideas about how to query that are now less muddled than
    they were.

(I'm still a bit confused about the relationship between pkg and ports, though...; but I'm _differently_ confused, so that's progress)

Thanks again,

Norman


----------



## nxg (Aug 16, 2022)

...but still


```
% pkg rquery %n-%v' depends on '%dn-%dv nss-pam-ldapd-sasl
nss-pam-ldapd-sasl-0.9.12_1 depends on openldap24-client-2.4.59_4
```
and that's the same on two different 13.1-RELEASE boxes.

I have nothing beyond the default aliases in `pkg.conf`, and `pkg -d` confirms that it's fetching from <http://pkgmir.geo.freebsd.org>.


----------



## monwarez (Aug 17, 2022)

Are you using quarterly or latest ? For me with latest I get


> nss-pam-ldapd-sasl-0.9.12_1 depends on openldap26-client-2.6.3


----------



## Erichans (Aug 17, 2022)

nxg said:


> (Context: This machine is an LDAP server, which since 11.x has
> required careful selection of things from the ports tree, in order to
> get the up-to-date openldap2X-server with needed options, [...]





nxg said:


> ...but still
> 
> ```
> % pkg rquery %n-%v' depends on '%dn-%dv nss-pam-ldapd-sasl
> ...


My guess is that those "two different 13.1-RELEASE boxes" versus the "LDAP server" do not have the same package settings (quarterly versus latest) in FreeBSD.conf. Check:

/etc/pkg/FreeBSD.conf
/usr/local/etc/pkg/repos/FreeBSD.conf
The setting in /etc/pkg/FreeBSD.conf is the "general-default" setting (preferably not to be changed by you). When present, /usr/local/etc/pkg/repos/FreeBSD.conf should contain local settings that are to be retained when updating the FreeBSD OS; please refer to the FreeBSD Handbook: 4.4.2. Quarterly and Latest Ports Branches

I do not have net/nss-pam-ldapd-sasl installed; however, on 12.3-RELEASE, I get:

```
% pkg rquery %n-%v' depends on '%dn-%dv nss-pam-ldapd-sasl
nss-pam-ldapd-sasl-0.9.12_1 depends on openldap24-client-2.4.59_4

<snap>

% pkg rquery %n-%v' depends on '%dn-%dv nss-pam-ldapd-sasl
nss-pam-ldapd-sasl-0.9.12_1 depends on openldap26-client-2.6.3
```
The second result is based on the _latest_ pkg repository—as indicated by monwarez. The first result is based on _quarterly_.



SirDice said:


> ports - FreeBSD ports tree
> 
> 
> 
> ...


My observation is that the change by this commit - Mk/bsd.ldap.mk: Use OpenLDAP 2.6 by default from `2022-07-08 08:06:07 +0000` according to this diff:

```
-DEFAULT_OPENLDAP_VER?=    24
+DEFAULT_OPENLDAP_VER?=    26
```
has been committed _after_ the quarterly branch of Q3 (=2022Q3) has been branched off the ports tree's main branch. This means that this commit _is incorporated_ in the ports tree's main branch but, it _is not incorporated_ in the 2022Q3 branch of the ports tree. Currently, the 2022Q3 branch of the ports tree is used for the _quarterly_ setting. The main branch of the ports tree is used for the _latest_ setting. 

The _quarterly_ setting will use the 2022Q4 branch when the next quarter has arrived. At Q4, first a (new) branch will be branched off from the ports tree's main branch, immediately after this it will be designated as the 2022Q4 branch and will be used by FreeBSD installs using the _quarterly_ setting. It will, of course, incorporate the change of the "old" commit from `2022-07-08 08:06:07 +0000`; because of that net/nss-pam-ldapd-sasl will then have openldap26-client as its dependency when using the _quarterly_ setting.*

Concluding: this commit with this particular timestamp is the reason that, at this moment, net/nss-pam-ldapd-sasl has a different dependency (openldap24-client versus openldap26-client), depending on which remote package repository is being used.


Generally speaking, especially from an admin standpoint of view, it would be beneficial to have all your FreeBSD systems on the same remote pkg repository. If you don't do that or if that is not possible then, that is one reason to consider using a local package server. If you mix ports and packages and you do not have a (very) good knowledge of the consequences of mixing ports and packages and individual settings that govern how they influence each other then, that is another reason to consider using a local package server.

A local package (build) server (with its own local repository) creates an extra level of indirection that enables you to effectively control the update process of all your FreeBSD installs. That means that the update process of this local repository with source code from the external FreeBSD ports repository is fully under your control, _independent_ from the remote FreeBSD ports tree's individual branches. The packages that have been built this way enable a uniform way by which the FreeBSD installs in a local network get their packages updated from this local repository.

Such a new configuration will require extra work to set up and an extra learning curve but the tools are there.

___
* edit: slightly reworded


----------



## mer (Aug 17, 2022)

monwarez said:


> Are you using quarterly or latest ? For me with latest I get


I think this is a key difference.  I run that pkg rquery on 13.1-RELEASE that I know is pointing at quarterly and get

`pkg rquery %n-%v' depends on '%dn-%dv nss-pam-ldapd-sasl
nss-pam-ldapd-sasl-0.9.12_1 depends on openldap24-client-2.4.59_4`

just like the OP.  Another reminder that quarterly lags latest, by design.
And ports trees track whatever you last updated to, whether it's portsnap or a git clone.

I'd like to also point out that Erichans has a lot of good info in post #9


----------



## nxg (Aug 18, 2022)

Thanks, Erichans and mer.

I think I'd managed to confuse myself in post #7.  I'm aware of the distinction between the quarterlies (being what's in pkg) and latest (in ports), so I shouldn't have been surprised at what I saw there. Each of the machines in question has the default `/etc/pkg/FreeBSD.conf` and no local tweaks.  My problem comes down to simultaneously wanting `openldap26-server`, which depends on `openldap26-client` (unsurprisingly) and `net/nss-pam-ldapd-sasl` which, in the current quarterly, depends on `openldap24-client`.  And if I'd stepped back and walked around the block again, I'd have seen that pkg and portmaster were in fact telling me that.

My immediate problem is therefore solved by building `net/openldap26-server`, `security/cyrus-sasl2-saslauthd` _and, crucially,_ `net/nss-pam-ldapd-sasl` from ports (and locking them and their immediate dependencies).  The command `portmaster --show-work` shows ports dependencies, and I can install most of the ‘ordinary’ dependencies (but, from experiment, not `net/nss-pam-ldapd-sasl`) from pkgs.

From what Erichans says, I'm optimistic that, come the next quarterly, because `openldap26-server` now includes the options I want and because `nss-pam-ldapd-sasl` depends on the right OpenLDAP, I'll be able to install what I need entirely from quarterly pkgs.  I'll therefore – despite Erichans's reassurance – be able to postpone learning about poudriere for a little longer (I think my bad experience with it before was because it was a hurried means to an end, which didn't finally work, as opposed to being a bit of strategic learning).

But more importantly, I think I now have a clearer idea of just what the manual is referring to when it says that ‘it is generally ill-advised’ to use pkg and ports together.  It's not some arcane issue of subtly incompatible APIs (which, though not impossible, is a problem you can probably get away with ignoring quite a lot of the time), but instead exactly this problem of confusing and subtly incompatible dependencies, which can only really be worked out by iterative, trial-and-error, work with `portmaster --show-work`, and then taking careful notes, to minimise the total amount of magic.

So: the learning:

  * one should repeatedly remind oneself of the relationship between pkg/quarterlies and ports/latest
  * `pkg query` and `rquery` are your friend
  * `portmaster --show-work` is what discloses the corresponding information for ports

(I'd still quite like a `--why-on-earth-would-you-want-to-do-that` option, though, a bit like `make -dm`).

Thanks again for very useful answers; I hope this was a useful question for the archive.


----------



## mer (Aug 18, 2022)

You are not the first one to get "bitten by" "confused by" ports and packages and quarterlies and latest.  And probably not the last.
Since my systems are "home use" I tend to follow -RELEASE and quarterly packages simply for ease of use.  I do pay attention to warnings and vulnerabilities, but look to see if I actually "care right now".



nxg said:


> and then taking careful notes, to minimise the total amount of magic.


Words of wisdom to any system admin.


----------

