# `pkg clean` seems to remove all cached packages?



## kpedersen (Jul 23, 2021)

Hi all,

According to the manpage, the command `pkg clean` should only remove cached packages that are not the very latest.

What I seem to be getting is that it is removing everything, almost as if the `-a` (clear all) flag is being used.

So if I run:


```
# pkg fetch wget
# pkg clear
$ ls /var/cache/pkg | wc -l

   >> 0
```

Is this the case for anyone else?

I am due that time of the year to do a complete grab of the 90gig repo and it takes so long on my connection that at least a couple of superfluous package revisions creep in


----------



## SirDice (Jul 23, 2021)

If I remember correctly someone reported that before, I'm not sure if they submitted a PR for it.



kpedersen said:


> What I seem to be getting is that it is removing everything, almost as if the `-a` (clear all) flag is being used.


That's basically what you're seeing, as if the `-a` option is applied. I've been able to reproduce it too, but since I always use `pkg clean -a` anyway I'm not bothered by it (I have a local repository thus no need for that cache).


----------



## kpedersen (Jul 23, 2021)

SirDice said:


> If I remember correctly someone reported that before, I'm not sure if they submitted a PR for it.


I had a quick search in the forums before posting. I couldn't spot anything. I will see if I can track down the PR. Otherwise I will log one.



SirDice said:


> but since I always use `pkg clean -a` anyway I'm not bothered by it (I have a local repository thus no need for that cache).


Yeah, I am trying to grab a local repo at the moment. I tend to just have one big folder and then use `pkg add <pkg>.tgz` just like in the old pre-pkg-ng days. I don't suppose you can suggest a way of clearing out excess versions of some of these packages? I.e when you sync yours, how do you eliminate older package archives that are no longer useful (or referenced in the database)?

Currently I can only think of creating a full local repo (with database and all) and then doing *another* `pkg fetch` from that. Since that is unchanging (and a local disk), that should simply not transfer the excess versions and flush them out that way. It seems like a bit more work though.


----------



## mer (Jul 23, 2021)

You're not the only one seeing that behavior, but just like SirDice I typically add -a on the command.
I don't have a local repo, so the only use for the cache in my situation is if upgrading a package causes something else to get reinstalled.
I saw that this morning, libgltf needed to get reinstalled because upgrading something caused shared lib to change, but since I did pkg clean last time, libgltf package needed to get pulled down.
Yes, extra work/load on the network, but a tradeoff I'm willing to put up with.

Oh, "chromium" install is still talking about python2.7


----------



## SirDice (Jul 23, 2021)

kpedersen said:


> I am trying to grab a local repo at the moment. I tend to just have one big folder and then use `pkg add <pkg>.tgz` just like in the old pre-pkg-ng days. I don't suppose you can suggest a way of clearing out excess versions of some of these packages?


I was actually thinking of writing a "repocopy" like tool that would make it possible to easily copy the FreeBSD repositories locally (and keep it updated). But I haven't gotten around to it yet. 



kpedersen said:


> I.e when you sync yours, how do you eliminate older package archives that are no longer useful (or referenced in the database)?


Poudriere takes care of that; poudriere-pkgclean(8). On my servers I don't keep a locally cached copy, it just takes up space. So after every pkg(8) run I simply run `pkg clean -a` to clean it all up.


----------



## kpedersen (Jul 23, 2021)

OK, so its a little naff but for now I will use the following to list the path of the packages in current repo:

`$ sqlite3 /var/db/pkg/repo-FreeBSD.sqlite "SELECT path FROM packages"`

`All/double-conversion-3.1.5.19.txz
All/php73-pdo_pgsql-7.3.29.txz
All/nyancat-1.5.2,1.txz
All/wmsun-1.05,1.txz
All/p5-Text-WideChar-Util-0.171.txz
All/php74-pecl-xdebug-3.0.4.txz
All/dotdotpwn-3.0.2_2.txz
All/py38-django-sekizai-1.1.0_1.txz
All/p5-Log-Syslog-Constants-1.02.txz
All/p5-Perl6-Export-Attrs-0.000006.txz
All/php73-pdo_odbc-7.3.29.txz
...`

And then I basically do a diff between this output and the `ls` (for the packages that may have obsolete versions).

A bit manual, but it will do for now.


----------



## VladiBG (Jul 23, 2021)

pkg-clean: fix logic for 'latest version' · freebsd/pkg@622b4ca
					

36dfb4895 adjusted the cache logic to use tilde as a separator, which broke pkg-clean's "latest version" logic. Update pkg-clean to use the correct separator, but this may be a good c...




					github.com


----------



## SirDice (Jul 23, 2021)

Ah, that explains why it broke too. Because I definitely remember it working the way it's mentioned in the man pages.


----------



## VladiBG (Jul 23, 2021)

On 13.0-RELEASE `pkg clean` is broken as it doesn't detect the latest version in the cache. I don't have CURRENT installation to test if it's working there.


----------



## SirDice (Jul 23, 2021)

VladiBG said:


> On 13.0-RELEASE `pkg clean` is broken as it doesn't detect the latest version in the cache. I don't have CURRENT installation to test if it's working there.


All versions use the same pkg(8) version, so I expect it to be broken in the same way on all versions.


----------



## VladiBG (Jul 23, 2021)

Ok i will update to _pkg 1.17.0_ or _pkg-devel 1.17.99.1_ and check again.


----------



## kpedersen (Jul 23, 2021)

There are a few things slightly iffy with the pkg-fetch manpages.

The `-U` argument states that without it, if no write access to database, then updating the repo database is silently ignored. However if you try to do a pkg fetch as a normal user, it gives you an error and terminates. You have to run it with `-U` for it to work:

`$ pkg fetch -U -o . -a`

The good news though is that the database isn't updated, so I can grab all packages. Do an update, grab remaining packages and use that output to remove the few redundant ones.

Though now I am also quite tempted to try _pkg 1.17.0_ and avoid this faff


----------

