# Can I avoid duplicating packages across multiple pkg repositories?



## tuaris (Jul 12, 2015)

I am using ports-mgmt/poudriere to create multiple pkg repositories for my systems. 

`poudriere bulk -j 10amd64 -z base -f base-pkglist`
`poudriere bulk -j 10amd64 -z basicweb -f basicweb-pkglist`
`poudriere bulk -j 10amd64 -z wallets -f wallets-pkglist`

There is a 'base' repository that will be provide a common set of packages across all the servers.  Then there are specific repositories that have packages according to the type of server a host will be.  For example, a 'webserver' repository that has Apache and PHP packages for web server type hosts and a 'wallets' repository that has a Bitcoin daemon for cryptocurrency wallet server type hosts.

On the systems I disable the FreeBSD repos by creating /usr/local/etc/pkg/repos/FreeBSD.conf with: 

```
FreeBSD: { enabled: no }
```

Then I have my repositories as follows:

Repository that all systems will have enabled.
/usr/local/etc/pkg/repos/Base.conf


```
Base: {
       url: "http://pkg.morante.net/base/${ABI}",
       enabled: yes
     }
```

Repository that *only *a web server will have.  
/usr/local/etc/pkg/repos/Basicweb.conf


```
Basicweb: {
       url: "http://pkg.morante.net/basicweb/${ABI}",
       enabled: yes
     }
```

Repository that *only *a Bitcoin/Cryptocurrency wallet server will have. 
/usr/local/etc/pkg/repos/Wallets.conf


```
Wallets: {
       url: "http://pkg.morante.net/wallets/${ABI}",
       enabled: yes
     }
```


My problem is that packages across the repositories are sometime duplicated.  For example all repositories will have a copy of lang/python27 packages due to dependencies.  It's not so much an issue in regards to space.  It's more of an issue with package building and updating.  Poudriere will need to build lang/python27 3 times.  

Then there is the issue of what will happen if (by some reason) I have different versions of a dependent packages in the repositories.  For example devel/icu version 53 in the 'wallets' repo and version 55 in the 'base' repo.  Packages in base need 55 and packages in wallets need 53.

Is it possible to have Poudriere 'use' packages from already built repositories instead of building a new copy to avoid this situation?


----------



## talsamon (Jul 12, 2015)

My idea is: I would make only one repo. I would create files (e.g. install_list) and put in , in the form category/ports what's should be installed. And install it with `pkg install '(cat install_list)'`. And `pkg update` only updates what is really installed on the specific system.


----------



## kpa (Jul 12, 2015)

It's not really achievable with the current implementation of ports-mgmt/poudriere because there's no "intelligence" to consider whether the sets intersect or not and in what proportions. The sets are assumed to be completely distinct with all packages from set A incompatible with set B and vice versa.

If you have ideas how to implement this make a feature request at:

https://github.com/freebsd/poudriere/issues


----------



## tuaris (Jul 14, 2015)

Thanks for the replies so far.  Sorry to hear there is no solution.

It seems I have run into the issue of conflicts in my most recent repository update.  For some reason, one repository has decided to start pulling in perl5-5.20.2 while the other continues to pull perl5-5-18.4.  Both repositories were rebuilt with the latest ports try yet.

This results in an odd issue:


```
The following 8 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
  perl5: 5.18.4_14 -> 5.20.2_5 [Base]

Installed packages to be REINSTALLED:
  webmin-1.760 [Base]
  p5-Net-SSLeay-1.70 [Base]
  p5-IO-Tty-1.12_1 [Base]
  p5-Authen-PAM-0.16_2 [Base]
  python27-2.7.10 [Base]
  libffi-3.2.1 [Base]
  indexinfo-0.2.3 [Base]
```

I try to reinstall the perl update and it wants to downgrade:


```
pkg upgrade -f perl5
Updating Base repository catalogue...
Base repository is up-to-date.
Updating Wallets repository catalogue...
Wallets repository is up-to-date.
All repositories are up-to-date.
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be DOWNGRADED:
  perl5: 5.20.2_5 -> 5.18.4_14 [Base]
```

Then I try to reinstall the webmin update and it wants to upgrade perl


```
pkg upgrade -f webmin
Updating Base repository catalogue...
Base repository is up-to-date.
Updating Wallets repository catalogue...
Wallets repository is up-to-date.
All repositories are up-to-date.
Checking integrity... done (0 conflicting)
The following 8 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
  perl5: 5.18.4_14 -> 5.20.2_5 [Base]

Installed packages to be REINSTALLED:
  webmin-1.760 [Base]
  p5-Net-SSLeay-1.70 [Base]
  python27-2.7.10 [Base]
  libffi-3.2.1 [Base]
  indexinfo-0.2.3 [Base]
  p5-IO-Tty-1.12_1 [Base]
  p5-Authen-PAM-0.16_2 [Base]
```


----------



## tuaris (Jul 14, 2015)

talsamon said:


> My idea is: I would make only one repo. I would create files (e.g. install_list) and put in , in the form category/ports what's should be installed. And install it with `pkg install '(cat install_list)'`. And `pkg update` only updates what is really installed on the specific system.



So you suggest that I create only a single repository for each system type that includes everything.  The only downside being that I would need to keep a more complicated pkg list file for every system.

I guess I was expecting pkg to behave in a way it was not originally designed for?


----------



## tuaris (Jul 15, 2015)

I think my conflict issue was resolved by running `poudriere pkgclean...`.


----------

