# Packages from poudriere's set custom repository to jail



## mecano (Apr 2, 2014)

Hello,
I have one ports-mgmt/poudriere jail with custom sets and I am trying to deploy sets into jails.
So ports-mgmt/pkg is said to allow to deploy packages to jails from the outside which is a thrilling feature. Note that I am not trying this over the network yet but all on the same box.
So I set custom repos corresponding to ports-mgmt/poudriere package directories like this :
`vim /usr/local/etc/pkg/repos/proxy.conf`

```
proxy: {                                                                                                                                                                             
  url : "file:///usr/local/poudriere/data/packages/92x64-default-proxy",
  enable : yes,
  signature_type : NONE,
  mirror_type : NONE
}
```
Then tried to install a package to jail :
`pkg -j proxy install -r proxy bash`


> Updating repository catalogue
> No valid repository found.


but
`pkg -vv`

```
Repositories:
  proxy: { 
    url             : "file:///usr/local/poudriere/data/packages/92x64-default-proxy",
    enabled         : yes
  }
```
and 
`ls /usr/local/poudriere/data/packages/92x64-default-proxy/`


> All             Latest          digests.txz     packagesite.txz


in /usr/local/etc/poudriere.d/make.conf I have WITH_PKGNG=yes
I have created an option directory for the set at /usr/local/etc/poudriere.d/proxy-options but left it empty for now.
If I try to local install with `pkg -j proxy add /usr/local/poudriere/data/packages/92x64-default-proxy/All/bash*`


> pkg: /usr/local/poudriere/data/packages/92x64-default-proxy/All/bash-4.3.0_1.txz: No such file or directory
> pkg: Was 'pkg install /usr/local/poudriere/data/packages/92x64-default-proxy/All/bash-4.3.0_1.txz' meant?
> 
> Failed to install the following 1 package(s): /usr/local/poudriere/data/packages/92x64-default-proxy/All/bash-4.3.0_1.txz


where bash-4.3.0_1.txz exists in All:
`ls -la /usr/local/poudriere/data/packages/92x64-default-proxy/All/bash-4.3.0_1.txz`


> -rw-r--r--  1 root  wheel  1234188 Apr  1 22:58 /usr/local/poudriere/data/packages/92x64-default-proxy/All/bash-4.3.0_1.txz



Running on FreeBSD 9.2-RELEASE-p3, ports-mgmt/pkg is not installed in target jail.
/bin/sh, /var/db/pkg and /var/cache/pkg are available in jail.
Any hint?


----------



## mecano (Apr 2, 2014)

So it looks like you need to have the repo mounted inside the jail as well as a repo configuration file, I was able to update the catalog for the jail after few steps.
Adding 
	
	



```
/usr/local/poudriere/data/packages/92x64-default-proxy/ /usr/jails/proxy/var/ports/packages nullfs ro 0 0
```
 to /etc/fstab.proxy
restart the jail.
`mkdir -p /usr/jails/proxy/usr/local/etc/pkg/repos`
setting /usr/jails/proxy/usr/local/etc/pkg/repos/proxy.conf with

```
proxy: {
  url : "file:///var/ports/packages",
  enable : yes,
  signature_type : NONE,
  mirror_type : NONE
}
```
and catalog update `pkg -j proxy update` works


> Updating repository catalogue
> digests.txz                                                                                                                              100%  928     0.9KB/s   0.9KB/s   00:00
> packagesite.txz                                                                                                                          100% 5724     5.6KB/s   5.6KB/s   00:00
> Incremental update completed, 13 packages processed:
> 0 packages updated, 0 removed and 13 added.


now trying `pkg -j proxy install -r proxy bash*` stops here 


> Updating repository catalogue
> pkg: No packages matching 'bash*' available in the repositories


It's like playing mouse and cat


----------



## SirDice (Apr 2, 2014)

mecano said:
			
		

> now trying `pkg -j proxy install -r proxy bash*` stops here
> 
> 
> > Updating repository catalogue
> ...


You need to escape the asterisk or the shell will try to evaluate it before handing over the results to the command.

`pkg -j proxy install -r proxy bash\*`

I usually disable the standard FreeBSD repository, just edit /etc/pkg/FreeBSD.conf and set enabled to "no".


----------



## kpa (Apr 2, 2014)

Check what `pkg -vv` outputs when run inside the jail, that should give you some clues of what is missing.


----------



## mecano (Apr 2, 2014)

SirDice said:
			
		

> mecano said:
> 
> 
> 
> ...



Thanks for the escaping tip @SirDice
I was planning to use `pkg -j proxy install -r proxy *.*` to install all packages from the repo with one simple command.  :OOO Is there any built in ports-mgmt/pkg? Something like an all switch maybe? Can't find anything on these in the man.
Thanks @kpa for joining in , I'm trying to not run or install ports-mgmt/pkg inside the jail since it has abilities to work on jail from the outside. I want to install the jail, run ports-mgmt/pkg on it and have my first jump in the jail be like sitting on my cosy good old living room (with the help of a customized sysutils/ezjail newjail, not even touching flavours).


----------



## SirDice (Apr 2, 2014)

mecano said:
			
		

> I was planning to use `pkg -j proxy install -r proxy *.*` to install all packages from the repo with one simple command.  :OOO Is there any built in ports-mgmt/pkg? Something like an all switch maybe? Can't find anything on these in the man.


No, it's not there. But you don't want everything anyway, as it will also install all the build dependencies that got packaged.


----------



## kpa (Apr 2, 2014)

Well you don't have to install ports-mgmt/pkg in the jail since it can be run inside a jail with the -j option. However, when run inside a jail it is not aware of the environment or files outside the jail and that's why there has to be a repository configuration file inside the jail as well as have the repository mounted in it. I asked you to check what `pkg -vv` outputs inside the jail because that command also lists the repository definitions that are in effect, if there's a mistake somewhere that's the place to go look for one.


----------



## mecano (Apr 2, 2014)

SirDice said:
			
		

> No, it's not there. But you don't want everything anyway, as it will also install all the build dependencies that got packaged.



Yes, yes I want everything! Because I'm using custom sets in ports-mgmt/poudriere I can deploy all packages from one particular set to one particular jail, so it's easier to deploy and update and it looks so flexible and faster to set up that way. I can have general sets for let's say a web server with databases/mysql55 and another one with databases/postgresql93, if I need a particular port for one jail only, I just create a new set and be done. With the help of sysutils/ezjail for populating base configuration files it looks like a great and efficient combo.



			
				kpa said:
			
		

> Well you don't have to install ports-mgmt/pkg in the jail since it can be run inside a jail with the -j option. However, when run inside a jail it is not aware of the environment or files outside the jail and that's why there has to be a repository configuration file inside the jail as well as have the repository mounted in it. I asked you to check what `pkg -vv` outputs inside the jail because that command also lists the repository definitions that are in effect, if there's a mistake somewhere that's the place to go look for one.



Are you sure? I understand ports-mgmt/pkg can be run from the outside against a jail but how can it be run from the jail itself without being installed? Or do you mean to run it from the outside like this
`pkg -j proxy -vv`
because
root@proxy:/ > `pkg -j proxy -vv` or `pkg -vv`
outputs


> The package management tool is not yet installed on your system.
> Do you want to fetch and install it now? [y/N]: N


And I don't want to install it in it if I don't need to :beergrin


----------



## SirDice (Apr 2, 2014)

mecano said:
			
		

> SirDice said:
> 
> 
> 
> ...


No, really you don't 



> Because I'm using custom sets in ports-mgmt/poudriere I can deploy all packages from one particular set to one particular jail, so it's easier to deploy and update and it looks so flexible and faster to set up that way. I can have general sets for let's say a web server with databases/mysql55 and another one with databases/postgresql93, if I need a particular port for one jail only, I just create a new set and be done. With the help of sysutils/ezjail for populating base configuration files it looks like a great and efficient combo.


Yes, sure I get that. But it's probably worthwhile to invest some time in a custom "meta-port". One that only has the things you need as a run dependency. Installing that package would trigger all the dependencies to be installed. Then you only get the stuff you really need to get it running, instead of everything but the kitchen-sink that was needed to build it.


----------



## mecano (Apr 2, 2014)

SirDice said:
			
		

> Yes, sure I get that. But it's probably worthwhile to invest some time in a custom "meta-port". One that only has the things you need as a run dependency. Installing that package would trigger all the dependencies to be installed. Then you only get the stuff you really need to get it running, instead of everything but the kitchen-sink that was needed to build it.



Yup that's what I'm doing, I leave dependencies checking to ports-mgmt/poudriere.
I use ports-mgmt/poudriere sets and only put what I know I need for a particular job in each set list (meta-ports).
I only plan on building for one os version/arch so things will be easier to maintain.
What is great with ports-mgmt/poudriere is that a set=a repo in the end.
So once all (these meta-ports and dependencies) has been built for that set, I want all of it because 'a poudriere set'='a pkg repo'='one or more jails tools' and I want all (and only) the tools that are needed for a particular job to be installed in the target jail. I won't have any junk I guess or am I missing something?


----------



## kpa (Apr 2, 2014)

I meant to run it outside the jail. This worked just fine for me on jail set up for testing:


```
freebsd10 ~ # pkg -j testjail -vv
...snip...

Repositories:
  FreeBSD: { 
    url             : "pkg+http://pkg.FreeBSD.org/freebsd:10:x86:32/latest",
    enabled         : yes,
    mirror_type     : "SRV",
    signature_type  : "FINGERPRINTS",
    fingerprints    : "/usr/share/keys/pkg"
  }
```

I tested installing editors/vim using the official repository and the installation didn't want to install ports-mgmt/pkg at all:


```
freebsd10 ~ # pkg -j testjail install vim
...
freebsd10 ~ # pkg -j testjail info pkg   
pkg: No package(s) matching pkg
freebsd10 ~ #
```


----------



## mecano (Apr 2, 2014)

SirDice said:
			
		

> instead of everything but the kitchen-sink that was needed to build it.



Do you mean things like devel/gmake and such will be installed with `pkg install`?
Previously I was 
`portmaster -g shells/bash` in a building jail.
then 
`portmaster -PP -D -B --packages-local --local-packagedir=/pkg pkgname` in the target jail.
ports-mgmt/portmaster is a very nice tool too and I didn't noticed that, and wouldn't expect that the kitchen-sink was brought over.
It is really like this with ports-mgmt/pkg?



			
				kpa said:
			
		

> Well you don't have to install ports-mgmt/pkg in the jail since it can be run inside a jail with the -j option.



that was the misleading part  When I red it, I understood "from the inside of the jail issue the command".



			
				kpa said:
			
		

> I meant to run it outside the jail. This worked just fine for me on jail set up for testing



Yes it works great  and you can have custom repo as well, really great!


----------



## mecano (Apr 2, 2014)

Ok sorry I'm really tired and I'm posting too fast, need to eat too 
@SirDice you mean that ports-mgmt/poudriere may create packages for things like devel/gmake when building a port that will then picked up by ports-mgmt/pkg and so I need a meta-ports list for my packages installation then? damn… I just could use the ports-mgmt/poudriere set list then? Can you feed ports-mgmt/pkg with such a thing? It's a one line per port list in the 'category/port name' format.


----------



## SirDice (Apr 3, 2014)

mecano said:
			
		

> @SirDice you mean that ports-mgmt/poudriere may create packages for things like devel/gmake when building a port that will then picked up by ports-mgmt/pkg and so I need a meta-ports list for my packages installation then?


Yes, that's what I meant. If you install _everything_ from the repository you will also install devel/gmake and all the other build dependencies.



> I just could use the ports-mgmt/poudriere set list then? Can you feed ports-mgmt/pkg with such a thing? It's a one line per port list in the 'category/port name' format.


Yeah, you could do that. I don't think pkg-install(8) has an option to read from a file but it should be relatively easy to script. Something like this might do the trick:

```
#!/bin/sh

cat my_package.lst | while read line
do
   pkg install $line
done
```

This may also work but not tested: `pkg install `cat my_package.lst``. But this may cause problems if the list is too long.


----------



## mecano (Apr 3, 2014)

SirDice said:
			
		

> Yeah, you could do that. I don't think pkg-install(8) has an option to read from a file but it should be relatively easy to script. Something like this might do the trick:
> 
> ```
> #!/bin/sh
> ...



Thanks for response @SirDice!
If I have a dozen of meta-ports and some with a lot of dependencies, having all these ports-mgmt/pkg instances at the (nearly) same time won't slow down things too much? Ok we are talking packages not ports so maybe not?
I was going the `awk` way, looks like ports-mgmt/pkg has a -g glob option (but does it mean multiple packages installation?), so I thought a one liner may work. Didn't try yet though.


----------

