# hierarchical jails



## leveche (May 29, 2019)

Hello all,

I have some rather naiive questions re: hierarchical jails. For a start, when I try to start a jail within a jail I get


```
root@outer:/ # jail -c inner 
mount: .: Operation not permitted
jail: test1: /sbin/mount -t devfs -oruleset=4 . /jail/inner/dev: failed
```

My setup is as follows:

/etc/jail.conf on the host:


```
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
mount.devfs;
host.hostname = "$name.mylab.example.org";
path = /jail/$name;
interface = em1;
ip4.addr = 10.1.1.$ip;

outer {
    children.max = 128;
    ip4.addr     = 10.1.1.4,10.1.1.5;
    allow.mount  = true;
    allow.mount.devfs  = true;
    allow.mount.zfs  = true;
}
```

In the 'outer' jail, I can verify that the `mount_devfs_allow` parameter is set, but not sure what the `security.jail.param.allow` settings are:


```
security.jail.mount_procfs_allowed: 0
security.jail.mount_zfs_allowed: 1
security.jail.mount_devfs_allowed: 1
security.jail.param.allow.mount.procfs: 0
security.jail.param.allow.mount.zfs: 0
security.jail.param.allow.mount.devfs: 0
security.jail.param.allow.mount.: 0
security.jail.param.devfs_ruleset: 0
security.jail.devfs_ruleset: 0
security.jail.mount_allowed: 1
```

Is there something obvious I'm missing?

In addition, I'm confused how the hierarchical jails are meant to work with zfs jailed property: would the child jails be able to mount the child zfs datasets of the 'outer' jail? The documentation suggests otherwise. Is the hierarchical jail feature used much at all, and what are the typical scenarios? For me, it seems appealing to arrange jails into hierarchies like work.{project1,project2,...}, labs.{test1, test2,...} etc - is there anyone on this forum who is using a similar setup?

Thanks!
leveche


----------



## zirias@ (May 30, 2019)

leveche said:


> /etc/jail.conf on the host:
> [...]


There's probably a lot missing. For my poudriere jail, I have the following settings:

```
children.max = 20;
     allow.mount;
     allow.mount.devfs;
     allow.mount.procfs;
     allow.mount.linprocfs;
     allow.mount.zfs;
     allow.mount.nullfs;
     allow.mount.tmpfs;
     allow.raw_sockets;
     allow.socket_af;
     allow.sysvipc;
     allow.chflags;
     enforce_statfs=1;
```

Of course, some of these options are not always needed, e.g. procfs and linprocfs. You'll need nullfs very often though, it's typically used for setting up a jail.



leveche said:


> not sure what the `security.jail.param.allow` settings are


These are sysctl(8)s setting default security settings for jails. I'd suggest leaving them all 0 (default), you can override them in /etc/jail.conf for individual jails. Deny by default and allow where needed is always the best policy for anything security-related.



leveche said:


> Is the hierarchical jail feature used much at all, and what are the typical scenarios?


I only use it to have ports-mgmt/poudriere in a jail, and this is the only usecase that actually comes to my mind. Most of the times, you just want to isolate a service, so nesting jails doesn't make much sense. But maybe you find another usecase ...


----------



## Arne (Aug 19, 2022)

leveche said:


> Is the hierarchical jail feature used much at all, and what are the typical scenarios? For me, it seems appealing to arrange jails into hierarchies like work.{project1,project2,...}, labs.{test1, test2,...} etc - is there anyone on this forum who is using a similar setup?



Yes, this is a very common use case. Today "normal" server hardware is extremly overpowered. We use some of our servers as "jail-farms" and bundle jails belonging to the same lab environment together under the lab master server. This helps to give stuff members the rights to restart the jails they need access to without compromising other lab environments.

VNET Jails work perfect inside hierarchical jails and helps to solve a few permisson problems.

Thinking only at poundriere is way too short. Poudriere is the special case here.


----------

