# Cannot mount ZFS delegated administration datasets in a jail



## asteriskRoss (Apr 26, 2014)

I have a machine running (`uname -srmi`) FreeBSD 9.2-RELEASE-p4 i386 GENERIC. The server hosts a number of jails installed and maintained using sysutils/ezjail.  One of my jails is a ZFS file server that receives ZFS snapshots as backups from my other machines.  I would like some isolation for the different machines here, such that one machine has access only to its own ZFS datasets.  One option for achieving this would be a separate jail for each machine, but this doesn't scale well and is somewhat inelegant.  The option I chose was to set up a different user in the jail for each machine that sends snapshots and to delegate ZFS administration to each user for its own dataset and descendants.  Unfortunately, I'm having trouble mounting datasets as a user within the jail.  Mounting the ZFS datasets within the jail as root appears to be working fine.  I have checked that ZFS administration delegation is working on the host system, outside of a jail. 

On the host system, I have the following in /etc/sysctl.conf:

```
# Allow ZFS mounting in the backup jail so it can receive ZFS datasets
security.jail.enforce_statfs=1
security.jail.mount_allowed=1
security.jail.mount_zfs_allowed=1

# Allow users to mount filesystems so ZFS administration within
# the backup jail may be delegated
vfs.usermount=1
```

In the sysutils/ezjail configuration file for the jail (/usr/local/etc/ezjail/backup_mydomain_com) I have set the following parameters (the ezjail script runs `zfs jail <jailname> <dataset>` and then passes the parameters to be used by /etc/rc.d/jail) :

```
[...]
export jail_backup_mydomain_com_zfs_datasets="tank1/backup"
export jail_backup_mydomain_com_parameters="allow.mount=\"true\" allow.mount.zfs=\"true\" enforce_statfs=1"
[...]
```

In the jail, I checked that all the parameters were set as I expected:

```
root@backup:~ # sysctl {security.jail.jailed,security.jail.enforce_statfs,security.jail.mount_allowed,security.jail.mount_zfs_allowed,vfs.usermount}
security.jail.jailed: 1
security.jail.enforce_statfs: 1
security.jail.mount_allowed: 1
security.jail.mount_zfs_allowed: 1
vfs.usermount: 1
```

From within the jail, as root, I have configured the following permission sets that will be delegated to each user for their dataset and descendants.  My intention is to prevent users from destroying the dataset configured for them but have lots of control over child datasets.

```
root@backup:~ # zfs allow tank1/backup
---- Permissions on tank1/backup -------------------------------------
Permission sets:
	@descendants clone,create,destroy,diff,hold,mount,mountpoint,promote,readonly,receive,release,rename,rollback,send,snapshot
	@local create,mount,receive,send
```

I have applied the permission sets to a dataset intended for use by one such user, fluxcapacitor:

```
root@backup:~ # zfs allow tank1/backup/fluxcapacitor
---- Permissions on tank1/backup/fluxcapacitor -----------------------
Local permissions:
	user fluxcapacitor @local
Descendent permissions:
	user fluxcapacitor @descendants
---- Permissions on tank1/backup -------------------------------------
Permission sets:
	@descendants clone,create,destroy,diff,hold,mount,mountpoint,promote,readonly,receive,release,rename,rollback,send,snapshot
	@local create,mount,receive,send
```

This dataset has a mount point in the user's home directory:

```
root@backup:~ # zfs list -o name,mountpoint,canmount tank1/backup/fluxcapacitor
NAME                        MOUNTPOINT                      CANMOUNT
tank1/backup/fluxcapacitor  /usr/home/fluxcapacitor/backup        on
```

I can mount the dataset as the jailed root with `zfs mount tank1/backup/fluxcapacitor`, but doing the same as the jailed fluxcapacitor user results in the following error:

```
fluxcapacitor@backup:~ % zfs mount tank1/backup/fluxcapacitor
cannot mount 'tank1/backup/fluxcapacitor': Insufficient privileges
```

The ZFS side of things appears to be working since the jailed user can create and destroy datasets under tank1/backup/fluxcapacitor but just not mount them.  For example:

```
fluxcapacitor@backup:~ % zfs create tank1/backup/fluxcapacitor/example
cannot mount 'tank1/backup/fluxcapacitor/example': Insufficient privileges
filesystem successfully created, but not mounted

fluxcapacitor@backup:~ % zfs list -r -o name tank1/backup/fluxcapacitor
NAME
tank1/backup/fluxcapacitor
tank1/backup/fluxcapacitor/example

fluxcapacitor@backup:~ % zfs destroy tank1/backup/fluxcapacitor/example

fluxcapacitor@backup:~ % zfs list -r -o name tank1/backup/fluxcapacitor
NAME
tank1/backup/fluxcapacitor
```

I saw there was a change in the zfs(8) man page between 9.2 and 10 on what the enforce_statfs parameter needed to be set to.  For 9.2:


> zfs jail jailid filesystem
> 
> Attaches the specified	filesystem to the jail identified by JID
> jailid.  From now on this file	system tree can	be managed from	within
> ...



And for 10:


> zfs jail jailid filesystem
> 
> Attaches the specified	filesystem to the jail identified by JID
> jailid.  From now on this file	system tree can	be managed from	within
> ...



I tried changing enforce_statfs to 0 but it did not resolve my issue and also had the expected and undesired effect of having to use the full host filesystem path for mounting ZFS datasets within the jail.

Does anyone have any ideas of what I'm missing or what I should try next?

--
*References*

 jail(8)
 zfs(8)
 ezjail documentation ([url=http://erdgeist.org/arts/software/ezjail/]http://erdgeist.org/arts/software/ezjail/)[/url]
 Thread 34598 "mounting zpool within a jail"
 Thread 37571 "Why permission denied for this ZFS user?"
 "Fwd: ZFS and jailed environments -- best practice"([url=http://lists.freebsd.org/pipermail/freebsd-hackers/2009-December/030161.html]http://lists.freebsd.org/pipermail/free ... 30161.html)[/url]
 "Delegation in ZFS with FreeBSD jails" ([url=http://blog.binarysolutions.dk/2010/08/22/delegation-in-zfs-with-freebsd-jails/]http://blog.binarysolutions.dk/2010/08/ ... bsd-jails/)[/url]


----------



## gmarcus (May 26, 2014)

*Re: Cannot mount ZFS delegated administration datasets in a*

*I*t seems we might have the same problem. I just posted mine here: https://forums.freebsd.org/viewtopic.php?f=48&t=46614. *T*ry changing the owner of the mount point to be your user 'fluxcapacitor'*.*

*H*ope it helps.


----------



## asteriskRoss (May 28, 2014)

*Re: Cannot mount ZFS delegated administration datasets in a *

Thanks for the reply, @gmarcus.  The directory for the mount point (/usr/home/fluxcapacitor/backup) is already owned by the jail's fluxcapacitor user (a member of the jail's fluxcapacitor group).  Is that what you meant?  I read your post (Thread 46614) but I am not using Access Control Lists (ACLs) in my configuration.


```
fluxcapacitor@backup:~ % zfs list -o name,mountpoint,canmount,jailed tank1/backup/fluxcapacitor
NAME                        MOUNTPOINT                      CANMOUNT  JAILED
tank1/backup/fluxcapacitor  /usr/home/fluxcapacitor/backup        on      on

fluxcapacitor@backup:~ % ls -ld backup
drwxr-xr-x  2 fluxcapacitor  fluxcapacitor  2 Apr 26 09:55 backup

fluxcapacitor@backup:~ % getfacl backup
# file: backup
# owner: fluxcapacitor
# group: fluxcapacitor
            owner@:rwxp--aARWcCos:------:allow
            group@:r-x---a-R-c--s:------:allow
         everyone@:r-x---a-R-c--s:------:allow

fluxcapacitor@backup:~ % zfs mount tank1/backup/fluxcapacitor
cannot mount 'tank1/backup/fluxcapacitor': Insufficient privileges
```

If I remove the /usr/home/fluxcapacitor/backup directory, and attempt to mount the ZFS dataset as the fluxcapacitor user, the directory is recreated, owned by fluxcapacitor, but the ZFS dataset is not mounted:

```
fluxcapacitor@backup:~ % ls -ld backup
drwxr-xr-x  2 fluxcapacitor  fluxcapacitor  2 May 28 10:28 backup

fluxcapacitor@backup:~ % rmdir backup

fluxcapacitor@backup:~ % ls -ld backup
ls: backup: No such file or directory

fluxcapacitor@backup:~ % zfs mount tank1/backup/fluxcapacitor
cannot mount 'tank1/backup/fluxcapacitor': Insufficient privileges

fluxcapacitor@backup:~ % ls -ld backup
drwxr-xr-x  2 fluxcapacitor  fluxcapacitor  2 May 28 10:28 backup
```

Any and all suggestions are welcome.


----------



## gmarcus (May 28, 2014)

*Re: Cannot mount ZFS delegated administration datasets in a*

*C*heck also the values of aclinherit and aclmode in the ZFS properties, they might need to be set to passthrough. Otherwise it is possible that you are not inheriting the ACL from backup into backup/fluxcapacitor.


----------



## asteriskRoss (May 29, 2014)

*Re: Cannot mount ZFS delegated administration datasets in a *

Thank you for the suggestion.  As I mentioned in my previous post, I'm not making use of filesystem ACLs for this configuration, only filesystem permissions and ZFS dataset permissions.  The same ZFS delegated administration set-up I have used works fine outside of the jail on the host system, but not inside the jail.  Perhaps it's time to consider submitting a Problem Report (PR).

I tried changing the aclinherit and aclmode properties as you suggested, but unfortunately it had no effect:

```
root@backup:~ # zfs set aclinherit=passthrough tank1/backup

root@backup:~ # zfs set aclmode=passthrough tank1/backup

root@backup:~ # zfs list -o name,aclinherit,aclmode
NAME                           ACLINHERIT      ACLMODE
tank1                          restricted      discard
tank1/backup                  passthrough  passthrough
tank1/backup/fluxcapacitor    passthrough  passthrough

root@backup:~ # su fluxcapacitor

fluxcapacitor@backup:/root % zfs mount tank1/backup/fluxcapacitor
cannot mount 'tank1/backup/fluxcapacitor': Insufficient privileges
```


----------

