# mount nullfs in a jail



## Alain De Vos (Oct 19, 2019)

Is it possible to mount nullfs within a jail ?
Mount produces : Operation not permitted
{PS: ravenports needs nullfs}


----------



## scottro (Oct 19, 2019)

It should be. That used to be our way of doing it, but in honesty, now we use standard jails.

http://srobb.net/nullfsjail.html  This is my old page on it. I'm not even sure when I wrote it but it was for 9.2 and worked at the time.
I'm sorry but then we went back to standard jails and I have no idea if this would still work.  I have to say, don't ask me, because, due to some medical 
annoyances I'm a bit too drugged to give intelligent answers these days and it was so long ago that I really don't remember anything about it.


----------



## T-Daemon (Oct 19, 2019)

Alain De Vos said:


> Is it possible to mount nullfs within a jail ?


Have look at this reply in this Thread 67891.


----------



## Alain De Vos (Oct 19, 2019)

I see it does not work.
As "ugly" workaround I just chroot into the jail and that works.


----------



## ShelLuser (Oct 19, 2019)

There's something else to consider... what are you mounting? If you're trying to mount something that's not available inside the jail then obviously that's not going to work. Otherwise....

Still, the fstab.jail (see mount.fstab inside /etc/jails.conf) is best way to resolve this.


----------



## D-FENS (Oct 19, 2019)

Alain De Vos said:


> Is it possible to mount nullfs within a jail ?
> Mount produces : Operation not permitted
> {PS: ravenports needs nullfs}


Mounting things inside the jail is not recommended, unless you want to let users inside a jail have a VM like experience.
Mostly you mount nullfs outside of the jail in the jail's fstab.
Here is how I do it:

```
# cat fstab
#FS                                             Mountpoint                            Type    Options     Dump  Pass
/basejails/basejail01/root/                     /jails/jail01/mnt/                    nullfs  ro,noatime  0     0
/jails/jail01/root/etc                          /jails/jail01/mnt/etc                 nullfs  rw          0     1
/jails/jail01/root/usr/home                     /jails/jail01/mnt/usr/home            nullfs  rw          0     1
/jails/jail01/root/usr/local/etc                /jails/jail01/mnt/usr/local/etc       nullfs  rw          0     1
/jails/jail01/root/tmp                          /jails/jail01/mnt/tmp                 nullfs  rw          0     1
/jails/jail01/root/var                          /jails/jail01/mnt/var                 nullfs  rw          0     1
/basejails/basejail01/root/usr/local/etc/rc.d   /jails/jail01/mnt/usr/local/etc/rc.d  nullfs  ro,noatime  0     2
/basejails/basejail01/root/var/db/pkg           /jails/jail01/mnt/var/db/pkg          nullfs  ro,noatime  0     2
```


```
# cat jail.conf | grep fstab
    mount.fstab = "$path/../fstab";
```

I actually don't use mount.fstab in jail.conf but mount them manually in scripts, hooked on exec.created and exec.prestop but it works the same way.
I switched to the hooks because for some reason the jail service left mounted nullfs-es after jail stop but if I unmounted them in advance, it complained with failures. So I just switched to fully manual. But theoretically it should work the same.


----------



## Lamia (Oct 22, 2019)

roccobaroccoSC said:


> Mounting things inside the jail is not recommended, unless you want to let users inside a jail have a VM like experience.
> Mostly you mount nullfs outside of the jail in the jail's fstab.
> Here is how I do it:
> 
> ...


You mind sharing the script for unmounting?

I could mount nullfs and a number of other filesystems in a jail but restarting such a jail sometimes is not possible. The error is often something like 'resource deadlock avoided' and that could only be fixed by restarting the host. Part of the problem is that other jails also have the same dir. (e.g. /var/db/freebsd-update & /usr/ports/distfiles)  mounted. Hence restarting one jail, which requires unmounting those dirs/fs, fails.

Kindly upload your script; there might be some clues in it.


----------



## Alain De Vos (Oct 22, 2019)

It was this I tried in a jail,








						quickstart freebsd · Ravenports/Ravenports Wiki
					

Universal package builder system. Contribute to Ravenports/Ravenports development by creating an account on GitHub.




					github.com


----------



## D-FENS (Nov 2, 2019)

Lamia said:


> You mind sharing the script for unmounting?
> 
> I could mount nullfs and a number of other filesystems in a jail but restarting such a jail sometimes is not possible. The error is often something like 'resource deadlock avoided' and that could only be fixed by restarting the host. Part of the problem is that other jails also have the same dir. (e.g. /var/db/freebsd-update & /usr/ports/distfiles)  mounted. Hence restarting one jail, which requires unmounting those dirs/fs, fails.
> 
> Kindly upload your script; there might be some clues in it.



Sure. Our script is not an extremely clean solution but it does the job well so far. It could be improved for sure.

```
#!/bin/sh -x

jail=$1
mntDir=$2

[ -n "$mntDir" ] || { echo "mntDir is empty." 1>&2; exit 3; }

echo "Unmounting dirs under $mntDir"

# unmount fdescfs
/sbin/umount "$mntDir/dev/fd"  || /sbin/umount -f "$mntDir/dev/fd"    || true

# unmount devfs
/sbin/umount "$mntDir/dev"    || /sbin/umount -f "$mntDir/dev"      || true

cat $mntDir/../fstab \
    | grep -v '^\s*#.*' \
    | sort -r -k 6 \
    | awk 'NF { print "/sbin/umount -t " $3 " " $2 " || /sbin/umount -f -t " $3 " " $2 " || true"; }' \
    | /bin/sh \
    #
```

And I call the hook like this in my jail.conf

```
exec.prestop   += "$path/../exe/hooks/prestop.sh   $name \"$path\"";
```

I have a "jail/mnt" directory, where I mount the stuff. I also have a "jail/fstab" file but I do not use "mount.fstab" in jail.conf. I mount manually in a similar script on the "exec.created" event.


----------



## Lamia (Nov 2, 2019)

roccobaroccoSC said:


> Sure. Our script is not an extremely clean solution but it does the job well so far. It could be improved for sure.
> 
> ```
> #!/bin/sh -x
> ...


Thanks. Sure, the script is for a manual mount. You mind sharing its twin i.e. the one for mounting. I will check if both can work in our case. The jails share the same mount dir e g. /var/db/freebsd-update and /var/ports[/dist].


----------



## D-FENS (Nov 9, 2019)

That's the script for mounting.
I execute this in "prestart".

```
#!/bin/sh -x

name=$1
mntDir=$2

isMounted=$(zfs get -H -o value mounted "zroot/jails/myjail/mnt")
if [ "$isMounted" = "no" ]; then
        zfs mount "zroot/jails/myjail/mnt"
fi

/sbin/mount -a -F "$mntDir/../fstab"

/sbin/mount -t devfs -oruleset=4 . "$mntDir/dev"
/sbin/mount -t fdescfs . "$mntDir/dev/fd"
```


----------

