# How To: Execute Firefox in a jail using iocage and ssh/jailme



## hukadan (Sep 26, 2015)

*Motivations*

The main reason to put a browser in a jail is quite simple : browsers cannot be trusted. They are too much exposed. Executing a browser inside a jail(8) is a way to be sure that the damages induced by a malicious software are contained (as much as possible). I decided to write this tutorial after posting in the Thread x11-applications-in-iocage-jails.53224 since I did not find any on the internet (including this forum). There is nothing new here but I just gathered information here and there to put everything together in one and the same place.

In this tutorial I will describe the steps allowing to execute a browser inside a jail(8). Of course, it can be adapted to other software such as e-mail clients. I will take Firefox (www/firefox) as browser but once again, it applies to others with almost no modification.

We will use different tools to achieve this goal. First, we will create a jail(8) using iocage(8) but you can do the same using other tools such as ezjail(8). We will also use pf(4) to NAT our jail(8) and nullfs(5) to share folders between the host and the jail(8). In order to launch the browser, I show two alternatives. With the first one, a ssh(1) server is used with X11 forwarding. With the second one, we use a tool called jailme() (sysutils/jailme) which does a work similar to jexec(8) but without needing the root credentials.

*1- Let's begin with the jail network*

In this section we will use pf(4) to NAT our jail(8). This configuration makes sure that my jails do not interfere with my house network and can also take advantage of my firewall settings.

Create a lo1 interface that will be used for the jail(8) network and assign an IP address to it.
`# ifconfig lo1 create
# ifconfig lo1 inet 10.0.0.254/24`

In order to make this change permanent, add the following lines to /etc/rc.conf.

```
cloned_interfaces="lo1"
ifconfig_lo1="inet 10.0.0.254 netmask 255.255.255.0"
```
Next, configure pf(4)  to NAT the future jail(8) by creating/editing the /etc/pf.conf file (this is a minimal configuration to NAT the jail and by no means a firewall configuration. You have to adapt it to your existing rules) and replacing re0 with your network interface name (you can have a list of your network interfaces with `ifconfig -l`).

```
ext_if="re0"
int_if="lo1"
localnet=$int_if:network

scrub in all fragment reassemble
set skip on lo0
set skip on lo1

#nat for jails
nat on $ext_if inet from $localnet to any -> ($ext_if)
```
Then launch pf(4) to activate this network.
`# pfctl -e -f /etc/pf.conf`

Do not forget to make sure that pf(4) is launched at boot.

```
pf_enable="YES"
pflog_enable="YES"
```

*2- jail(8) creation using iocage(8)*

Create a jail(8) named (more exactly tagged) injail with an IP address `10.0.0.1` and a hostname injail.
`# iocage create hostname=injail tag=injail ip4_addr="lo1|10.0.0.1"
# iocage set hostname=injail injail`

Since mount_nullfs(8) will be used in the next part to mount jail(8) folders, it can be convenient to use the hack88 hack (introduced in 1.6.0) on our jail(8).
`# iocage set hack88=1 injail`

Then make sure the jail(8) is launched at boot. This is done in two steps. First, set the boot property of the jail(8) to *on* (this is not a jail(8) property _per se_ but an iocage(8) specific property).
`# iocage set boot=on injail`

Then modify the /etc/rc.conf file.

```
iocage_enable="YES"
```
The last step is to launch the jail(8), attach to it and create the jailuser user using pw(1).
`# iocage start injail
# iocage console injail
# pw user add jailuser -s /bin/tcsh -m
# passwd jailuser`

If you plan to use jailme(), make sure that the username and user id match between host and jail(8) when creating this user (to know your user id, use id(1)).

*3- Get the jail(8) "Firefox ready"*

If you want to use Firefox, you have to install it.
`# iocage console injail
# pkg install -y firefox`

Firefox needs devel/dbus to run. So make sure this service is started at jail boot by editing the /etc/rc.conf file on the jail.

```
dbus_enable="YES"
```
Then, start the service.
`# service dbus start`

*4- jailme or ssh(1)*

Now, you have the choice between two solutions. The first one requires a perfect match between the jail and the host for both the username and the user id. We also have to set xhost(1) in order to accept connections from the jail(8) to the host X server. The second one requires to setup a ssh(1) server on the jail(8), to install xauth(1), to modify the host file of the jail(8) and use public key authentication (this is not compulsory but it is really convenient).

Two points worth mentioning before going on :

if your version of x11-servers/xorg-server is equal or greater than 1.17, you have to make sure that the Xorg server listen to tcp. Using startx(1), the command is `% startx -- -listen tcp`.
if you plan to use multiple jails to launch multiple Firefox instances, do not forget to add the `--new-instance` option each time you start Firefox to avoid strange behavior like starting Firefox on one jail and see it executes on a another one (see this for more details).

*4-1 jailme*

jailme() works as jexec(8) except that you do not need root credentials to use it. It will let you execute a command in the jail(8) as normal user provided that your username and your id are identical on both the host and the jail(8). The _trick_ here is to execute an X11 application on the jail(8) and make it use the X server on the host. For this to happen, you have to modify the access to your X server with the xhost(1) command. Since our jail(8) as the IP address `10.0.0.1`, allow only this address.
`% xhost + inet:10.0.0.1`

Add the following line to your ~/.xinitrc in order to allow the jail(8) when you login.

```
xhost + inet:10.0.0.1
```
If you do not use ~/.xinitrc in your setup, I am sure there is another way to make this happen. Of course, we need to install jailme() on the host.
`# pkg install -y jailme`.

jailme() takes as argument the jail id or jail name and the command you want to launch. The problem is that the jail id can change and the jail name set automaticaly by iocage(8) is rather long and cannot be changed (as far as I know). So make a small script named `injail` that allows you to directly issue `injail [I]command[/I]`.

```
#! /bin/sh
jail_hostname=injail
jail_id=$(jls -h jid host.hostname | grep $jail_hostname | cut -w -f 1)
jailme $jail_id $@
```
That's it. You can try it.
`% injail firefox`

If you are not interested in the ssh(1) setup, you can skip the following paragraph and go to the one about sharing folders.

*4-2 ssh(1) and X11 forwarding*

With this solution, we use the well known X11 forwarding capabilities of ssh(1). For that, we need to configure the ssh(1) server on the jail. First attach to the jail(8).
`# iocage console injail`

Edit the /etc/ssh/sshd_config file and add the following lines.

```
X11Forwarding yes
X11UseLocalhost no
```

Since we do not use localhost, make sure that the jail(8) can resolve its hostname by adding the following line to /etc/hosts on the jail.

```
10.0.0.1  injail
```
The last step is to make sure that the ssh(1) server is started at jail boot adding the following line to /etc/rc.conf file.

```
sshd_enable="YES"
```
Then, start the ssh server.
`# service sshd start`.

We also need to install xauth(1) to do X11 forwarding.
`# pkg install -y xauth`

Then, come back to the host and configure de client side of ssh(1). It is quite boring to have to type password each time. So setup a public key authentication.
`% ssh-keygen -t rsa -b 4096 -C "jailuser@example.com" -f ~/.ssh/injail
% ssh jailuser@10.0.0.1 mkdir .ssh
% cat ~/.ssh/injail.pub | ssh jailuser@10.0.0.1 'cat >> .ssh/authorized_keys'`

We also have to tell the ssh(1) client to use this public key and to activate the X11 forwarding. Edit/create the ~/.ssh/config file.

```
Host injail
  Hostname 10.0.0.1
  Port 22
  User jailuser
  IdentityFile ~/.ssh/injail
  ForwardX11 yes
  ForwardX11Trusted yes
```
Check that everything works.
`% ssh injail firefox`

*5- Sharing folders between the host and the jail(8) (optional)*

It can be convenient to grab files you just downloaded on the jail(8) to use them on the host. nullfs(8) will be used to achieve this. First check the mount point of the jail.
`# iocage get mountpoint injail`

For those who already use iocage(8), you can see that this mount point is shorter than usual. This is due to the hack88 we set earlier. With this information, edit the /etc/fstab file.

```
$JAIL_MOUNT_POINT/root/home/jailuser /home/hostuser/whereyouwant nullfs  rw,late  0 0
```
$JAIL_MOUNT_POINT has to be replaced by the mount point you got previously. The late option is here to avoid problems such as this one Thread nullfs-via-etc-fstab-on-zfs-only-system.11454.

We can check if everything is set properly.
`# mount /home/hostuser/whereyouwant
% ls /home/hostuser/whereyouwant`

You can also choose not to mount the entire jailuser home but only one folder (I personaly mount a Documents subfolder of the jailuser home on the host). Anyway, now you have access to your files in the jail(8) from the host.

*6 - Sharing sound devices (optional)*

The internet is nice, but with sound it is even better and jails can share devices with the host including sound devices. This done by editing the devfs.rules(1). The default ones, including jails, can be found in the /etc/default/devfs.rules file. If the default settings are fine for _classic_ jails, it is not enough for a jail(8) executing a browser. Therefore, non default devfs.rules(1) settings have to be created. Create/edit the /etc/devfs.rules file and add the following lines.

```
# Devices usually found in a desktop jail for sound.
#
[devfsrules_desktop_jail=5]
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add path zfs unhide
add path 'mixer*' unhide
add path 'dsp*' unhide
```
This is simply the default settings plus the two last lines that give the jail(8) access to sound devices. Set the new devfs.rules(1) to your jail(8).
`# iocage set devfs_ruleset=5 injail`

And restart the jail(8) (not using the soft restart)
`# iocage stop injail
# iocage start injail`

You should be able to play sound using your jailed browser now.

*Afterthoughts*

I have tried the different steps of this tutorial thoroughly in order to make sure that if you follow them this should work for you as well. However, each system is different and may be this tutorial will not work right away for you. Please, do not hesitate to post if you are facing problems following this tutorial.

There is for sure smarter ways to achieve what I just described. I would really like to know those smarter ways so I can improve this tutorial and my FreeBSD knowledge. So do not hesitate to question my setup. We could also image a script that take a snapshot of the jail(8) before starting the browser and rollback to that snapshot when you exit so you always start with a _clean_ system.

Concerning browsers, it would be nice to change "_Firefox ready_" by "_Browser ready_", so if you try this with other browsers, do not hesitate to come here and tell me. I will modify this tutorial accordingly.

Why only browsers ? Because it is the only peace of software I jailed so far. I can imagine that other software would need additional/different devices shared between the host and the jail(8). A word processor for instance would not need sound but could need a printer device (/dev/ulpt0 for instance). Therefore, do not hesitate to share your devfs.rules(1) for other software.

Finally, this is my first HowTo. So if some parts are not clear enough for you or you think I skipped an important step and therefore this tutorial need some clarifications, I would happily add them to this tutorial.

*EDITS :*

_2015-09-07_ :

sections have been rearranged : optional steps have been put at the end ;
multiple Firefox instances issue added : comment added on the `--new-instance` option.

new *Xorg* version issue added : need to start  Xorg(1) server with `-listen tcp` option starting from version *1.17* ;
*$DISPLAY* variable not strictly needed : if you attach your jail(8) using iocage(8), you need to set it. Otherwise, you can skip this step. The initial text that just followed the jailme() script in the jailme section was :


> You can now use it to access to a shell on the jail(8) and set the $DISPLAY variable so the jail(8) uses the host X server. `% injail /bin/tcsh`. From there, edit the ~/.cshrc file by adding the following line
> 
> ```
> setenv  DISPLAY 10.0.0.254:0.0
> ...


----------



## hukadan (Sep 26, 2015)

(In case some further steps need to be added)


----------



## ivosevb (Sep 27, 2015)

Excellent post. Just change `pkg install -y auth` in xauth.


----------



## ivosevb (Sep 27, 2015)

hukadan said:


> iocage set hostname=injail


aAnd here i get "ERROR: Missing property or UUID!", so
`iocage set hostname=injail [I]jail_UUID[/I]`


----------



## hukadan (Sep 27, 2015)

Sorry for that, I forgot the jail(8) tag. It is now corrected.


----------



## yggdrasil (Sep 27, 2015)

> jailme() takes as argument the jail id and the command you want to launch. The problem is that the jail id can change. So make a small script named injail that allows you to directly issue injail command.


Not quite true (anymore):


> usage: jailme jid|jailname command [...]


Works well for me


----------



## hukadan (Sep 27, 2015)

yggdrasil said:


> Not quite true (anymore):
> 
> Works well for me



Well, I did not mention it because iocage(8) set automatically a rather long jail name, too long to be remembered, and it is not possible to change it (I suppose it is used by iocage(8) somehow and should not be change anyway).  But you are right, it is worth mentioning and I corrected the tutorial.


----------



## yggdrasil (Sep 28, 2015)

hukadan said:


> iocage(8) set automatically a rather long jail name, too long to be remembered, and it is not possible to change it


Really? I'm currently using cbsd, but could imagine switching to iocage in the future, so this surprises me a bit.


----------



## hukadan (Sep 29, 2015)

yggdrasil said:


> Really? I'm currently using cbsd, but could imagine switching to iocage in the future, so this surprises me a bit.



You should not take my word for it . But last time I checked, the jail name was not accessible with the get/set command. But iocage(8) uses tags which are quite similar to name in a sense.


----------



## yggdrasil (Oct 28, 2015)

Quick question: I'd like to have multiple firefox jails, maybe one on the main system. Unfortunately, when the main system firefox is already running, a `jailme $JID firefox` runs another main system instance  Any know how to fix this?


----------



## hukadan (Oct 28, 2015)

The `--new-instance` option should do the trick. If not, try with the `--no-remote` option. For more details, read this and `firefox --help`.


----------



## yggdrasil (Oct 29, 2015)

Thank you 

But now I have a new problem, that I already had in the past every now and then: sometimes, without any apparent reason, trying to execute a graphical program in a jail results in a "cannot open display" error.
Right now for example. I just started the computer, wanted to execute a new firefox instance from a jail via `jailme 2 firefox --new-instance`, yielding a "Error: cannot open display: :0.0". Experimenting with other DISPLAY variables (setting it to $MAIN_IP:0.0 for example) doesn't change this.
`xhost` lists the corresponding IP address, so it should work, and actually it did work before.
As an alternative, I tried the ssh method: `ssh -Y 192.168.0.101 firefox --new-instance` yields an error stating that Firefox is already running, but unresponsive. Inside the jail, there is no Firefox process running, only on the main system.

Any Ideas what I'm overlooking?


----------



## hukadan (Oct 30, 2015)

If you followed the tutorial, could you please connect to the jail and check the value of your $DISPLAY variable.
`% jailme you_jail_id /bin/tcsh`
And once in the jail (this is my output.).

```
% echo $DISPLAY
10.0.0.254:0.0
```
If you did not follow the tutorial, which part did you skip/adapt ?


----------



## yggdrasil (Oct 30, 2015)

I adapted the IP to be directly deployed on my re0 interface.

```
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
    ether 74:d0:2b:9d:4c:46
    inet 192.168.0.12 netmask 0xffffff00 broadcast 192.168.0.255
    inet 192.168.0.102 netmask 0xffffffff broadcast 192.168.0.102
    inet 192.168.0.101 netmask 0xffffffff broadcast 192.168.0.101
    inet 192.168.0.103 netmask 0xffffffff broadcast 192.168.0.103
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
```
That's once my Main IP + three Jails. Therefore, no pf is running. pinging the jails and logging in from the host via ssh works.

`xhost` looks like this:

```
yggdrasil@midgard:~ % xhost
access control enabled, only authorized clients can connect
INET:192.168.0.103
INET:192.168.0.102
INET:192.168.0.101
INET6:localhost
INET:localhost
```

I followed the iocage part, adapting to my interface/IPs, skipped the shared folder part.

Inside the 10.2 jail:

```
yggdrasil@banking:/ % echo $DISPLAY
192.168.0.102:0.0
yggdrasil@banking:/ % midori
midori - Cannot open display:
…
yggdrasil@banking:/ % echo $DISPLAY
192.168.0.12:0.0
yggdrasil@banking:/ % midori
midori - Cannot open display:
…
yggdrasil@banking:/ % echo $DISPLAY
:0.0
yggdrasil@banking:/ % midori
midori - Cannot open display:
…
yggdrasil@banking:/ % echo $DISPLAY
:0
yggdrasil@banking:/ % midori
midori - Cannot open display:
```

Since it worked before with these jails and xhost settings, I'm thoroughly  confused...


----------



## hukadan (Oct 30, 2015)

yggdrasil said:


> pinging the jails and logging in from the host via ssh works.


Does it work the other way around (ping the host from the jails - you have to allow raw socket for that). That's just a shot in the dark (and also because of the 0xffffffff netmask).


----------



## yggdrasil (Oct 30, 2015)

Yep, that works.
Before I followed your writeup (before my recent reinstall), I had a simple banking jail set up, that just worked. No need to even modify the DISPLAY variable, only adding to the xhost, that was it. Then `jailme`, or `sudo jexec` would just work. Until it wouldn't suddenly for no discernible reason. Now, even a full restart of the system doesn't reset whatever is going wrong.


----------



## yggdrasil (Oct 31, 2015)

Update:
It seems this may be caused by the xorg-server version in the Latest package set. I thought to remember that after switching from Quarterly to Latest it still worked, but that seems to be wrong. In a VM it worked (Quarterly, xorg-server 1.14), but after switching to Latest (xorg-server 1.17), it stopped working.


----------



## hukadan (Nov 1, 2015)

yggdrasil said:


> No need to even modify the DISPLAY variable


Apparently, you are right. Thank you for pointing this out. I will check if it does not break anything to remove this step and modify the tutorial accordingly.


yggdrasil said:


> but after switching to Latest (xorg-server 1.17), it stopped working.


As far as I understood, Xorg went from `-listen tcp` as default option to `-nolisten tcp`. So you have to make sure that Xorg starts with the first option again. It worked for me (I ran into this problem today).


----------



## yggdrasil (Nov 2, 2015)

Thank you very much, now it works again as promised 

In case someone else is using good old xdm: you have to edit /usr/local/lib/X11/xdm/Xservers and simply add the `-listen tcp` to the X server command specified there.


----------



## bobk48 (Dec 1, 2015)

I don't get why you put the jail on 10.0.0
In other words, why do you need a jail network?
Doesn't that isolates the jail from the 192.168.0 subnetwork on your home network, and make 10.0.0 unreachable from both the host and your home network?
So if you ssh into the jail from your host or another computer on your network or the Internet, what IP do you use?
Also, yggdrasil adds 192.168.0.10x on his host network adapter, how does he `ssh` into the jail from his host or another computer or the Internet, with what IP? When I use his technique, I can only get into the host, not any of the other IP's I have set up.


----------



## hukadan (Dec 2, 2015)

Hi,


bobk48 said:


> I don't get why you put the jail on 10.0.0


I just picked that one but I could have chosen some other address range as long as they are non routable addresses. In fact, the first time I read a blog about jails sharing one public IP, the network used was this one. Since then, I use it myself.



> In other words, why do you need a jail network?


The jails are only used by the host. They do not provide services to the home network. I do not see the need for them to appear on my home network. You can chose to do otherwise like yggdrasil did. It is up to you.



> Doesn't that isolates the jail from the 192.168.0 subnetwork on your home network


If by isolate you mean that they are not visible from the home network, that's precisely what I want for the reasons explained just before.



> and make 10.0.0 unreachable from both the host and your home network?


Not from the host. The host can still reach the jails (as explained in the HowTo) but that's true for the home network.



> So if you ssh into the jail from your host (...)what IP do you use?


Like explained in the HowTo, I use the jail(8) IP.



> So if you ssh into the jail from (...) another computer on your network or the Internet, what IP do you use?


In this case, I would use the IP address of the host and implement a rule in the host firewall to redirect the connection to the proper jail(8) (not described in the HowTo since it did not make sense to me in that case).

Your questions are more related to jail networking in general rather than the HowTo itself. If you have a general problem on jail networking you should search in the forum and if you do not find your answer, then post your question in the proper section.


----------



## Maxnix (Feb 11, 2016)

Thank you hukadan. 

I tried your tutorial with the following browsers:

opera -Works fine.
links - Works fine (both graphical and text mode).
qupzilla - Starts, but both address and search bars are not shown correctly. Browser's Interface responds, but I have problems writing and interacting with web pages.
I'm using sysutils/jailme to start them.


----------



## hukadan (Feb 11, 2016)

Hi Maxnix,

Thank you for your feed back. I tried www/qupzilla-qt4 and I saw the problem you described. I have to admit that I did not investigate too much on this since I saw that www/qupzilla-qt5 worked fine. Have you tried that one ?


----------



## Maxnix (Feb 12, 2016)

Just tried it on both the host system and the jailed one. It works, but stressing it a bit causes a core dump due to "illegal hardware instruction" (on both). But it is not a news for me, other QT5-related programs also had this problem on my system some time ago (in fact I use their QT4 version).


----------



## Maxnix (Mar 11, 2016)

For those who use X forwarding with ssh, here a way to speedup the rendering of forwarded programs: http://xmodulo.com/how-to-speed-up-x11-forwarding-in-ssh.html


----------



## yggdrasil (Nov 3, 2016)

Revisiting this nice thread after my recent 11.0 reinstall. I originally followed the jailme route, but decided to switch to the ssh-based one because having to use xhost felt wrong. But I still seem to need xhost, even when using ssh. Without it, I get an "Invalid MIT-MAGIC-COOKIE-1 keyUnable to init server: Broadway display type not supported: browser:10.0 Error: cannot open display: browser:10.0" error. I followed the instructions, and don't remember having to use xhost when I last used X11 forwarding (which admittedly was some while back), so I'm wondering what is going on here.


----------



## chrbr (Nov 3, 2016)

The howtodo says that it needs xauth(1) for the ssh method. It is working for me as described. I am not sure if things have changed with FreeBSD 11.0. I am still using FreeBSD 10.3. I guess you just might have overlooked a small detail somewhere.


----------



## chrbr (Jan 9, 2017)

hukadan said:


> Then, come back to the host and configure de client side of ssh(1). It is quite boring to have to type password each time. So setup a public key authentication.
> % ssh-keygen -t rsa -b 4096 -C "jailuser@example.com" -f ~/.ssh/injail
> % ssh jailuser@10.0.0.1 mkdir .ssh
> % cat ~/.ssh/injail.pub | ssh jailuser@10.0.0.1 'cat >> .ssh/authorized_keys'


I have stumbled over ssh-copy-id(1) in a text book. This could handle the task of line 2 and 3. I wonder if there is at least a single person who knows all the nice tools we have. It is not me . 


chrbr said:


> I am not sure if things have changed with FreeBSD 11.0. I am still using FreeBSD 10.3.


By the way, just to answer my own question, the nice guide works fine on FreeBSD11.0. Merci beaucoup , hukadan.


----------



## hlindh (Mar 23, 2017)

Thanks for the useful guide. Some minor differences encountered (FreeBSD 11), though:

instead of pfctl, I did `service pf start` and `service pflog start`
before doing anything with iocage, I had to `iocage fetch`
to get hostname set correctly, I had to `iocage set host_hostname=myjail myjail`. Otherwise it would just stubbornly default to the jail UUID, which for instance prevented X11 forwarding from working


----------



## michael_hackson (Apr 26, 2018)

Hello! I thank you very much for this awesome guide and I'd like to necro this since I stumbled upon some issues.
I can start Links in textmode but as soon as I do:
`jailme $myjail firefox --new-instance`

I get:

```
JavaScript error: jar:file:///usr/local/lib/firefox/omni.ja!/components/XULStore.js, line 70: Error: Can't find profile directory.
```
Together with a GTK-window stating:
Your Firefox profile cannot be loaded. It may be missing or inaccessible.

I tried copying my ~/.mozilla folder over to a /usr/home directory in my jail but the problem was still there.
Does anyone have any ideas?

I don't need to have Xorg or something like that installed in the jail since it's supposed to use the system's own Xorg, yes?


----------



## chrbr (Apr 27, 2018)

I start www/firefox by `/usr/local/sbin/jailme fox firefox 2>/dev/null &` using x11/dmenu. I have just tried, it works from the command line, too.
EDIT: I checked out that it works when starting from an completely empty home in the jail. And it will use the system's Xorg.


----------



## michael_hackson (Apr 27, 2018)

Hi! Today I noticed that the first Firefox session (of the Xsession), in jail, gave the code:

```
libGL error: failed to open drm device: No such file or directory
```

And through a quick search I got reminded that the devices are related to /etc/devfs.rules so what I did was to put: `add path 'dri*' unhide` in the jail section of /etc/devfs.rules.

Problem solved.


----------



## debguy (Jun 25, 2018)

on linux all that would do no good.  bsd i'm not sure.

changing root (jail) ONLY CHANGES __DEFAULT disk.  the root application running can still mount (drives, memory, etc) just like processes running in the booted /

doing that assumes you'll limit access to a non-root user inside the jail.  but you can do that without a jail: it's called: LOGIN.  (noting you can log in as a new user for each application your run, if you  like to keep apps separated)

jails are very useful if your getting around /lib "dependency issues" though, to provide alternate roots.

freebsd has added some new features but they aren't meant to provide what your thinking.  there isn't documentation on "jail breaks" for no reason.  the fact is "jails" are a misnomer that lead people to believe they are something they are not.  (or rather not yet - there are some who firmly believe chroot(1) should be hacked until it works like a jail %100 

now for freebsd ...  the kernel and libc may not make "every options" changeable on a per user / per process basis, but *might* make "copies" of the feature per / (per root).  so: if you can filter network access using libc (or using kernel options) per user do that.  only provide a whole disk copy of a new libc/login root if your libc / kernel simply don't support the (filter option) your looking for as a regular option (not requiring a whole mini OS disk copy)

what that means is: if your (root user) can set ulimit, if he/she can firewall based on application or based on UID, or of your user can do the equivalent by adding personal firewall rules (added to roots post-op), that's all you need.


----------



## debguy (Jun 25, 2018)

I have a problem with Motivation here.

I've run linux with a firewall, (freebsd a little), redhat debian slackare, other, and sometimes found a glitch had my firewall left open (NOT suggested).

I've never had a system freaked by my web browser or otherwise in 30 years.

So I have problems with a suggestion people should run their desktop inside a jail.  I don't think it will be helpful in general: but it will make their life more complicated surely.

the worst i've had browser wise is

#1  hacks doing irreversible changes causing forced upgrades (ie, website requiring html6 support, broken on html7, that have no alternate html5 page - bad web HACKS)

#2  a debian linux webbrowser that consumed memory until it crashed: firefox-2.0 (they compiled it incorrecly).


----------



## debguy (Jun 25, 2018)

> cute fuzzy wrote: "Just tried it on both the host system and the jailed one. It works, but stressing it a bit causes a core dump due to "illegal hardware instruction" (on both). But it is not a news for me, other QT5-related programs also had this problem on my system some time ago (in fact I use their QT4 version)."

ditto.  i tried qupzilla and it'd been ruined by versional issues in Qt that were unresolvable by lib version/depends.  in linux LFS this is.  it was fresh, i liked seeing what the gz author did - amazing he got as far as he did.  (note: it's a fresh project not intended to be a full web solution)

That Qt is NOT APPLE QT which apple still makes and puts in new products.  That's kde (german/novell/anthem) Qt they copylefted from Apple's free 1.x release to the public to help Win95 users.  it contained NO gui at the time.  the GUI is totally KDE related, not Qt at all


----------



## debguy (Jun 26, 2018)

i hate to add on more here but one more thought.

by watching traffic i'm sure that china (by ip) and others continually polled my PC's ports (portscanned) and tried to upload viruses to my FTP server and APACHE server

lesson here is: follow BSD's security guide essentials (ie, freebsd says to enable a firewall, and if they haven't already done it for you, do it).  if they say "don't visit promiscous web sites" or sites that issue warnings, then don't   if you see bogus messages: just close the browser popup window.  the simple essentials will do fine.  don't open PDF (unless in safe mode) or Word documents you find on the internet unless your running as a un-elevated user and $HOME doesn't contain your valued save files (or you have a backup).

beyond that, i wouldn't worry unless your running servers that due to configuration allow "login", "upload", or something where a virus could be activated.

yes attacks are run but if you follow the essentials that are mentioned - your covered for home use

(if you ever develop on a sensitive server system by then you'll know what to do after reading many docs.  but really: you don't run web browsers on a sensitive system because they're inherently broken ware)


----------



## reddy (Jan 20, 2019)

debguy said:


> (if you ever develop on a sensitive server system by then you'll know what to do after reading many docs.  but really: you don't run web browsers on a sensitive system because they're inherently broken ware)



So basically, in your first message you say that you do not see the point because you never had any problem caused by a browser. Then in your third message you admonish us to never run a browser on a sensitive system. So maybe in the your fifth message you will acknowledge that this how-to was well advised.

As far as the how-to is concerned, one way to improve performance over ssh could be to use a very small key size. I am not sure what's the lowest size allowed by sshd is but use the absolute minimum since this is not even exposed to the outside world.

I personally think that web browsers have become quite reliable sandboxes at this point, we're far from the ActiveX days. What I mistrust is user's judgement with downloaded files (same for email attachments). Therefore, such a jail to be useful should also include LibreOffice, a PDF viewer and an image viewer so that files can be checked in a secured environment before being used by the host.

In a corporate environment I would have an intermediary jail scan the files using an antivirus before copying them to a location mounted by the host. I would also recreate the browser jail every time to start from a fresh system at every session. There could also be a browser in the host restricted to internal and whitelisted websites, while the jail browser would be able browse the whole internet.

I find jails so inspiring, they have already become one of my preferred features of FreeBSD. There seems to be no limit to what can be achieved in terms of having a system exactly matching your needs.

Great how-to!


----------



## itsthosestonesman (Jun 2, 2019)

Wanting to achieve something similar, I read through this thread, but believe I have found a simpler solution, taking some inspiration from debguy's earlier post.  Perhaps my solution is not as secure as running the web browser in a jail, however it may well be 'good enough' and has minimum complexity, requiring neither a jail, nor having to use ssh, nor having to xhost + the display server.

Let's recap what I was trying to do when I found this useful and interesting thread.  I have a single computer running freebsd 11  and X11.  I want to have a 'clean' userid that is used for doing things like online banking.  And I want to have a 'dirty' userid that can be used for general browsing anywhere on the web that might be exposed to picking up website browser exploits.

So I start by making two userids: 'clean' and 'dirty'.  They both get separate home directories, and I set up the permissions so that neither can gain access to each others home directories, eg:-

drwxr-x---  22 clean clean   1536  2 Jun 20:26 clean
drwxr-x---  18 dirty dirty  3072  2 Jun 20:35 dirty

Assuming the machine is set up to run an X display manager (eg xdm), login as 'dirty'.
The X server is hence started as a process owned by 'dirty' userid.
Now dirty can start X clients like a browser without any further issue.
Most of time on the machine is spent logged in as 'dirty', but if I want to do something like read email or do some online banking, I would prefer to login as 'clean' and run a separate instance of the browser.

We can arrange for the 'clean' userid to also be able to run X clients on the same X display.
In an xterm, login as 'clean' by running 

# login clean

Once logged in as 'clean', it will be found that attempts to run a client (eg another xterm) will be
rejected by the display server due to failed authorisation.

We don't need to resort to ssh'ing to localhost or unlocking the X server using xhost +;
there is instead a simpler method using xauth.  The 'dirty' userid already has an xauth security
token to be allowed to run clients on the X server it started.  This token can be exported by 'dirty' userid
and imported by 'clean' userid, which will then allow 'clean' to run clients on the X server owned  by 'dirty'.

To set this up do the following:
1. in an xterm logged in as 'dirty', run
# xauth extract - $DISPLAY > /tmp/myauth
(Note that $DISPLAY should already have been set to :0)

2. In a separate xterm logged in as 'clean', run the command
# xauth merge /tmp/myauth
# export DISPLAY=:0 (I'm running bash)

Delete /tmp/myauth afterwards.

And hey presto, the 'clean' userid is now able to start any X client it desires running on the X server
owned by 'dirty' userid.  So the 'clean' userid can now start a separate instance of the web browser.

Because there are two isloated home directories, each with their own browser cache and config,
there is no collision in the filesystem.  And ps shows that the two web browser instances result in process trees owned by the two different userids, so there should be address space isolation between the two browser instances.  Now 'clean' can go ahead and do his online banking, and 'dirty' can go mess around looking at general websites, in isolation from each other.

The nice thing about this approach is its simplicity; we don't need a jail, we don't need to ssh back in, and we don't need to open the server to other connections using xhost +.  We just use the filesystem permissions and xauth security provided by the X server.  I'm not qualified to say whether an approach like this would be safe from exploits like spectre and meltdown, but it might be good enough to keep a first level of isolation between websites that you want to keep private and general browsing.  Especially if you don't go searching for dodgy warez sites in the 'dirty' browser ;-)

I would be interested in people's thoughts on this approach...


----------



## itsthosestonesman (Jun 2, 2019)

And before anyone spots my obvious mistake...
I should have said: if I login over the X display manager (xdm,wdm etc..) the X server process itself is owned by root, not the 'dirty' id.  Of course if some website code running in the 'dirty' browser manages to get root then all bets are off, but I'm not sure a jail gives any greater protection under those circumstances, as was pointed out earlier...


----------



## itsthosestonesman (Jun 2, 2019)

Minor annoyance number 1 is that if the X server is re-started, the 'clean' userid must re-import the xauth token before it can start clients again.  This can likely be scripted, I'll have a think.

Minor annoyance number 2 is that 'clean' must start his clients manually from a terminal eg by typing 'firefox'.  Running a second X server eg using Xnest or Xephyr may provide a nice way to give 'clean' his own complete desktop in a separate (potentially fullscreen) window.  I'll have a play with that tomorrow and post an update.


----------



## itsthosestonesman (Jun 3, 2019)

I played around a bit more, I found that Xephyr provides a nice way to make a nested X server, within which a second instance of the window manager can be started.  What this essentially provides is two display servers used by two different userids on the system; both are able to run separate instances of the desktop and web brower.  We can then choose to keep all activities that might pick up web exploits in one browser/userid and reserve the second browser/userid for 'safe' websites like online banking.  So I think this approximates the original idea of running the untrusted web browser in a jail or sandbox.  The scope of any exploits run in that untrusted browser are limited to the access rights of the userid who owns its pid.  As long as any exploit loaded from the web into its browser can't get root.

And the ultimate step... if we really, really don't trust this dirty web browser, is to run it on a separate dedicated machine, and present its display surface on the same desktop through the nested x-server over ssh.


----------



## rootbert (Mar 8, 2020)

I am currently running 12.1-RELEASE and tried accomplish the same setup. I encountered the same error as yggdrasil. The solution was: in the jailed environment the /etc/hosts file needs the ip+hostname of the jail so it can resolve itself, and in sshd_config in the jail we need to set "X11UseLocalhost no" - then xforwarding works. However, I'd like to get jailme to work, I get 


```
Unable to init server: Could not connect: Abstract UNIX domain socket addresses not supported on this system
Error: cannot open display: :0.0
```

I am running xdm on ttyv8 and then "exec startxfce4" in ~/.xinitrc so I wonder how I can run "startx -- -listen tcp" (or whatever I have to do to get that functionality)


----------



## rootbert (Mar 8, 2020)

hoorrray, two steps later my system is working as expected, how nice! The jailed applications, mainly browsers, now run via "ssh -Y", via jailme or with xephyr. Two things to remark:
*) on the hardware I added my hostname + IP-address to /etc/hosts
*) /usr/local/etc/X11/xdm/Xservers now contains the (only) the line: ":0 local /usr/local/bin/X -listen tcp :0"


----------



## rootbert (Apr 22, 2020)

did anyone manage to get this setup running with vnet jail + jailme - is it even possible if the jail has its own ip stack?


----------



## john_rambo (Nov 1, 2021)

This method is not working under FreeBSD 13.0-STABLE. For example the PF rules for adding NAT is getting rejected. Please make the necessary changes to your tutorial & make it compatible for FreeBSD 13.0-STABLE.


----------



## rootbert (Nov 2, 2021)

you have probably missed something; it's working here as expected.


----------



## john_rambo (Nov 2, 2021)

rootbert said:


> you have probably missed something; it's working here as expected.


I have created a jail but using Bastille but unfortunately Iam still stuck. Please read this https://forums.freebsd.org/threads/bastille-jail-no-sound-in-firefox.82683/#post-539968


----------



## cmoerz (Jan 8, 2022)

rootbert said:


> Unable to init server: Could not connect: Abstract UNIX domain socket addresses not supported on this system


You'll like need to do `xhost +` on your host; otherwise, you can't properly connect to your X11. Alternatively, you can transfer the magic cookie, I believe.


----------

