# Example jail configuration causes host to lose network



## Wout (Oct 14, 2014)

Using FreeBSD 10-RELEASE, when following the jail(8) jail.conf example, it causes the host to lose network when starting the jail.


```
testjail {
   path	= /tmp/jail/testjail;
   mount.devfs;
   host.hostname = testhostname;
   ip4.addr = 192.0.2.100;
   interface = ed0;
   exec.start =	"/bin/sh /etc/rc";
   exec.stop = "/bin/sh	/etc/rc.shutdown";
}
```

The host is using one static IP address on the same interface as is passed to the jail.

After removing the _interface_ option, everything works fine. It does not seem to be required.
Is this expected behaviour? If so, shouldn't this be removed from the example to avoid mistakes?

First time I try out the jail.conf method, luckily on a local machine P


----------



## wblock@ (Oct 14, 2014)

Does the host actually have an ed0 interface?


----------



## Wout (Oct 14, 2014)

No, but the above example is just a copy from jail(8). My version has _path_, _host.hostname_, _ip4.addr_ and _interface_ options changed.

To be complete, my jail.conf contains:

```
testjail {
	path = /usr/jails/test;
	mount.devfs;
	host.hostname = testhostname;
	ip4.addr = 10.0.2.15;
	interface = em0;
	exec.start = "/bin/sh /etc/rc";
	exec.stop = "/bin/sh /etc/rc.shutdown";
}
```

And my rc.conf contains:

```
ifconfig_em0="inet 10.0.2.15 netmask 255.255.255.0"
defaultrouter="10.0.2.2"
jail_enable="YES"
```


----------



## wblock@ (Oct 15, 2014)

So you mean that merely setting the interface in jail.conf causes the host's interface to go down?


----------



## SirDice (Oct 15, 2014)

Not surprisingly as the jail and the host want the same IP address. You have to set a _different_ IP address on the jail. Think of the jail as a separate machine, you can't have two machines on the network with the same IP address.


----------



## Wout (Oct 15, 2014)

@wblock@: Yes, it seems so. Without the interface option, everything works as expected. Please, try it, but not on a remote machine 

@SirDice: I do not believe that to be true. A jail and host may share the same IP address. It is even required as the host has to have this IP assigned to a interface.


----------



## usdmatt (Oct 15, 2014)

> @SirDice: I do not believe that to be true. A jail and host may share the same IP address. It is even required as the host has to have this IP assigned to a interface.



You're wrong on the "required" front. The host and jail ideally need different IP addresses. There is no problem with assigning multiple IP addresses to one interface, which is what will happen. The interface will accept traffic to both addresses. Packets going to the IP assigned to the host will be delivered to the host, and packets received by the interface for the jail's IP will be passed to the jail.

Edit: Clarification needed - It does appear possible to run a jail on the same IP as the host, but I suspect you have to be very careful about the services running in the jail to make sure none overlap services running on the host. It makes sense to use different IP addresses in most cases.

Edit 2: You also need to make sure that any network services running on the host are restricted to just the host's IP. Looking on my machine for example it looks like ssh defaults to accepting connections on all addresses (*.22) which would need to be changed to only accept on the host's IP. Otherwise running ssh on both host and jail would cause problems.


----------



## SirDice (Oct 15, 2014)

Wout said:
			
		

> A jail and host may share the same IP address.


Multiple jails can share the same IP address, the host cannot.



> It is even required as the host has to have this IP assigned to a interface.


A jail can have zero or more IP addresses. It's not required to set an IP address.


----------



## Wout (Oct 15, 2014)

I agree it is uncommon, and probably bad practice. Normally I also assign IP aliases to my jails. But it surely is possible. If the host and jail would use the same port on a shared IP address, the host will have preference. I do not find anything in jail(8), nor in the Handbook, that says it is not allowed to use the host's primary IP address.

Just found out that when I add a IP alias to the host, there is no problem using the _interface_ option in jail.conf. But it does not matter which IP is assigned to the jail.

So it seems when using the interface option in jail.conf, it is *required* to have at least one alias assigned to that interface, or the interface goes down on jail start.

Edit: Typo


----------



## Wout (Oct 15, 2014)

SirDice said:
			
		

> Multiple jails can share the same IP address, the host cannot.



I am running a system doing the very same thing right now. Even without talking about the "primary" IP address, when a jail uses an IP address, it must the assigned to the host as well (as alias or primary). So they always share IP addresses. To use this IP for services on the host, is probably not a good idea.



			
				SirDice said:
			
		

> A jail can have zero or more IP addresses. It's not required to set an IP address.



I agree on this one.


----------



## SirDice (Oct 15, 2014)

Wout said:
			
		

> SirDice said:
> 
> 
> 
> ...


Having thought it through a bit more, you are absolutely right.


----------



## Wout (Oct 15, 2014)

SirDice said:
			
		

> Having thought it through a bit more, you are absolutely right.



Thanks. You do have a valid point. When using aliases, the interface problem does not occur.

I do not see any reason why the interface option is in the example, and as it causes the interface to go down when not having aliases, maybe it is best to have it removed from the jail(8) example?

Should this be reported to the jail maintainers? I believe this to be pretty dangerous §jr

Edit: Trying to follow those proper formatti rules.


----------



## kpa (Oct 15, 2014)

The example should have a clarification that the address used should be an interface alias if shared with the real host. Many parts of the handbook are unfortunately written in style and with the expectations that the reader knows his/her TCP/IP, not a good assumption in a handbook meant for beginners.


----------



## SirDice (Oct 15, 2014)

Wout said:
			
		

> Should this be reported to the jail maintainers? I believe this to be pretty dangerous §jr


It won't hurt to report it. Looking at the bug reports I think you should file it under "Documentation". I see a lot more requests for changes in manual pages there.


----------



## nakal (Oct 15, 2014)

An interface alias will be automatically generated by the jail initialization routine. A jail start with an assigned IP will setup the interface new and cause all connections dropped on this IP. This is a well-known behavior. Not sure if it is a bug, because jails are contained within an assigned IP for a reason. Assigning the IP of the host will probably cause much more problems than just dropped connections. If this works at all, it will also be perhaps bad for security (haven't thought about this, because I don't run jails like this).

But I have the feeling that the OP has got a different problem, which he tries to solve by re-using an IP. (S)he hasn't told us what he is trying to do.


----------



## kpa (Oct 15, 2014)

nakal said:
			
		

> An interface alias will be automatically generated by the jail initialization routine.



There are actually two syntaxes for specifying the address. This form will not create the interface alias automatically:


```
ip4.addr = 192.0.2.100;
```

This one will create the alias automatically on jail start up:


```
ip4.addr = ed0|192.0.2.100;
```


----------



## Wout (Oct 15, 2014)

@nakal: No problem here, I was just trying out the jail.conf way of starting jails. I was doing this on a local machine, which does not have any aliases.

So if I understand correctly, when using the interface option, you do not need to have aliases assigned to the host before starting the jail. As they will be created by the jail command.
If you do not set the interface option, you need to create those aliases yourself.

This is all good, but the strange part is when you assign the primary address to the jail, the interface goes down on jail start *except* when you have assigned a random alias to the interface before the jail command runs. In this case, I can use the primary address on the jail.

Bad practice or not, the interface option is a little dangerous.


----------



## wblock@ (Oct 15, 2014)

Trying to address several questions:

Setting interface and address separately behaves differently than using the "interface|address" form.  Right?  The entries on ip4.address and ip4.saddrsel imply there is some automatic behavior going on that might complicate this.

The Handbook tries to walk the line of not talking down to the reader while still being basic enough to cover most questions.  It's not easy, and much of the time some knowledge has to be assumed.  Because jails are not truly separate machines, networking is a little trickier there anyway.  And if we repeat the warning about not using duplicate IP addresses everywhere it might apply, it will be repeated all throughout the Handbook.

Yes, corrections to man pages should be filed as documentation bugs.


----------



## Wout (Oct 15, 2014)

wblock@ said:
			
		

> Setting interface and address separately behaves differently than using the "interface|address" form.  Right?  The entries on ip4.address and ip4.saddrsel imply there is some automatic behavior going on that might complicate this.



No, they behave the same. Adding the interface by either interface option or interface|address syntax in ip4.address results in automatic creating and destroying of IP aliases.

To problem lies when you (either by mistake, or on purpose) assign the host's primary IP address to the jail. Than the automatic behavior results in the interface going down. While this is certainly not recommended, it is a valid thing to do without using "automatic IP aliasing" by jail.



			
				wblock@ said:
			
		

> The Handbook tries to walk the line of not talking down to the reader while still being basic enough to cover most questions.  It's not easy, and much of the time some knowledge has to be assumed.  Because jails are not truly separate machines, networking is a little trickier there anyway.  And if we repeat the warning about not using duplicate IP addresses everywhere it might apply, it will be repeated all throughout the Handbook.



The Handbook does not suggest using jail.conf, nor the interface option, only jail(8) does. But it might do so in the future.



			
				wblock@ said:
			
		

> Yes, corrections to man pages should be filed as documentation bugs.



I shall report it, and refer to this topic.


----------

