# VNET jails+vlan



## sko (Dec 14, 2016)

Hello,

I've started testing with some vnet enabled jails on 11.0-RELEASE and I can't seem to get the network working properly with VLANs.

I want to connect vnet-interfaces for different VLANs to the jails, so the authority over which jail has access to what VLAN is on the hosts side (which also simplifies jail configuration).
The host is also member of one or more of these VLANs, so the host and the jails should be able to connect with one another and of course to other hosts on the same VLAN.

So for e.g. VLAN4 it should look like this:

```
|         | - vnet4:N - jail vnet4
| lagg0 | - | vlan4 | - | bridge4 | - vnet4:N - jail vnet4
                        |         | - vnet4:N - jail vnet4
```

Following the same logic as for bridging a VPNs tun-interfaces to vlans on each end (for interconnecting a subnet for our telephone system over multiple sites), I created the VLAN interface with the physical interface as vlandev, added the vlan to a bridge and assigned the hosts IP to the bridge. The jails' vnet interfaces get added to the bridge by iocage on jail startup.

But with vnet interfaces the forwarding on the bridge seems to work only between vnet and physical interfaces, not for vlan interfaces.

- If I assign the hosts IP to the bridge, communication between host and jails is possible, but no traffic at all will reach out from any bridge member trough the VLAN interface.
- If I assign the hosts IP to the VLAN interface, it can reach out to the VLAN, but no longer connect to the jails.

The only way I found to get the jails to reach the VLAN outside the host, is to assign the physical interface to the bridge, and create a vlan interface within the jail. Communication with the host is still not possible though, except when I use the bridge as the vlandev for the host's vlan interface.

This would resemble the following layout:

```
|         | - vlan4(host)
            |         | - vnet0:N - jail vnet0 - vlan4
| lagg0 | - | bridge0 | - vnet0:N - jail vnet0 - vlan4
            |         | - vnet0:N - jail vnet0 - vlan4
```
Which would undermine my initial goal of keeping the VLAN assignment on the hosts side, which greatly simplifies changes in network layout or migration of jails to other hosts.

I still think I've just missed some option or sysctl knob for the first setup to work properly, but couldn't find any hint in the manpages for bridge, vlan or vnet.
`net.inet.ip.forwarding` is set to 1, bridge filtering is disabled; all net.link.bridge.* options are set to 0.

Anyone running the above mentioned configuration? Am I just m[i|e]ssing something (up) or is this really a restriction/missing feature for bridge/vlan/vnet interfaces?


----------



## HugoMcKinley (Jul 6, 2018)

Hi,
did you find a solution for this problem?
I searched a lot and read many pages, but it seems to me, that this is not solved in FreeBSD-11, or I'm wrong?


----------



## sko (Jul 9, 2018)

I haven't found any solution to get this to work. Additionally at some patchlevel occasional kernel panics when tearing down vnet interfaces were re-introduced, so I had to revert to non-vnet jails anyways.

We've now mostly switched to smartOS zones for everything where virtualized interfaces are needed. All jails are directly attached to the vlan interface of the host to simplify configuration inside and outside of the jail.
I'm still waiting for projectFIFOs vmadm port to become somewhat stable and production-ready, then I'll re-evaluate the need (and actual benefits) of running vnet jails in our environment. So far the combination of smartOS Zones and non-vnet-jails was sufficient and easy to manage, but with vmadm I could manage both platforms through the same interface...


----------

