# VLAN bridge networking issue



## DaHerm (Jun 20, 2022)

Hi there !

I am new to this board and relatively new to freebsd, so please be kind if i have done something wrong.
I am learning 

I have 2 physical FreeBSD Hosts in my Network. Both of them are running several jails.

Also i have multiple VLANs in my Network.

The Jails are connected to virtual Bridges which are based on VLAN Interfaces.

Also i have a Management Network on VLAN11, on which my physical Hosts are running as well as several Management Jails (Monitoring).

My Problem is, that every jail on every Bridge can connect to every other jail in the same vlan and on every other physical machine outside the host, except for vlan 11. My Monitoring Jail can't connect to physical Hosts on VLAN11.

This is my Network Configuration:


```
### Link aggregation of both physical NICs
cloned_interfaces="lagg0 bridge2 bridge3 bridge6 bridge11 vlan2 vlan3 vlan6 vlan11"
ifconfig_em0="up"
ifconfig_em1="up"
ifconfig_lagg0="laggproto lacp laggport em0 laggport em1 up"

### VLAN Config
ifconfig_vlan11="vlandev lagg0 vlan 11 up"
ifconfig_vlan2="vlandev lagg0 vlan 2 up"
ifconfig_vlan3="vlandev lagg0 vlan 3 up"
ifconfig_vlan6="vlandev lagg0 vlan 6 up"

### Default Gateway
defaultrouter="172.16.11.1"

### Bridge Interfaces
ifconfig_bridge11="inet 172.16.11.4/24 addm vlan11 up"
ifconfig_bridge2="addm vlan2 up"
ifconfig_bridge3="addm vlan3 up"
ifconfig_bridge6="addm vlan6 up"
```

I believe that the Traffic on VLAN11 is not routed outside of the Host, because the Host has an IP-Address on Bridge11, but i don't know how to solve it.

What have i done wrong ?


----------



## SirDice (Jun 20, 2022)

DaHerm said:


> My Problem is, that every jail on every Bridge can connect to every other jail in the same vlan and on every other physical machine outside the host, except for vlan 11. My Monitoring Jail can't connect to physical Hosts on VLAN11.


Is your monitoring (connected to bridge11, did I understand that correctly?) able to connect to any of the other jails on the other bridges? But nothing 'outside' of the machine? Is VLAN 11 passed on the switch ports? VLAN 2, 3 and 6 obviously are, because you can connect to machines outside of the host.


----------



## DaHerm (Jun 21, 2022)

SirDice said:


> Is your monitoring (connected to bridge11, did I understand that correctly?) able to connect to any of the other jails on the other bridges? But nothing 'outside' of the machine? Is VLAN 11 passed on the switch ports? VLAN 2, 3 and 6 obviously are, because you can connect to machines outside of the host.


Hi !

Yes thats correct, but only because it is allowed from my dedicated firewall to connect to other VLANs.
Only Jails on VLAN11 / bridge11 cant connect to Physical Hosts in VLAN11 outside of the host, but it can connect to other VLANs outside of the Host.
VLAN11 seems not to be routed outside.


----------



## SirDice (Jun 21, 2022)

DaHerm said:


> Only Jails on VLAN11 / bridge11 cant connect to Physical Hosts in VLAN11 outside of the host, but it can connect to other VLANs outside of the Host.
> VLAN11 seems not to be routed outside.


It is being routed, at least to some extend. Or else you wouldn't be able to connect to the hosts on the other VLANs either. So routing of forward and the return traffic seems to work here.

If you do a tcpdump(1) on the bridge11, can you see the ARP requests for the 'external' hosts on VLAN 11 from the monitoring host?


----------



## DaHerm (Jun 21, 2022)

If i do `tcpdump -i bridge11 arp` on my first host, there is no output.
If i do that from the Monitoring Jail it says: `tcpdump: (there are no BPF devices)`

But i have to correct something.
My Monitoring Jail is sitting on Bridge2 with VLAN2 instead of Bridge11 wih VLAN11.
It's a bit complicated network configuration for me, please excuse me for overseeing it.

Edit:
I think it is something like this:
Host: "Oh, i know that this Network is on my Bridge11 i dont have to look outside"


----------



## DaHerm (Jul 11, 2022)

Hi !
I solved my Problem.
It was caused by asymmetric Routing.
Host 1 did know Host 2 in his Routing Table and routed directly over the Switch without going over the Firewall. 
Host 2 did not know Host 1 and routed over the Firewall, which dropped the Packets.

I solved my Problem with providing vnet on my jails via ansible, so that every Jail has its own Network Stack and Routing Table.


----------

