# Problems understandig routing between VMs and Jails



## capra (Aug 25, 2021)

Hi all, 

I have a single FreeBSD12 host, which I'm using to run a bunch of (iocage-managed) Jails, as well as some (vm-bhyve-managed) virtual machines. I'm also using vxlan for some network-separation, where my Server and my (OPNSense) router are the endpoints. 
(see picture network.png for schematic topology) 
It works fine with all traffic that leaves the physical server (for Jails as well as VMs). 
Now here comes the problem:
Traffic between Jails and VMs seems to be asymmetrical, seen in traceroute, as well as observable via tcpdump


```
traceroute from Jail (10.44.0.20) to VM (192.168.1.142)

traceroute to 192.168.1.142 (192.168.1.142), 64 hops max, 40 byte packets
 1  poudriere.capra.local (192.168.1.142)  1.348 ms  0.333 ms  0.256 ms

and vice versa

root@poudriere:~ # traceroute 10.44.0.20
traceroute to 10.44.0.20 (10.44.0.20), 64 hops max, 40 byte packets
 1  fw01 (192.168.1.1)  0.391 ms  0.404 ms  0.206 ms
 2  10.44.0.20 (10.44.0.20)  0.251 ms  0.232 ms  0.222 ms
```

I guess this is due to the fact, that I'm not using VNET Jails, so the Host is sending the traffic from the alias-ip, which is assigned to the jail directly into the virtual switch. 
The other way arround passes the physical router, I tried to create a set of pictures to visualize the path my traffic takes. 

Now my question is: "physical Server" does not have the option "gateway_enabled" set in rc.conf, is there a way to fix that without implementing a whole set if firewall rules in my server?


----------

