# bhyve, if_bridge and real IP for VM



## SemFLY (Apr 6, 2017)

Server on Hosting VDS/VPS
HOST SYSTEM FreeBSD 11 Release (last update - use subversion)
Provider give IP addresses 200.100.200.100/29 (example adresses)
Gateway 200.100.200.97
DNS 8.8.8.8




*Host SYSTEM*


```
# uname -simnr
FreeBSD srv-vds 11.0-RELEASE-p8 amd64 SRV-VDS

Use bhyve
[CODE]# kldload vmm
```

# ifconfig
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=98<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
        ether 00:50:57:7d:f7:f2
        inet 200.100.200.98 netmask 0xfffffff8 broadcast 200.100.200.255
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active

# ifconfig tap0 create
# ifconfig bridge0 create
# ifconfig bridge0 addm em0 addm tap0 up
# ifconfig
tap0: flags=8902<BROADCAST,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80000<LINKSTATE>
        ether 00:bd:ee:6e:fa:00
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: no carrier
        groups: tap
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:ba:05:cb:3e:00
        nd6 options=9<PERFORMNUD,IFDISABLED>
        groups: bridge
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 1 priority 128 path cost 20000
        member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 6 priority 128 path cost 2000000

# sysctl -a | grep net.link.bridge
net.link.bridge.ipfw: 0
net.link.bridge.allow_llz_overlap: 0
net.link.bridge.inherit_mac: 0
net.link.bridge.log_stp: 0
net.link.bridge.pfil_local_phys: 0
net.link.bridge.pfil_member: 0
net.link.bridge.ipfw_arp: 0
net.link.bridge.pfil_bridge: 0
net.link.bridge.pfil_onlyip: 0[/CODE]

When guest boot and open shell
On the HOST enter commands

```
# ifconfig tap0 up
# ifconfig bridge0 up
```


Guest FreeBSD 11.0

```
# cat bhyve_freebsd
#!/bin/sh

bhyve -c 1 -s 7,fbuf,tcp=200.100.200.98:5920,wait \
-s 0,hostbridge \
-s 3,ahci-hd,/vmdata/virtual.hdd \
-s 4,ahci-cd,/vmdata/FreeBSD-11.0-RELEASE-amd64-disc1.iso \
-s 10,virtio-net,tap0 \
-s 31,lpc -l bootrom,/vmdata/BHYVE_UEFI.fd \
-m 2G -H -w fbsd
```


Guest boot and open shell


```
# ifconfig vtnet 200.100.200.101 255.255.255.248
# ifconfig
vtnet0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80028<VLAN_MTU,JUMBO_MTU,LINKSTATE>
        ether 00:a0:98:a1:89:eb
        inet 200.100.200.101 netmask 0xfffffff8 broadcast 200.100.200.255
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet 10Gbase-T <full-duplex>
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo
```

Yes, gateway for VM will be host and use this address (when try use gateway 200.100.200.97 no ping too)

```
# route add default 200.100.200.98
```

Guest system

```
# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
^C
--- 8.8.8.8 ping statistics ---
33 packets transmitted, 0 packets received, 100.0% packet loss
```

no ping.....


Maybe I'm missing something or Hosting VDS/VPS must turn on some options ?
Help please


----------



## aragats (Apr 6, 2017)

SemFLY said:


> Yes, gateway for VM will be host and use this address (when try use gateway 200.100.200.97 no ping too)


If you're using a bridge, the gateway is the same as for the host.
If you want to use your host as a gateway, you must use NAT instead of a bridge.
In a case when your host network interface is wireless that's the only way (well, almost ;-) ).


----------



## usdmatt (Apr 6, 2017)

Is there a specific reason for using the host as the gateway? Traffic will go out via VM -> Host -> Router but will come back Router -> VM ("physically" through the host of course). Seems an odd setup.

It's entirely possible that the hosting provider have restricted this is some way, although they usually just disable VT-x in the host. (Edited: thought host was a real server but it's a vm?)

You don't have any firewall rules that may affect the traffic (or you've already tested with the firewall off) right?
Do you have connectivity between the guest and host?


----------



## SemFLY (Apr 6, 2017)

aragats said:


> If you're using a bridge, the gateway is the same as for the host.
> If you want to use your host as a gateway, you must use NAT instead of a bridge.
> In a case when your host network interface is wireless that's the only way (well, almost ;-) ).


Yes, i try gateway is the same as for the host, but does't work ))) and i will try host for gateway ))



usdmatt said:


> Is there a specific reason for using the host as the gateway? Traffic will go out via VM -> Host -> Router but will come back Router -> VM ("physically" through the host of course). Seems an odd setup.
> 
> It's entirely possible that the hosting provider have restricted this is some way, although they usually just disable VT-x in the host. (Edited: thought host was a real server but it's a vm?)
> 
> ...



Yes, on VDS/VPS usually turn off VT-x.
But i send email to support and for me turn on VT-x technology on my VDS.

Firewall is off (ipwf and pf - no in kernel)
From host ping it's ok

From guest on host ping - yes
But in internet - no


----------



## aragats (Apr 6, 2017)

SemFLY said:


> vtnet0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
> options=80028<VLAN_MTU,JUMBO_MTU,LINKSTATE>
> ether 00:a0:98:a1:89:eb inet 200.100.200.101 netmask 0xfffffff8 broadcast 200.100.200.255


Does it mean that the guest receives its IP address from a DHCP server?
If yes, that means the bridge is working correctly and you should be able to access the gateway.


----------



## SemFLY (Apr 6, 2017)

aragats said:


> Does it mean that the guest receives its IP address from a DHCP server?
> If yes, that means the bridge is working correctly and you should be able to access the gateway.


No... on VDS doesn't use DHCP server, only manual set IP

*Host machine - tcpdump*
Look at this, ping from guest to internet
no answer to VM guest

```
# tcpdump -i em0 -v -p icmp
tcpdump: listening on em0, link-type EN10MB (Ethernet), capture size 262144 bytes
19:52:26.491722 IP (tos 0x0, ttl 64, id 36008, offset 0, flags [none], proto ICMP (1), length 84)
    200.100.200.101 > google-public-dns-a.google.com: ICMP echo request, id 20738, seq 0, length 64
19:52:27.555201 IP (tos 0x0, ttl 64, id 36009, offset 0, flags [none], proto ICMP (1), length 84)
    200.100.200.101 > google-public-dns-a.google.com: ICMP echo request, id 20738, seq 1, length 64
19:52:28.618975 IP (tos 0x0, ttl 64, id 36010, offset 0, flags [none], proto ICMP (1), length 84)
    200.100.200.101 > google-public-dns-a.google.com: ICMP echo request, id 20738, seq 2, length 64
19:52:29.680158 IP (tos 0x0, ttl 64, id 36011, offset 0, flags [none], proto ICMP (1), length 84)
```


----------



## SemFLY (Apr 6, 2017)

what do you think ?
may be some filtering on hosting provider(router hardware - example mac control) VDS or no, because this only my VDS area ?


----------

