# FreeBSD+uCARP not working on KVM Hypervisor



## fraenki (Oct 17, 2013)

Hi,

I'm running two FreeBSD 9.1 VMs on a KVM Hypervisor. While this basically works quite well, I can't get uCARP to work. First I thought it must be a multicast issue (because I consider multicast to be black magic), but a quick test shows (I assume) that multicast is working:

On VM 1 I started a multicast server:


```
vm1# iperf -s -u -B 224.0.67.67 -i 1
------------------------------------------------------------
Server listening on UDP port 5001
Binding to local address 224.0.67.67
Joining multicast group  224.0.67.67
Receiving 1470 byte datagrams
UDP buffer size: 41.1 KByte (default)
------------------------------------------------------------
[  3] local 224.0.67.67 port 5001 connected with 10.10.10.2 port 65530
[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams
[  3]  0.0- 1.0 sec   126 KBytes  1.03 Mbits/sec   0.165 ms    0/   88 (0%)
[  3]  1.0- 2.0 sec   126 KBytes  1.03 Mbits/sec   0.163 ms    0/   88 (0%)
[  3]  2.0- 3.0 sec   128 KBytes  1.05 Mbits/sec   0.162 ms    0/   89 (0%)
[  3]  3.0- 4.0 sec   129 KBytes  1.06 Mbits/sec   0.122 ms    0/   90 (0%)
[  3]  4.0- 5.0 sec   128 KBytes  1.05 Mbits/sec   0.174 ms    0/   89 (0%)
[  3]  0.0- 5.0 sec   640 KBytes  1.05 Mbits/sec   0.176 ms    0/  445 (0%)
[  3]  0.0- 5.0 sec  1 datagrams received out-of-order
```

And on VM 2 I started a multicast client:


```
vm2# iperf -c 224.0.67.67 -u --ttl 5 -t 5
------------------------------------------------------------
Client connecting to 224.0.67.67, UDP port 5001
Sending 1470 byte datagrams
Setting multicast TTL to 5
UDP buffer size: 9.00 KByte (default)
------------------------------------------------------------
[  3] local 10.10.10.2 port 65530 connected with 224.0.67.67 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 5.0 sec   640 KBytes  1.05 Mbits/sec
[  3] Sent 446 datagrams
```

On my KVM hypervisor I can see these packets going through:


```
kvm# tcpdump -i vnet13 -s 1500 -n -X host 224.0.67.67
11:14:58.113280 IP 10.10.10.2.10745 > 224.0.67.67.commplex-link: UDP, length 1470
11:14:58.113457 IP 10.10.10.2.10745 > 224.0.67.67.commplex-link: UDP, length 1470
11:14:58.143342 IP 10.10.10.2.10745 > 224.0.67.67.commplex-link: UDP, length 1470
```

As you can see from the iperf output, 445 out of 446 datagrams could be received by VM 1. That's why I assume multicast is basically working.

The problem is that uCARP/VRRP is not working :-( I can see the VRRP advertisements on my KVM hypervisor as well:


```
kvm# tcpdump -i vnet13 -s 1500 -n -X  |grep -i vrrp
tcpdump: WARNING: vnet13: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vnet13, link-type EN10MB (Ethernet), capture size 1500 bytes
11:17:46.386437 IP 10.10.10.2 > 224.0.0.18: VRRPv2, Advertisement, vrid 2, prio 0, authtype none, intvl 1s, length 36
11:17:47.353269 IP 10.10.10.2 > 224.0.0.18: VRRPv2, Advertisement, vrid 2, prio 0, authtype none, intvl 1s, length 36
11:17:48.363266 IP 10.10.10.2 > 224.0.0.18: VRRPv2, Advertisement, vrid 2, prio 0, authtype none, intvl 1s, length 36
```

... but they never reach the interface of the other FreeBSD VM. I don't think it's a FreeBSD issue, but maybe someone has seen this before and may give some advice how to tweak KVM or possibly iptables to make (u)CARP work. I've already tried pfSense (with native CARP) and it's the same problem.

Thanks
- Frank


----------



## fraenki (Oct 17, 2013)

OK, this was easy to solve. My KVM Hypervisor is oVirt... I wasn't aware that they introduced the network-filters feature which prevents guests from spoofing other mac-addresses than these which were assigned by the oVirt engine.

I just needed to disable this anti-spoofing feature. Now uCARP and CARP is working as expected  

More information is available from this post on the ovirt-users mailinglist.


----------

