# lagg interface throughput



## da1 (Mar 11, 2018)

Hello everyone,

Long time no seen, good to be back. Hope y'all are doing just fine.

I've got myself an Hp Proliant DL380 G6 running 11.1-RELEASE (UFS) with 4x bce() NIC's. I've connected all of them to an Hp Procurve 1810 switch and defined a trunk (see first and second attachment) but I only get a maximum of 2Gbit/s throughput with iperf. I've tried different settings but cannot exceed that speed. I have iperf running on 4x different ports (5555, 5556, 5557 and 5558) on the DL380 G6 but it makes no difference if I connect each client to another port. I've also tried running test on 2x iperf ports only. On the DL380 G6, I disabled TSO and LRO, changed the MTU to 9000, changed net.link.lagg.default_use_flowid to "0", same results. All 4x servers have different IP's and are connected to different ports in the switch.

INFO
------

On the FreeBSD side I have this:

```
bce0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
        options=c00bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE>
        ether 1c:c1:de:21:35:a0
        hwaddr 1c:c1:de:21:35:a0
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
bce1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
        options=c00bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE>
        ether 1c:c1:de:21:35:a0
        hwaddr 1c:c1:de:21:35:a2
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
bce2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
        options=c00bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE>
        ether 1c:c1:de:21:35:a0
        hwaddr 1c:c1:de:21:35:a4
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
bce3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
        options=c00bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE>
        ether 1c:c1:de:21:35:a0
        hwaddr 1c:c1:de:21:35:a6
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
        options=c00bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE>
        ether 1c:c1:de:21:35:a0
        inet 192.168.6.180 netmask 0xffffff00 broadcast 192.168.6.255
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: active
        groups: lagg
        laggproto lacp lagghash l2,l3,l4
        laggport: bce0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
        laggport: bce1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
        laggport: bce2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
        laggport: bce3 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
```

The other servers are 1x HP Proliant DL380 G7 and 3x Dell PE 1950 running VmWare (FreeBSD vm) and bare-metal OpenBSD 6.2 respectively. See 3rd attachment for the diagram (missing the 4th OpenBSD 6.2 client)

Where is the limitation?


----------

