# FreeBSD Networking Benchmarks



## Phishfry (Dec 15, 2016)

I found a nice site for some FreeBSD networking benchmarks.

https://github.com/ocochard/netbenches
Look for the results directory and scroll all the way to the bottom of the page.
Not a huge offering of platforms but some popular routing solutions.

Here is a nice example of Netgate SG4860 performance shown by each revision for 2015.

HP and Chelsio really pumping some packets.


----------



## LVLouisCyphre (May 30, 2022)

The last two pages are in the 404 zone.  I bought a pair of Chelsio I believe to be T5 ASICs from eBay that I'm testing.  They come up beautifully with the cxgbe driver.

All four interfaces are up and pingable after giving them respective IP addresses in their respective subnet.


```
cxgbe0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
cxgbe1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
cxgbe2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
cxgbe3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
```

Running a CDP/LLDP daemon which I strongly encourage if you use good network plumbing to aide in troubleshooting.



Spoiler: CDP output.





```
(*redacted *) >sh cdp n det
-------------------------
Device ID: (*redacted*)
Entry address(es):
  IP address: (*redacted*)
Platform: Cisco WS-C3550-12T,  Capabilities: Switch IGMP
Interface: GigabitEthernet1/0/52,  Port ID (outgoing port): GigabitEthernet0/12
Holdtime : 137 sec

Version :
Cisco IOS Software, C3550 Software (C3550-IPSERVICESK9-M), Version 12.2(44)SE6, RELEASE SOFTWARE (fc1)
Copyright (c) 1986-2009 by Cisco Systems, Inc.
Compiled Mon 09-Mar-09 20:28 by gereddy

advertisement version: 2
Protocol Hello:  OUI=0x00000C, Protocol ID=0x0112; payload len=27, value=00000000FFFFFFFF010221FF000000000000000D658CFF00FF0000
VTP Management Domain: 'bsdworks'
Native VLAN: 1
Duplex: full
Management address(es):
  IP address: (*redacted*)

-------------------------
Device ID: (*redacted*)
Entry address(es):
  IP address: (*redacted*)
Platform: FreeBSD 13.1-RELEASE amd64,  Capabilities: Host
Interface: GigabitEthernet1/0/4,  Port ID (outgoing port): cxgbe3
Holdtime : 170 sec

Version :
FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC amd64

advertisement version: 2
Duplex: full
Management address(es):

-------------------------
Device ID: (*redacted*)
Entry address(es):
  IP address: (*redacted*)
Platform: FreeBSD 13.1-RELEASE amd64,  Capabilities: Host
Interface: GigabitEthernet1/0/3,  Port ID (outgoing port): cxgbe2
Holdtime : 170 sec

Version :
FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC amd64

advertisement version: 2
Duplex: full
Management address(es):

-------------------------
Device ID: (*redacted*)
Entry address(es):
  IP address: (*redacted*)
Platform: FreeBSD 13.1-RELEASE amd64,  Capabilities: Host
Interface: GigabitEthernet1/0/2,  Port ID (outgoing port): cxgbe1
Holdtime : 169 sec

Version :
FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC amd64

advertisement version: 2
Duplex: full
Management address(es):

-------------------------
Device ID: (*redacted*)
Entry address(es):
  IP address: (*redacted*)
Platform: FreeBSD 13.1-RELEASE amd64,  Capabilities: Host
Interface: GigabitEthernet1/0/1,  Port ID (outgoing port): cxgbe0
Holdtime : 168 sec

Version :
FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC amd64

advertisement version: 2
Duplex: full
Management address(es):
```


----------



## SirDice (May 30, 2022)

LVLouisCyphre said:


> The last two pages are in the 404 zone.


It was posted 6 years ago.


----------



## LVLouisCyphre (May 30, 2022)

SirDice said:


> It was posted 6 years ago.


I know, I can tell time and how ntp works.  I was wondering if there was updated information.


----------



## olivier (May 30, 2022)

Few months old, because ENOTIME.
I should take some time for a 13.0 vs 13.1.

Mellanox:








Chelsio:


----------



## Phishfry (May 30, 2022)

So olivier  you still advise disabling Hyperthreading for maximum performance?



> 16 Cores E5-2697A v4


I disabled Hyperthreading because my router is internet facing and all those related bugs.
Plus I have plenty of real cores like you.

Are you using the Intel CPU microcode update port?
I would really like to see the differences there and how hyperthreading performs.


----------



## olivier (May 31, 2022)

Disabling hyperthreading is no more as useful as it was, mainly since the introduction of machdep.hyperthreading_intr_allowed (Allow interrupts on HTT logical CPUs):

Impact on 16c/32t with 32 NIC queues.
Impact on an old 8 cores 10Gb
And no, I'm not using Intel CPU microcode (but try to have up-to-date EFI/BIOS on my machines).


----------



## LVLouisCyphre (Jun 1, 2022)

That's impressive.  Unfortunately I only have 1GbE here in my small network and only have a pair of sexa-core CPUs via X5675s only giving me a dozen cores. 

That's the nice thing about FreeBSD since the beginning.  You can throw together some wicked packet engines if you select the right hardware.

Are there any benchmarks using Cisco FEC, LACP or PAgP on the various supported multi GbE NICS?  Are there some NICS that are more lagg friendly and are better suited for lagg?  If there's a better discussion thread on using lagg and the selection of multiport NICs for lagg, please link me in the right direction and I'll continue the discussion there.

I plan on running an etherchannel with my Chelsio quad port NICs and using that primarily for moving data.  The pair of Chelsios I purchased do 1/10GbE so I have an upgrade path when used Cisco 10GbE switches start flooding the market at good prices.  The onboard Intel em NICs will be stand by interfaces via carp and administrative access to the servers.


----------



## Sergei_Shablovsky (Jun 1, 2022)

olivier said:


> Disabling hyperthreading is no more as useful as it was, mainly since the introduction of machdep.hyperthreading_intr_allowed (Allow interrupts on HTT logical CPUs)



Dear Olivier!

If I remember correctly, at 2018 You make ~70pages “Tuning FreeBSD for routing and firewalling” PDF presentation on AsiaBSDcon.

Do You have updated version of this presentation, modern CPUs, modern NICs?

And another one question: from which reason You not using Intel microcode patch?

P.S. Thank You for perfect research! Good luck!


----------



## Sergei_Shablovsky (Jun 30, 2022)

For anyone who interested in network testing, there are another great open source tool TRex Realistic Traffic Generator https://trex-tgn.cisco.com/

[UPDATE] Some users note there are some disadvantages of this tool, so better to Googling on this forum or others network-specified...


----------



## bgrant (Jul 2, 2022)

Sergei_Shablovsky said:


> For anyone who interested in network testing, there are another great open source tool TRex Realistic Traffic Generator https://trex-tgn.cisco.com/


If anyone wants to take it on, feel free, but here is what I found.









						Multi-protocol traffic generator (Seagull fork for FreeBSD, TRex ?)
					

Hi FreeBSD Gurus!  Please recommend me a multi-Protocol traffic generator tool like Seagull.  I more interesting in automation and profilings, like Saegull have. BTW Seagull is very powerful tool!  Thank You all!  P.S. I read on official page that “Seagull entirely coded in C++”, so may be...




					forums.freebsd.org


----------

