# 66.6% of speed. Trial+error & research has yielded nothing.



## Anywhere (Apr 21, 2014)

Update: as per suggestion from freenode/#freebsd I tried running pfsense. The thought was that it´s probably a driver issue. I get the same speed and I guess Im going for openbsd then. Shame though. Still odd that I would get the same low exact same speed with two different kinds of interfaces though. 

So I recently got 100Mbit fiber at home. For someone like me who has poor health this is a godsend upgrade since its rather cheap at around 12 Euros/month with the possibility to upgrade to 1Gbit for 22 Euros per month. Celebrating this I decided to upgrade my longtime Tomato router which has functioned superbly and decided to get the Asrock C2550D4I sporting two i210 intel NICs + another intel NIC  CT 9301CT. 

The install worked beautifully, in the 10 years or so since I used this last it´s improved a lot and everything went like a breeze. However when I try to FTP things from ftp.sunet.se (1mS no packet loss) I only get ~6.66MiB/S instead of the 10 I get from Windows. I even installed OpenBSD on the same machine and also get 10 there too. So, since I usually do my homework fairly well I checked out https://wiki.freebsd.org/NetworkPerformanceTuning and https://calomel.org/freebsd_network_tuning.html. The latter seems about the same as the first with a more thorough explanation and propositions at which value to set things. This gains me slight increase landing me at  7.5MiB/s. The FTP program takes around 18% CPU on one of the cores. I have also tried ncftp (as opposed to the default client). I also tried all 3 NICs. 

`Iperf` localhost:


```
iperf -c localhost
------------------------------------------------------------
Client connecting to localhost, TCP port 5001
TCP window size: 47.8 KByte (default)
------------------------------------------------------------
[  3] local 127.0.0.1 port 56147 connected with 127.0.0.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  13.1 GBytes  11.2 Gbits/sec
```

If I let `iperf` run for a few hours I get half that BTW, works fine after restarting the server though. 

/boot/loader.conf

```
hw.igb.max_interrupt_rate="32000"
net.inet.tcp.tcbhashsize="65536"
kern.ipc.nmbjumbop="246339"
```
/etc/rc.conf

```
ifconfig_em0="10.0.0.3 netmask 255.255.255.0"
defaultrouter="10.0.0.1"
```
/etc/sysctl.conf

```
dev.igb.0.fc=0
dev.igb.1.fc=0
dev.em0.0.fc=0
kern.ipc.maxsockets=25600
kern.maxfiles=500
net.inet.ip.redirect=0
net.bpf.optimize_writers=1
kern.random.sys.harvest.ethernet=0
kern.random.sys.harvest.point_to_point=0
kern.random.sys.harvest.interrupt=0
net.inet.raw.maxdgram=9216
net.inet.raw.recvspace=9216
```

Edit: here´s my ifconfig em0.

```
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
        ether 68:05:ca:23:8a:5e
        inet 10.0.0.3 netmask 0xffffff00 broadcast 10.0.0.255
        inet6 fe80::6a05:caff:fe23:8a5e%em0 prefixlen 64 scopeid 0x1
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
```

Output from vmstat -i:

```
interrupt                          total       rate
irq23: ehci0                        1668         19
cpu0:timer                         11415        134
irq256: em0:rx 0                  109311       1286
irq257: em0:tx 0                   64067        753
irq258: em0:link                       2          0
irq270: ahci1                       3857         45
irq272: ahci3                        168          1
cpu1:timer                         30727        361
cpu3:timer                         24518        288
cpu2:timer                         11448        134
Total                             257181       3025
```
I really hope we can get this working I guess I could use another OS but I would -really- like to use FreeBSD. The other BSD´s don't attract me much and Linux.. well no thanks.


----------



## Anywhere (Apr 21, 2014)

*Re: 66.6% of speed. Trial+error & research has yielded nothi*

Wow this is frustrating. I´m currently running OpenBSD on that machine, but it just feels so poor compared to FreeBSD. Please feel free to ask questions or if  you have any hints or thoughts that might not be totally relevant.. that's welcome too.


----------



## SirDice (Apr 22, 2014)

*Re: 66.6% of speed. Trial+error & research has yielded nothi*

I would suggest removing all those 'optimizations' from /etc/sysctl.conf. FreeBSD does a very good job of determining a lot of those values automatically. You usually only have to tweak settings if you're running into specific situations. 

How many cards are in the machine? I see some optimizations for igb(4) but you only mentioned an em(4) interface?


----------



## junovitch@ (Apr 23, 2014)

*Re: 66.6% of speed. Trial+error & research has yielded nothi*

Seconded about the above; premature tuning is only going to make things worse down the line.  For example, this line: 

```
kern.maxfiles=500
```
This is a tiny fraction of the default and running more than a few busy processes will probably hit this limit.

It's not entirely clear what you are doing to test.  For one, `iperf -c localhost` tells you very little since you aren't actually passing traffic through the device.  It would make sense to run that test as client A -> router - > client B or at least client -> router. 

Now you also said you bought this to replace a router but then speak of using Windows on it for testing as well as OpenBSD.  Is this all still the same device?  Do you just have this plugged directly into your WAN connection?  Are you sure that normal deviations in speed aren't a factor?  As in you didn't test FreeBSD in the evening during prime video streaming hours and the others at an off hour.


----------



## Anywhere (Apr 23, 2014)

*Re: 66.6% of speed. Trial+error & research has yielded nothi*

Thanks a lot for replying guys. I only edited the sysctl.conf and rc.conf after having run into a wall of low performance. 

It seems however that both `ftp` and `ncftp`, the two primary programs I used to test my connection outside my LAN with, are somehow limited to 6.66~ Mbit/s. When using several instances of either one or my dear old `lftp` it shows the expected bandwidth. Boy am I happy I really wanted to run FreeBSD on this and it seems I got my wish.


----------



## trh411 (Apr 23, 2014)

*Re: 66.6% of speed. Trial+error & research has yielded nothi*



			
				Anywhere said:
			
		

> Thanks a lot for replying guys. I only edited the sysctl.conf and rc.conf after having run into a wall of low performance.
> 
> It seems however that both `ftp` and `ncftp`, the two primary programs I used to test my connection outside my LAN with, are somehow limited to 6.66~ Mbit/s. When using several instances of either one or my dear old `lftp` it shows the expected bandwidth. Boy am I happy I really wanted to run FreeBSD on this and it seems I got my wish.


Were you always testing to the same FTP site? If so, perhaps the site throttles all connections to preserve bandwidth? Just a thought.


----------



## Anywhere (Apr 24, 2014)

*Re: 66.6% of speed. Trial+error & research has yielded nothi*

Yes I was always running towards ftp.sunet.se. They don´t throttle afaik. I have been using it for 15 years or so, and its the site I pinged and checked pl against. Its the most stable connection I know of next to funet.fi also a university site.


----------

