# Single thread download over FreeBSD router - slow speed.



## Sobaka (Sep 15, 2022)

Hello everyone.
On the router (gateway) work FreeBSD with 10G network card.
And the problem is in single thread download over router, but in multi download speed increases so much.
Typicaly scheem:


DOWNLOAD FROMDOWNLOAD TOSPEED, Mbyte/sTHREAD (wget ftp://......)It's OK?ClientRouter90+1OkRouterClient90+1OkServerRouter180+1OkRouterServer180+1OkClientServer10..201not OK, slowServerClient10..201not OK, slowClientServer80+manyOkServerClient80+manyOk




> Client -> Router: 90Mbyte/s (OK)
> Router->Client  : 90Mbyte/s (OK)
> Server -> Router: 180Mbyte/s (OK)
> Router-> Server: 180Mbyte/s (OK)
> ...


And some info about systems:

```
2X CPU: Intel(R) Xeon(R) CPU E5-2689 0 @ 2.60GHz (2593.56-MHz K8-class CPU)
```


```
any of this sysctl does not matter:
net.link.ifqmaxlen=8192
net.isr.defaultqlimi=4096
net.isr.numthreads=4
hw.sfxge.tx_dpl_get_non_tcp_max=10240
net.graph.recvspace=128000
net.graph.maxdgram=64000
net.inet.ip.fw.one_pass=0
net.inet.icmp.icmplim=1500
kern.ipc.nmbclusters=65536

net.inet.flowtable.enable=0
sysctl net.bpf.maxinsns=1024
sysctl net.bpf.maxbufsize=1024000
sysctl net.bpf.bufsize=8192


net.inet.tcp.rfc1323=1
kern.ipc.maxsockbuf=16777216
net.inet.tcp.sendspace=1048576
net.inet.ip.intr_queue_maxlen=1000
net.inet.ip.dummynet.hash_size=256
kern.ipc.maxsockbuf=900000000
net.inet.tcp.sendspace=300000000
net.inet.tcp.recvspace=300000000
net.inet.udp.recvspace=300000000
net.inet.tcp.recvspace=1048576
```

Some IPFW rules, but from/to CLIENT and SERVER rules are first
Any CPU core not load over 50%.
Always Free memory.
Average traffic 200Kpps.
Network card driver sfxge
BGP working with 150 000 entries in routing table
Version: FreeBSD  12.2-RELEASE r366954 GENERIC  amd64


Where starting find problem?


----------



## jbo (Sep 15, 2022)

Hello & Welcome to this FreeBSD community!

In general, with these sorts of things you'd be best off providing a lot more details - especially how you acquired those numbers (eg. how you made those measurements).

benchmarks/iperf3 is also a good utility for these sorts of endeavors.


----------



## SirDice (Sep 15, 2022)

Sobaka said:


> Version: FreeBSD 12.2-RELEASE r366954 GENERIC amd64


Besides not having _any_ security patches installed, keep in mind that 12.2-RELEASE is EoL since March 2022.


----------



## Sobaka (Sep 15, 2022)

jbodenmann said:


> Hello & Welcome to this FreeBSD community!
> 
> In general, with these sorts of things you'd be best of providing a lot more details - especially how you acquired those numbers (eg. how you made those measurements).
> 
> benchmarks/iperf3 is also a good utility for these sorts of endeavors.


Its so simply:  wget [in router I also use --bind-address] ftp://.....bigfile.avi
As ftp server i use standart ftpd on SERVER and ROUTER, in CLIENT I use windows 7 build in ftp server.
new CLIENT2 (bhyve@ROUTER at some network as CLIENT freebsd 13.1):



iperf 3 (as server side)iperf 3 (as client side)Transfer @ 10secBitrateRetrROUTERSERVER2.96 GBytes
2.96 GBytes2.54 Gbits/sec
2.54 Gbits/sec1237
-ROUTERCLIENT21013 MBytes
1013 MBytes850 Mbits/sec
849 Mbits/sec0
-SERVERROUTER3.12 GBytes
2.84 GBytes2.68 Gbits/sec
2.40 Gbits/sec961
-SERVERCLIENT21.10 GBytes
1.10 GBytes942 Mbits/sec
942 Mbits/sec215
-CLIENT2ROUTER1.30 GBytes
1.02 GBytes1.11 Gbits/sec
843 Mbits/sec0
-CLIENT2SERVER108 MBytes
108 MBytes90.7 Mbits/sec
88.7 Mbits/sec7406
-

and raw output (to many letters):


> root@SERVER # iperf3 -c ROUTER
> Connecting to host ROUTER, port 5201
> [  5] local SERVER port 42475 connected to ROUTER port 5201
> [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
> ...


But: 
	
	



```
[root@CLIENT] #wget ftp://SERVER.......   will take about 100Mbits/sec (as start peak at 150-180, and quikly decrease to 100)
```


----------



## Sobaka (Sep 15, 2022)

SirDice said:


> Besides not having _any_ security patches installed, keep in mind that 12.2-RELEASE is EoL since March 2022.


Ohc... I know it. So to short life time of the 12.x
I can try 13.1. I'll do it tonight next weekend, becouse system in production.


----------



## SirDice (Sep 15, 2022)

Sobaka said:


> So to short life time of the 12.x


_major_ versions have a minimum of 5 year support. But only the last _minor_ version of a _major_ branch is supported. FreeBSD 12 is supported until June 2024.


----------



## Sobaka (Sep 15, 2022)

I found iperf for windows:
This *host *limited  to 1gb - 150-200mbit/s to other client = 800-850mbit/s max.

In single-download over iperf: 200mbit/s (its so slow)
In multi-download over iperf in 10 streams : 836mbit/s (its wery well)


> C:\iperf3.exe -c SERVER
> Connecting to host SERVER, port 5201
> [  4] local CLIENT port 62125 connected to SERVER port 5201
> [ ID] Interval           Transfer     Bandwidth
> ...






> iperf -c SERVER -P 10
> skipp some text.....





> [ ID] Interval           Transfer     Bandwidth
> [  4]   0.00-10.00  sec  83.1 MBytes  69.7 Mbits/sec                  sender
> [  4]   0.00-10.00  sec  82.9 MBytes  69.6 Mbits/sec                  receiver
> [  6]   0.00-10.00  sec   102 MBytes  85.7 Mbits/sec                  sender
> ...


----------

