# Not Getting Full Bandwith with 9.0-RC1



## Persephone (Nov 25, 2011)

I have 9.0-RC1 running on a machine with a Realtek 8111E on the motherboard. I ran Windows 7 on it with no problems for some time before installing FreeBSD on it recently. I have a standard home Internet setup: Computer -> Router -> Modem.


```
uname -a
FreeBSD computer 9.0-RC1 FreeBSD 9.0-RC1 #0: Tue Oct 18 18:51:43 UTC 2011
root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64
```


```
ifconfig re0
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=389b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_UCAST,
WOL_MCAST,WOL_MAGIC>
        ether f3:1c:51:29:9a:ed
        inet 192.168.1.100 netmask 0xffffff00 broadcast 192.168.1.255
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
```

I have no trouble grabbing an IP address from the router. My connection is solid and doesn't drop. Webbrowsing works well. However, for any large file transfers I am not getting the full bandwidth that I am when booted into Windows 7 on the same machine. The data rate appears to constantly peak and drop between full bandwidth and nothing.

Is there a way I can try to diagnose what is going wrong?


----------



## bbzz (Nov 25, 2011)

Persephone said:
			
		

> The data rate appears to constantly peak and drop between full bandwidth and nothing.



That's how packets are serialized, actually. You are either sending full speed or nothing trough interface.
What I'm trying to say is, how do you know you are not getting proper speed? What did you look at? You didn't provide any details.

What does:
`$ systat -if 5`
show?


----------



## Persephone (Nov 26, 2011)

bbzz said:
			
		

> What I'm trying to say is, how do you know you are not getting proper speed? What did you look at? You didn't provide any details.



I made it clear in my post. Same hardware in Windows 7 - I am able to max out my bandwidth when downloading. Boot into FreeBSD and I am getting nowhere near the K/sec from the very same sources. When looking at network monitor like KDE's System Monitor, the K/sec is constantly peaking at max and then dropping down to zero and then right back up to max. Transfer times for the very same sized files are ending up taking up much longer as a result.

Ping times are exactly the same. No lost packets are begin reported lost. Something is causing file transfers to constantly stutter or stall.


----------



## bbzz (Nov 27, 2011)

Besides the point that it's slower you didn't give any additional details, is what I was getting at.
In other words it could be number of different things, quite possible issue with Realtek itself.

I would first upgrade to 9-RC2. 

What does
`# netstat -i`
says after a transfer?

What about
`# netstat -s -p tcp`

If you have another machine you could try to do local transfer, see what happens.

What kind of filesystem are you running? Maybe it has nothing to do with network issue at all. Download speed could be limited by hard disk performance, for example.


----------



## Persephone (Nov 27, 2011)

Thanks for the reply.

I have now upgraded to 9.0RC2:


```
FreeBSD slack-bsd 9.0-RC2 FreeBSD 9.0-RC2 #0: Sat Nov 12 18:35:25 UTC 2011     
[email]root@farrell.cse.buffalo.edu[/email]:/usr/obj/usr/src/sys/GENERIC  amd64
```
`netstat -s -p tcp`

```
tcp:
        219857 packets sent
                959 data packets (486946 bytes)
                6 data packets (4268 bytes) retransmitted
                26 data packets unnecessarily retransmitted
                0 resends initiated by MTU discovery
                121844 ack-only packets (727 delayed)
                0 URG only packets
                0 window probe packets
                95974 window update packets
                1074 control packets
        307177 packets received
                1941 acks (for 485496 bytes)
                321 duplicate acks
                0 acks for unsent data
                278449 packets (398593622 bytes) received in-sequence
                64 completely duplicate packets (14452 bytes)
                0 old duplicate packets
                0 packets with some dup. data (0 bytes duped)
                26445 out-of-order packets (37977939 bytes)
                0 packets (0 bytes) of data after window
                0 window probes
                62 window update packets
                20 packets received after close
                0 discarded for bad checksums
                0 discarded for bad header offset fields
                0 discarded because packet too short
                644 discarded due to memory problems
        565 connection requests
        1 connection accept
        0 bad connection attempts
        0 listen queue overflows
        0 ignored RSTs in the windows
        557 connections established (including accepts)
        615 connections closed (including 6 drops)
                154 connections updated cached RTT on close
                154 connections updated cached RTT variance on close
                8 connections updated cached ssthresh on close
        9 embryonic connections dropped
        1930 segments updated rtt (of 1953 attempts)
        66 retransmit timeouts
                0 connections dropped by rexmit timeout
        0 persist timeouts
                0 connections dropped by persist timeout
        0 Connections (fin_wait_2) dropped because of timeout
        0 keepalive timeouts
                0 keepalive probes sent
                0 connections dropped by keepalive
        94 correct ACK header predictions
        277603 correct data packet header predictions
        1 syncache entry added
                0 retransmitted
                0 dupsyn
                0 dropped
                1 completed
                0 bucket overflow
                0 cache overflow
                0 reset
                0 stale
                0 aborted
                0 badack
                0 unreach
                0 zone failures
        1 cookie sent
        0 cookies received
        73 hostcache entries added
                0 bucket overflow
        0 SACK recovery episodes
        0 segment rexmits in SACK recovery episodes
        0 byte rexmits in SACK recovery episodes
        7 SACK options (SACK blocks) received
        29090 SACK options (SACK blocks) sent
        0 SACK scoreboard overflow
        0 packets with ECN CE bit set
        0 packets with ECN ECT(0) bit set
        0 packets with ECN ECT(1) bit set
        0 successful ECN handshakes
        0 times ECN reduced the congestion window
```


----------



## Persephone (Nov 27, 2011)

I have narrowed the problem down a bit further.

I am using: http://www.thinkbroadband.com/download.html as a test download because it reliably saturates my DSL connection. If I boot into Windows 7 and download the 1 GB test file I get an almost perfect 300k a sec transfer. I now see that when I first boot into FreeBSD I get the same solid 300k a sec transfer. However, at some point something causes the net connection to go into a bad state where transfers bounce between 5k and 300k with the majority of time at 5k.

*/etc/rc.d/netif restart* doesn't help. Only restarting the machine will get the net connection back to working correctly.

I don't know what exactly is triggering the problem yet. 

This is my *dmesg | grep ^re* output:


```
real memory  = 8589934592 (8192 MB)
re0: <RealTek 8168/8111 B/C/CP/D/DP/E PCIe Gigabit Ethernet> port 0xde00-0xdeff mem 0xfbdff000-0xfbdfffff,0xfbdf8000-0xfbdfbfff irq 18 at device 0.0 on pci7
re0: Using 1 MSI-X message
re0: turning off MSI enable bit.
re0: Chip rev. 0x2c800000
re0: MAC rev. 0x00000000
re0: Ethernet address: f3:1c:51:29:9a:ed
re0: link state changed to DOWN
re0: link state changed to UP
real memory  = 8589934592 (8192 MB)
re0: <RealTek 8168/8111 B/C/CP/D/DP/E PCIe Gigabit Ethernet> port 0xde00-0xdeff mem 0xfbdff000-0xfbdfffff,0xfbdf8000-0xfbdfbfff irq 18 at device 0.0 on pci7
re0: Using 1 MSI-X message
re0: Chip rev. 0x2c800000
re0: MAC rev. 0x00000000
re0: Ethernet address: f3:1c:51:29:9a:ed
```


----------



## DutchDaemon (Nov 28, 2011)

Persephone, format your posts, thanks.


----------



## yongari@ (Nov 28, 2011)

Because you have high number of out of order TCP segments I'm guessing you're suffering from a TCP reassembly bug. This bug was already fixed in HEAD(r228016).
Try applying the patch at the following URL.

http://svnweb.freebsd.org/base/head...view=patch&r1=227309&r2=228016&pathrev=228016


----------



## Persephone (Nov 28, 2011)

yongari,

I applied the path and am running it now. I will post back after I've tested it for a while.


----------



## Persephone (Nov 28, 2011)

Internet downloads are working perfectly now.

Thank you yongari.


----------



## kpa (Nov 29, 2011)

The patch makes a huge difference in NFS performance, I get about 10 MB/sec reads over 100Mbit line with the patch applied and without the patch the numbers were sometimes as low as 10-20 percent of that.


----------

