# The router with FreeBSD 9.0 hangs in most cases. Need help



## r00ty (Oct 26, 2012)

Hey Guys,

I have a box with FreeBSD 9.0 at home. Nothing special, using this as router that supports about 4-6 devices _(Me and my relatives)_. 

My ISP gives me 100Mb/s (real value). Usually everything is ok, but after I start to download some file with my torrent client at my desktop PC, my local network is become pretty unstable.
So when I said _unstable_, I meant:

1) All other devices suddenly lost connection to gateway.
2) At my PC with torrent client I have ability to communicate with gateway, but it's very-very slow and that downloading file, torrent client just drop speed from 10MB/s to 2-3MB/s. So right after start to download something it works pretty nice with speed 9-10MB/s but it's take only 20-30 seconds, than kaboom.
3) My gateway console is freezing when this happens.

BTW, some times, in 10% cases, it works fine with 9-10MB/s speed, but It's really only 10% :\

Network schema:
 ---> Switch ---> Router (FreeBSD) ---> ISP

Router HW:

```
CPU: Intel(R) Pentium(R) 4 CPU 2.00GHz (2019.93-MHz 686-class CPU)

real memory  = 536870912 (512 MB)
avail memory = 515694592 (491 MB)
```

ifconfig

```
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=2098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
        ether 00:07:e9:89:cf:5b
        inet x.x.x.x netmask 0xffffff00 broadcast x.x.x.x
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=2098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
        ether 00:07:e9:0a:c8:67
        inet 10.72.0.14 netmask 0xfffffff0 broadcast 10.72.0.15
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=3<RXCSUM,TXCSUM>
        inet 127.0.0.1 netmask 0xff000000
```

I like ipfw, so I decided to build NAT with this:
*KERNEL NAT* + IPFW.

IPFW Rules

```
-f flush

add allow ip from any to any via lo0
add deny ip from any to 127.0.0.0/8
add deny ip from 127.0.0.0/8 to any

add allow ip from any to any via em1

nat 1 config log if em0 reset same_ports deny_in redirect_port tcp 10.72.0.1:60701 60701

add nat 1 ip from any to any via em0
```

/etc/rc.conf

```
...
#
# Network
#
hostname="vault88.log.kiev.ua"
ifconfig_em0=" inet x.x.x.x netmask 255.255.255.0 -rxcsum"
#ifconfig_em0=" inet x.x.x.x netmask 255.255.255.0"
#ifconfig_em1=" inet 10.72.0.14 netmask 255.255.255.240"
ifconfig_em1=" inet 10.72.0.14 netmask 255.255.255.240 -rxcsum"
defaultrouter="x.x.x.x"
gateway_enable="YES"
rinetd_enable="YES"
#dummynet_enable="YES" [ ALREADY COMPILED WITH KERNEL ]
#
# Firewall
#
firewall_enable="YES"
#firewall_nat_enable="YES" [ ALREADY COMPILED WITH KERNEL ]
firewall_type="/etc/firewall"
#firewall_logging="YES"
#firewall_quiet="YES"
#
# TCP/IP options
#
tcp_extensions="YES"
tcp_drop_synfin="YES"
icmp_drop_redirect="YES"
icmp_log_redirect="NO"
...
```

/etc/sysctl.conf

```
net.inet.ip.fw.one_pass=1

net.inet.tcp.blackhole=1
net.inet.udp.blackhole=0

net.inet.ip.redirect=0
net.inet.icmp.drop_redirect=1
net.inet.icmp.log_redirect=0
net.inet.tcp.drop_synfin=1

net.inet.tcp.maxtcptw=40960

p1003_1b.sem_nsems_max=10000

kern.ipc.somaxconn=32768
kern.ipc.nmbclusters=524288
net.inet.ip.intr_queue_maxlen=8192
kern.ipc.maxsockbuf=8388608
kern.ipc.maxsockets=204800
net.inet.tcp.sendspace=3217968
net.inet.tcp.recvspace=3217968

net.inet.ip.fastforwarding=1

kern.maxfiles=400000
kern.maxfilesperproc=200000
kern.maxvnodes=2000000

kern.random.sys.harvest.ethernet=0
kern.random.sys.harvest.point_to_point=0
kern.random.sys.harvest.interrupt=0

net.inet.raw.maxdgram=16384
net.inet.raw.recvspace=16384
```

I tried to tune it, but it really gave me only that 30 seconds of stable work.
Also I played with -rxcsum, but it didn't any effect at all.

vmstat -z
_attached_

uname -a

```
FreeBSD vault88.log.kiev.ua 9.0-RELEASE FreeBSD 9.0-RELEASE #1: Wed Apr  4 19:56:50 EEST 2012     root@vault88.log.kiev.ua:/usr/obj/usr/src/sys/VAULT88  i386
```

KERNEL VAULT88 is attached.

So I am really confused... Will be appreciated for any help.


----------



## SirDice (Oct 26, 2012)

First try turning off all the TCP options and sysctls.


----------



## marwis (Oct 26, 2012)

r00ty said:
			
		

> /etc/rc.conf
> 
> ```
> ifconfig_em0=" inet x.x.x.x netmask 255.255.255.0 -rxcsum"
> ...



You might want to turn off TXCSUM, as well.  Details here (ipfw nat and em interface rxcsum problem):



> >Fix:
> Turn off RXCSUM,TXCSUM on em adapter


----------



## r00ty (Oct 26, 2012)

Awesome!

I truncated /etc/sysctl.conf and left there only:
net.inet.ip.fw.one_pass=1

Also I removed from /etc/rc.conf

```
tcp_extensions="YES"
tcp_drop_synfin="YES"
icmp_drop_redirect="YES"
icmp_log_redirect="NO"
```

And now it's works perfect. Need to investigate wtf with my hands and with this tuning :e .

Thanks Guys.


----------



## phoenix (Oct 26, 2012)

marwis said:
			
		

> You might want to turn off TXCSUM, as well.  Details here (ipfw nat and em interface rxcsum problem):



Note:  this was fixed in 9.0.


----------

