# PF; rdr rules; unable to keep the connection



## _martin (Aug 7, 2014)

I've described the similar problem here already:  http://forums.freebsd.org/viewtopic.php?f=7&t=44953 . I did suggest the workaround too. 

Recently I shared somewhat big files (~30GB) over http. FreeBSD 10/PF/jails/apache22 under the hood. I encountered the same problem as in VPN connection described above - download was aborted prematurely. When I looked at traffic with tcpdump I saw incoming RST packet from the FreeBSD server just before connection was dropped. Web server itself is working (meaning semi-static html contents is displayed correctly). 
I had to move apache out of jail (to avoid using rdr in PF) to make it all work. 

I see this behavior in FreeBSD 10. Have any of you encountered the similar problem ? 
Setup is the following: 

ext_if - some public IP on external interface
int_if / loopback - internal or dummy interface 

basic PF skeleton: 


```
# LOCAL_NET - local net on dummy/internal iface
# IP_PUB - nat-to IP (on ext_if)
# PORT_WWW -  port to rdr
# IP_JAIL_WEBSERVER  - IP from LOCAL_NET

nat pass on $ext_if from $LOCAL_NET to any -> $IP_PUB
rdr pass on $ext_if proto tcp to $IP_PUB port $PORT_WWW -> $IP_JAIL_WEBSERVER

pass in all
pass out all
```
As it is vacation period I didn't have much time to do deeper tests on my machine. I will try to do deeper tests before submitting PR. I am, however,  curious if somebody else did encounter the same problem.


----------



## kpa (Aug 7, 2014)

I can't remember where I read it but it might be that the rdr pass construction doesn't imply keep-state on the firewall rule that it creates like you would expect. Try moving the firewalling of the NAT'ed connections to separate rules.


----------



## _martin (Aug 8, 2014)

To my understanding keep-state means to keep the status of the connection in a status table. This is mainly due to performance and a security. The way I see it it should not matter if I do have keep-state or not (in the sense that session should not be closed prematurely in either setup). Sure I can avoid nat pass and pass the traffic in filtering section; I'll try to do it during my tests.


----------



## _martin (Aug 10, 2014)

@kpa Hm, so I did some tests today. It seems that nat pass is OK, but rdr pass is a trouble here. I cut the pf.conf to the very basic as following (public IP changed here):


```
ext_if="em0"

IP_PUB="192.0.2.1"
IP_JAIL_SANDBOX="192.168.252.1"
NET_LO252="192.168.252.0/24"
PORT_JAIL_SANDBOX_SSH="5222"

set skip on lo0
scrub in all

nat pass on $ext_if from $NET_LO252 to any -> $IP_PUB
rdr pass on $ext_if proto tcp to $IP_PUB port $PORT_JAIL_SANDBOX_SSH -> $IP_JAIL_SANDBOX port 22

block in all
pass out all

# (XXX: keep the door open!)
pass in on $ext_if proto tcp from any to $IP_PUB port 22
```
I have a FreeBSD machine running a jail which I access from outside to a custom port. Local address is bind to a custom loopback interface. Config above does NOT work (i.e. RST is sent at random) on my machine (10.0-RELEASE-p6 GENERIC). However, if I do a slight modification:


```
ext_if="em0"

IP_PUB="192.0.2.1"
IP_JAIL_SANDBOX="192.168.252.1"
NET_LO252="192.168.252.0/24"
PORT_JAIL_SANDBOX_SSH="5222"

set skip on lo0
scrub in all

nat pass on $ext_if from $NET_LO252 to any -> $IP_PUB
rdr on $ext_if proto tcp to $IP_PUB port $PORT_JAIL_SANDBOX_SSH -> $IP_JAIL_SANDBOX port 22

block in all
pass out all

# pass sandbox rdr to IP_JAIL_SANDBOX
pass in quick proto tcp from any to $IP_JAIL_SANDBOX port 22

# (XXX: keep the door open!)
pass in on $ext_if proto tcp from any to $IP_PUB port 22
```
Now the connection is working perfectly. 

To make it worse though, I can't reproduce this behavior in VM. Right now the main difference is that my VM was on the same network as a machine I was doing tests from (but should that matter ? ) opposed to my server which I'm reaching behind the gateway.


----------



## devil_devil (Mar 7, 2015)

Hi all,

I can confirm that today we had the same problem with the "rdr pass" rules in our pf.conf/FreeBSD 10.1 STABLE
`uname -K`

```
1001510
```
`uname -U`

```
1001510
```
I can also confirm that in FreeBSD 9 STABLE these rules just work perfectly!


----------



## _martin (Mar 8, 2015)

devil_devil The sad thing is that I tried to reproduce this issue in 3 different ways: 


0) in VM
1) on a totally different physical HW
2) on a same type of a HW (server) where problem occurred, just different node

Neither of these scenarios helped - issue didn't occur. *0) 1)* examples would make me think OK, it's something more HW-related, maybe Intel's driver ? Hard to say. But when I tested the *2) *scenario I was really bumped. True that uptime was a bit small (a day or so), but this issue does occur on my HW even after reboot, maybe few minutes (traffic) later. 

My network adapter is Intel 82579LM on a S1200BT motherboard.


----------

