# Port redirection on 2 WAN setup



## jailed (Jul 23, 2013)

Hello,

I have a server acting as a router. I use FreeBSD 9.1 (i386) and PF on it. It has three network interfaces. Two of them are for WAN and remaining one is for LAN. I have no problem with NAT. It's working without problem. The problem is, I cannot accept incoming TCP connections from the second WAN.

Here's the necessary parts of my pf.conf

```
table <network_1> persist { 192.168.0.1, 192.168.0.2, 192.168.0.50, 192.168.0.51, 192.168.0.52, 192.168.0.53 }
table <network_1_area> persist { 192.168.0.0/24 }

table <network_2> persist { 192.168.0.3, 192.168.0.4, 192.168.0.5, 192.168.0.20 }

wan_2_tcp_incoming = "{ 80 }"

icmp_types = "{ echoreq, unreach }"

set block-policy return
set loginterface $wan_1
set state-policy if-bound
set skip on lo0

# NAT
nat on $wan_1 from <network_1> to any -> $ip_1
nat on $wan_2 from <network_2> to any -> $ip_2

rdr pass on $wan_1 proto tcp from any to $ip_1 port 80 -> 192.168.0.2 port 80
rdr pass on $wan_1 proto tcp from any to $ip_1 port 8000 -> 192.168.0.2 port 8000
rdr pass on $wan_1 proto tcp from any to $ip_1 port 443 -> 192.168.0.2 port 443
rdr pass on $wan_1 proto tcp from any to $ip_1 port 2222 -> 192.168.0.1 port 22

rdr pass log on $wan_2 proto tcp from any to $ip_2 port 80 -> 192.168.0.3 port 80 # Not working

pass in on $lan all
block in log on ste0 all
block in log on ste1 all

pass in on $wan_1 inet proto tcp from any to $ip_1 port $wan_1_tcp_incoming
pass in on $wan_1 inet proto udp from any to $ip_1 port $wan_1_udp

# pass in log on $wan_2 reply-to ( $wan_2 $gw_2 ) proto tcp from any to $ip_2 port $wan_2_tcp_incoming

pass in inet proto udp from any port 53 to any

pass inet proto icmp all icmp-type $icmp_types

pass in on $lan route-to ( $wan_1 $gw_1 ) from <network_1> to !<network_1_area> keep state
pass in on $lan route-to ( $wan_2 $gw_2 ) from <network_2> to !<network_1_area> keep state

pass in on $wan_1 route-to ( $wan_2 $gw_2 ) from <network_2> keep state
pass in on $wan_2 route-to ( $wan_1 $gw_1 ) from <network_1> keep state

pass out on $wan_1 route-to ( $wan_2 $gw_2 ) from $wan_2 to any keep state
pass out on $wan_2 route-to ( $wan_1 $gw_1 ) from $wan_1 to any keep state

pass out all

# END
```

The redirection on the port 80 of $wan_2 is not working. The packet reaches to 192.168.0.3 but the reply doesn't reach to the source address. When I disable port forwarding and directly bind httpd to $ip_2 on the router, it still does not reply to the source.

I will be very happy if anyone can help me solve this problem.

Thanks.


----------



## SirDice (Jul 23, 2013)

jailed said:
			
		

> The packet reaches 192.168.0.3 but the reply doesn't reach to the source address. When I disable port forwarding and directly binding httpd to $ip_2 on the router, it still does not reply to the source.


This is most likely a routing problem. Packets coming in on WAN2 might have their replies sent out on WAN1 depending on where your default gateway is set to.


----------



## kpa (Jul 23, 2013)

Isn't reply-to tailor made for this situation?


----------



## SirDice (Jul 23, 2013)

kpa said:
			
		

> Isn't reply-to tailor made for this situation?



It should, yes. Although I never quite understood how to use it properly. I tend to prevent situations like this from happening


----------



## gkontos (Jul 23, 2013)

This appears to be a routing issue. Try running a tcpdump to make sure.

You can create multiple route paths using setfib(1)() but you also need to compile your kernel with:


```
options         ROUTETABLES=4
```

Then you add each gateway: `# setfib 1 route add -net default 10.0.0.1`. I had done this in the past but I can't recall exactly how...


----------



## jailed (Jul 23, 2013)

Thank you for your replies. Previously, I tried the rtable method, but it didn't work for me. I couldn't understand how it works. There's no enough documentation on the Internet. I have also tried with OpenBSD, but the results were the same.

I created all the routing rules correctly with the route-to command. Different IP's on the same LAN can use different ISPs with my configuration. Only this port redirection does not work. I think it's easy to solve with PF's route-to and reply-to commands. I don't understand how the reply-to command really works, so that I couldn't solve the last piece of the job.

I've seen some examples with tag and tagged rules. This can help too, but there's not enough documentation with this topics too.

I'm also considering buying a hardware router/switch. But I'm not sure which router can help me for this setup. I have two Internet connections for now. But I can increase the amount of it in the future. So, using a server with PF is a cheaper solution for now.


----------



## kpa (Jul 23, 2013)

The reply-to should be applied to the filter rules that accept the redirected traffic. So something like what you have already done but have commented out:


```
rdr log on $wan_2 proto tcp from any to ($wan_2) port 80 -> 192.168.0.3 port 80 
...
pass in log on $wan_2 reply-to ( $wan_2 $gw_2 ) proto tcp from any to 192.168.0.3 port 80
```

I took out the pass from the rdr because it creates another filter rule that competes with the rule below it.


----------



## DutchDaemon (Jul 23, 2013)

Yes, reply-to is certainly the answer. I just made my router/firewall dual-homed yesterday, and the second WAN interface (i.e. not the default gateway) needed reply-to in its pass in rules.

E.g.


```
pass in quick on $second_wan_if reply-to ( $second_wan_if $second_wan_ip ) inet proto udp from any to $second_wan_if port 53 keep state
```


----------



## jailed (Jul 25, 2013)

Thank you for your replies. But it's not working.

Here's the latest version of my pf.conf:

```
table <network_1> persist { 192.168.0.1, 192.168.0.2, 192.168.0.50, 192.168.0.51, 192.168.0.52, 192.168.0.53 }
table <network_1_area> persist { 192.168.0.0/24 }

table <network_2> persist { 192.168.0.3, 192.168.0.4, 192.168.0.5, 192.168.0.20 }

wan_2_tcp_incoming = "{ 80 }"

icmp_types = "{ echoreq, unreach }"

set block-policy return
set loginterface $wan_1
set state-policy if-bound
set skip on lo0

# NAT
nat on $wan_1 from <network_1> to any -> $ip_1
nat on $wan_2 from <network_2> to any -> $ip_2

# RDR
rdr pass on $wan_1 proto tcp from any to $ip_1 port 80 -> 192.168.0.2 port 80
rdr pass on $wan_1 proto tcp from any to $ip_1 port 8000 -> 192.168.0.2 port 8000
rdr pass on $wan_1 proto tcp from any to $ip_1 port 443 -> 192.168.0.2 port 443
rdr pass on $wan_1 proto tcp from any to $ip_1 port 2222 -> 192.168.0.1 port 22

rdr log on $wan_2 proto tcp from any to $ip_2 port 80 -> 192.168.0.3 port 80

pass in on $lan all
pass out all
block in on $wan_1 all
block in on $wan_2 all

pass inet proto icmp all icmp-type $icmp_types keep state

pass in on $wan_1 inet proto tcp from any to $ip_1 port $wan_1_tcp_incoming

pass in log quick on $wan_2 reply-to ( $wan_2 $ip_2 ) inet proto tcp from any to 192.168.0.3 port 80 keep state

pass in on $lan route-to ( $wan_1 $gw_1 ) from <network_1> to !<network_1_area> keep state
pass in on $lan route-to ( $wan_2 $gw_2 ) from <network_2> to !<network_1_area> keep state

pass out on $wan_1 route-to ( $wan_2 $gw_2 ) from $wan_2 to any keep state
pass out on $wan_2 route-to ( $wan_1 $gw_1 ) from $wan_1 to any keep state
```

pf.log output:

```
00:00:00.249600 rule 10..16777216/0(match): pass in on ste1: aa.bb.cc.dd.1739 > 192.168.0.3.80: Flags [S], seq 761507850, win 65535, options [mss 1460,nop,wscale 1,nop,nop,sackOK], length 0
```


----------



## jailed (Jul 27, 2013)

I've just resolved the problem. I wanted to share the result in order to help others who may read this thread.

@kpa, your reply was certainly the real answer. The problem with my rules was the state-policy. When I remove state-policy definition, everything works perfectly.

@DutchDaemon, reply-to should contain the gateway IP and packets should go to the target IP as it shown on @kpa's post.

Thank you all who participated this thread to help me with sharing their valuable knowledge and time.


----------



## DutchDaemon (Jul 27, 2013)

My reply-to rules work just fine, so I 'should' not do anything, I believe. Perhaps you misunderstood my pf.conf line. Here it is translated:


```
pass in quick on bge0 reply-to (bge0 10.10.0.1) inet proto udp from any to 10.10.0.13 port = domain keep state
```

The server itself has 10.10.0.13 as its IP address, and 10.10.0.1 is its gateway. This is how packets are statefully returned to a non-default gateway


----------



## jailed (Jul 28, 2013)

@DutchDaemon,

I'm deeply sorry if I hurt you accidentally. I didn't talked about your ruleset and I've never said you should do anything. What I've said is "it should be like that for my ruleset to work in this case".

Thanks.


----------

