# tcp reset on reply to nat'd address



## dove (Nov 8, 2009)

I have ipfw running on a freebsd box with two NICs.

The "external" NIC is really facing a dumb router that is
configured to just pass everything through.  That subnet is 192.168.0/24.

The "internal" subnet is actually a routable real internet
block, let's say _a.b.c_/24.

The firewall has its "internal" IP set to _a.b.c.d_ on
interface em1.
"external" IP set to 192.168.0.2 with the dumb router
at 192.168.0.1 on interface em0.

I also have some internal hosts that are unregistered RFC
1918 addresses.  I am trying to NAT these IPs and leave
the routable registered IPs untouched.

Rules:

```
extif=em1
intif=em0
pubnet=[I]a.b.c[/I]
privnet=10.0.0.0/8
ipfw deny log ip from 192.168.0.0/24 to any in recv $intif
ipfw add divert natd log all from $privnet to any out via $extif
ipfw add allow tcp from any to [I]a.b.c.e[/I] 22 via $extif
ipfw add deny log tcp from any to any in recv $extif setup
ipfw add allow tcp from any to any setup
ipfw add allow tcp from any to any established
ipfw add allow udp 53 to [I]a.b.c[/I].0/24
ipfw add 65500 deny log ip from any to any
```
and natd running as [cmd=]natd -v -n em0[/cmd]

tcpdump on both interfaces looking at external host '_E.F.G.H_' is showing this when I try to ssh to it from the internal 10.0.0.2 box:

```
16:52:16.207810 10.0.0.2.45642 > [I]E.F.G.H[/I].22: S 3095885078:3095885078(0) win 65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 288974989 0,sackOK,eol> (DF)
16:52:16.209071 [I]a.b.c.d[/I].45642 > [I]E.F.G.H[/I].22: S 3095885078:3095885078(0) win 65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 288974989 0,sackOK,eol> (DF)
16:52:16.271086 [I]E.F.G.H[/I].22 > [I]a.b.c.d[/I].45642: S 1381661810:1381661810(0) ack 3095885079 win 5792 <mss 1380,nop,nop,timestamp 2738478757 288974989,nop,wscale 2> (DF)
16:52:16.271214 [I]a.b.c.d[/I].45642 > [I]E.F.G.H[/I].22: R 3095885079:3095885079(0) win 0
```

As you can see the rewrite is done correctly on the outgoing divert, and the external host responds.  But the firewall is generating a tcp reset.

It's as if it has forgotten about the diverted socket and doesn't have an open socket for it anymore.

Hints would be wonderful, please.


----------



## DutchDaemon (Nov 8, 2009)

I'm a bit out of the loop wrt ipfw, but I seem to be missing stateful rules here, which would explain why return traffic is refused. Maybe statefulness is built in/implied now (like in pf), in which case I wasn't here  I'm not sure whether the 'established' rule works on its own like that, i.e. without check-state and keep-state somewhere else.


----------



## dove (Nov 8, 2009)

I tried adding keep-state to the 'divert' rule (just appended it to the line) and check-state on the line after it.  natd -v is now reporting an 'In' rewrite, but the tcp reset happens exactly the same as before.

Where's the right place to put the *state directives?


----------



## DutchDaemon (Nov 8, 2009)

ipfw(8) is really the place to go 
See the STATEFUL FIREWALL and EXAMPLES sections.


----------



## dove (Nov 8, 2009)

Yeah, I've been over the man page (and various howtos and blogs).  I'm not finding the right magic, and could use a hint about what I'm doing wrong and/or an explanation of why the reset is happening.  The handbook has an example, but the entire internal net is nat'd.  In my case, I only want the RFC 1918 addresses nat'd, not the public ones.  Hopefully someone can interpret the tea leaves here.


----------



## DutchDaemon (Nov 8, 2009)

If you have a sufficently new ipfw, NAT is built in. No need for the divert stuff.

http://www.freebsd.org/cgi/man.cgi?query=ipfw&sektion=8#NETWORK_ADDRESS_TRANSLATION_(NAT) (see EXAMPLES)

And, as far as I can remeber, the check-state rule is usually the first ipfw add rule, and keep-state flags are usually added to all allow rules.


----------



## dove (Nov 8, 2009)

Thanks for the ideas.
Moving check-state to the top (or anywhere I tried it) and adding keep-state to all the allow rules didn't help.  I still get the tcp reset.


----------



## DutchDaemon (Nov 8, 2009)

Please post an ascii network diagram (put 
	
	



```
tags around it). Your 'external' and 'internal' networks, IPs, and NICs confuse the hell out of me .. You appear to have no routing for the 10/8 network, and to be NAT'ing that network without it being directly connected to (present on) any of your interfaces. One normally NAT's a network on interface_a to interface_b. You could try adding a static route to network 10/8, pointing to the interface it's behind.
```


----------



## dove (Nov 8, 2009)

DutchDaemon said:
			
		

> Please post an ascii network diagram (put
> 
> 
> 
> ...


----------



## DutchDaemon (Nov 8, 2009)

Can you ping (or otherwise connect to) 10.0.0.2 from the IPFW box? If 1.2.3.4 sends a tcp-reset, it means it doesn't know how to handle the connection back to the 10/8 network.


----------



## dove (Nov 8, 2009)

Yes.  Before I tried adding the NAT, I had rules to stop all RFC 1918 addresses from going beyond the firewall.  And those deny rules (logged) are hit.

Here's the traceroute...

```
% traceroute -n 10.0.0.2
traceroute to 10.0.0.2 (10.0.0.2), 64 hops max, 44 byte packets
 1  1.2.3.5  0.361 ms  0.265 ms  0.247 ms
 2  10.0.0.2  0.734 ms  0.729 ms  0.750 ms
```

It's the rules I need help with I think (or natd args?).  If I do need keep-state, I'm unclear where to put it (and check-state) in my rule set previously shown.  As I said, I tried keep-state on the divert rule and then separately on all the allow rules, with check-state at various places with no luck.

Here is the rule set again with rule numbers so it's easier to reference them in this thread and the fake IP numbers from the ASCII diagram plugged in instead of symbolic letters...



```
extif=em1
intif=em0
pubnet=1.2.3
privnet=10.0.0.0/8
ipfw deny 100 log ip from 192.168.0.0/24 to any in recv $intif
ipfw add 200 divert natd log all from $privnet to any out via $extif
ipfw add 300 allow tcp from any to 1.2.3.10 22 via $extif
ipfw add 400 deny log tcp from any to any in recv $extif setup
ipfw add 500 allow tcp from any to any setup
ipfw add 600 allow tcp from any to any established
ipfw add 700 allow udp 53 to 1.2.3.0/24
ipfw add 65500 deny log ip from any to any
```


----------



## dove (Nov 8, 2009)

Clarification...
I said the RFC 1918 deny rules "are hit".  I meant to say "were hit" (when I had them in there).  In the interest of simplicity to isolate this reset problem, I pulled those rules (and some others) out.


----------



## Const (Nov 10, 2009)

Try to log every IPFW rule including the final "deny all". Sometimes it's difficult to see a little issue in the system you've built. IPFW will show you how the packet comes in, how it comes out, wether it's blocked by some rules or anything else


----------



## dove (Nov 11, 2009)

As you can see in the rules, all deny entries are logged.  None of them are hit when the tcp reset occurs.  I even added logging for all 'allow' entries briefly and there was no interesting information there (incoming reply from the external host is logged, and there is no deny thereafter).


----------



## aragon (Nov 12, 2009)

For simplicity's sake, it might be better to do your NAT on 10.0.0.1.

The problem with your current setup is your divert rule.  It is only diverting traffic that exits $extif.  You need to remember that natd needs to do the aliasing of outgoing traffic _and_ the dealiasing of incoming traffic.  You are not passing incoming traffic back through natd to dealias it, so ipfw is rejecting it.


----------



## dove (Nov 13, 2009)

Okay, let's reformulate the question to do it on 10.0.0.1 then.

What would be the rules if I wanted to NAT only traffic to and from 10.0.0.2 and none of the other hosts on 10/8?


----------



## aragon (Nov 14, 2009)

dove said:
			
		

> What would be the rules if I wanted to NAT only traffic to and from 10.0.0.2 and none of the other hosts on 10/8?


Something like:


```
ipfw add divert natd from 10.0.0.2 to any out via $extif
ipfw add divert natd from any to me in via $extif
```


----------

