# NAT not working



## thegolum35 (Nov 25, 2012)

Hi,

Here is my network:  

Internet----- BOX (192.168.1.1/24) -----  (.40 ; fxp0)  SERVER (192.168.50.1/24; lo1) 

There is a jail with the 192.168.50.1 ip.


I can't dig google.com. The paquets from the jail are natted but the answer isn't redirected to it, because dig tells me "timeout".


```
# tcpdump -i fxp0 port 53

09:52:23.760845 IP 192.168.1.40.50956 > google-public-dns-a.google.com.domain: 11540+ A? google.fr. (27)
09:52:23.804847 IP google-public-dns-a.google.com.domain > 192.168.1.40.50956: 11540 3/0/0 A 74.125.230.88, A 74.125.230.87, A 74.125.230.95 (75)
```

Here is my pf.conf


```
nat on fxp0 from lo1:network to any -> (fxp0)

pass log all keep state
```

I also noticed that ICMP paquets have no problem, I can ping a server on the Internet.
Plus, why is there nothing displayed when listening on lo1, diging google.fr for example ?

Thank you, G0llum.


----------



## bbzz (Nov 25, 2012)

Because the packets are rewritten as they go to on real interface; the packets don't actually go anywhere inside lo1 interface, it's just passed up the TCP stack, etc.

Same thing when echo packets destined for re0 arrive at interface re1, for example. Those packets are not passed to re0.

Not sure about timeout, it looks like you got your answer back from google server.


----------



## thegolum35 (Nov 26, 2012)

How may I fix this ?


----------



## bbzz (Nov 26, 2012)

With logical deduction.
More info would help. I can't see why there's timeout when you get your reply back.


----------



## gqgunhed (Nov 27, 2012)

Watch your fxp0 with tcpdump when working inside the jail. Packets from the jail/lo1 must leave the box via fxp0 if routing is configured correctly.

Also check your pf-log as packets fly by (any blocks/drops)?
Maybe it's a routing issue, maybe something where UPD gets blocked.

The tcpdump snippet seems to show DNS-traffic from your server itself?


----------



## mamalos (Nov 27, 2012)

thegolum35 said:
			
		

> ...
> I also noticed that ICMP paquets have no problem, I can ping a server on the Internet.
> Plus, why is there nothing displayed when listening on lo1, diging google.fr for example ?
> ...


Can you telnet(1) a server on port 80 on the Internet (cause you're saying that you can ping(8) hosts from within your jail, which is a bit strange, because default configuration isn't allowing this, you need to set security.jail.param.allow.raw_sockets=1 to achieve this)? If so, your routing and natting is working just fine and your problem is either related to UDP packets not being sent correctly, or at your /etc/resolv.conf (...also, maybe, on the dig(1) command you're  issuing).


----------



## thegolum35 (Nov 28, 2012)

gqgunhed said:
			
		

> Watch your fxp0 with tcpdump when working inside the jail. Packets from the jail/lo1 must leave the box via fxp0 if routing is configured correctly.



It is, I can see the paquets leave via fxp0.



			
				gqgunhed said:
			
		

> Also check your pf-log as packets fly by (any blocks/drops)?
> Maybe it's a routing issue, maybe something where UPD gets blocked.



It shows the request pass, but not the answer.



			
				gqgunhed said:
			
		

> The tcpdump snippet seems to show DNS-traffic from your server itself?



Nope, my server's DNS aren't google's. It's because of NAT.


----------



## thegolum35 (Nov 28, 2012)

mamalos said:
			
		

> Can you telnet(1) a server on port 80 on the Internet (cause you're saying that you can ping(8) hosts from within your jail, which is a bit strange, because default configuration isn't allowing this, you need to set security.jail.param.allow.raw_sockets=1 to achieve this)? If so, your routing and natting is working just fine and your problem is either related to UDP packets not being sent correctly, or at your /etc/resolv.conf



I had allowed ICMP paquets so that I could find what the issue was. And a telnet times out too..

I really have no idea.


----------



## bbzz (Nov 28, 2012)

Are those two lines the only pf rules you have?
Can you try setting skip on loopback.


----------



## thegolum35 (Nov 28, 2012)

Yeap. It doesn't fix the problem..


----------



## bbzz (Nov 28, 2012)

Can you post the output of
`# pfctl -sa`
when you try to resolve IP and initiate connection.

What is shown on *pflog* interface? 

What happens when you make jail run on external IP? Say 192.168.1.100. Remove nat.

Give more info.


----------



## mamalos (Nov 29, 2012)

I had an analogous issue with one server having a Marvel NIC (msk(4)) and I was advised to give the -rxcsum option on my *ifconfig* command -which fixed the problem- due to a buggy hardware implementation of checksum offloading (check out this thread). Even though the PR was msk(4) related, you never know, it may work for you too.


----------



## kpa (Nov 29, 2012)

I had the same problem on a ppc mac mini that has a NIC that uses the gem(4) driver, the -rxcsum option solved the problem. 

For reference, here is the mailing list post:

http://lists.freebsd.org/pipermail/freebsd-stable/2012-August/069126.html


It looks like fxp(4) suffers from the same problem.


----------



## mamalos (Nov 29, 2012)

@kpa, thanx for providing the freebsd.org related link, I couldn't find it in my google search.


----------



## thegolum35 (Nov 30, 2012)

bbzz said:
			
		

> Can you post the output of
> `# pfctl -sa`
> when you try to resolve IP and initiate connection.
> 
> ...




```
pfctl -sa
```


```
Serveur# pfctl -sa
No ALTQ support in kernel
ALTQ related functions disabled
TRANSLATION RULES:
nat on fxp0 inet from 192.168.50.0/24 to any -> (fxp0) round-robin

FILTER RULES:
pass log all flags S/SA keep state

STATES:
all tcp 192.168.1.40:22 <- 192.168.1.29:35041       ESTABLISHED:ESTABLISHED
all udp 192.168.1.40:123 -> 88.190.27.54:123       MULTIPLE:MULTIPLE
all udp 239.255.255.250:1900 <- 192.168.1.1:32792       NO_TRAFFIC:SINGLE
all udp 192.168.1.40:123 -> 88.190.17.126:123       MULTIPLE:MULTIPLE
all udp 192.168.1.40:123 -> 193.55.167.2:123       MULTIPLE:MULTIPLE
all tcp 192.168.1.40:22 <- 192.168.1.29:40824       ESTABLISHED:ESTABLISHED
all udp 192.168.1.40:16116 -> 192.168.1.1:53       MULTIPLE:SINGLE
all udp 192.168.1.40:33117 -> 192.168.1.1:53       MULTIPLE:SINGLE
all udp 192.168.1.40:65226 (192.168.50.1:59171) -> 8.8.8.8:53       MULTIPLE:SINGLE

INFO:
Status: Enabled for 0 days 00:05:23           Debug: Urgent

State Table                          Total             Rate
  current entries                        9               
  searches                             537            1.7/s
  inserts                               77            0.2/s
  removals                              68            0.2/s
Counters
  match                                 77            0.2/s
  bad-offset                             0            0.0/s
  fragment                               0            0.0/s
  short                                  0            0.0/s
  normalize                              0            0.0/s
  memory                                 0            0.0/s
  bad-timestamp                          0            0.0/s
  congestion                             0            0.0/s
  ip-option                              1            0.0/s
  proto-cksum                            0            0.0/s
  state-mismatch                         0            0.0/s
  state-insert                           0            0.0/s
  state-limit                            0            0.0/s
  src-limit                              0            0.0/s
  synproxy                               0            0.0/s

TIMEOUTS:
tcp.first                   120s
tcp.opening                  30s
tcp.established           86400s
tcp.closing                 900s
tcp.finwait                  45s
tcp.closed                   90s
tcp.tsdiff                   30s
udp.first                    60s
udp.single                   30s
udp.multiple                 60s
icmp.first                   20s
icmp.error                   10s
other.first                  60s
other.single                 30s
other.multiple               60s
frag                         30s
interval                     10s
adaptive.start             6000 states
adaptive.end              12000 states
src.track                     0s

LIMITS:
states        hard limit    10000
src-nodes     hard limit    10000
frags         hard limit     5000
tables        hard limit     1000
table-entries hard limit   200000

TABLES:

OS FINGERPRINTS:
700 fingerprints loaded
```


```
tcpdump -n -e -i pflog0
```


```
22:17:20.052143 rule 0..16777216/0(match): pass out on fxp0: 192.168.1.40.64480 > 8.8.8.8.53: 59270+ A? google.fr. (27)
```

If I put the jail on the host's network, it simply works.

problem solved with -rxcsum option.

Thank all of you.


----------



## bbzz (Nov 30, 2012)

Nvm. Guess it's solved then.


----------

