# PF and GRE tunnel



## roman90731 (Aug 9, 2013)

I'm trying to set up a GRE tunnel between a FreeBSD server and a CentOS server. I want the FreeBSD server to be the one with the hidden IP. I'm having trouble making the connections I need. Only port 80 seems to work. I have PF disabled, and no firewall running on CentOS. I will be using placeholders for the IP addresses.

_x.x.x.x will be FreeBSD's IP
y.y.y.y will be CentOS's IP_

Here are my telnets from FreeBSD to CentOS:

```
# telnet y.y.y.y 80
Trying y.y.y.y...
Connected to y.y.y.y.static.xxxxxx.com.
Escape character is '^]'.
```


```
# telnet y.y.y.y 27960
Trying y.y.y.y...
telnet: connect to address y.y.y.y: Connection refused
telnet: Unable to connect to remote host
```

Here are my telnets from CentOS to FreeBSD:

```
# telnet x.x.x.x 80
Trying x.x.x.x4...
Connected to x.x.x.x.
Escape character is '^]'.
```


```
# telnet x.x.x.x 27960
Trying x.x.x.x...
telnet: connect to address x.x.x.x: Connection refused
```


Here are the applicable rules in CentOS iptables:

```
# Generated by iptables-save v1.4.7 on Thu Aug  8 22:43:38 2013
*filter
:INPUT ACCEPT [1]
:FORWARD ACCEPT [9]
:OUTPUT ACCEPT [60]
:l - [0]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT 
-A INPUT -p icmp -j ACCEPT 
-A INPUT -i lo -j ACCEPT 
-A INPUT -p gre -j ACCEPT 

## SSH
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT 

## Add your ports you want to accept
-A FORWARD -p tcp -m tcp --dport 80 -m state --state NEW,RELATED,ESTABLISHED 
-A FORWARD -p tcp -m tcp --dport 27960 -m state --state NEW,RELATED,ESTABLISHED 
-A FORWARD -p udp -m udp --dport 80 -m state --state NEW,RELATED,ESTABLISHED 
-A FORWARD -p udp -m udp --dport 27960-m state --state NEW,RELATED,ESTABLISHED 


## Default reject non-defined stuff
#-A FORWARD -j REJECT --reject-with icmp-host-prohibited
-A INPUT -j REJECT --reject-with icmp-host-prohibited

COMMIT

# Completed on Thu Aug  8 22:43:38 2013
# Generated by iptables-save v1.4.7 on Thu Aug  8 22:43:38 2013
*nat
:PREROUTING ACCEPT [2]
:POSTROUTING ACCEPT [0]
:OUTPUT ACCEPT [1]

## Add your ports that live on the FreeBSD server here
-A PREROUTING -p tcp -m tcp --dport 80 -m state --state NEW,RELATED,ESTABLISHED -j DNAT --to-destination 10.0.0.1:80 
-A PREROUTING -p tcp -m tcp --dport 27960-m state --state NEW,RELATED,ESTABLISHED -j DNAT --to-destination 10.0.0.1:27960
-A PREROUTING -p udp -m udp --dport 80 -m state --state NEW,RELATED,ESTABLISHED -j DNAT --to-destination 10.0.0.1:80 
-A PREROUTING -p udp -m udp --dport 27960 -m state --state NEW,RELATED,ESTABLISHED -j DNAT --to-destination 10.0.0.1:27960

-A POSTROUTING -j MASQUERADE 
COMMIT
# Completed on Thu Aug  8 22:43:38 2013
```

My `ifconfig` on FreeBSD (IPv6 replaced with 0s):

```
# ifconfig
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
        ether 00:1b:24:5b:eb:7c
        inet x.x.x.x netmask 0xfffffffc broadcast x.x.x.x
        inet6 0000::000:0000:0000:0000%em0 prefixlen 64 scopeid 0x1
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
em1: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
        ether 00:1b:24:5b:eb:7d
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: no carrier
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 0000::1%lo0 prefixlen 64 scopeid 0x8
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=141<UP,RUNNING,PROMISC> metric 0 mtu 33152
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
gre0: flags=9051<UP,POINTOPOINT,RUNNING,LINK0,MULTICAST> metric 0 mtu 1476
        tunnel inet 1x.x.x.x --> y.y.y.y
        inet 10.0.0.1 --> 10.0.0.2 netmask 0xfffffffc
        inet6 fe80::21b:24ff:fe5b:eb7c%gre0 prefixlen 64 scopeid 0xa
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
```

My `ifconfig` on CentOS:

```
# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:16:3C:B9:49:E7
          inet addr:y.y.y.y  Bcast:y.y.y.y  Mask:255.255.255.255
          inet6 addr: 0000::000:0000:000:0000/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:45984 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8182 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:3519237 (3.3 MiB)  TX bytes:2175342 (2.0 MiB)
          Interrupt:11 Base address:0x2000

gre5      Link encap:UNSPEC  HWaddr C6-37-6A-0D-00-00-E1-8E-00-00-00-00-00-00-00-00
          inet addr:10.0.0.2  P-t-P:10.0.0.2  Mask:255.255.255.252
          UP POINTOPOINT RUNNING NOARP  MTU:1476  Metric:1
          RX packets:1198 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1376 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1037684 (1013.3 KiB)  TX bytes:131590 (128.5 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:1 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:112 (112.0 b)  TX bytes:112 (112.0 b)
```


----------



## roman90731 (Aug 9, 2013)

I've noticed that when the server is running I can telnet from CentOS to FreeBSD but not the other direction.


----------



## junovitch@ (Aug 15, 2013)

Can you post the FreeBSD /etc/rc.conf and CentOS /etc/sysconfig/ifcfg-xxx bits as well?


----------



## junovitch@ (Aug 16, 2013)

The main reason I ask is because of `ifconfig` on CentOS

```
inet addr:10.0.0.2  P-t-P:10.0.0.2  Mask:255.255.255.252
```

It doesn't make sense that both are showing the same address and I need to see more of the picture to understand your setup. 


Additionally, your firewall rules need work.  For example.

```
## Add your ports you want to accept
-A FORWARD -p tcp -m tcp --dport 80 -m state --state NEW,RELATED,ESTABLISHED 
-A FORWARD -p tcp -m tcp --dport 27960 -m state --state NEW,RELATED,ESTABLISHED 
-A FORWARD -p udp -m udp --dport 80 -m state --state NEW,RELATED,ESTABLISHED 
-A FORWARD -p udp -m udp --dport 27960-m state --state NEW,RELATED,ESTABLISHED
```

You need to add the target to jump (-j ACCEPT).  Leaving it empy means per `man iptables` that means "matching the rule will have no effect on the packet's fate, but the counters on the rule will be incremented".


```
-A PREROUTING -p tcp -m tcp --dport 80 -m state --state NEW,RELATED,ESTABLISHED -j DNAT --to-destination 10.0.0.1:80 
-A PREROUTING -p tcp -m tcp --dport 27960-m state --state NEW,RELATED,ESTABLISHED -j DNAT --to-destination 10.0.0.1:27960
-A PREROUTING -p udp -m udp --dport 80 -m state --state NEW,RELATED,ESTABLISHED -j DNAT --to-destination 10.0.0.1:80 
-A PREROUTING -p udp -m udp --dport 27960 -m state --state NEW,RELATED,ESTABLISHED -j DNAT --to-destination 10.0.0.1:27960
```

I would recommend adding an -i eth0 so that your CentOS box doesn't send packets right back to where they came from.  As of right now any packets leaving the FreeBSD box destined to port 80 and 27960 will be reflected right back because of the DNAT.  This is assuming my guess is correct that only your eth0 side hosts should be forwarded through the tunnel.

Lastly, did you enable routing in CentOS's /etc/sysctl.conf?

```
net.ipv4.ip_forward = 1
```


----------



## roman90731 (Aug 18, 2013)

junovitch said:
			
		

> Can you post the FreeBSD /etc/rc.conf and CentOS /etc/sysconfig/ifcfg-xxx bits as well?




```
ifconfig_em0="inet x.x.x.x netmask 255.255.255.252"
defaultrouter="x.x.x.x"
hostname="nc12.censored.com"
sshd_enable="YES"
fsck_y_enable="YES"
named_enable="YES"
ntpdate_flags="europe.pool.ntp.org"
ntpdate_enable="YES"
ntpd_enable="YES"
syslogd_flags="-s -b 127.0.0.1"
powerd_enable="NO"
#powerd_options="-d adaptive -p adaptive -n adaptive"
zfs_enable="YES"
#firewall_enable="YES"
#firewall_script="/etc/ipfw.rules"
#firewall_logging="YES"
apache22_enable="YES"
```

Sorry, I couldn't find the file you requested on the CentOS machine.

And no, I didn't add forwarding to the sysctl.conf file. I actually paid the host to set this up for me and they failed, then I had a friend set up on FreeBSD and CentOS but only those two ports worked. The host still failed to help me set it up and basically ripped me off. The entire idea was the host's to use a tunnel like this, knowing I was using FreeBSD which they consider "crap" and claim to be inexperienced in. I guess the host thinks whatever they don't understand is crap...

Anyways, I gave up on this as I was tired of fooling with it and wanted to forget the fact I wasted $35 for the CentOS server and the setup fee without a positive result. I didn't receive an answer until 6 days after I posted the thread, so I figured nobody knew what the problem was. I do appreciate that you took the time to answer me and try to help me out. Maybe later I can get this same setup going on two FreeBSD machines with the new host I plan to move to.

Thanks again.


----------



## junovitch@ (Aug 18, 2013)

Sorry your plans didn't work out.  This is a FreeBSD forum so I would expect a little bit less than usual response when it comes to multi-vendor questions.  I'm an RHCSA for work and do FreeBSD on home stuff so I figured I could take a stab at it.  The files I was looking for were /etc/sysconfig/network-scripts/ifcfg-xxx where xxx is the interface.


----------



## roman90731 (Aug 19, 2013)

junovitch said:
			
		

> Sorry your plans didn't work out.  This is a FreeBSD forum so I would expect a little bit less than usual response when it comes to multi-vendor questions.  I'm an RHCSA for work and do FreeBSD on home stuff so I figured I could take a stab at it.  The files I was looking for were /etc/sysconfig/network-scripts/ifcfg-xxx where xxx is the interface.



Yeah, it's understandable. Here are the contents of the files you requested:

/etc/sysconfig/network-scripts/ifcfg-eth0

```
DEVICE=eth0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=dhcp
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME="System eth0"
```

/etc/sysconfig/network-scripts/ifcfg-lo

```
DEVICE=lo
IPADDR=127.0.0.1
NETMASK=255.0.0.0
NETWORK=127.0.0.0
# If you're having problems with gated making 127.0.0.0/8 a martian,
# you can change this to something else (255.255.255.255, for example)
BROADCAST=127.255.255.255
ONBOOT=yes
NAME=loopback
```


----------

