# FreeBSD Policy Based Routing with ipfw nat + fwd using 2 or more Poor Man's ssh VPNs



## lucdig (Apr 20, 2019)

Hello, thanks to the posts that I found on this forum, I could implement a gateway in FreeBSD that allows me to do flexible policy routing through different interfaces.

I'm going to share.

System: FreeBSD freebsd 12.0-STABLE FreeBSD 12.0-STABLE r346132 *NEWKERNEL  *amd64

*NEWKERNEL *compiled with:

```
include GENERIC
ident NEWKERNEL

options IPFIREWALL
options IPFIREWALL_DEFAULT_TO_ACCEPT
options IPFIREWALL_NAT
options LIBALIAS
```


/etc/rc.conf: `gateway_enable="YES"`
/etc/sysctl.conf: `net.inet.ip.fw.one_pass=0` _ipfw doesn’t exit at the first match, it continues processing to the next rule_

Network Interface: `ifconfig hn0`
`<local_ip_hn0> = 172.19.167.89 (FreeBSD)
<local_netmask_hn0> = 0xfffffff0 = 255.255.255.240 = 28`


Routing: `netstat -nr4`

```
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            172.19.167.81      UGS         hn0
127.0.0.1          link#1             UH          lo0
172.19.167.80/28   link#2             U           hn0
172.19.167.89      link#2             UHS         lo0
```


Initial ipfw state: `ipfw list`

```
65535 allow ip from any to any
```


Using 2 ssh connections to setup 2 tun interfaces. <remote host1> and <remote host2> are Linux, so the commands are ip link set… ip addr add… iptables -t nat…

```
ssh -x -l root -p <remote_port1> -f -T -N -C -M -S /tmp/test100 -o Tunnel=yes -w 100:100 <remote_host1>
ssh -x -l root -p <remote_port1> -T -S /tmp/test100 <remote_host1> 'ip link set dev tun100 up'
ssh -x -l root -p <remote_port1> -T -S /tmp/test100 <remote_host1> 'ip addr add 10.100.100.2/32 peer 10.100.100.1/32 dev tun100'
ssh -x -l root -p <remote_port1> -T -S /tmp/test100 <remote_host1> 'iptables -t nat -A POSTROUTING -s 10.100.100.1/32 -j MASQUERADE'
ifconfig tun100 10.100.100.1/32 10.100.100.2

ssh -x -l root -p <remote_port2> -f -T -N -C -M -S /tmp/test99 -o Tunnel=yes -w 99:99 <remote_host2>
ssh -x -l root -p <remote_port2> -T -S /tmp/test99 <remote_host2> 'ip link set dev tun99 up'
ssh -x -l root -p <remote_port2> -T -S /tmp/test99 <remote_host2> 'ip addr add 10.99.99.2/32 peer 10.99.99.1/32 dev tun99'
ssh -x -l root -p <remote_port2> -T -S /tmp/test99 <remote_host2> 'iptables -t nat -A POSTROUTING -s 10.99.99.1/32 -j MASQUERADE'
ifconfig tun99 10.99.99.1/32 10.99.99.2
```


Configuring nat on tun99 and tun100:

```
ipfw nat 100 config if tun100
ipfw nat 99 config if tun99
```


Adding ipfw rules to nat-and-route *HTTP generated by FreeBSD to tun99* and *HTTPS generated by FreeBSD to tun100*, nat-and-route* icmp traffic generated by external host to 8.8.8.8 through tun100*:

```
00001 skipto 65534 tcp from <local_ip_hn0> to < remote_host1> < remote_port1> out
00001 skipto 65534 tcp from <local_ip_hn0> to < remote_host2> < remote_port2> out
00002 skipto 65534 tcp from < remote_host1> < remote_port1> to <local_ip_hn0> in
00002 skipto 65534 tcp from < remote_host2> < remote_port2> to <local_ip_hn0> in
00003 skipto 65534 ip from 172.19.167.80/28 to 172.19.167.80/28

00010 skipto 61100 tcp from 172.19.167.89 to any 443 out
00011 skipto 60099 tcp from 172.19.167.89 to any 80 out
00012 skipto 61100 icmp from 172.19.167.82 to 8.8.8.8 out

40099 nat 99 ip from any to any in via tun99
40100 nat 100 ip from any to any in via tun100

50000 skipto 65534 ip from any to any

60099 nat 99 ip from any to any out
60100 fwd 10.99.99.2 ip from any to any out
60101 skipto 65534 ip from any to any

61100 nat 100 ip from any to any out
61101 fwd 10.100.100.2 ip from any to any out
61102 skipto 65534 ip from any to any

65535 allow ip from any to any
```


Rules 1 and 2: *don’t change the routing for the 2 ssh connections*:

```
00001 skipto 65534 tcp from <local_ip_hn0> to < remote_host1> < remote_port1> out
00001 skipto 65534 tcp from <local_ip_hn0> to < remote_host2> < remote_port2> out
00002 skipto 65534 tcp from < remote_host1> < remote_port1> to <local_ip_hn0> in
00002 skipto 65534 tcp from < remote_host2> < remote_port2> to <local_ip_hn0> in
```


Rule 3:* don’t change the routing between hosts of the same local network*:

```
00003 skipto 65534 ip from 172.19.167.80/28 to 172.19.167.80/28
```


Rules 40XXX: packets coming in via tun99 and tun100 must be processed by the nat instances, *checking if some entry exists in the internal nat table*.

```
40099 nat 99 ip from any to any in via tun99
40100 nat 100 ip from any to any in via tun100
```


Rule 50000: after nat for the incoming packets via tun99 or tun100, no more processing is needed, *skipto allow*

```
50000 skipto 65534 ip from any to any
```


Rules 60XXX: *nat *through interface tun99, *forward *to the peer ip address of tun99, *skipto allow*

```
60099 nat 99 ip from any to any out
60100 fwd 10.99.99.2 ip from any to any out
60101 skipto 65534 ip from any to any
```


Rules 61XXX: *nat *through interface tun100, *forward *to the peer ip address of tun100, *skipto allow*

```
61100 nat 100 ip from any to any out
61101 fwd 10.100.100.2 ip from any to any out
61102 skipto 65534 ip from any to any
```


Rules 10, 11: traffic generated by 172.19.167.89 - outgoing HTTP to tun99, outgoing HTTPS to tun100

```
00010 skipto 61100 tcp from 172.19.167.89 to any 443 out
00011 skipto 60099 tcp from 172.19.167.89 to any 80 out
```


Rule 12: icmp traffic generated by 172.19.167.82 (172.19.167.82 has FreeBSD 172.19.167.89 as default gateway) to 8.8.8.8 is nat-ed and sent through interface tun100

```
00012 skipto 61100 icmp from 172.19.167.82 to 8.8.8.8 out
```

You can add ipfw rules for flexible Policy Routing between rule 3 and 40099. Be careful with the order of the rules, because net.inet.ip.fw.one_pass=0.

Hope it helps, enjoy


----------

