# Unable to get a multi-homed gateway to route properly



## decuser (Jun 30, 2020)

I have added a nic to my system and I am trying to use my freebsd 12.1 box as a gateway complete with dhcp and dns. I enabled the gateway in /etc/rc as per handbook, but I'm not able to ping my ISP router through the gateway. Any help is appreciated.

The configuration is as follows:

ISP Router (192.168.1.254) connected to internet.

Freebsd 12.1 Machine:
nic 1 - ue0, 192.168.1.1 (connected to isp router's on 192.168.1.254)
nic 2 - em0, 192.168.55.1 (connected to switch in home office)

Macpro running Mojave:
en0, 192.168.55.31 (dhcp, connected to switch in home office, default gateway is 192.168.55.1)

On Freebsd:
/etc/rc.conf


```
ifconfig_ue0="inet 192.168.1.1/24"
ifconfig_em0="inet 192.168.55.1/24"
defaultrouter="192.168.1.254"
gateway_enable="YES"
```

Ping the ISP router:

```
ping 192.168.1.254
PING 192.168.1.254 (192.168.1.254): 56 data bytes
64 bytes from 192.168.1.254: icmp_seq=0 ttl=64 time=0.960 ms
```
Ping the gateway:

```
ping 192.168.55.1
PING 192.168.55.1 (192.168.55.1): 56 data bytes
64 bytes from 192.168.55.1: icmp_seq=0 ttl=64 time=0.049 ms
```

Display the routing table:

```
netstat -r
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            192.168.1.254      UGS         ue0
localhost          link#2             UH          lo0
192.168.1.0/24     link#3             U           ue0
192.168.1.1        link#3             UHS         lo0
192.168.55.0/24    link#1             U           em0
192.168.55.1       link#1             UHS         lo0
```

Everything looks good to me from the freebsd side, now for the mac...

On Mac:

Ping the gateway:

```
ping 192.168.55.1
PING 192.168.55.1 (192.168.55.1): 56 data bytes
64 bytes from 192.168.55.1: icmp_seq=0 ttl=64 time=0.255 ms
```

Ping the ISP router:

```
ping 192.168.1.254
PING 192.168.1.254 (192.168.1.254): 56 data bytes
Request timeout for icmp_seq 0
...
```

Hmm... that's odd...

Trace the route to the gateway:

```
traceroute 192.168.55.1
traceroute to 192.168.55.1 (192.168.55.1), 64 hops max, 52 byte packets
1  192.168.55.1 (192.168.55.1)  0.586 ms  0.277 ms  0.288 ms
```

Trace the route to the ISP router:

```
traceroute 192.168.1.254
traceroute to 192.168.1.254 (192.168.1.254), 64 hops max, 52 byte packets
1  192.168.55.1 (192.168.55.1)  0.542 ms  0.280 ms  0.281 ms
2  * *
```

So, what's up. Did I miss a configuration setting?


----------



## Jose (Jun 30, 2020)

What does `sysctl net.inet.ip.forwarding` report?


----------



## decuser (Jun 30, 2020)

net.inet.ip.forwarding: 1


----------



## genneko (Jun 30, 2020)

Hi,
I guess ping packets from Mac reached the ISP router but the router cannot send them back because it doesn't have a route to 192.168.55.0/24 subnet.
The router might send them out to the ISP and the packets are dropped there as their destination is a private address.


----------



## decuser (Jul 1, 2020)

genneko That makes sense. I added a cascaded router configuration (told the router about my router and subnet) to the ISP router and the mac responded to pings from the network. But, it won't ping to the internet. That said, the ISP docs claim the cascaded router configuration is for static ips assigned by the ISP, so I'm not sure this is how I ought to configure, but maybe it's progress.


----------



## decuser (Jul 1, 2020)

In thinking this through, I thought I'd have another go at explaining my goals - maybe they're what's out of whack and I need to adjust my expectations.

All I really want to do is situate my FreeBSD server between my ISP's provided Router and my LAN (eventually as a firewall, but for the time being as a simple router):






Then from Lan Device 1or 2, I'd like to be able to ping 8.8.8.8 and get the response back. My understanding from Handbook Gateways and Routes Page was that I'd assign an IP to the ISP Router facing interface in it's network, set the ISP Router IP as the default gateway, assign an IP to the LAN facing interface in its network and enable the gateway in /etc/rc.conf.

Does this sound reasonable?


----------



## decuser (Jul 1, 2020)

OK. After looking around, some more I added the following to /etc/rc.conf:

```
pf_enable="YES"
pf_rules="/etc/pf.conf"
pf_flags=""
```

and then in /etc/pf.conf:

```
ext_if="em0"
int_if="ue0"
set skip on lo
nat on $ext_if inet from !($ext_if) -> ($ext_if:0)
```

This seemed to fix the problem, but danged if I understand why this is needed. I clearly need to read up on NAT.


----------



## decuser (Jul 1, 2020)

Oh, and it doesn't require any hacking on the ISP router!


----------



## genneko (Jul 1, 2020)

Congratulations!  It looks like the ISP router only performs NAT on the packets from its directly connected subnet (192.168.1.0/24).

Another option would be putting the ISP router into bridge mode and making the FreeBSD box the only router for your network. You can avoid double NAT with this but I'm not sure if it's possible.
~


----------



## Jose (Jul 1, 2020)

decuser said:


> This seemed to fix the problem, but danged if I understand why this is needed. I clearly need to read up on NAT.


It's needed because the ISP's router will not perform NAT on packets from the 192.168.55.0/24 network. It will only perform NAT on packets from the 192.168.1.0/24 network. Set up your Freebsd gateway as a bridge, and you'll be able to use the 192.168.1.0/24 network everywhere.


----------

