# Wireguard Triple NAT Route Out



## kane.nod (Nov 24, 2020)

First of all hi everyone!

I have an intentional VPN chain between 3 nodes, they are all connected and I can ping and mtr from 10.0.0.2 to 10.0.2.1 however its not passing any connections from the 10.0.X.X network to the external network on node 3. I am wondering what type of routing I need to implement to make it work.

The idea here was a multi-hop wireguard config. If i can fix the outbound routing this setup is complete.

--pf.conf--

```
ext_if="wg1"
int_if="wg0"
set skip on lo0
scrub in all
nat on $ext_if from $int_if:network to any -> ($ext_if)
pass from { lo0, $int_if:network } to any keep state
```

--rc.conf-section

```
gateway_enable="YES"
pf_enable="YES"
pf_rules="/etc/pf.conf"
wireguard_enable="YES"
wireguard_interfaces="wg0 wg1"
```
For Node 3 the wg1 is not present, otherwise the pf.conf/rc.conf are identical and net.inet.ip.forwarding=1

Node 1

--wg0.conf--

```
[Interface]
Address = 10.0.0.1/24
ListenPort = 8888
PrivateKey = <KEYREMOVED>

[Peer]
PublicKey = <KEYREMOVED>
PresharedKey = <KEYREMOVED>
AllowedIPs = 10.0.0.2/32
```

--wg1.conf--

```
[Interface]
Address = 10.0.1.2/24
ListenPort = 2222
PrivateKey = <KEYREMOVED>
DNS = 1.1.1.1
Table = off
PostUp = route -q -n add -inet 10.0.0.0/16 -interface wg1
PostUp = pfctl -F all -f /etc/pf.conf
PostUp = ping -c 1 10.0.1.1
PostDown = route -q -n delete -inet 10.0.0.0/16 -interface wg1
PostDown = pfctl -F all

[Peer]
PublicKey = <KEYREMOVED>
PresharedKey = <KEYREMOVED>
AllowedIPs = 0.0.0.0/0, ::/0
Endpoint = <IPREMOVED>:2222
```
Node 2

--wg0.conf--

```
[Interface]
Address = 10.0.1.1/24
ListenPort = 2222
PrivateKey = <KEYREMOVED>

[Peer]
PublicKey = <KEYREMOVED>
PresharedKey = <KEYREMOVED>
AllowedIPs = 10.0.1.2/32
```
--wg1.conf--

```
[Interface]
Address = 10.0.2.2/24
ListenPort = 3333
PrivateKey = <KEYREMOVED>
DNS = 1.1.1.1
Table = off
PostUp = route -q -n add -inet 10.0.0.0/16 -interface wg1
PostUp = pfctl -F all -f /etc/pf.conf
PostUp = ping -c 1 10.0.2.1
PostDown = route -q -n delete -inet 10.0.0.0/16 -interface wg1
PostDown = pfctl -F all

[Peer]
PublicKey = <KEYREMOVED>
PresharedKey = <KEYREMOVED>
AllowedIPs = 0.0.0.0/0, ::0
Endpoint = <IPREMOVED>:3333
```
Node 3

--wg0.conf--

```
[Interface]
Address = 10.0.2.1/24
ListenPort = 3333
PrivateKey = <KEYREMOVED>

PostUp = pfctl -F all -f /etc/pf.conf
PostUp = ping -c 1 -S 10.0.2.1 1.1.1.1
PostDown = pfctl -F all

[Peer]
PublicKey = <KEYREMOVED>
PresharedKey = <KEYREMOVED>
AllowedIPs = 10.0.2.2/32
```


----------



## SirDice (Nov 24, 2020)

kane.nod said:


> however its not passing any connections from the 10.0.X.X network to the external network on node 3.


Look with tcpdump(1). I'm betting there are connections going to the external network but that external network doesn't know where to route the return traffic to. Routing has to be correctly configured to work both ways.


----------



## kane.nod (Nov 25, 2020)

Trying a different approach. bridge0 doesn't work with wg0 wg1.

How can I switch wg0 and wg1 (route iface to iface?)


----------



## VladiBG (Nov 25, 2020)

Create only one VPN tunnel interface and put all clients as peers to it. Then use the routing instead of NAT.

Server:

```
[Interface]

Address = 10.0.0.1/24
ListenPort = 8888
PrivateKey = <KEYREMOVED>

[Peer]

PublicKey = <KEYREMOVED>
PresharedKey = <KEYREMOVED>
Endpoint = <Client 1 Public IP>:<PORT>
AllowedIPs = 10.0.0.2/32

[Peer]

PublicKey = <KEYREMOVED>
PresharedKey = <KEYREMOVED>
Endpoint = <Client 2 Public IP>:<PORT>
AllowedIPs = 10.0.0.3/32

[Peer]

PublicKey = <KEYREMOVED>
PresharedKey = <KEYREMOVED>
Endpoint = <Client 3 Public IP>:<PORT>
AllowedIPs = 10.0.0.4/32
```

Client1

```
[Interface]
PrivateKey = XXX
Address = 10.0.0.2/32

[Peer]
PublicKey = XXX
AllowedIPs = 10.0.0.0/24
PersistentKeepalive = 30
```

Client2

```
[Interface]
PrivateKey = XXX
Address = 10.0.0.3/32

[Peer]
PublicKey = XXX
AllowedIPs = 10.0.0.0/24
PersistentKeepalive = 30
```

Client3

```
[Interface]
PrivateKey = XXX
Address = 10.0.0.4/32

[Peer]
PublicKey = XXX
AllowedIPs = 10.0.0.0/24
PersistentKeepalive = 30
```










						GitHub - pirate/wireguard-docs: 📖 Unofficial WireGuard Documentation: Setup, Usage, Configuration, and full example setups for VPNs supporting both servers & roaming clients.
					

📖 Unofficial WireGuard Documentation: Setup, Usage, Configuration, and full example setups for VPNs supporting both servers & roaming clients. - GitHub - pirate/wireguard-docs: 📖 Unofficial Wir...




					github.com


----------



## kane.nod (Dec 1, 2020)

I solved the problem.

For reference to anyone looking to do multi-hop effectively without too much complexity:

NginX stream module / upstream was used on each hop before the regular wireguard VPN config simply forwarding udp packets to the VPN port. The issue was checksum features stuttering the connection. This is also a solution for Wireguard Download speed being slower than Upload speed.

Once established properly (10-15 seconds) this connection feeds 35-50 Mbit/s on a 100Mbit/s port.

`ifconfig ix0 xxx.xxx.xxx.xxx netmask 255.255.255.xxx -txcsum -rxcsum -tso -lro -vlanhwtso`

nginx.conf

```
load_module /usr/local/libexec/nginx/ngx_stream_module.so;

user  www;
worker_processes  1;

events {
    worker_connections  1024;
}

# NODE-FORWARD1
stream {
    server {
        listen <IPREMOVED>:8888 udp;
        proxy_pass w;
    }
    upstream w {
        server <IPREMOVED>:8888;
    }
}
```


----------

