# Strongswan + IPv6



## Veeny (Jun 23, 2021)

Folks,
this is not an emergency, but I'm quite stumped, so I'd like to know if anyone here has a clue.

Here is my problem:
My IPv6 (let's assume I have a /64 prefix ‘P’) is set like this: the fibre modem, which is also the default router, has address P::1; it is connected by a short patch cord to my FreeBSD server, on interface re0, address P::2/112.
The server has another net interface, ale0, which every machine on the local network is connected to (this allow the server to act as a NAT in IPv4, DHCP server and some other services). ale0 has the address P::1:0/96
Connecting re0 to ale0 is bridge0, whose address is P:ffff::/128.

The server also run strongswan and serves as a VPN peer for ‘road warriors’ machines. The configuration seems to be correct: IPv4 VPN works correctly, and the remote machine also gets an IPv6 address from a pool P:0:1::/96, distinct from re0 and ale0 ranges. leftsubnet is ::/0 so all IPv6 traffic is routed through the VPN.

Now the problem is that I have no outside IPv6 connectivity from VPN clients. I can ping6 the server (either P::2 or P::1:0), the machines on the ale0 interface, but nothing seems to get through from re0. That is to say, a tcpdump -i re0 icmp6 shows ICMP6 requests sent, but the fibre modem responds with ICMP6 NS to (let's assume the address of a VPN client is) P:0:1:0:1, and gets no response, so I assume the ICMP6 response packets are just lost there. I can't even ping the fibre modem (P::1).

Should I set a rule for the bridge not to forward to ale0 packets sent to the VPN pool? If yes, how? Any other idea that could make the VPN IPv6 outside connectivity work?

Thanks a bunch!
V

EDIT: Oh, I forgot to say I have rtadvd running on ale0, advertising the fibre modem as the router.


----------



## covacat (Jun 23, 2021)

you have to proxyarp the warriors ?


----------



## SirDice (Jun 23, 2021)

covacat said:


> you have to proxyarp the warriors ?


IPv6 doesn't use ARP.



Veeny said:


> My IPv6 (let's assume I have a /64 prefix ‘P’) is set like this: the fibre modem, which is also the default router, has address P::1; it is connected by a short patch cord to my FreeBSD server, on interface re0, address P::2/112.
> The server has another net interface, ale0, which every machine on the local network is connected to (this allow the server to act as a NAT in IPv4, DHCP server and some other services). ale0 has the address P::1:0/96
> Connecting re0 to ale0 is bridge0, whose address is P:ffff::/128.


This looks like a giant entangled mess of different prefix-lengths. Everything is supposed to be on the same /64 (you're bridging everything).


----------



## covacat (Jun 23, 2021)

SirDice said:


> IPv6 doesn't use ARP.


then it has to advertise a route for the warriors on re0
modem has to know how to reach them (and they are not directly reachable)


----------



## SirDice (Jun 23, 2021)

covacat said:


> modem has to know how to reach them (and they are not directly reachable)


Like I said, it's a big entangled mess of different prefix-lengths when everything is supposed to be on the same layer 2 network. I'm actually surprised it works at all. You can't just cut up an IPv6 prefix in the same way you would do with IPv4, it doesn't work that way.


----------



## Veeny (Jun 23, 2021)

I'm no past master at IPv6, but yeah, IPv6 uses a combination of ‘link local addresses’ which contain the MAC addresses of the hosts and special packets such as NS to find out the correspondence between MAC and IPv6 addresses without using ARP.

I'm using different prefixes because I don't need all the /64, and I wish to keep my IPv6 readable, and simplify routing. TBH, I'm not sure why the standard policy is to allocate /64 to clients, where /96 would be more than a thousand times sufficient in many cases. But that's beside the point.

Fact is, if I don't create the bridge, it doesn't work. I can't get FreeBSD to properly forward packets from ale0 to re0 and vice-versa. inet6.gateway is set to 1, rtadvd runs, so I'm a bit at a loss.



SirDice said:


> Like I said, it's a big entangled mess of different prefix-lengths when everything is supposed to be on the same layer 2 network. I'm actually surprised it works at all. You can't just cut up an IPv6 prefix in the same way you would do with IPv4, it doesn't work that way.


Do you think I should use a /64 prefix everywhere?


----------



## SirDice (Jun 23, 2021)

The FreeBSD server (with re0, ale0 and the bridge(4)) and everything connected on the LAN (via ale0) are all on the same network (layer 2), so it doesn't make sense to split it up on layer 3.



Veeny said:


> TBH, I'm not sure why the standard policy is to allocate /64 to clients,


That's pretty much standard. Either a /64 or a /48. The /48 can be split up into multiple /64 networks. rtadvd(8) works with /64 by default too. 



Veeny said:


> Fact is, if I don't create the bridge, it doesn't work.  rtadvd(8) works with /64 by default too.  I can't get FreeBSD to properly forward packets from ale0 to re0 and vice-versa.


When you bridge everything you're basically putting everything on the same network (at layer 2). Without the bridge you will need to use subnets (IP routing) and you're going to need to enable routing (`gateway_enable` and `ipv6_gateway_enable`).


----------



## Veeny (Jun 23, 2021)

SirDice said:


> When you bridge everything you're basically putting everything on the same network (at layer 2). Without the bridge you will need to use subnets (IP routing) and you're going to need to enable routing (`gateway_enable` and `ipv6_gateway_enable`).


Yeah, I agree wholeheartedly with that. The problem is, when I ‘dismantle’ the bridge, the machines on ale0 cannot reach the outside world anymore. Forwarding from ale0 to re0 seems to work fine, but once again I get those pesky ICMP6 NS packets on re0 back from my default router asking who has whatever IPv6 address is pinging the outside, no one answers to those, and the packets get chucked.

I've setup rtadvd on re0 in order to make it advertise for the full /64 range. No change. This lead me to suspect something might be wrong in my fibre router. Normally, if rtadvd correctly sends RA packets on re0 specifying the server handles all the allowed ipv6 range, the next hop (fibre router) should automatically forward any packet to it and not emit and NS packet, right?

EDIT: here are the relevant lines in rtadvd.conf:


```
re0:\
        :maxinterval=15:\
        :addr="P::2":prefixlen#64:\
        :rtprefix="P::":rtlen#64:\
        :raflags="mo":
```


----------



## SirDice (Jun 23, 2021)

Any chance you can set up the fibre "modem" in bridge mode? I assume it's configured as a router (it spits out a RFC1918 network on your side). Then you can connect re0 directly to the modem and get your _external_ IP there. Then it's just a simple case of configuring the FreeBSD host for the routing, firewalling, NAT, DHCP, DNS and whatever else you need for the network attached to ale0. (I have a similar construct but using a cable modem in bridge mode).


----------



## Veeny (Jun 23, 2021)

SirDice said:


> Any chance you can set up the fibre "modem" in bridge mode? I assume it's configured as a router (it spits out a RFC1918 network on your side). Then you can connect re0 directly to the modem and get your _external_ IP there. Then it's just a simple case of configuring the FreeBSD host for the routing, firewalling, NAT, DHCP, DNS and whatever else you need for the network attached to ale0. (I have a similar construct but using a cable modem in bridge mode).


Unfortunately I can't get inside that modem. It's being installed by my ISP and they don't relinquish any kind of control on it (apart from the on/off switch).
The reason is that the fibre not only carries Internet data traffic, but also VoIP, and the modem muxes/demuxes the different services.


----------



## covacat (Jun 23, 2021)

try ndp -ns ip6_of_warrior mac_of_re0 proxy


----------



## SirDice (Jun 23, 2021)

Veeny said:


> Unfortunately I can't get inside that modem. It's being installed by my ISP and they don't relinquish any kind of control on it (apart from the on/off switch).


Yeah, they tend to do that. I seriously hate that. Can't do anything on my cable modem either, switching between routing and bridge mode is all the configuration I can do on it. At least they let me switch it to bridge mode. 



Veeny said:


> The reason is that the fibre not only carries Internet data traffic, but also VoIP, and the modem muxes/demuxes the different services.


As does my cable modem. It also has wifi built-in. In bridge mode the phone (VoIP) still works but I can't use the wifi of the cable modem anymore. I don't need the wifi on the cable modem as I have my own AP.


----------



## Veeny (Jun 23, 2021)

SirDice said:


> Yeah, they tend to do that. I seriously hate that. Can't do anything on my cable modem either, switching between routing and bridge mode is all the configuration I can do on it. At least they let me switch it to bridge mode.


I mean, I can reasonably imagine they don't want ‘users’, even ‘power users’ to mess up with their hardware, but you’re right, they could at least let us access some basic configuration parameters.



covacat said:


> try ndp -ns ip6_of_warrior mac_of_re0 proxy


Thanks for 'ndp', a command I wasn’t aware of. Very useful.
So, at this point I turned off the bridge.
I have kern.features.inet6: 1 and net.inet6.ip6.forwarding: 1, plus "ipv6_gateway_enable" in /etc/rc.conf, which I believe is enough at the kernel level.

When I try to ping6 freebsd.org from an ale0 machine, nothing happens. The ICMP echo packet is correctly forwarded from ale0 to re0, but I get the same behaviour on re0 with one unanswered NS query for each ICMP6 echo packet.

More baffling, ndp -a shows a correct entry for the ale0 machine, so the v6 address is correctly registered in the v6 routing table, but the server doesn't answer the NS request for that machine on re0.

Question is: is it because the machine is on ale0 and the query arrives on re0? In other words, there's no proxying of v6 addresses across networks?


----------



## covacat (Jun 23, 2021)

run everything with /64,  leave bridge on
remove ip6 from ale0
then your lan computers should be able to use inet6
you may get some icmp redirects if ale0 connected devices use freebsd box as default route
when this part works try to play with ndp to  make the road warriors work


----------



## Veeny (Jun 23, 2021)

Really at this point I wonder if it's not the v6 stack in the modem which is broken. 

As matter of fact, I can ping6 my home machine from an ale0 machine, it's just that the ICMP6 response cannot get further than the modem. So the IPv6 gateway works. It is as if the modem ignores the RA messages sent on re0.

Unfortunately I have no other machine connected on re0, so I cannot really test that hypothesis. I'll set the bridge up again, and next time I go to my office I'll try and reset the modem and see what happens.


----------



## Veeny (Jul 17, 2021)

I'm still stuck with this:

If I configure my re0 interface with *prefix::/64*, ale0 with *prefix::1:0/112* and no bridge, the VPN clients (*prefix::4:0/112*) can ping the ale0 clients, but nothing flows to re0 (neither the VPN clients, nor the machines on the local network can talk to the outside world: the packets are correctly sent out, but the replies are not routed – the gateway repeatedly sends NS packets on re0 corresponding to the pinging address to which the server never replies. I've tried to add the v6 addresses manually using ndp(8), but to no avail. The server remains dumb. Sending RA over re0 doesn't work either.

If I set a bridge between re0 and ale0, the local machines on ale0 now can talk to the outside world, as the NS packets are forwarded from re0 to ale0, but the VPN clients are isolated. They can talk to the server which runs strongswan, but to no other machine. Once again, the packets are correctly sent out, but the replies are dropped because of the same unanswered NS packets.

Any idea why the kernel does not NS-proxy for the VPN clients (or the local machines when the bridge isn't up)?

EDIT: I was able to find a workaround using a NAT (!)



> nat on ale0 from $vpn_range_v6 to $int_range_v6 -> $ale0_inet6
> nat on re0 from $vpn_range_v6 to ! $int_range_v6 -> $re0_inet6



With $vpn_range_v6 being the CIDR address of the VPN pool, and $int_range_v6 the CIDR address of the local network.


----------

