# Fallback to secondary gateway on ICMP Destination Unreachable



## alecdotninja (Mar 25, 2019)

Hello,

I apologize in advance if this question has already been answered. I did some Googling and searching of the form, but I'm new enough to FreeBSD that I doubt I am searching for the correct terms.

For cost reasons, I am operating a Tor exit node in a data center that is single-homed on Cogent. Unfortunately, Cogent does not have routes to the full IPv6 internet (most notably Google).

I have been able to restore access to Google by using static routes and tunnelbroker.net from HE, but this is an awkwardly manual process.

What I would really like is to attempt to send all IPv6 traffic to the Cogent gateway and fallback to the HE gateway when I receive ICMP Destination Unreachable from Cogent.


```
14:30:57.771735 5c:45:27:dc:8b:c1 (oui Unknown) > 00:25:90:0d:d1:18 (oui Unknown), ethertype IPv6 (0x86dd), length 118: (hlim 62, next-header ICMPv6 (58) payload length: 64) 2001:550:2:2a::2f:1 > 2001:49f0:d03f:1::1337: [icmp6 sum ok] ICMP6, destination unreachable, unreachable route he.net
```

Is something like this possible with FreeBSD (without getting BGP involved)?

For reference, here is my current networking configuration:


```
ifconfig_igb0="inet 198.255.44.98 netmask 255.255.255.248"
defaultrouter="198.255.44.97"
ifconfig_igb0_ipv6="inet6 2001:49f0:d03f:1::1337/64"

gif_interfaces="gif0"
gifconfig_gif0="198.255.44.98 184.105.250.46"
ifconfig_gif0_ipv6="inet6 2001:470:39:76e::2 2001:470:39:76e::1 prefixlen 128 deprecated"
ifconfig_gif0_alias0_ipv6="inet6 2001:470:4b:76e::1337/64"

ipv6_static_routes="google1 google2 google3 google4 google5 google6 google7 google8 google9 google10 google11 google12 google13 google14"
ipv6_route_google1="2001:4860:: -prefixlen 32 2001:470:39:76e::1"
ipv6_route_google2="2401:fa00:: -prefixlen 32 2001:470:39:76e::1"
ipv6_route_google3="2404:6800:: -prefixlen 32 2001:470:39:76e::1"
ipv6_route_google4="2600:1900:: -prefixlen 28 2001:470:39:76e::1"
ipv6_route_google5="2605:ef80:: -prefixlen 32 2001:470:39:76e::1"
ipv6_route_google6="2607:f8b0:: -prefixlen 32 2001:470:39:76e::1"
ipv6_route_google7="2620:0:1000:: -prefixlen 40 2001:470:39:76e::1"
ipv6_route_google8="2620:120:e000:: -prefixlen 40 2001:470:39:76e::1"
ipv6_route_google9="2620:15c:: -prefixlen 36 2001:470:39:76e::1"
ipv6_route_google10="2800:3f0:: -prefixlen 32 2001:470:39:76e::1"
ipv6_route_google11="2a00:1450:: -prefixlen 32 2001:470:39:76e::1"
ipv6_route_google12="2a00:79e0:: -prefixlen 32 2001:470:39:76e::1"
ipv6_route_google13="2a03:ace0:: -prefixlen 32 2001:470:39:76e::1"
ipv6_route_google14="2c0f:fb50:: -prefixlen 32 2001:470:39:76e::1"

ipv6_defaultrouter="2001:49f0:d03f:1::1"
```

Thank you in advance for any help that you are willing to provide!

-- Alec


----------



## alecdotninja (Mar 27, 2019)

I'm sorry to bump this thread, but in case there is anyone else out there with this issue, I want to document a workaround that I am using now.

I had hoped to be able to get this working with pf but there are two problems there I wasn't able to solve:

Dynamically adding missing networks/prefixes to a table based on incoming ICMP6 packets
Dynamically routing traffic to another gateway based on the destination's address being in a table
Instead, I've hacked together a small script that listens for the correct ICMP6 messages and adds the static routes dynamically:

```
#!/usr/bin/env ruby
# frozen_string_literal: true

EDGE_ROUTER = "2001:550:2:2a::2f:1"
ALT_GATEWAY = "2001:470:39:76e::1"

# Redirect missing routes (according to EDGE_ROUTER) to ALT_GATEWAY.
each_unreachable_route(EDGE_ROUTER) do |network, prefix_length|
  add_static_route(network, prefix_length, ALT_GATEWAY)
end

# If we made it this far, something went wrong; exit with EXIT_FAILURE.
Kernel.exit(false)

# Setup some abstractions so the code can be ｓｅｍａｎｔｉｃ.
BEGIN {
  UNREACHABLE_ROUTE_FILTER = <<~FILTER
    src %{router_ip} &&
    icmp6 &&
    icmp6[icmptype] == 1 &&
    icmp6[icmpcode] == 0
  FILTER

  UNREACHABLE_ROUTE_LINE_REGEXP = %r{
    ^
    (?<timestamp>[\d:\.]+)\s
    IP6\s
    (?<source>[\da-f:]+)\s>\s(?<destination>[\da-f:]+):\s
    ICMP6,\s
    destination\sunreachable,\s
    unreachable\sroute\s(?<unreachable_route>[\da-f:]+),\s
    length\s(?<length>\d+)
    $
  }x

  def each_unreachable_route(router_ip)
    filter = UNREACHABLE_ROUTE_FILTER % { router_ip: router_ip }

    IO.popen(["tcpdump", "-l", "-nn", "--", filter], in: File::NULL) do |stdout|
      stdout.each do |line|
        parsed_line = UNREACHABLE_ROUTE_LINE_REGEXP.match(line)

        unless parsed_line
          warn "Skipping unexpected output from tcpdump: #{line}"

          next
        end

        yield parsed_line[:unreachable_route],
              parsed_line[:length]
      end
    end
  end

  def add_static_route(network, prefix_length, gateway)
    system "route",
           "-6", "add",
           "-net", network,
           "-prefixlen", prefix_length,
           gateway
  end
}
```

The biggest problem with this approach is that it takes two tries to connect to a missing route. There's also currently no mechanism that expires the routes.

If anyone knows how to make this work with pf or has any suggestions for improvements, I would appreciate the feedback. Otherwise, I plan to use this for the time being.


----------

