# Route certaintraffic through VPN tunnel



## twarkie (Apr 1, 2009)

I have access to a VPN tunnel which I set up with openVPN in FreeBSD 7.1. The tunnel itself works perfectly. Now I want to route some traffic into it. I want to start with bittorrent traffic but I can't figure out how to make it's traffic take the tunnel instead of the default route.

Is it possible to make a certain application (i.e. rtorrent) to use a specific interface and/or use a specific route?


----------



## SirDice (Apr 1, 2009)

Set the default gateway to the VPN tunnel. Torrent traffic will get sent all over the world. Hence there's no specific network to base the routing information on. Routing is based on IP addresses, not on applications.


----------



## DutchDaemon (Apr 1, 2009)

You could try binding rtorrent to the tunnel IP in .torrent.rc. That should only expose the VPN network (and gateway) to it, making it its default route.


----------



## twarkie (Apr 2, 2009)

Binding rtorrent to my VPN IP doesn't seem to work, the traffic is still routed through my regular connection.

Any other way to get rtorrent to use the tunnel?


----------



## bobveznat (Apr 3, 2009)

Turns out that the bind IP doesn't have anything to do with routing. The bind IP is the address used in the source field of the IP packet.

The route used is based on the destination address of the IP packet. Notice this is inline with `netstat -r` output, `route add` and even `route get`.

You can get a bit more tricky with routing in FreeBSD 7 (7.1 maybe?) with the multiple routing table feature that was introduced. With multiple tables you can have multiple default routes. Which table gets used is controllable via both setfib(1) and setfib(2). setfib(1) allows you to say that a given process uses a given routing table. There were also setfib options added to ipfw so that a packet matching a rule can be pushed out a specific routing table.


----------



## twarkie (Apr 9, 2009)

Thank you bobveznat! Your information solved my problem. I now use setfib to lock rtorrent to a routing table. 

For future reference, this is what I had to do:

* Recompile the kernel with the following option: "options ROUTETABLES=2"
* Add a default inte the second routing table: "setfib -1 route add default <ip>"
* Start the application that should use the routing table with: "setfib -1 <application>"

Everthing works great no"


----------



## twarkie (Apr 27, 2009)

Ok I have a slight problem. Turns out that incoming traffic doesn't reach rtorrent. I see it with dcpdump but it seems to stop there. Maybe I missed something? 

I have also tried to add this rule to pf:
pass quick on $tun_if inet proto tcp from any to any rtable 1

But that didn't change anything.


----------



## bobveznat (Apr 27, 2009)

Are you using RFC 1918 addresses? Do you need to do port forwarding in whatever device is doing NAT?

I use ipfw for my firewall and when I have problems like this I turn logging on for my deny rule. This puts a message into /var/log/security for each packet that is blocked and let's me see things I didn't think were happening.


----------



## twarkie (Apr 27, 2009)

Well, rtorrent is running on my gateway (it' a all-in-all server) so it shouldn't need any port forwarding, it's connected right to the Internet without anything in the way. But maybe I misunderstood you?

The port is indeed "functional" because it works fine when I try the same procedure without setfib, so that the traffic is routed through the default interface.


----------



## bobveznat (Apr 27, 2009)

What do you mean by incoming traffic? Incoming connection attempts? Or replies to your outgoing packets?


----------



## twarkie (Apr 27, 2009)

Incoming connection attempts. Replies works like it should.


----------



## bobveznat (Apr 27, 2009)

Is the destination IP for the packets you're seeing in tcpdump the correct IP (i.e. the IP your torrent client is supposed to be using?)

Does sockstat(1) show your torrent client using the right IP and port?


----------



## twarkie (Apr 28, 2009)

Ok, here comes some debugging data. I tried to connect to port 39926.

Output of sockstat | grep rtorrent


```
matte    rtorrent   28166 280tcp4   1.2.124.171:55451     173.73.99.69:58124
matte    rtorrent   28166 316tcp4   1.2.124.171:55452     84.248.150.219:51390
matte    rtorrent   28166 335tcp4   1.2.124.171:55453     189.54.229.100:51413
matte    rtorrent   28166 336tcp4   1.2.124.171:55454     24.255.106.101:55015
matte    rtorrent   28166 351tcp4   1.2.124.171:55459     158.38.88.102:62342
matte    rtorrent   28166 352tcp4   1.2.124.171:55460     87.89.61.33:62348
matte    rtorrent   28166 354tcp4   1.2.124.171:55462     78.86.141.97:23208
matte    rtorrent   28166 357tcp4   1.2.124.171:55465     98.209.63.212:38561
matte    rtorrent   28166 358tcp4   1.2.124.171:55466     82.75.193.117:64519
matte    rtorrent   28166 367tcp4   1.2.124.171:55475     77.102.71.123:56067
matte    rtorrent   28166 369tcp4   1.2.124.171:55477     66.188.27.160:55055
matte    rtorrent   28166 387tcp4   1.2.124.171:55495     72.12.141.202:28791
matte    rtorrent   28166 391tcp4   1.2.124.171:55499     71.117.208.180:36976
matte    rtorrent   28166 392tcp4   1.2.124.171:55500     174.88.104.130:51397
matte    rtorrent   28166 404tcp4   1.2.124.171:55544     78.106.177.218:27840
matte    rtorrent   28166 405tcp4   1.2.124.171:55545     213.65.188.84:55983
matte    rtorrent   28166 406tcp4   1.2.124.171:55546     71.231.41.9:62954
```


Output of tcpdump -n -i tun0


```
22:51:57.742516 IP 208.113.181.158.47622 > 1.2.124.171.39926: S 1900548817:1900548817(0) win 5840 <mss 1368,sackOK,timestamp 964686539 0,nop,wscale 7>
22:51:58.741600 IP 208.113.181.158.47625 > 1.2.124.171.39926: S 1922982417:1922982417(0) win 5840 <mss 1368,sackOK,timestamp 964686789 0,nop,wscale 7>
22:51:59.761736 IP 208.113.181.158.47632 > 1.2.124.171.39926: S 1935878960:1935878960(0) win 5840 <mss 1368,sackOK,timestamp 964687043 0,nop,wscale 7>
```

1.2.124.171 is my internal VPN ip, so that seems to be correct.

I really appreciate your help!


----------



## bobveznat (Apr 28, 2009)

In that sockstat output I don't see where rtorrent is actually bound to a listen socket? For example, on one of my hosts here is Apache:

```
www      httpd      1708  3  tcp46  *:80                  *:*
```

There we see that Apache is bound to all interfaces on port 80.

Here's named with a UDP socket to show that even UDP sockets show up here.

```
bind     named      1359  512udp4   192.168.1.3:53        *:*
```

rtorrent would need to be there somewhere if it were to accept incoming connections. Did you somehow leave out that info from the paste? Or is rtorrent not actually listening?


----------



## twarkie (Apr 28, 2009)

Oh, sorry about that. Somehow, I missed the first line, do'h 


```
matte    rtorrent   29777 4  tcp4   *:39926               *:*
```

So it is indeed listening.


----------



## bobveznat (Apr 28, 2009)

I'm left assuming that your firewall is breaking things. I'd turn on logging and verify that the packets coming in have the attributes you're trying to match in your rule.


----------



## twarkie (Apr 28, 2009)

Ok, I enabled pflog and performed some debugging.


```
$ sudo tcpdump -n -e -ttt -i pflog0 -s 256
1. 997873 rule 11/0(match): pass in on tun0: 78.145.254.238.2384 > 1.2.124.134.39925: S 4051716295:4051716295(0) win 65535 <mss 1368,nop,nop,sackOK>
1. 043151 rule 11/0(match): pass in on tun0: 94.208.254.178.58647 > 1.2.124.134.39925: S 1326161159:1326161159(0) win 8192 <mss 1368,nop,nop,sackOK>
6. 592374 rule 11/0(match): pass in on tun0: 151.47.73.167.2169 > 1.2.124.134.39925: S 3535810382:3535810382(0) win 65535 <mss 1368,nop,nop,sackOK>
```


```
$ sudo  pfctl -sr -vv
@11 pass log quick on tun0 inet proto tcp all flags S/SA modulate state queue(q_def, q_pri) rtable 1
  [ Evaluations: 948       Packets: 4713      Bytes: 1329564     States: 93    ]
  [ Inserted: uid 0 pid 2282 ]
```

...so the rule is matched ok, but it doesn't matter if it is enabled or not, there's no different.

Am I the first one trying out the new setfib functionality? There are no other information on the Internet so really have no other place to look.

Maybe there's a problem with Openvpn? But I feel like I googled for hours and hours without finding any useful information. Perhaps the tunnel needs to be bridged (tap)?


----------



## bobveznat (Apr 28, 2009)

Well now I just don't know. I don't think this is a setfib problem.

I do see your pf rule says something about rtable? Is that the same as setfib for ipfw? You shouldn't need that on the receiving side unless this host is a router (which it isn't, at least its not routing this traffic on to another host).

Perhaps that is somehow confusing things? I'd trying writing a regular old rule to simply pass in the traffic and not mess with the routing table at all.


----------



## twarkie (Apr 28, 2009)

Well, I just tried to set the VPN gateway to the default route (in the default routing table) and then started rtorrent without setfib. That worked perfectly. But when I use rtorrent with the second routing table (using setfib) it stoppes working. I must have misses something with setfib or it's broken some how.

About the pf rule, I have tried every combination. Disabling pf completely, adding different pass rules, with and without rtable etc etc. Nothing has worked so far.


----------



## twarkie (Apr 28, 2009)

Oh, for the record, I have also tried with apache instead of rtorrent just to check if it was rtorrent's fault. But it wasn't


----------



## bobveznat (Apr 28, 2009)

Oh. So maybe the kernel is getting the SYN and trying to reply with the SYN+ACK but it's going out the wrong interface?

Have you run tcpdump on the other interfaces to see if wonky SYN+ACK packets are being sent out?


----------



## twarkie (Apr 28, 2009)

bobveznat, you are my new god!


```
$ sudo tcpdump -n -i fxp0 port 39927
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on fxp0, link-type EN10MB (Ethernet), capture size 96 bytes
21:22:34.259496 IP 1.2.124.111.39927 > 208.113.181.158.53722: S 4264271094:4264271094(0) ack 2277152839 win 65535 <mss 1368,nop,wscale 3,sackOK,timestamp 1641556618 984958359>
21:22:34.940404 IP 1.2.124.111.39927 > 189.27.73.214.3560: S 853564184:853564184(0) ack 965093301 win 65535 <mss 1368,nop,wscale 3,sackOK,eol>
21:22:35.252024 IP 1.2.124.111.39927 > 208.113.181.158.53734: S 2932002775:2932002775(0) ack 2293408718 win 65535 <mss 1368,nop,wscale 3,sackOK,timestamp 2810777111 984958609>
21:22:36.252163 IP 1.2.124.111.39927 > 208.113.181.158.53744: S 1789207156:1789207156(0) ack 2304939826 win 65535 <mss 1368,nop,wscale 3,sackOK,timestamp 2796398840 984958859>
21:22:37.259505 IP 1.2.124.111.39927 > 208.113.181.158.53722: S 4264271094:4264271094(0) ack 2277152839 win 65535 <mss 1368,nop,wscale 3,sackOK,timestamp 1641556618 984958359>
21:22:37.939512 IP 1.2.124.111.39927 > 189.27.73.214.3560: S 853564184:853564184(0) ack 965093301 win 65535 <mss 1368,nop,wscale 3,sackOK,eol>
21:22:38.251514 IP 1.2.124.111.39927 > 208.113.181.158.53734: S 2932002775:2932002775(0) ack 2293408718 win 65535 <mss 1368,nop,wscale 3,sackOK,timestamp 2810777111 984958609>
21:22:39.251523 IP 1.2.124.111.39927 > 208.113.181.158.53744: S 1789207156:1789207156(0) ack 2304939826 win 65535 <mss 1368,nop,wscale 3,sackOK,timestamp 2796398840 984958859>
21:22:43.259555 IP 1.2.124.111.39927 > 208.113.181.158.53722: S 4264271094:4264271094(0) ack 2277152839 win 65535 <mss 1368,nop,wscale 3,sackOK,timestamp 1641556618 984958359>
21:22:43.939557 IP 1.2.124.111.39927 > 189.27.73.214.3560: S 853564184:853564184(0) ack 965093301 win 65535 <mss 1368,nop,wscale 3,sackOK,eol>
21:22:44.251561 IP 1.2.124.111.39927 > 208.113.181.158.53734: S 2932002775:2932002775(0) ack 2293408718 win 65535 <mss 1368,nop,wscale 3,sackOK,timestamp 2810777111 984958609>
21:22:45.251572 IP 1.2.124.111.39927 > 208.113.181.158.53744: S 1789207156:1789207156(0) ack 2304939826 win 65535 <mss 1368,nop,wscale 3,sackOK,timestamp 2796398840 984958859>
21:22:46.337999 IP 1.2.124.111.39927 > 94.255.184.85.58416: S 2584706853:2584706853(0) ack 2502558933 win 65535 <mss 1368,sackOK,eol>
21:22:47.039152 IP 1.2.124.111.39927 > 208.113.181.158.53821: S 3045899886:3045899886(0) ack 2477652692 win 65535 <mss 1368,nop,wscale 3,sackOK,timestamp 3877236234 984961555>
21:22:48.044923 IP 1.2.124.111.39927 > 208.113.181.158.53835: S 740791687:740791687(0) ack 2481550772 win 65535 <mss 1368,nop,wscale 3,sackOK,timestamp 2341374172 984961805>
21:22:49.036503 IP 1.2.124.111.39927 > 208.113.181.158.53850: S 2094285704:2094285704(0) ack 2501862752 win 65535 <mss 1368,nop,wscale 3,sackOK,timestamp 19391780 984962055>
21:22:49.329948 IP 1.2.124.111.39927 > 94.255.184.85.58416: S 2584706853:2584706853(0) ack 2502558933 win 65535 <mss 1368,sackOK,eol>
21:22:50.038606 IP 1.2.124.111.39927 > 208.113.181.158.53821: S 3045899886:3045899886(0) ack 2477652692 win 65535 <mss 1368,nop,wscale 3,sackOK,timestamp 3877236234 984961555>
21:22:51.044618 IP 1.2.124.111.39927 > 208.113.181.158.53835: S 740791687:740791687(0) ack 2481550772 win 65535 <mss 1368,nop,wscale 3,sackOK,timestamp 2341374172 984961805>
21:22:52.035622 IP 1.2.124.111.39927 > 208.113.181.158.53850: S 2094285704:2094285704(0) ack 2501862752 win 65535 <mss 1368,nop,wscale 3,sackOK,timestamp 19391780 984962055>
```

The traffic was indeed going out the wrong way. So I simply added a reply-to rule to pf and YES, it works! 


```
pass in quick on $tun_if reply-to $tun_if inet proto tcp from any to any flags S/SA modulate state queue (q_def, q_pri)
```

And the result

```
OK! Port 39927 is open and accepting connections.You will be able to receive incoming BitTorrent connections.
```

Thank you once again!


----------



## bobveznat (Apr 28, 2009)

ha. It's funny how in hindsight this crap is so simple.


----------

