# Bypassing OpenVPN for certain ports



## cschiewek (Aug 28, 2013)

Hello all,

I have OpenVPN up and running and working. However, I want incoming traffic on port 22 not to use the VPN. I'd also like outgoing 80 and 443 to not use the VPN either.

I'm guessing the solution requires PF to get working, but I've read the documentation and I'm even more confused now.

Thoughts? Guidance?  

Many thanks!

Curtis


----------



## SirDice (Aug 28, 2013)

Where is the VPN connecting to? Incoming connections will connect to your external IP address, not to your VPN end-point.


----------



## junovitch@ (Aug 28, 2013)

Are you running OpenVPN as a server or client?  As a server packets are just going to where the route table says.  For incoming packets it will be to the external IP and outgoing packets will only use the VPN only if they are destined for VPN endpoints.  As a client, it will depend on whether the server is pushing route statements or a gateway redirect statement.  Please provide more details or a network map that can explain what you are looking to do.


----------



## cschiewek (Aug 28, 2013)

The VPN is running as a client to a VPN service.  Incoming connections are still hitting my external IP, but when I try to SSH to it, it times out.

I assumed it was because outgoing packets were being redirected over the VPN.  Is that not the case?  Why isn't SSH working then?  And it's not just SSH, it's any listening port.

The VPN server is adding the following route and gateway statements, where x.x.x.x is my VPN server:


```
add net x.x.x.x: gateway 192.168.1.1
Wed Aug 28 18:21:24 2013 /sbin/route add -net 0.0.0.0 10.7.1.113 128.0.0.0
add net 0.0.0.0: gateway 10.7.1.113
Wed Aug 28 18:21:24 2013 /sbin/route add -net 128.0.0.0 10.7.1.113 128.0.0.0
add net 128.0.0.0: gateway 10.7.1.113
Wed Aug 28 18:21:24 2013 /sbin/route add -net 10.7.0.1 10.7.1.113 255.255.255.25
```

Here's what my OpenVPN configuration looks like:


```
client
dev tun
proto tcp
remote x.x.x.x 80
resolv-retry infinite
nobind
ns-cert-type server
cipher AES-256-CBC
comp-lzo
verb 3
```

Originally, I tried to get only a handful applications to work over the VPN tunnel in a jail, but I could not get it to work (See here).  

So I thought instead, just leave the VPN connection on at all times, and tell whatever services I want to bypass the VPN (SSH, HTTP, HTTPS, FTP, etc).

Thanks!


----------



## junovitch@ (Aug 29, 2013)

OK.  Redirecting all traffic is probably perfect for a company laptop that connects while on the road so that everything gets encrypted, be that DNS to the company LAN, file sharing, or access to the Intranet.  For a system that is also a server that is using an OpenVPN client that may not be the best.  The reason why packets are being output over the VPN is because the most favorable route is through that tunnel.  It will take precedence over the original route that I would guess comes from your local DHCP server.  I would suggest reading up on these two options and see what works best for you.

From an OpenVPN man page (http://linux.die.net/man/8/openvpn)

```
--route-noexec
Don't add or remove routes automatically. Instead pass routes to --route-up script using environmental variables.
--route-nopull
When used with --client or --pull, accept options pushed by server EXCEPT for routes.
When used on the client, this option effectively bars the server from adding routes to the client's routing table, however note that this option still allows the server to set the TCP/IP properties of the client's TUN/TAP interface.
--up cmd
Shell command to run after successful TUN/TAP device open (pre --user UID change). The up script is useful for specifying route commands which route IP traffic destined for private subnets which exist at the other end of the VPN connection into the tunnel.
```
With these you can ignore the routes being pushed and do your own thing.  Your use case will probably have to determine how you go about it.  I've used --up scripts to push additional routes and force IPv6 addressing onto a point-to-point OpenVPN tunnel with success but haven't experimented with the others.

Another option would be to force synchronous routing with something like the following:
/etc/pf.conf

```
lan_if="em0" # (your LAN physical interface)
lan_inet_gw="192.168.1.1" # (your LAN gateway IP)
pass in quick on $lan_if reply-to ($lan_if $lan_inet_gw)
```

The reply-to will force replies to go out the interface they come in on.  I would advise reading up about this some more as I've only begin testing this for my uses but haven't turned it on full time.


----------



## cschiewek (Aug 29, 2013)

OK, so if I opt for the first option, and stop routes from getting created by the VPN server, can I then setup routes that are port specific (say 25 for SMTP)?  

I didn't think it was possible to add routes based on ports?

Thanks so much for your help!


----------



## cschiewek (Aug 29, 2013)

Alright, so I just tested option 2, the pf rules, and it's working.  I can now access SSH, HTTP, HTTPS, etc. when the VPN is connected.  

Now, what rule would I have to add to tell specific outgoing ports to use my 192.168.1.1 gateway?  Say I want outoing 80 and 443 traffic to not use the VPN? 

Thanks again! You rule!


----------



## junovitch@ (Aug 30, 2013)

Try "catching" the packets leaving on the VPN and pushing them somewhere else with a route-to statement.

Idea:
/etc/pf.conf

```
pass out on tun0 route-to (em0 192.168.1.1) inet proto tcp from any to any port { 80, 443 }
```

I'm not sure if PF will error out on reboot if the tunnel hasn't been created yet.  I'm guessing it might if PF starts before OpenVPN is started.  You can force a tunnel interface being created upon boot by doing something like this so that it will be there regardless of OpenVPN being enabled or not.  

/etc/rc.conf

```
cloned_interfaces="tun"
```


----------



## kpa (Aug 30, 2013)

That won't work. You absolutely must catch the traffic you want to route-to to a different gateway on the inbound side of filtering. When the packets are destined to leave out via an interface it's too late to re-route them.


----------



## cschiewek (Aug 31, 2013)

Hey @kpa,

Could you give me an example rule?


----------



## junovitch@ (Sep 1, 2013)

The rule should work as is.  Although catching it on the inbound side would be preferred there is no inbound side for packets originated from the server itself.


----------



## cschiewek (Sep 1, 2013)

It didn't work.  My pf.conf looks like this:


```
lan_if = "re0"
lan_inet_gw = "192.168.1.1"

pass in quick on $lan_if reply-to ($lan_if $lan_inet_gw)
pass out on tun0 route-to (re0 192.168.1.1) inet proto tcp from any to any port { 80, 443 }
```

pf starts up fine without throwing errors, but when I `curl google.ca` it times out.


----------



## cschiewek (Sep 1, 2013)

I'm guessing something else is the problem.  I've disabled the rule, restarted pf, and curl is still timing out.  I'm confused


----------



## junovitch@ (Sep 1, 2013)

It's starting to make sense to me now.  Let me explain what I am doing to test it.

10-CURRENT VM em0 (192.168.1.1) <=> em1 (192.168.1.2) 9.1-RELEASE VM em0 (DHCP 10.100.82.129) <=> LAN

I am using two VMs with multiple NICs on the middle one to test out what is happening.  Without PF on, running `nc 192.168.102.13 80` and typing get pulls down the web page of an internal address I have.  With the PF rules enabled, that connection times out.  `tcpdump -vvi em0` on the 10-CURRENT VM shows that it is receiving packets with a source of 10.100.82.129 and destination of 192.168.102.13.

The rule does work but now I see that it's using the original egress interface's IP address as we shove it out the other interface.  I suppose one way to fix that would be to put a static route on the host to the IP of the tun0 interface so that packets would know how to get back.  That just seems kind of ugly though.  I'll have to read up on this some more.

I have more experience with IPtables and Cisco ACLs and have been meaning to do some more reading on PF.  Thanks for the patience in the meantime, hopefully we can get to the bottom of this.


----------



## junovitch@ (Sep 2, 2013)

I'm having no luck with this.  I can see that some things are working properly when I am testing in a VM but when I do it on my actual dual NIC file server it doesn't work.  For some reason there are millions of searches to the state table for packets and my loopback is getting multiple gigabytes of data.  Normally the console locks up and I've had to power cycle the box twice.  Oddly my NFS mounts are fully accessible and pings work fine.  But I can't get to the web server on it while it is locked up nor SSH.  I'm sorry but I don't have any suggestions right now.


----------



## throAU (Sep 4, 2013)

Just as an aside - are you sure you want to do this?

If you have a network that is important enough to secure via VPN, then consider that if you are allowing simultaneous connection by a client to both the internet, and your internal network, you have established an alternative path into your network, that will bypass your edge firewall.

Sure, it's unlikely, but one has to ask - what are you trying to achieve here (Yes, skip VPN for SSL/SSH, but why exactly?  Just performance?), and is it worth the additional risk, above?

That said, as with any routing issue, be sure to check both sides of the connection.  I haven't had a detailed look at your configuration, but given the state table growth mentioned, it sounds like you're perhaps bypassing the VPN on the client's end, but not on the server's end.  Also, why are you using a netmask of 128.0.0.0 on some of your routes?


----------

