# Bridge with hosted services?



## mkernalcon (Jan 9, 2019)

I manage a small business network. Our core router is running pfSense, with two WAN connections (each is a static IP with only one address) and a bunch of VLANS, etc. We will soon be hosting our own (low-traffic - maybe 300 hits a month) web server and authoritative DNS. My concern is that when I have to run a pfSense update, that the DNS will become unavailable (HTTP will be unavailable too, but I can easily time this to avoid any interruptions).

So, I'd like to take our secondary WAN, and put a pure FreeBSD box in between the WAN and the router. I envision this passing all traffic straight through itself, and only catching DNS on the outside, and maybe SSH on the inside (but I can figure out a different management solution).

I can certainly do this with a (mostly) 1:1 NAT - give this box the WAN IP address, then give the other side a network on a unique subnet, and simply pass everything through except DNS. If it's not too difficult, though, I'd like to avoid this, and keep my WAN IP on the router.

So, I'm thinking that I may be able to exploit bridge for this. Directly pass through every incoming packet that's not bound for my.ip.ad.dr:53, and catch it if it is. I believe this doesn't even have to be stateful. If I wanted ssh on the inside, I could do something similar, but catch packets bound for my.ga.te.way:22 instead.

I'm not too familiar with bridging, and I've never touched it on BSD. Seems like I'll have a problem getting named to listen, since this computer won't really have an ip address, unless I can make some weird internal interface.

Am I totally barking up the wrong tree here, or is what I'm describing possible? If so, how do I go about it?


----------



## SirDice (Jan 9, 2019)

mkernalcon said:


> I manage a small business network. Our core router is running pfSense, with two WAN connections (each is a static IP with only one address) and a bunch of VLANS, etc. We will soon be hosting our own (low-traffic - maybe 300 hits a month) web server and authoritative DNS. My concern is that when I have to run a pfSense update, that the DNS will become unavailable (HTTP will be unavailable too, but I can easily time this to avoid any interruptions).


The industry standard, tried and tested, solution is to set up a high availability installation, in this case by using TWO pfSense machines (the same holds true for Cisco/HP/Juniper/whatever routers, switches and firewalls). That way you can take one offline for updates (or hardware maintenance) while the other takes over the functionality (in some cases even seamlessly, without any interruption to existing sessions).

https://www.netgate.com/docs/pfsense/highavailability/configuring-high-availability.html

I suggest asking on the pfSense forums for people who have experience with this kind of setup if you need help with this.

Don't try to over-engineer a SPOF (Single Point Of Failure) by adding another SPOF.


----------



## mkernalcon (Jan 9, 2019)

Yes, I had looked into this, and it's really my ideal. However, I was hoping to get away with using a computer that was not powerful enough to take over as a full router for this particular redundancy (particularly with the upcoming AES-NI requirements in pfSense 2.5). I'll still play with the idea, as it is definitely the "right way" to do this.

My theoretical setup isn't a single point of failure - if the router goes down, this box still serves my "essential services", and if this box goes down, the router still works just without the second WAN link. I don't necessarily disagree with you on "over-engineered" though. Just want to see if it's possible.


----------



## SirDice (Jan 9, 2019)

mkernalcon said:


> My theoretical setup isn't a single point of failure - if the router goes down, this box still serves my "essential services"


If the router goes down there is no connection any more between your LAN and the suggested FreeBSD bridge. Your pfSense also doubles as your core router, so your entire network is disconnected into a collection of individual VLANs with no way in or out of each subnet. Been there, done that. Not a happy place. Being disconnected from the internet is the least of your worries, your entire network will stop functioning.


----------



## mkernalcon (Jan 9, 2019)

SirDice said:


> If the router goes down there is no connection any more between your LAN and the suggested FreeBSD bridge. Your pfSense also doubles as your core router, so your entire network is disconnected into a collection of individual VLANs with no way in or out of each subnet. Been there, done that. Not a happy place. Being disconnected from the internet is the least of your worries, your entire network will stop functioning.


True... but I'll only do updates when no one's on the network. And as every good admin knows, unexpected hardware failures don't ever happen 

...anyway, you've convinced me to try to set up HA even if the secondary is massively underpowered (limp mode, essentially). But, for the sake of curiosity, is what I described possible?


----------



## SirDice (Jan 9, 2019)

I just drove home from work, had a beer, and let your situation sink in a bit. I believe I have an idea, maybe even a solution. A bridge isn't going to work, that's not what a bridge is for. You're going to need a lot more than that. The idea I have partially comes from the before referenced, complete pfSense, HA solution. The keyword here is carp(4). 

Assuming each subnet uses the .1 address as a gateway and DNS, you can configure pfSense to use the .1 address as its CARP VIP address, the 'physical' address of the pfSense router will become .2. Your backup FreeBSD router will use .3 and also set its CARP VIP to .1. pfSense's FreeBSD version X.y is the same as FreeBSD's own versions so this should work. Create the same vlan(4) interfaces on FreeBSD and enable routing. A basic routing table will keep everything connected. You can probably copy pfSense's PF configuration, maybe tweak it a bit for its specific purpose, and use that directly on FreeBSD. That will take care of NAT and some basic firewalling you might need. DNS same thing, or settle for local-unbound(8).

Switching between the pfSense or FreeBSD router can be done "on-the-fly" and even automatically by controlling carp(4). It will kill all running sessions because there's no sync between the two PFs. But that's not really important at that time anyway. The idea is to stay online, working as much as possible, and a minor interruption isn't a big deal.  You will indeed be in "limp mode" but I'm sure it could be made to work.


----------



## mkernalcon (Jan 9, 2019)

Hmm, this is interesting. It certainly fixes my problem with not having AES-NI when pfSense finally updates, and I reckon it should give me more flexibility than a second pfSense box (especially since I probably don't want to copy the config exactly, and that it would be much easier for this second box to only have one WAN interface). I realized, though, that my biggest issue with CARP is that I only have one public IP per WAN (one is a /30, one is a /27 that I share with other customers). pfSense doesn't seem to have figured out how to give the secondary access to the internet with single-ip carp.

However, here's my thought: CARP the VLAN interfaces as you described. Set my default gateway to the pfSense router (so that I can keep this machine updated easier, etc.). On a CARP switchover, it's pretty safe to assume the entire router is down, so I bring up the WAN interface and reset to that gateway. I'm really hoping it's all quick enough to not cause my IP conflict issues on the WAN interface.. I suppose if it becomes a problem, I can add another network card to this machine, and set it up as a bridge between WAN and the pfSense WAN interface, then on CARP switchover, disable the bridge and bring up the WAN as a real interface. I'm not overcomplicating this, right?

pfsync is way out of the question, especially with how different the network configurations will be, but killing states is not the worst thing in the world - it's much better than it is now. Not better enough to let me update pfSense during work hours, but definitely good enough that I can update at night and not worry about my hosted services dropping.


----------

