# FreeBSD Home Router Question...or two.



## n00balert (Jun 22, 2013)

Hi _g_uys,

I have a home router/firewall/DHCP-server/DNS-forwarder set up at home that is giving me some grief and since I'm not a networking expert I'll pose the questions to the good people here in hopes that they can find some peace of mind for me 

This is what my setup looks like; 

The cable modem (Cisco DCP3825 in bridged mode) connected to nfe0 on the FreeBSD server. The cable modem gets a DHCP public IP from ISP and also (sadly) turns on a local DHCP server that gives out IPs in the 192.168.100.x range. The FreeBSD server then has a NIC em0 that is connected to a LAN switch, to which my LAN clients are also connected. Some of my LAN clients are static and others "should" get their IP from the FreeBSD DHCP server in the 10.0.0.x range. I'm using PF as firewall/NAT and have set up rules as described below. I'm running the latest/updated FreeBSD 9.1-RELEASE-p3.

Problem 1: Sometimes my LAN clients (especially when they're "reconnected") get DHCP assigned IP address from my cable modem! Instead of my FreeBSD server. i.e. they get address in 192.168.100.x range instead of 10.0.0.x range. This has happened to Mac, Windows and Linux clients, all the same. How do I "block" my cable modem's DHCP offering or broadcast from being passed through the FreeBSD server? Maybe it has something to do with my PF rules?

Problem 2: It's not really a problem, rather a question; As you can see from the rules below, I have bridge0 set up as internal NIC to which my PF rules apply. Reason for this is that for now I only have LAN clients connected to em0 (only member of bridge0) but later on I'll be adding a wifi card as AP (once I find a compatible card, hopefully soon) to the bridge so both my LAN and WiFi clients can get DHCP addresses. However, I want to assign addresses in different IP space for both types of clients. e.g. LAN clients coming off the LAN switch to em0 should get DHCP IPs in range 10.0.0.x ..and.. WiFi clients coming from wlan0 get DHCP IPs in the range 192.168.1.x - Both these NICs are members of bridge0. Is this doable? If yes, how? Is that something I have to set up on the DHCP server itself? Based on MAC addresses of bridge0 members (em0, wlan0) MAC addresses? This is just my guess. 

Here are my PF rules:

```
ext_if="nfe0"
int_if="bridge0"
tcp_services="{ 22 }"
icmp_types="echoreq"
mynix="10.0.0.10"

set block-policy return
set loginterface $ext_if
set skip on lo
scrub in
nat on $ext_if from !($ext_if) -> ($ext_if:0)
nat-anchor "ftp-proxy/*"
rdr-anchor "ftp-proxy/*"
rdr pass on $int_if proto tcp to port ftp -> 127.0.0.1 port 8021
block in
pass out
anchor "ftp-proxy/*"
antispoof quick for { lo $int_if }
pass in on $int_if inet proto tcp from any to $int_if port $tcp_services
pass in inet proto icmp all icmp-type $icmp_types
pass quick on $int_if no state
```
Thank you for reading.

Hoping to get some answers 

N00B.


----------



## wblock@ (Jun 22, 2013)

First, try to shut off the cable modem DHCP server.  It should have a web config page.  If that can't be done, DHCP can probably be blocked at the firewall.


----------



## n00balert (Jun 22, 2013)

Hi @wblock@, Unfortunately it doesn't allow me to turn it off. I can't even login to the modem (default user/pass doesn't work) in bridged mode.

How do I bock it at the firewall? That's the question.


----------



## n00balert (Jun 22, 2013)

*M*aybe I should try _the_ firewall forum?


----------



## DutchDaemon (Jun 22, 2013)

Try blocking ports 67/68 UDP in both directions.


----------



## tingo (Jun 22, 2013)

Problem 2: another way to solve it would be not to use bridge at all, but let your wireless AP be on its own subnet (as you have described it). The DHCP server is perfectly capable of serving more than one subnet, and you can control which interface gets what.


----------



## Anonymous (Jun 22, 2013)

Usually DHCP is limited to the broadcast domain into which the client is connected.

I understood from your description (please correct me), that the external interface nfe0 gets the public IP from the cable modem (bridge mode). All your clients are physically connected to the internal interface em0, by the way of a switch. em0 itself is placed into a bridge.

Now, I can't understand, how the DHCP server of your modem could be in the same broadcast domain as the internal interfaces  on your server, let it be em0, wlan0, or bridge0.

Please show the output of the following command on your server: `# ifconfig`

Did you connect the modem also to said switch?

Does the modem have an active WLAN AP, which is accessed somehow by your clients?


----------



## shaqan (Jun 22, 2013)

Why don't you make your life easier and simply install pfSense? It's also based on FreeBSD and makes use of pf.


----------



## n00balert (Jun 22, 2013)

Hi @tingo, That's a good idea, but it also means that I have to add separate firewall/NAT rules for wlan0. Maybe it's a good thing that I do  I'll definitely think about it. 

Hi @Rolfheinrich, modem is directly connected to FreeBSD and you're spot on in your understanding of my setup. I am puzzled myself, how can that be?

`$ ifconfig -a`


```
sk0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=80009<RXCSUM,VLAN_MTU,LINKSTATE>
	ether 00:11:d8:4c:3d:90
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet autoselect (none)
	status: no carrier
nfe0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
	options=8210b<RXCSUM,TXCSUM,VLAN_MTU,TSO4,WOL_MAGIC,LINKSTATE>
	ether 00:11:d8:4c:2d:ef
	inet xx.xx.xx.xxx netmask 0xfffffc00 broadcast 255.255.255.255
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9000
	options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
	ether 68:05:ca:14:5c:86
	inet6 fe80::6a05:caff:fe14:5c86%em0 prefixlen 64 scopeid 0x5 
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
pflog0: flags=141<UP,RUNNING,PROMISC> metric 0 mtu 33152
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
pfsync0: flags=0<> metric 0 mtu 1500
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	syncpeer: 0.0.0.0 maxupd: 128
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128 
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x8 
	inet 127.0.0.1 netmask 0xff000000 
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
	ether 02:d3:4d:da:36:00
	inet 10.0.0.254 netmask 0xffffff00 broadcast 10.0.0.255
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
	maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
	root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
	member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 5 priority 128 path cost 2000000
```

Hi @shaqan, I could but I'm also using this box as a ZFS-NFS-Samba-media server. and DIYO is more fun?  Well, not right now it isn't, I'm stuck 

@Rolfheinrich, as far as I understand, in bridged mode, the WLAN portion of the modem gets turned off, only my LAN clients are exhibiting this behavior (I don't have any WLAN clients yet). I ran `arp -a` on my freebsd FreeBSD server but don't see any indication (that I can tell) of 192.168.100.x or modem's DHCP server;

`arp -a`

```
? (10.0.0.102) at c8:2a:14:19:89:82 on bridge0 expires in 736 seconds [bridge]
myhost.xxxxxxxxxxx.com (10.0.0.254) at 02:d3:4d:da:36:00 on bridge0 permanent [bridge]
xxxxxxxxxxx.xx.shawcable.net (xx.xx.xx.xxx) at 00:11:d8:4c:2d:ef on nfe0 permanent [ethernet]
? (xx.xx.xx.x) at 00:1d:70:cc:ac:d9 on nfe0 expires in 1199 seconds [ethernet]
```


----------



## n00balert (Jun 22, 2013)

DutchDaemon, if I did that, wouldn't I be blocking all DHCP traffic?


----------



## n00balert (Jun 22, 2013)

*I*s it also possible that this is because of my LAN broadcast domain 10.0.0.0? If I changed it to 10.10.10.0 - maybe that'd help?


----------



## kpa (Jun 22, 2013)

Broadcast domain is not defined by the addresses used but by the way you interconnect different networking equipment. It is the largest continuous *E*thernet segment a broadcast can travel in the network before it hits a device that does not forward the *E*thernet frames across the device (a router for example). Bridges extend the *E*thernet segment and do not stop broadcasts.


----------



## n00balert (Jun 23, 2013)

@kpa, Understood. Thanks. What is it in my route that I'm doing wrong then?

Hi @tinga, I take my answer back; I didn't have to use bridging in my setup, and I should have my DHCP server assign addresses in two different range_s_. I'd separate them as below;


```
int_if="{em0 wlan0}"
```

Thanks!

ps: sorry Admins, I just read the rules


----------



## J65nko (Jun 23, 2013)

IMHO your bridge configuration is complicating things. Just forget about using a bridge.This will also stop the cable modem from giving out addresses via DHCP to your LAN clients.


----------



## kpa (Jun 23, 2013)

Definitely get rid of the bridge as @J65nko suggests above. If you want to have the FreeBSD machine as a DHCP server/DNS forwarder/proxy etc. it's easiest to set it up as a router. Bridged setup would require that the LAN clients would have to have the next "hop" address as their gateway, not the IP address of the machine that implements the bridge and that makes everything very complicated.


----------



## Anonymous (Jun 23, 2013)

@@J65nko and @kpa, I guess you are talking about this bridge:



			
				n00balert said:
			
		

> ```
> ...
> bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
> ether 02:d3:4d:da:36:00
> ...



Even if a bridge(4)() with the only one pier em0 is somehow useless until some other piers are added (like wlan0, as the OP stated), I fail to see, how this would have any influence on the routing/NAT provided by pf(4)() between the external interface nfe0 and the internal interface bridge0.

If pf happens to be confused by a bridge within the internal broadcast domain, then it got a big issue here.

Perhaps the OP wants to try ipfw(8)().


----------



## n00balert (Jun 24, 2013)

Hello guys, I took bridge0 out of my setup and so far haven't had any DHCP issues yet. I agree though PF is probably getting confused with bridging. Thanks all for your replies.


----------



## n00balert (Jun 24, 2013)

@rolfheinrich, I'm happy to switch to ipfw but not sure if that'd be any more or less beneficial. I invested a lot of time already learning a few things about pf - don't really have any motivation to move to ipfw unless there were obvious benefits.


----------



## Anonymous (Jun 24, 2013)

n00balert said:
			
		

> @rolfheinrich, I'm happy to switch to ipfw but not sure if that'd be any more or less beneficial. I invested a lot of time already learning a few things about pf - don't really have any motivation to move to ipfw unless there were obvious benefits.



Both, pf and ipfw are mature firewalls. IMHO, it is basically a matter of taste which one to choose.

Perhaps I should have been more precise with my suggestion. In the course of troubleshooting (and in general experimental planning), the most descent approach is orthogonal testing of the variables at its different levels. In your case, 2 variables [bridge : firewall] at 2 levels each [on|off : pf|ipfw], this would have meant cross-checking, i.e. 4 experiments:
	
	



```
#       bridge        firewall    result
1.       on             pf         nOK
2.       on            ipfw        ?
3.       off            pf         OK
4.       off           ipfw        ? (presumably OK)
```

You did #1 and #3, the latter of which worked out for you, so there is no other reason than perhaps a scientific one to perform the remaining tests.

So, please keep pf as long as it works for you.


----------



## n00balert (Jun 24, 2013)

Update: I spoke too soon. The problem is still here even with no bridging. I hibernated a Windows client and when it came back up it had received an IP address from the modem's DHCP server. I then rebooted it and the same thing happened. It's when I turned it off and turned it back on that it was able to get an IP from FreeBSD's DHCP server.

I was also experimenting with Xen on a Linux installation - I created a xenbr0 interface with DHCP and noticed while it was coming up it had discovered both 10.0.0.254 and 192.168.100.1 as DHCP servers and taken 192.168.100.1 as its DHCP server and acquired an address. I have a strong feeling that this would exhibit the same behavior as the windows client.

@rolfheinrich, I guess I'm gonna going to have to give ipfw a shot. Unless there is something else wrong with my setup.


----------



## wblock@ (Jun 25, 2013)

Why not just fix the PF rules?  The firewall is not the problem, you'll need a working ruleset either way.


----------



## Anonymous (Jun 25, 2013)

n00balert said:
			
		

> @rolfheinrich, I guess I'm going to have to give ipfw a shot. Unless there is something else wrong with my setup.



@wblock is right, the kind of the firewall is not the problem, however, I cannot be of any help in fixing the pf ruleset, I am an ipfw guy. Just in case, you want to give it a shot, here comes a quick and dirty, basic open ipfw/NAT setup -- the test shouldn't take more than _five_ min_utes_.

Create a file named ipfw_quick.sh and enter the following content:

```
#!/bin/sh
kldload ipfw_nat.ko
ipfw flush
ipfw nat 1 config if nfe0 reset
ipfw add allow ip from any to any via em0
ipfw add allow ip from any to any via lo0
ipfw add deny ip from any to any not antispoof in
ipfw add nat 1 ip from any to any via nfe0
ipfw add 65534 allow ip from any to any
```

`# chmod 500 ipfw_quick.sh`
.
Disable pf.
.
Start the ipfw firewall using the just created shell script
`# ipfw_quick.sh`
.
Do your tests.
.
Disable ipfw and re-enable pf by simply restarting your FreeBSD server.


----------



## n00balert (Jun 27, 2013)

Hello @rolfheinrich,

Thank you for answering and the script; I have tried your script and the behavior is still the same. This is what I get from a Linux client;

`sudo /etc/init.d/networking stop && sudo /etc/init.d/networking start`

```
[warn] not deconfiguring network interfaces: network file systems still mounted. ... (warning).
[....] Configuring network interfaces...
Waiting for xenbr0 to get ready (MAXWAIT is 32 seconds).
Internet Systems Consortium DHCP Client 4.2.2
Copyright 2004-2011 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/

Listening on LPF/xenbr0/f4:6d:04:48:4d:33
Sending on   LPF/xenbr0/f4:6d:04:48:4d:33
Sending on   Socket/fallback
DHCPDISCOVER on xenbr0 to 255.255.255.255 port 67 interval 6
DHCPREQUEST on xenbr0 to 255.255.255.255 port 67
DHCPOFFER from 192.168.100.1
DHCPNAK from 10.0.0.254
DHCPDISCOVER on xenbr0 to 255.255.255.255 port 67 interval 5
DHCPREQUEST on xenbr0 to 255.255.255.255 port 67
DHCPOFFER from 192.168.100.1
DHCPNAK from 10.0.0.254
DHCPACK from 192.168.100.1
bound to 192.168.100.10 -- renewal in 14 seconds.
done.
```

But this doesn't always happen, once I power client off - power it back on, it gets an IP from the FreeBSD Server, even when restarting networking.

This is what my dhcpd.conf looks like;

`more /usr/local/etc/dhcpd.conf`

```
# dhcpd.conf
#
# Sample configuration file for ISC dhcpd
#

# option definitions common to all supported networks...
option domain-name "xxxxxxxxxxxx.com";
option domain-name-servers 10.0.0.254;
option subnet-mask 255.255.255.0;

default-lease-time 600;
max-lease-time 7200;

# Use this to enble / disable dynamic dns updates globally.
#ddns-update-style none;

# If this DHCP server is the official DHCP server for the local
# network, the authoritative directive should be uncommented.
authoritative;

# Use this to send dhcp log messages to a different log file (you also
# have to hack syslog.conf to complete the redirection).
log-facility local7;

# This is a very basic subnet declaration.

subnet 10.0.0.0 netmask 255.255.255.0 {
  range 10.0.0.100 10.0.0.150;
  option routers 10.0.0.254;
}
```


----------



## n00balert (Jun 27, 2013)

@wblock@, I'm trying to figure out that part.


----------



## MisterDX (Jul 31, 2013)

You could remove your router and have FreeBSD obtain a lease directly from your ISP as you do not seem to use the router for anything else.


----------



## n00balert (Jul 31, 2013)

Hi @MisterDX,

Thanks for responding. However, my "router" is a cable modem that connects to the ISP over COAX then connects to the FreeBSD server over ethernet.

I thought the issue would be resolved by firewall rules. However, I upgraded FreeBSD to -HEAD and added all my DHCP clients as static clients in dhcpd.conf - and that seems to have "fixed" my issue.


----------



## throAU (Aug 1, 2013)

Why do you have a bridge interface configured in FreeBSD?

If I understand your situation correctly, this is not required and most definitely not desirable, as you are trying to use your FreeBSD machine as a router, not a bridge.  This is why your clients are seeing the cable modem's DHCP server - you're bridging the traffic to it.

If you're still getting cable modem assigned IPs on your clients after you think you have turned bridging off, then either bridging is still working, or it is an old lease - make sure to flush the leases out on the client with whatever commands are appropriate for the client OS.

There is no way your clients should be hitting the cable modem for DHCP if bridging is not enabled, whether or not you have a firewall or not.  I.e., the problem is not firewall rule related.

You should not need to upgrade to -HEAD to fix this, this is basic functionality that I've personally used in FreeBSD 4... I suspect it has started working after the -HEAD upgrade due to lease timeouts or such - not because you're running a different OS.


----------



## n00balert (Aug 1, 2013)

@throAU, It's puzzling for me too. ISP interface i.e. nfe0 is not part of any bridge. It comes up on its own with DHCP enabled.


----------



## throAU (Aug 1, 2013)

You have configured a bridge0 interface though.

Why?  If it isn't needed, I'd get rid of it entirely.


----------



## n00balert (Aug 2, 2013)

I'm bridging eth0 wlan0 tap0 for ease of management. (Laziness in other words).


----------



## throAU (Aug 2, 2013)

n00balert said:
			
		

> I'm bridging eth0,wlan0,tap0 for ease of management. (Laziness in other words).



LOL, at least you admit it  Pro-tip though:  Laziness on initial setup bites you down the track. However, that aside...

If you're definitely still seeing DHCP traffic between your modem and your clients (you should be able to verify this by running `wireshark` (or equivalent text mode packet capture) on nfe0 and watching for DHCP when you plug in or reboot a client machine) I suspect you may have run into a bug where FreeBSD is still bridging to nfe0 despite not being included in the bridge.

I suspect what you are doing (bridging to some interfaces, but not all) is not a common FreeBSD usage scenario, and it would not surprise me to see a bug in that situation.


----------



## MisterDX (Aug 2, 2013)

I thought you wanted different IP space (subnets) for WLAN and LAN clients? In that case, you should set up DHCP with specific settings for each NIC em0 and wlan0 (and disable bridging) but I think you need to fix routing first. 

Can you post a 'traceroute to google.com' from either your LAN-facing interfaces or better yet your Windows or MAC system? Both from the 10.xx and 192.xx leases if possible, and can you confirm that both leases have internet connectivity simultaneously or is it just one or the other?

The nfe0 needs to be in 'route' mode. In that case, broadcasts from your em0 and wlan0 would not get to your Cisco device and as a result not offer a lease.


----------



## n00balert (Aug 3, 2013)

@throAU, if what you're saying is true, then I will probably check at some point with Wireshark to see what's really going on.

Hi @MisterDX, I did, correct. But I dropped the idea for laziness/ease of management. With modem lease (192.168.100.x) I don't get Internet connectivity.

`traceroute` from Mac client;

```
myosx:~ n00b$ traceroute google.ca
 traceroute: Warning: google.ca has multiple addresses; using 173.194.33.31
 traceroute to google.ca (173.194.33.31), 64 hops max, 52 byte packets
  1  myhost.xxxxxxxx.com (10.0.0.254)  1.159 ms  0.669 ms  0.723 ms
  2  * * *
  3  xx.xx.xxx.xxx (xx.xx.xxx.xxx)  15.786 ms  10.752 ms  15.954 ms
  4  rc2bb-tge0-0-0-0.vc.shawcable.net (66.163.69.141)  13.753 ms  17.734 ms    15.820 ms
  5  rc2wt-pos1-0.wa.shawcable.net (66.163.76.142)  17.684 ms  27.036 ms  36.023 ms
  6  72.14.196.254 (72.14.196.254)  15.061 ms  26.132 ms  13.898 ms
  7  66.249.94.212 (66.249.94.212)  16.696 ms  14.867 ms  44.494 ms
  8  209.85.253.24 (209.85.253.24)  23.591 ms  15.107 ms  13.935 ms
  9  sea09s01-in-f31.1e100.net (173.194.33.31)  13.796 ms  14.209 ms  15.259 ms
```

`traceroute` from FreeBSD server:

```
$ traceroute google.ca
traceroute: Warning: google.ca has multiple addresses; using 173.194.33.23
traceroute to google.ca (173.194.33.23), 64 hops max, 52 byte packets
 1  * * *
 2  xx.xx.xxx.xxx (xx.xx.xxx.xxx)  10.252 ms  14.974 ms  16.105 ms
 3  rc2bb-tge0-4-0-0.vc.shawcable.net (66.163.69.45)  9.403 ms  19.356 ms  15.680 ms
 4  66.163.77.78 (66.163.77.78)  13.431 ms  15.976 ms  14.553 ms
 5  72.14.195.254 (72.14.195.254)  39.550 ms  13.458 ms  13.541 ms
 6  66.249.94.212 (66.249.94.212)  12.040 ms  13.498 ms  17.573 ms
 7  209.85.253.24 (209.85.253.24)  13.361 ms  13.387 ms  28.282 ms
 8  sea09s01-in-f23.1e100.net (173.194.33.23)  13.372 ms  15.431 ms  12.498 ms
```

I should probably remind you that the issue hasn't happened again, yet.

Thanks for your time and interest!


----------

