# FreeBSD 10 router, pf, problems with mpd5 as a pppoe server



## kktr (Oct 6, 2014)

Hi,

I am trying to add mpd5 to my existing FreeBSD 10 router so that it can act as a PPPoE server. Including relevant configurations from my setup. Hopefully some flaws can be found which I would be grateful for pointing out:

rc.conf

```
## hostname
hostname="freebsd"

## Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"

## network
ifconfig_fxp0="inet aaa.bbb.ccc.ddd netmask 255.255.255.0 group wanif"
ifconfig_fxp1="inet 10.10.10.1 netmask 255.255.255.0 group lanif" 
ifconfig_fxp1_alias0="inet 10.10.20.1 netmask 255.255.255.0 group lanif"
defaultrouter="aaa.bbb.ccc.1" 

## ssh
ssh_enable="YES"

## pf
pf_enable="YES"

## gateway
gateway_enable="YES"
```

pf.conf

```
#note: I am making use of ifconfig groups here

table <ssh_abuse> persist

set skip on lo0
set block-policy return

scrub all fragment reassemble

nat on wanif from !(wanif) to any -> (wanif)

antispoof quick for fxp1
antispoof quick for fxp2

block all

block in quick on wanif from <ssh_abuse>

pass in quick on wanif inet proto icmp from any to wanif keep state
pass in quick on wanif proto tcp from any to wanif port { http, https }
pass in quick on wanif proto tcp from any to wanif port ssh flags S/SA keep state \
(max-src-conn 5, max-src-conn-rate 10/30, overload <ssh_abuse> flush)

pass out quick on wanif keep state
pass out quick on !wanif from any to !(wanif:network) keep state

# might use altq in the future individually on each ip address thus the separate rules for the time being
pass in quick on lanif from 10.10.10.2 to any keep state
pass in quick on lanif from 10.10.10.3 to any keep state
pass in quick on lanif from 10.10.20.2 to any keep state
pass in quick on lanif from 10.10.20.3 to any keep state
```

/etc/ethers
I generate a static ARP for my both networks (10.10.10.1/24 and 10.10.20.1/24) and only include the allowed MAC addresses. Everything else is blocked by a zeroed MAC. File is loaded by `arp -f /etc/ethers`.

```
10.10.10.1		00:00:00:00:00:00
10.10.10.2		C8:E2:EB:E1:E1:FB
10.10.10.3		CE:1A:82:14:A3:9D
10.10.10.4		00:00:00:00:00:00
# ... skipped 
10.10.10.254	00:00:00:00:00:00

10.10.20.1		00:00:00:00:00:00
10.10.20.2		7A:EB:60:3D:39:46
10.10.20.3		81:30:36:B2:9E:D0
10.10.20.4		00:00:00:00:00:00
# ... skipped
10.10.20.254	00:00:00:00:00:00
```

At this point things work great. To connect to the router from the LAN all it takes is to set[]up a proper configuration (and have the correct MAC address), e.g
IP address: 10.10.20.2
mask:        255.255.255.0
gateway:   10.10.20.1
DNS:           8.8.8.8

So I decided to add a PPPoE server by installing and configuring mpd5. 

```
pkg install mpd5-5.7_1
echo 'mpd_enable="YES"' >> /etc/rc.conf
mv /usr/local/etc/mpd5/mpd.conf.sample mpd.conf
mv /usr/local/etc/mpd5/mpd.secret.sample mpd.secret
service mpd5 start
```

mpd.secret

```
test123         test123         10.10.10.2
```

mpd.conf

```
startup:
        set user foouser bartemporary admin

        # configure the console
        set console self 127.0.0.1 5005
        set console open

        # configure the web server
        set web self 0.0.0.0 5006
        set web open

pppoe_shared:
        create link template L_pppoe pppoe
        set link action bundle B_pppoe
        set link disable pap
        set link disable eap
        set link enable chap
        set link mtu 1492
        set link mru 1492 
        set link keep-alive 10 80

pppoe1:
        create link pppoe1_link L_pppoe
        set pppoe iface fxp1
        set pppoe service pppoe1_service
        set link enable incoming 

default:
        create bundle template B_pppoe 

        set iface enable tcpmssfix
 
        set iface enable proxy-arp
        set iface route default

        # so that i can reference the ngX interface from pf as 'lanif'
        set iface group lanif
 
        set ipcp enable vjcomp 
        set ipcp ranges 10.10.20.1 10.10.20.0/24   
        set ipcp dns 8.8.8.8 208.67.220.220

        load pppoe_shared
        load pppoe1
```

When connecting to the PPPoE server I use pppoe1_service as the service name, and 'test123' as user and password.

*PPPoE connection initiation*

```
Incoming PPPoE connection request via vtnet1: for service "pppoe1_service" from C8:E2:EB:E1:E1:FB
No free PPPoE link with requested parameters was found
[pppoe1_service] PPPoE: connection successful
[pppoe1_service] Link: UP event
[pppoe1_service] LCP: Up event
[pppoe1_service] LCP: state change Starting --> Req-Sent
[pppoe1_service] LCP: SendConfigReq #1
[pppoe1_service]   PROTOCOMP
[pppoe1_service]   MRU 1492
[pppoe1_service]   MAGICNUM 48e14fb2
[pppoe1_service]   AUTHPROTO CHAP MSOFTv2
[pppoe1_service] LCP: rec'd Configure Request #2 (Req-Sent)
[pppoe1_service]   MRU 1492
[pppoe1_service]   MAGICNUM 8b83f8ce
[pppoe1_service] LCP: SendConfigAck #2
[pppoe1_service]   MRU 1492
[pppoe1_service]   MAGICNUM 8b83f8ce
[pppoe1_service] LCP: state change Req-Sent --> Ack-Sent
[pppoe1_service] LCP: SendConfigReq #2
[pppoe1_service]   PROTOCOMP
[pppoe1_service]   MRU 1492
[pppoe1_service]   MAGICNUM 48e14fb2
[pppoe1_service]   AUTHPROTO CHAP MSOFTv2
[pppoe1_service] LCP: rec'd Configure Reject #2 (Ack-Sent)
[pppoe1_service]   PROTOCOMP
[pppoe1_service] LCP: SendConfigReq #3
[pppoe1_service]   MRU 1492
[pppoe1_service]   MAGICNUM 48e14fb2
[pppoe1_service]   AUTHPROTO CHAP MSOFTv2
[pppoe1_service] LCP: rec'd Configure Ack #3 (Ack-Sent)
[pppoe1_service]   MRU 1492
[pppoe1_service]   MAGICNUM 48e14fb2
[pppoe1_service]   AUTHPROTO CHAP MSOFTv2
[pppoe1_service] LCP: state change Ack-Sent --> Opened
[pppoe1_service] LCP: auth: peer wants nothing, I want CHAP
[pppoe1_service] CHAP: sending CHALLENGE #1 len: 21
[pppoe1_service] LCP: LayerUp
[pppoe1_service] CHAP: rec'd RESPONSE #1 len: 61
[pppoe1_service]   Name: "test123"
[pppoe1_service] AUTH: Trying INTERNAL
[pppoe1_service] AUTH: INTERNAL returned: undefined
[pppoe1_service] CHAP: Auth return status: undefined
[pppoe1_service] CHAP: Response is valid
[pppoe1_service] CHAP: Reply message: S=F6902B188DA9BB9A7DE71B438BF6B3D71432CF87
[pppoe1_service] CHAP: sending SUCCESS #1 len: 46
[pppoe1_service] LCP: authorization successful
[pppoe1_service] Link: Matched action 'bundle "B_pppoe" ""'
[pppoe1_service] Creating new bundle using template "B_pppoe".
[B_pppoe-1] Bundle: Interface ng0 created
[pppoe1_service] Link: Join bundle "B_pppoe-1"
[B_pppoe-1] Bundle: Status update: up 1 link, total bandwidth 64000 bps
[B_pppoe-1] IPCP: Open event
[B_pppoe-1] IPCP: state change Initial --> Starting
[B_pppoe-1] IPCP: LayerStart
[B_pppoe-1] IPCP: Up event
[B_pppoe-1] IPCP: state change Starting --> Req-Sent
[B_pppoe-1] IPCP: SendConfigReq #1
[B_pppoe-1]   IPADDR 10.20.11.1
[B_pppoe-1]   COMPPROTO VJCOMP, 16 comp. channels, no comp-cid
[B_pppoe-1] IPCP: rec'd Configure Request #3 (Req-Sent)
[B_pppoe-1]   IPADDR 0.0.0.0
[B_pppoe-1]     NAKing with 10.10.10.2
[B_pppoe-1]   PRIDNS 0.0.0.0
[B_pppoe-1]     NAKing with 8.8.8.8
[B_pppoe-1]   SECDNS 0.0.0.0
[B_pppoe-1]     NAKing with 208.67.220.220
[B_pppoe-1] IPCP: SendConfigNak #3
[B_pppoe-1]   IPADDR 10.10.10.2
[B_pppoe-1]   PRIDNS 8.8.8.8
[B_pppoe-1]   SECDNS 208.67.220.220
[B_pppoe-1] IPCP: rec'd Configure Reject #1 (Req-Sent)
[B_pppoe-1]   COMPPROTO VJCOMP, 16 comp. channels, no comp-cid
[B_pppoe-1] IPCP: SendConfigReq #2
[B_pppoe-1]   IPADDR 10.20.11.1
[B_pppoe-1] IPCP: rec'd Configure Request #4 (Req-Sent)
[B_pppoe-1]   IPADDR 10.10.10.2
[B_pppoe-1]     10.10.10.2 is OK
[B_pppoe-1]   PRIDNS 8.8.8.8
[B_pppoe-1]   SECDNS 208.67.220.220
[B_pppoe-1] IPCP: SendConfigAck #4
[B_pppoe-1]   IPADDR 10.10.10.2
[B_pppoe-1]   PRIDNS 8.8.8.8
[B_pppoe-1]   SECDNS 208.67.220.220
[B_pppoe-1] IPCP: state change Req-Sent --> Ack-Sent
[B_pppoe-1] IPCP: rec'd Configure Ack #2 (Ack-Sent)
[B_pppoe-1]   IPADDR 10.20.11.1
[B_pppoe-1] IPCP: state change Ack-Sent --> Opened
[B_pppoe-1] IPCP: LayerUp
[B_pppoe-1]   10.20.11.1 -> 10.10.10.2
[B_pppoe-1] IFACE: Add route 0.0.0.0/0 10.10.10.2 failed: File exists
[B_pppoe-1] IFACE: Up event
[B_pppoe-1] IFACE: Add group lanif to ng0
```

`ifconfig -v`

```
ng0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1492
        inet 10.10.10.1 --> 10.10.10.2 netmask 0xffffffff
        inet6 fe80::14bd:34c4:df14:bd58%ng0 prefixlen 64 scopeid 0x5
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lanif
```

As can be seen the connection was established, but the device that made the PPPoE connection has no internet access.

The remote device that connected via PPPoE reports:
name: test123
local IP: 10.10.10.2
remote IP: 10.10.10.1
DNS1: 8.8.8.8
DNS2: 208.67.220.220

Any ping from the PPPoE device (10.10.10.2) fails.

However I can ping 10.10.10.2 from my router fine.

`arp -f /etc/ethers` now reports an (error?) message

```
cannot intuit interface index and type for 10.10.10.2
```

I have to admit I am a bit clueless about PPPoE and there's obviously something I have done wrong.

I am not quire sure what is the purpose of set ipcp ranges, set iface enable proxy-arp and set iface defaulr route in mpd.conf despite reading the documentation. But I have a feeling the reason my PPPoE has no internet access is due to these settings.

I would be grateful if someone could pin-point the mistake(s) I have made.


----------



## SirDice (Oct 6, 2014)

*Re: FreeBSD 10 router, pf, problems with mpd5 as a pppoe ser*

I would suggest removing /etc/ethers, and let the system figure it out. It's going to cause all sorts of strange problems.


----------



## kktr (Oct 6, 2014)

*Re: FreeBSD 10 router, pf, problems with mpd5 as a pppoe ser*

So after playing around with it here's the things I apparently got wrong:

a) I should not be assigning an ip address to the pppoe device that is already a part of my network card (configured via ifconfig, in my case 10.10.10.1/24 and 10.10.20.1/24)

This has resulted in a cascade of errors

1) When the PPPoE connection was initiated, the set iface route default command complained that a default route already exists for ip address 10.10.10.2
2) Reloading the static arp table via arp -f /etc/ethers resulted in cannot intuit interface index and type for 10.10.10.2 error since technically the ng0 interface has arp disabled
3) Finally the antispoof quick for fxp1 rule in pf was blocking internet access from the pppoe device. Network 10.10.10.1/24 is defined on fxp1 but an ip belonging to that network (10.10.20.2) was moving packets on a different interface - ng0.

The solution is to e.g change mpd.secret

```
test123    test123     10.10.30.2
```

and change `set ipcp ranges` in mpd.conf to e.g
set ipcp ranges aaa.bbb.ccc.ddd 10.10.30.0/24

(where aaa.bbb.ccc.ddd is my router ip)

Now since the network 10.10.30.0/24 is not defined on the router ( ip addresses from that network are created on demand on each pppoe connection (taken from mpd.conf) - interface ng0, ng1... etc )
I get to keep a static arp mapping in /etc/ethers so there's no fear that someone can easily spoof his ip address without a correct mac. This problem also does not exist for the 10.10.30.0/24 pppoe network because the connections are created on demand and for that to happen a proper password and user needs to be provided.


----------

