# PF state key linking mismatch



## mlager (Dec 11, 2011)

I've got a raw GIF tunnel set up between an 8.2-RELEASE system and an 9.0-RC3 system. I haven't yet setup IPSEC to encrypt the traffic, so this is just a raw IP-IP tunnel. The tunnel function properly, each side can connect to eachother's network, so it seems I'm good with my firewall settings, GIF setup, and static routes.

Only on the 9.0-RC3 system, I see a lot of errors for connections that read:


```
pf: state key linking mismatch! dir=OUT, if=re1, stored af=2, a0: B.B.B.B, a1: A.A.A.A, proto=4, found af=2, a0: 172.16.1.2:80, a1: 172.16.2.1:52102, proto=6.
```

I don't see these errors on 8.2-RELEASE and the error seems to disrupt network performance. Is this a known issue in 9.0-RC3, or do I have a setup issue? Here is my configuration on each endpoint, I've masked public IP addresses as A.A.A.A and B.B.B.B:


```
[B]ENDPOINT 1:[/B]

[file]/etc/rc.conf:[/file]
gif_interfaces="gif0"
gifconfig_gif0="A.A.A.A B.B.B.B"
ifconfig_gif0="inet 172.16.1.1 172.16.2.1 netmask 255.255.255.0"
static_routes="tslbell"
route_tslbell="-net 172.16.2.0/24 172.16.2.1"

[file]/etc/pf.conf:[/file]
# MACROS
ext_if="re0"
int_if="re1"
internal_net="172.16.1.0/24"

# NORMALIZATION
scrub in all

# NETWORK ADDRESS TRANSLATION
nat on $ext_if from $internal_net to any -> ($ext_if)

# FILTERING
set skip on gif0

pass in all
pass out all

block in log all
pass quick on lo0 all
pass quick on $int_if all

# ENABLE INBOUND ICMP
pass in on $ext_if proto icmp all keep state

pass out on $ext_if proto { tcp, udp, icmp } all keep state
```


```
[B]ENDPOINT 2:[/B]

[file]/etc/rc.conf:[/file]
gifconfig_gif0="B.B.B.B A.A.A.A"
ifconfig_gif0="inet 172.16.2.1 172.16.1.1 netmask 255.255.255.0"
static_routes="belltsl"
route_belltsl="-net 172.16.1.0/24 172.16.1.1"


[file]/etc/pf.conf:[/file]
# MACROS
ext_if="lagg0"
int_if="bge0"
internal_net="172.16.2.0/24"

# NORMALIZATION
scrub in all

# NETWORK ADDRESS TRANSLATION
nat on $ext_if from $internal_net to any -> ($ext_if)

# FILTERING
set skip on gif0

pass in all
pass out all

block in log all
pass quick on lo0 all
pass quick on $int_if all

# ENABLE INBOUND ICMP
pass in on $ext_if proto icmp all keep state

pass out on $ext_if proto { tcp, udp, icmp } all keep state
```

Any help identifying the issue here would be great. I'd think if I had a major setup issue, connectivity wouldn't exist between the two endpoints, but they do, so I'm a bit stuck. Obviously this is something at the PF level, but I don't have anything really fancy in PF. I've tried removing the "scrub in all" lines but that doesn't work either.

Thanks again.


----------



## mlager (Dec 19, 2011)

FYI, because of a lack of information about this issue, and my inability to find anything on the web, I've logged a bug report. This is the link:

http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/163208

Thx.


----------



## gkontos (Feb 2, 2013)

It seems that the problem still exists. I submitted a followup.


----------



## J65nko (Feb 2, 2013)

```
set skip on gif0

[color=blue]pass in all[/color]
pass out all

[color=blue]block in log all[/color]
pass quick on lo0 all
pass quick on $int_if all
```

pf uses a last match wins strategy so this version below is equivalent:

```
set skip on gif0

[color=blue]#pass in all[/color]
pass out all

[color=blue]block in log all[/color]
pass quick on lo0 all
pass quick on $int_if all
```

And because all traffic on the loopback and internal interface is allowed, this can be simplified to

```
set skip on gif0
set skip on lo0
set skip on $int_if

[color=blue]#pass in all[/color]
pass out all

[color=blue]block in log all[/color]
#pass quick on lo0 all
#pass quick on $int_if all
```
It seems that the security policy is to allow all connections initiated from the "inside" and block connections initiated from the "outside". If this is correct, this can be even more simplified to:


```
# allow all gif0, lo0 and internal interface traffice
set skip on gif0
set skip on lo0
set skip on $int_if

# allow all outgoing connections (initiated from 'inside") and the reply packets
pass out quick all

# block and log everything else
block log all
```
Or am I missing something?


----------



## gkontos (Feb 2, 2013)

J65nko said:
			
		

> Or am I missing something?




```
ext_if="tun0"
int_if="re0"
icmp_types = "{ echorep, echoreq, timex, paramprob, unreach code needfrag }"
vpn_gateway = "{XXXX:XXXX:1001:5f00::86}"

scrub in on $ext_if all fragment reassemble

set skip on lo0

block in log quick from 10.1.1.0/24 to any
block in log

pass out log on $ext_if keep state
pass log on $int_if

pass in inet proto icmp all icmp-type $icmp_types
pass on $ext_if proto icmp6 all

pass in log on $ext_if proto esp from $vpn_gateway to any 
pass in log on $ext_if proto ah from $vpn_gateway to any
pass in log on $ext_if proto ipencap from $vpn_gateway to any
pass in log on $ext_if inet6 proto udp from $vpn_gateway to any port 500
```

I hope not 


```
Feb  2 22:23:34 hp kernel: pf: state key linking mismatch! dir=OUT, if=re0, stored af=28, a0: 2607:fc50:1001:5f00::86, a1: 2a02:580:8f00:2c00::2093, proto=50, found af=2, a0: 10.30.1.140:54088, a1: 10.1.1.1:22, proto=6.
Feb  2 22:23:34 hp kernel: pf: state key linking mismatch! dir=OUT, if=re0, stored af=28, a0: 2607:fc50:1001:5f00::86, a1: 2a02:580:8f00:2c00::2093, proto=50, found af=2, a0: 10.30.1.140:54088, a1: 10.1.1.1:22, proto=6.
Feb  2 22:23:34 hp kernel: pf: state key linking mismatch! dir=OUT, if=re0, stored af=28, a0: 2607:fc50:1001:5f00::86, a1: 2a02:580:8f00:2c00::2093, proto=50, found af=2, a0: 10.30.1.140:54088, a1: 10.1.1.1:22, proto=6.
Feb  2 22:23:34 hp kernel: pf: state key linking mismatch! dir=OUT, if=re0, stored af=28, a0: 2607:fc50:1001:5f00::86, a1: 2a02:580:8f00:2c00::2093, proto=50, found af=2, a0: 10.30.1.140:54088, a1: 10.1.1.1:22, proto=6.
Feb  2 22:23:34 hp kernel: pf: state key linking mismatch! dir=OUT, if=re0, stored af=28, a0: 2607:fc50:1001:5f00::86, a1: 2a02:580:8f00:2c00::2093, proto=50, found af=2, a0: 10.30.1.140:54088, a1: 10.1.1.1:22, proto=6.
Feb  2 22:23:34 hp kernel: pf: state key linking mismatch! dir=OUT, if=re0, stored af=28, a0: 2607:fc50:1001:5f00::86, a1: 2a02:580:8f00:2c00::2093, proto=50, found af=2, a0: 10.30.1.140:54088, a1: 10.1.1.1:22, proto=6.
```


----------



## J65nko (Feb 3, 2013)

According to one of the main pf developers in http://marc.info/?l=openbsd-misc&m=131210750223652&w=2 you can simply ignore these messages


----------



## gkontos (Feb 3, 2013)

J65nko said:
			
		

> According to one of the main pf developers in http://marc.info/?l=openbsd-misc&m=131210750223652&w=2 you can simply ignore these messages



I do ignore them believe me! But my logs are filled with those messages 

As a side note. It appears that gif(4)() is not honored at all in PF. 

[CMD=""]gkontos@bsd-vpn:/home/gkontos % ping 10.30.1.1[/CMD]

```
PING 10.30.1.1 (10.30.1.1): 56 data bytes
64 bytes from 10.30.1.1: icmp_seq=0 ttl=64 time=225.892 ms
64 bytes from 10.30.1.1: icmp_seq=1 ttl=64 time=225.731 ms
64 bytes from 10.30.1.1: icmp_seq=2 ttl=64 time=226.655 ms
64 bytes from 10.30.1.1: icmp_seq=3 ttl=64 time=225.531 ms
```

Capture from 10.30.1.1:

[CMD=""]root@hp:/home/gkontos # tcpdump -n -i gif0[/CMD]

```
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on gif0, link-type NULL (BSD loopback), capture size 65535 bytes
13:56:45.146833 IP 10.1.1.3 > 10.30.1.1: ICMP echo request, id 52747, seq 0, length 64
13:56:45.146857 IP 10.30.1.1 > 10.1.1.3: ICMP echo reply, id 52747, seq 0, length 64
```

[CMD=""]root@hp:/home/gkontos # tcpdump -n -e -ttt -i pflog0  host 10.1.1.3[/CMD]

```
tcpdump: WARNING: pflog0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size 65535 bytes
```

Nothing in the PF logs which makes it impossible to filter inbound traffic from the other side.


----------



## gkontos (Feb 3, 2013)

Modifying usr/src/sys/contrib/pf/net/pf.c did the trick at least for the messages:


```
else {
                /* mismatch. must not happen.
                printf("pf: state key linking mismatch! dir=%s, "
                    "if=%s, stored af=%u, a0: ",
                    dir == PF_OUT ? "OUT" : "IN", kif->pfik_name, a->af);
                pf_print_host(&a->addr[0], a->port[0], a->af);
                printf(", a1: ");
                pf_print_host(&a->addr[1], a->port[1], a->af);
                printf(", proto=%u", a->proto);
                printf(", found af=%u, a0: ", b->af);
                pf_print_host(&b->addr[0], b->port[0], b->af);
                printf(", a1: ");
                pf_print_host(&b->addr[1], b->port[1], b->af);
                printf(", proto=%u", b->proto);
                printf(".\n");[B]*/[/B]
                return (-1);
        }
```


----------



## dvl@ (Sep 10, 2014)

This is still a problem on FreeBSD 9.3


----------

