# PF Optimization



## cj (Jul 5, 2013)

I want to optimize my FreeBSD pf NAT firewall and my questions are: 

What exactly controls how much RAM pf will use for state tables and other data it may use in RAM? Also, are there any must-have configurations settings for traffic on the ISP level?

I have two FreeBSD 9.1, i5, 16 GB of RAM machines with 6 NICs each. Two NICs in one LAGG group (inside) and two more in another LAGG group (outside), using carp and one interface for pfsync and the last one for the management interface. I have the current stats below for the active firewall but I want to optimize for 700,000 table entries. Right now I'm using half that with 60% idle CPU and around 1.1 TB of data inbound daily.



```
Mem: 52M Active, 34M Inact, 1134M Wired, 1226M Buf, 14G Free

load averages:  2.87,  2.67,  2.63     
58.5% idle
```


```
root@ # pfctl -si
Status: Enabled for 3 days 18:29:48           Debug: Urgent

Interface Stats for lagg2             IPv4             IPv6
  Bytes In                  15352428125696                0
  Bytes Out                  1939415056495                0
  Packets In
    Passed                     12771296874                0
    Blocked                        6968223                0
  Packets Out
    Passed                      7750839635                0
    Blocked                         527616                0

State Table                          Total             Rate
  current entries                   339464               
  searches                     51784113403       158950.3/s
  inserts                        381900325         1172.2/s
  removals                       381660861         1171.5/s
Counters
  match                          366022632         1123.5/s
  bad-offset                             0            0.0/s
  fragment                           33642            0.1/s
  short                               6501            0.0/s
  normalize                           9315            0.0/s
  memory                                 0            0.0/s
  bad-timestamp                    1154571            3.5/s
  congestion                             0            0.0/s
  ip-option                            210            0.0/s
  proto-cksum                            0            0.0/s
  state-mismatch                   2343282            7.2/s
  state-insert                      525918            1.6/s
  state-limit                            0            0.0/s
  src-limit                              0            0.0/s
  synproxy                               0            0.0/s
```



```
root@ # vmstat -z
ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP

UMA Kegs:               208,      0,     105,      14,     105,   0,   0
UMA Zones:              896,      0,     105,       3,     105,   0,   0
UMA Slabs:              568,      0,    5301,       5,    5722,   0,   0
UMA RCntSlabs:          568,      0,   11573,       5,   11573,   0,   0
UMA Hash:               256,      0,       2,      13,       3,   0,   0
16 Bucket:              152,      0,     153,      22,     153,   0,   0
32 Bucket:              280,      0,     169,      13,     169,   0,   0
64 Bucket:              536,      0,     145,       2,     145,  57,   0
128 Bucket:            1048,      0,   12311,       1,   12311, 763,   0
VM OBJECT:              232,      0,   37569,     287,  354587,   0,   0
MAP:                    232,      0,       7,      25,       7,   0,   0
KMAP ENTRY:             120, 531185,      41,     486,   11045,   0,   0
MAP ENTRY:              120,      0,    1793,     377,  872259,   0,   0
fakepg:                 120,      0,       0,       0,       0,   0,   0
mt_zone:               4112,      0,     302,       9,     302,   0,   0
16:                      16,      0,    2658,     534,  515209,   0,   0
32:                      32,      0,    3491,     650, 1373615,   0,   0
64:                      64,      0,   12422,    1634,40462387557,   0,   0
128:                    128,      0,    8287,     616,   71550,   0,   0
256:                    256,      0,     913,     647,  245440,   0,   0
512:                    512,      0,    1279,     303,  111756,   0,   0
1024:                  1024,      0,      77,     179,   56581,   0,   0
2048:                  2048,      0,    5146,     364,   39633,   0,   0
4096:                  4096,      0,     308,     133,   28234,   0,   0
Files:                   80,      0,     139,     311, 2637289,   0,   0
TURNSTILE:              136,      0,     286,      74,     286,   0,   0
umtx pi:                 96,      0,       0,       0,       0,   0,   0
MAC labels:              40,      0,       0,       0,       0,   0,   0
PROC:                  1184,      0,      73,      89,   19934,   0,   0
THREAD:                1128,      0,     242,      43,     275,   0,   0
SLEEPQUEUE:              80,      0,     286,     120,     286,   0,   0
VMSPACE:                392,      0,      54,      76,   19898,   0,   0
cpuset:                  72,      0,      95,      55,     111,   0,   0
audit_record:           960,      0,       0,       0,       0,   0,   0
mbuf_packet:            256,      0,   20453,    1691,23430797474,   0,   0
mbuf:                   256,      0,    1025,    1161,19543754432,   0,   0
mbuf_cluster:          2048, 128768,   22145,     821,3283952055,   0,   0
mbuf_jumbo_page:       4096, 262144,       0,      90,    5908,   0,   0
mbuf_jumbo_9k:         9216,  64000,       0,       0,       0,   0,   0
mbuf_jumbo_16k:       16384,  32000,       0,       0,       0,   0,   0
mbuf_ext_refcnt:          4,      0,       0,       0,       0,   0,   0
g_bio:                  232,      0,       0,    4512,  295500,   0,   0
ttyinq:                 160,      0,     300,     156,     735,   0,   0
ttyoutq:                256,      0,     157,     128,     384,   0,   0
ata_request:            328,      0,       0,       0,       0,   0,   0
ata_composite:          336,      0,       0,       0,       0,   0,   0
VNODE:                  480,      0,  119594,     214, 2262197,   0,   0
VNODEPOLL:              112,      0,       0,       0,       0,   0,   0
NAMEI:                 1024,      0,       0,      96, 7696461,   0,   0
S VFS Cache:            108,      0,  100127,   24151, 1174240,   0,   0
STS VFS Cache:          148,      0,       0,       0,       0,   0,   0
L VFS Cache:            328,      0,   25648,   58136, 1144443,   0,   0
LTS VFS Cache:          368,      0,       0,       0,       0,   0,   0
NCLNODE:                568,      0,       0,       0,       0,   0,   0
DIRHASH:               1024,      0,    5398,      54,    5400,   0,   0
pipe:                   728,      0,       4,      86,    9183,   0,   0
Mountpoints:            792,      0,       5,      10,       5,   0,   0
ksiginfo:               112,      0,     184,     872,   17776,   0,   0
itimer:                 344,      0,       0,      22,       1,   0,   0
pfsrctrpl:              152, 400000,       0,       0,       0,   0,   0
pfrulepl:               936,      0,     167,     173,     462,   0,   0
pfstatepl:              288, 600002,  237842,  128498,393787401,   0,   0
pfstatekeypl:           288,      0,  330002,  174268,570051213,   0,   0
pfstateitempl:          288,      0,  330005,  174291,567485893,   0,   0
pfaltqpl:               240,      0,       0,       0,       0,   0,   0
pfpooladdrpl:            88,      0,     142,     236,     384,   0,   0
pfrktable:             1296,  10002,      33,      99,     296,   0,   0
pfrkentry:              160, 600000,      61,     179,     459,   0,   0
pfrkcounters:            64,      0,       0,       0,       0,   0,   0
pffrent:                 32,  10100,       0,     909, 7457768,   0,   0
pffrag:                  80,      0,       0,     540, 3321766,   0,   0
pffrcache:               80,  10035,       0,       0,       0,   0,   0
pffrcent:                24,  50022,       0,       0,       0,   0,   0
pfstatescrub:            40,      0,  400147,  200453,412251111,   0,   0
pfiaddrpl:              120,      0,       0,       0,       0,   0,   0
pfospfen:               112,      0,     700,     125,    5600,   0,   0
pfosfp:                  40,      0,     410,     430,    3280,   0,   0
KNOTE:                  128,      0,       8,     195,    5940,   0,   0
socket:                 680,  25602,      34,      86,   10562,   0,   0
ipq:                     56,   1638,       0,       0,       0,   0,   0
udp_inpcb:              392,  25600,       6,      84,    8193,   0,   0
udpcb:                   16,  25704,       6,     666,    8193,   0,   0
tcp_inpcb:              392,  25600,       9,      61,     152,   0,   0
tcpcb:                  976,  25600,       9,      39,     152,   0,   0
tcptw:                   72,   5150,       0,     250,      28,   0,   0
syncache:               152, 1048600,       0,     125,      20,   0,   0
hostcache:              136,  15372,       0,     112,      13,   0,   0
tcpreass:                40,   8064,       0,     252,       4,   0,   0
sackhole:                32,      0,       0,     606,    1886,   0,   0
sctp_ep:               1376,  25600,       0,       0,       0,   0,   0
sctp_asoc:             2288,  40000,       0,       0,       0,   0,   0
sctp_laddr:              48,  80064,       0,     360,     175,   0,   0
sctp_raddr:             704,  80000,       0,       0,       0,   0,   0
sctp_chunk:             136, 400008,       0,       0,       0,   0,   0
sctp_readq:             104, 400032,       0,       0,       0,   0,   0
sctp_stream_msg_out:    112, 400026,       0,       0,       0,   0,   0
sctp_asconf:             40, 400008,       0,       0,       0,   0,   0
sctp_asconf_ack:         48, 400032,       0,       0,       0,   0,   0
ripcb:                  392,  25600,       0,      50,       8,   0,   0
unpcb:                  240,  25600,      18,      78,    2196,   0,   0
rtentry:                200,      0,     108,      63,     179,   0,   0
pfsync:                  88,      0,       0,     462,  817608,   0,   0
selfd:                   56,      0,     145,     737, 6054404,   0,   0
SWAPMETA:               288, 116519,       0,       0,       0,   0,   0
FFS inode:              168,      0,  119556,     212, 2262109,   0,   0
FFS1 dinode:            128,      0,       0,       0,       0,   0,   0
FFS2 dinode:            256,      0,  119556,     219, 2262109,   0,   0
```


----------

