# Initial setting of hw.intr_storm_threshold too low for new processors?



## torqueturns (Aug 18, 2009)

While upgrading FreeBSD and hardware, we've run into issues with 10Gig ethernet drivers and expected processing power until changing the value of hw.intr_storm_threshold from the default of 1000 to 40000.

Both "systat -v 1" and "vmstat -i" confirm that even the processor timer seems to be generating 2000 interrupts/second, so is the inital value hampering the processing power of the new systems? 


This machine is pretty idle running FBSD 7.2-RELEASE-p3

```
> vmstat -i
interrupt                          total       rate
irq1: atkbd0                           8          0
irq6: fdc0                            10          0
irq15: ata1                        13713          8
irq52: amr0                         3549          2
irq59: em1                           434          0
cpu0: timer                      3421359       1999
cpu3: timer                      3421368       1999
cpu1: timer                      3421368       1999
cpu2: timer                      3421325       1999
Total                           13703134       8008
```

This machine is a coreI7 940 machine with some ftp load.
7.2-RELEASE-p2 of the OS.


```
> vmstat -i
interrupt                          total       rate
irq19: fwohci0+                  9923210         28
irq41: ixgb0                  1319628798       3805
cpu0: timer                    693204729       1999
irq257: re1                       291212          0
cpu6: timer                    693194426       1999
cpu1: timer                    693194695       1999
cpu2: timer                    693194565       1999
cpu5: timer                    693194407       1999
cpu4: timer                    693194409       1999
cpu7: timer                    693194425       1999
cpu3: timer                    693194567       1999
Total                         6875409443      19827
```

Same Corei7 940 only "systat -v 1" output


```
2 users    Load  3.17  2.82  2.66                  Aug 18 15:53

Mem:KB    REAL            VIRTUAL                       VN PAGER   SWAP PAGER
        Tot   Share      Tot    Share    Free           in   out     in   out
Act   32260    5916   247400     8108 1380704  count
All  255640    9944  4705480    83488          pages
Proc:                                                            Interrupts
  r   p   d   s   w   Csw  Trp  Sys  Int  Sof  Flt  95346 cow   29328 total
  2         152      121k 116k 334k  13k  417 110k   5903 zfod        fwohci0+ 1
                                                          ozfod 13331 ixgb0 41
24.2%Sys   2.3%Intr  7.0%User  0.0%Nice 66.5%Idle        %ozfod  1999 cpu0: time
|    |    |    |    |    |    |    |    |    |    |       daefr     4 re1 irq257
============+>>>>                                   96292 prcfr  1999 cpu6: time
                                       302 dtbuf   125430 totfr  1999 cpu1: time
Namei     Name-cache   Dir-cache    100000 desvn          react  1999 cpu2: time
   Calls    hits   %    hits   %     78996 numvn          pdwak  1999 cpu5: time
   47607   41650  87                 24998 frevn          pdpgs  1999 cpu4: time
                                                       15 intrn  1999 cpu7: time
Disks   ad4   ad6   ad8                            277316 wire   2000 cpu3: time
KB/t   0.00  0.00  0.00                             26824 act
tps       0     0     0                           1320476 inact
MB/s   0.00  0.00  0.00                                   cache
%busy     0     0     0                           1379156 free
```


----------



## Alt (Aug 19, 2009)

I think at this interrupt rate better to use polling(4)


----------



## torqueturns (Aug 20, 2009)

Alt said:
			
		

> I think at this interrupt rate better to use polling(4)



My understanding of polling was that you could not use it with an SMP capable kernel.


----------



## Alt (Aug 21, 2009)

> Remove recently added note about DEVICE_POLLING not working with SMP.
> Remove warning from kern_poll.c to allow DEVICE_POLLING to be built with SMP.


That was more than 4 years ago =)
I recommend to use it, and, if this servers is mainly router, you can set `sysctl kern.polling.idle_poll=1` to increase throughput. Just google some about tuning and you can really rock with this hardware 

p.s. `hw.intr_storm_threshold` its not number of interrupts/sec, so values with your cpu clock is ok =)


----------



## torqueturns (Aug 21, 2009)

Great!

I will try polling the interface..

The machine is being used to generate ftp traffic, which flows thru our RF router.  We are trying to scale to 10,000 concurrent connections.


----------



## Alt (Aug 21, 2009)

Well, your need to tune mbufs, tcp and disk sysctls then =)


----------



## torqueturns (Aug 25, 2009)

I have done that already using a variation of the information found at this url

```
"http://rerepi.wordpress.com/2008/04/19/tuning-freebsd-sysoev-rit/"
```

I have even tuned the transmit buffer space in the ixgb driver.  It seems that the last tweak was the sysctl variable hw.intr_storm_threshold, once I bumped it up, everything is running.  No more issues with the 10Gig ethernet card.  

I had just wondered about the relation of what that sysctl variable had on the amount of interrupts reported by vmstat -i or systat -v 1, for the newer processors.


----------

