# PCI Express card driver load and unload takes too much time



## steevan (Sep 25, 2018)

Hello Folks,

We have  a PCI express card for data processing to achieve 25 to 30 Gbps .
Recently we have been facing a issue in one of the server . System information is given below:

Supermicro motherboard-X11DPH-TQ motherboard
hw.model: Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz  ( Dual CPU with total 20 cores)
16 GB RAM

In this system the same PCIe card works fine in RHEL 7.5 and the driver load and unload also works fine.
Our driver usually takes about 20 seconds to load and approx 20 seconds to unload.

However, on this particular system when FreeBSD ( tried with 11.1 and 11.2   Release) is used, it takes about 2 to 4 minutes to load and about 8 to 30 minutes to unload.
During unload it looks like the system freezes completely and I can not run any commands to find out what is happening .

The same driver works fine in other servers like Dell T620 ( Xeon 12 core CPUs) and desktops with FreeBSD 11.1 and 11.2 and 10.4 .

Also I ran couple of Phoronix tests on this SuperMicro server. In one particular test I see that the time taken is too much.
The test suite is OsBench. In this, the thread creation test shows average time taken to create threads is 9000 usec .  On the other hand we have servers and desktops in our lab and in that this same thread creation takes only 20 to 30 usec .

Any pointers about what could be wrong with the system or our PCIe card driver  ?

Thanks,
Steevan


----------



## steevan (Nov 22, 2018)

I tried FreeBSD 12 Beta 3 version  on this server with Xeon Gold 5115 CPU.
All these problems have disappeared.  Thread creation time has improved greatly .
It is now  below 100 usec ( in FreeBSD 11.x  9000+ usec)

Also I had another issue  with FreeBSD 11.x which related to contigmalloc and contigfree .
Because of this too much time taken by contigfree driver unload was taking 20 minutes.
Actually contigfree was taking too much time on FreeBSD 11.x  on this same server with Xeon Gold 5115 CPU.

In FreeBSD 12 Beta3  also contigfree takes much more time compared to contigmalloc.
However  when I compare the  values to FreeBSD 11.x number I can see huge improvement in FreeBSD 12 Beta 3   .
Because of this contigfree issue my driver unload  used to take 5 to 20 minutes  in FreeBSD 11.x.
Now  my driver takes only a few seconds to load and a few seconds to unload in FreeBSD 12. BEta 3.

Hence it looks like the problem is with FreeBSD 11.x.  
I am still waiting for  final release version of FreeBSD 12.0

Regards,
Steevan


----------



## ShelLuser (Nov 22, 2018)

Why would you want to unload (and eventually re-load?) a driver?


----------

