# Installations hangs with PER6 HW RAID card, can we disable the loading of the megaraid driver?



## jonakarl (Sep 27, 2018)

I started this thread in the pfsense forum (original thread) but was redirected here (the problem are identical with "vanilla" freeBSD 11.2).  

In short: 
When we try to install pfsense 2.4.3 or freeBSD 11.2 on our server it stops after it has loaded the Megaraid lsi driver.

We have configured the raid card with 1 virtual drive and the installer finds the drive as seen when first booting and listing attached drives (and so does linux and eg freedos).

However when starting the install the kernel tries to load the megaraid driver and then it "hangs".  I suspect (without knowing) this is becasue the driver tries to "take over" the raid card for some reason.
We have tried to disable the megaraid "module" but it seams to built into the kernel and whatever command we tried to disable the driver have failed.

In short I have two questions, can we disable the driver with the current build of the installer or do we need to rebuild the kernel (without support for the megaraid driver ?
And related why is this driver necessary at all as the disks will present themselves as "bios" disks, ie if we do not load the driver will the install not detect the virtual disk at all?

Best regards
Jonas


----------



## SirDice (Sep 27, 2018)

I would try enabling the mrsas(4) driver instead:

```
Using /boot/device.hints (as mentioned below), the user can provide a
     preference for the mrsas driver to detect a MR-Fusion card instead of the
     mfi(4) driver.

           hw.mfi.mrsas_enable="1"

     At boot time, the mfi(4) driver will get priority to detect MR-Fusion
     controllers by default.  Before changing this default driver selection
     policy, LSI advises users to understand how the driver selection policy
     works.  LSI's policy is to provide priority to the mfi(4) driver to
     detect MR-Fusion cards, but allow for the ability to choose the mrsas
     driver to detect MR-Fusion cards.

     LSI recommends setting hw.mfi.mrsas_enable="0" for customers who are
     using the older mfi(4) driver and do not want to switch to mrsas.  For
     those customers who are using a MR-Fusion controller for the first time,
     LSI recommends using the mrsas driver and setting
     hw.mfi.mrsas_enable="1".
```


----------



## jonakarl (Sep 28, 2018)

Hi,

We have tried that before with no luck but to be sure I dropped to a prompt at the boat loader and executed
set hw.mfi.mrsas_enable="1"
boot

However the error is exactly the same as when not using this command (where the boot hangs can be seen in the attached picture).

The annoying thing is that the bootloader reports it has found the drive as disk0 (with the correct size) before I get to the boot menu. There should in my mind therefore be no need to load any driver for the raid card.
I am 100% postive that it is the lsi driver as if I disable the raid card in bios, the boot does not hang but then of course I do not have any disk available to install on either.   

I am on the latest FW version of the raid card and bios.


----------



## SirDice (Sep 28, 2018)

jonakarl said:


> The annoying thing is that the bootloader reports it has found the drive as disk0 (with the correct size) before I get to the boot menu.


The bootloader uses the BIOS to find the disk. 


> There should in my mind therefore be no need to load any driver for the raid card.


The kernel and the OS itself don't use anything from the BIOS.


----------



## jonakarl (Oct 1, 2018)

SirDice said:


> The bootloader uses the BIOS to find the disk.
> 
> The kernel and the OS itself don't use anything from the BIOS.



So if I understand you correctly there is no way to make bsd use the bios disk even if I disable the lsi driver? 

I am not complaining but since this, in our use case, would render the raid card pretty useless I trying to see what options we have to press forward in this issue.
As I see now we have three option; do not use raid, change distro/kernel (ie move to linux) or find another raid card. The last option is probably the best but if it is true as you say and bsd cannot/do not respect the (bios) drive configured by the card, I am unsure if another HW would fix the problem.


----------



## Terry_Kennedy (Oct 3, 2018)

jonakarl said:


> So if I understand you correctly there is no way to make bsd use the bios disk even if I disable the lsi driver?
> 
> I am not complaining but since this, in our use case, would render the raid card pretty useless I trying to see what options we have to press forward in this issue.
> As I see now we have three option; do not use raid, change distro/kernel (ie move to linux) or find another raid card. The last option is probably the best but if it is true as you say and bsd cannot/do not respect the (bios) drive configured by the card, I am unsure if another HW would fix the problem.


The "BIOS disk" uses INT 13 which is a 16-bit real mode interface. No modern operating system (that I know of) uses it for anything other than bootstrapping the system, at which time the OS disk driver takes over.

Is this PERC6 controller in a Dell chassis, or is it a card that was installed in some non-Dell chassis? Which card is it, exactly, and what firmware version is it running? What system is it in (Dell model number, or non-Dell brand / model)? It looks like the current firmware is 6.3.3-0002, which consists of:

```
Component        Current Version
Package            6.3.3-0002
Firmware        1.22.52-1909
Bootblock        1.00.00.01-0011
Ctrl-R            1.02-015B
```


----------



## Tim L (Oct 4, 2018)

Is that screenshot from when you stopped the boot and did set hw.mfi.mrsas_enable="1" at the loader prompt?  Since it says "mfi0:" it is still using the mfi(4) driver.  You would see lines starting with mrsas0: if it was using the mrsas(4) driver.


----------



## SirDice (Oct 5, 2018)

Indeed. Good catch. 

Case in point:

```
AVAGO MegaRAID SAS FreeBSD mrsas driver version: 06.712.04.00-fbsd
mrsas0: <AVAGO Invader SAS Controller> port 0x6000-0x60ff mem 0xc7300000-0xc730ffff,0xc7200000-0xc72fffff irq 32 at device 0.0 numa-domain 0 on pci4
mrsas0: Using MSI-X with 24 number of vectors
mrsas0: FW supports <96> MSIX vector,Online CPU 24 Current MSIX <24>
mrsas0: FW supports SED
mrsas0: MSI-x interrupts setup success
```

Reading up I noticed the man page mentions adding a line to device.hints. But I do believe that's incorrect as entries in device.hints(5) all start with hint.*. I have it in /boot/loader.conf:

```
hw.mfi.mrsas_enable=1
```


----------



## ucomp (Oct 6, 2018)

jonakarl said:


> (original thread) ..........    :
> -----jonakarl: ----
> While we both have sata and USB neither will suffice (single point of failure). Of course we could do software raid over multiple USB or SATA drives but then we need to take the server down if a disk fails which is not an option (with the raid card we have configured a RAID 1 with 4 spare drives so basically 5 drives can fail before "shit hits the fan").
> ---


you really should use ZFS on a compatible HBA or direct attached SATA .  Perc 6 is the wrong card because not flashable to IT-Mode(afaik).


----------

