# DELL H755 RAID support



## dkline201 (Feb 16, 2022)

We have been using DELL Server systems with FreeBSD since it at version 7.  FreeBSD support for DELL hardware has always been great,  up until recently.  When DELL released their 14 Gen systems (PowerEdge R740),  we found that there was NO support for the H740 RAID board.  But we were able to replace the H740 with the H730 in those sytems and all was fine.  Now DELL has released the 15 Gen systems with the H755 RAID interface.  Also,  the H730 PCIe RAID board is now EOL...  Which would not matter anyway, since the 12 bay RAID backplane connectivity would not support the PCIe H730 anyway.  I cannot find anything on the FreeBSD 13 HCL or forums about the H740/H755 RAID.   When I try to boot from a FreeBSD 13 (BIOS or UEFI - tried both),  it starts to load,  so it seeing the RAID1 boot drive pair,  but then fails with a 'mount root',  and does not show any available drives for 'tank'.   ANY SUGGESTIONS?

Thanks for taking the time to read this...

Dale Kline


----------



## dkline201 (Feb 16, 2022)

P.S.,  there DOES seem to be Linux support for this H755 RAID configuration,  but we cannot go to a new operating system.  After over 10 years,  we are tied into FreeBSD.


----------



## dkline201 (Feb 16, 2022)

P.P.S.   I did try loading (ugh) Windows 10 on the stsyem,  and then Ubuntu 20.  Both of these worked fine, and accessed other RAID0 drives.  so I know the hardware is good,  as long as it is not FreeBSD...  :-(


----------



## richardtoohey2 (Feb 16, 2022)

I don't know about the H755 (but interested because looking at the R6515) but I've got two Dell R640s working fine with RAID-5 on H740 controllers e.g.

```
# MegaCli -AdpAllInfo -aAll
                                   
Adapter #0

==============================================================================
                    Versions
                ================
Product Name    : PERC H740P Mini
Serial No       : 0AV024L
FW Package Build: 51.13.1-3662
```

EDIT: I'm running 13.0-RELEASE on both R640s.


----------



## suntzu00 (Feb 16, 2022)

Dell R740 here with PERC H740P. Running FreeBSD 12.3 without any issues. H755 isn't listed in mrsas but then again H740 is missing as well.  you should definitely check the mailing lists. 

```
# MegaCli -AdpAllInfo -aAll

Adapter #0

==============================================================================
                    Versions
                ================
Product Name    : PERC H740P Mini
Serial No       : 7CM019Q
FW Package Build: 51.14.0-3900
```


----------



## dkline201 (Feb 17, 2022)

Did some checking on the DELL H730P and H740P.  Both used the LSI SAS3108 chipset.  So the RAID support should have been similar,  if not the same.   We were then using FreeBSD10.3,  which could explain why the H740P did not work for us in the PowerEdge R640 and R740xd systems.
The PowerEdge R450 and R750 use the newer LSI SAS3916 Tri-mode (whatever that is?) ROC chipset which is an entirely new architecture.
Windows 10 and Ubuntu installs (for testing hardware) both install and boot up fine on this system...


----------



## SirDice (Feb 17, 2022)

You might need to enable mrsas(4), it's possible mfi(4) takes precedence but fail to attach to the card. Some LSI chipsets only work with mfi(4), some with both mfi(4) and mrsas(4) and the latest cards only work with mrsas(4).


```
Using /boot/device.hints (as mentioned below), the	user can provide a
     preference	for the	mrsas driver to	detect a MR-Fusion card	instead	of the
     mfi(4) driver.

	   hw.mfi.mrsas_enable="1"
```


----------



## 6502 (Feb 17, 2022)

Why not simply ignore the hardware RAID and use separate HDD/SSDs with ZFS? Most hardware RAID controllers have one main disadvantage - the controller has no backup. And if you are using e.g. RAID5 configuration and RAID controller fails, your server stops working until you find the same controller.


----------



## SirDice (Feb 17, 2022)

6502 said:


> Why not simply ignore the hardware RAID and use separate HDD/SSDs with ZFS?


Lots of modern RAID cards have a JBOD setting nowadays.


----------



## dkline201 (Feb 17, 2022)

There is a 'non-RAID' configure setting in BIOS,  but I am having a hard time figuring it out without wiping the boot drive in the process.   :-(   Fortunately,  these are just 'dupes' of working system drives for testing.   Time to try that again...


----------



## SirDice (Feb 17, 2022)

Nothing wrong with sticking to UFS on hardware RAID if that's what you've been using for all those years before.


----------



## dkline201 (Feb 23, 2022)

UPDATE:  Finally figured out the DELL RAID setup  -  no more Ctl-R on boot...   RAID configuration is now in BIOS (F2) setup under Device Configuration.  I changed the boot drive from RAID0 to JBOD.    Boot drive is FBSD 13.0 UEFI boot.  (This 'same drive' boots in previous model DELL PE R740xd with H730 RAID board,  without problems.  The new PE R450 uses the H755 with LSI sas3916 Trimode ROC chipset.)

No help.  It starts OK,  going through POST checks, finds one non-RAID drive, starts booting from Drive C:,  shows the FreeBSD splash page,  runs another five seconds,  and crashes with the "Fatal trap 12 error while in kernel mode".     "Fault code = supervisor read instruction, page not present" (picture attached)

I still believe this is a driver problem with the new H755 RAID board.  It does not seem that anyone is working on drivers for ths chipset.    The system boots fine off a Windows10 drive,  and an Ubunbtu 20 drive.

Any suggestions would be appreciated.  We can't be the only DELL <> FreeBSD users out here?  THANKS....


----------



## covacat (Feb 23, 2022)

without kernel debugging symbols it's hard to tell anything


----------



## ralphbsz (Feb 23, 2022)

The LSI 3916 chip is used in lots of other boards, sold by Lenovo (IBM) and Supermicro, plus obviously Broadcom's own boards. Those chips have been out for years. It seems unplausible that nobody else has used those chips with FreeBSD, in general. On the other hand, for using exactly that Dell H755 board with release 13, you might be the first one.

But let me ask a question for a quick check: Have you tried booting FreeBSD on this machine from something else (like a USB stick or some other port that doesn't use the LSI chip)? That would verify that there isn't something more basic broken here. You can do that both with and without the Dell H755 board inserted, and see that you at least get that up and running.


----------



## richardtoohey2 (Feb 23, 2022)

There's lots of mixing up of Dell server codes and RAID controller codes in your messages.

Your text talks of the H755, but your screenshot says PE R640 - so with the H740?

I've got Dell R430s and R640s running H730 and H740 with FreeBSD 13.0 RELEASE using hard drives and SSDs, but all in RAID-1 or RAID-5 mode so haven't used JBOD.

I've not yet got a server with the new H755 - is that what you are really interested in, or something to do with the earlier generation(s) or both?

It could certainly be the case that no-one has written a FreeBSD driver for the newer hardware - I think LSI supplied the FreeBSD driver for the previous RAID generations and maybe they haven't for the H755 etc.


----------



## SirDice (Feb 23, 2022)

dkline201 said:


> No help. It starts OK, going through POST checks, finds one non-RAID drive, starts booting from Drive C:, shows the FreeBSD splash page, runs another five seconds, and crashes with the "Fatal trap 12 error while in kernel mode". "Fault code = supervisor read instruction, page not present" (picture attached)


Did you enable the mrsas(4) driver by adding `hw.mfi.mrsas_enable="1"` to /boot/loader.conf?


----------



## dkline201 (Feb 23, 2022)

This system that is failing is the PE R450 with H755 RAID  -  15th Gen DELL just released  -  our first system at this 15th Gen.    I saved that screen shot above with the wrong system name...

_"I think LSI supplied the FreeBSD driver for the previous RAID generations and maybe they haven't for the H755 etc."  >  _I really do believe this is the case. :-(    How do we convince someone to look at it?  DELL has notified us that the H730 is going EOL.

I have PE R730xd 13th Gen,  and R740xd 14th Gen,  both using the H730 RAID board.  (We had trouble three years ago with the H740 RAID,  which seems to have been resolved now.)  These ran fine in the past with FBSD 10.3,  and are running OK now with FBSD 13.0.

I tried running the FBSD 13.0 installer USB on the R450,  and it crashes at the same place with exactly the same error.



Checking on that mrsas enable right now_,  _but probably won't have that answer til tomorrow morning.   It is now heading for 5:00pm_._


----------



## richardtoohey2 (Feb 23, 2022)

I suspect that yes, the H755 hasn't got a driver yet but might be more a question for the mailing lists.

Looks like support was added to recognise the controller:






						src - FreeBSD source tree
					






					cgit.freebsd.org
				




... but I don't know enough to find out if a driver exists yet (you could try 14.0?)

I was days away from ordering a R6515 so your questions about the H755 very pertinent for me.  Might have to see if I can get a R640 instead and stay on the previous generation until there's a definite answer on support for the new RAID controller.


----------



## dkline201 (Feb 23, 2022)

THANK YOU!  I will follow up in the morning with this!!!


----------



## ralphbsz (Feb 23, 2022)

dkline201 said:


> This system that is failing is the PE R450 with H755 RAID  ...
> _"I think LSI supplied the FreeBSD driver for the previous RAID generations and maybe they haven't for the H755 etc." ..._




No, there is some confusion here. I suspect that a Dell H755 card is nothing but a LSI/Broadcom/Avago 3916 chip. Probably all Dell does is to make the PC board and the connector, buy the chip from Broadcom, minimally modify the firmware of the chip (probably mostly to change the model name and number to be "Dell H755"), and then ship it. That is how most HBA/RAID cards are done. The support questions is 99% about FreeBSD support of the LSI 3916.



> How do we convince someone to look at it?


FreeBSD is a free operating system, and support is mostly friendly people on the Internet giving you tips (like SirDice above). If you can find a concrete bug in FreeBSD, you could file a bug report ... but saying "it crashes but I don't know where and why" is not a concrete bug. I also suspect (without direct knowledge) that LSI/Broadcom funds the software engineer(s) who writes the mrsas driver; if you were a large Broadcom customer, you could contact them for engineering support (BTDT, got the T-shirt). You could in theory contact Dell, but my educated guess is that they'll say "FreeBSD who? not our problem, we support Linux and Windows". And my educated guess is that if you contact Broadcam directly, they'll say "you're not a customer".



> I tried running the FBSD 13.0 installer USB on the R450,  and it crashes at the same place with exactly the same error.[


Was that with or without the LSI-based HBA in the system? If it was without, then the R755 is not the problem. If it was with, then it is possible that the R755 is the cause of the problem, even when not booting from it. In that case, after checking for the mfi versus mrsas question, I would get hold of a kernel with debug symbols and look at the stack traceback when it crashes.

By the way, somewhat unrelated: I always thought that the standard RELEASE kernel has symbol names; when I get stack tracebacks, there are function names there. Or am I hallucinating?


----------



## richardtoohey2 (Feb 23, 2022)

You are right, there's not a specific driver for the "H730" or "H740", there are drivers for LSI/Broadcom/next-owner that support the Dell-rebranded LSI chipsets.

There's stuff like this showing that LSI very definitely involved in the mrsas driver at one point: 



			'LSI - MR-Fusion controller driver <mrsas> patch and man page' - MARC
		


_LSI will continue to update <mrsas> driver in future in timely bases._


----------



## Ferrari-Dude (Feb 24, 2022)

Hi Dale,

I have several 15th generation Dell servers in production with that same controller running Windows Server 2019.  We had some boot reliability problems as well and it turned out to be an early revision of firmware that shipped with the controller.  

I used the embedded Dell Lifecycle Controller (available via F10 at POST time) to update the firmware to a more recent revision.  This cured our boot reliability problem.  
Windows is not FreeBSD but maybe the same issue applies more generally.

-Don


----------



## 6502 (Feb 24, 2022)

I avoid DELL brand after replaced a HDD of Laptop. It was necessary to disassemble nearly all parts (including keyboard and display) to reach the HDD.


----------



## SirDice (Feb 24, 2022)

6502 said:


> I avoid DELL brand after replaced a HDD of Laptop. It was necessary to disassemble nearly all parts (including keyboard and display) to reach the HDD.


Consumer, of-the-shelf, laptops aren't comparable to enterprise server grade hardware. Those things are typically built like a tank and pretty much all components are usually easily replaced, you don't even need a screwdriver nowadays. In some cases components can even be replaced without powering down the system.


----------



## 6502 (Feb 24, 2022)

You are saying that DELL engineers for "consumer" laptops are not good but for servers are very good? I am not sure. Equivalent HP servers are better or at least do not have such problems with RAID card support, etc.


----------



## SirDice (Feb 24, 2022)

6502 said:


> You are saying that DELL engineers for "consumer" laptops are not good but for servers are very good?


No, enterprise grade server hardware is designed to be serviceable (well, most of them anyway). Consumer laptops are designed to look pretty, not to be tampered with by us mere mortals.


----------



## dkline201 (Feb 24, 2022)

_skipping the last three comments about laptops........._

*As 'richard...' above noted*,  DELL (part #H... here) branded RAID boards have used LSI Broadcom chipsets back as far as the H700 RAID board,  maybe older.   It seems that Dell had failed to identify the chipset up to the H730P boards.  So looking for drivers seems to be mainly a matter of looking for a chipset # rather than the DELL PERC H###.

LSI and DELL both are supporting the H755 with SAS3916 chipset with drivers for Windows, VMware,  and Linux.   I successfully installed Windows 10 and Ubuntu 20 onto RAID1 sets in the PE R750 with the H755 RAID board.   But support for FreeBSD has not been directly provided by LSI or DELL as far back as I can trace the LSI connection.
( except for the <mrsas> a few years back for the H720 or H730,  I forget which. )  We have been using DELL servers going back into the PE 2900 and 2950 era.   FreeBSD RAID drivers were not hard to find then.

I downloaded that 'src - FreeBSD source tree',  but I don't know how to interpret the 'PERC H755' section.  Does this mean it is 'to be done',  'in progress',  or 'available'?


----------



## dkline201 (Feb 24, 2022)

Oh,  and one other comment to SirDice.  DELL Servers are designed to be 'easily serviced'.  There was one time that I had a DELL Tech here to replace the HDD backplane,  He took the system almost completely apart without even a screwdriver.  Everything was 'clip in' - just a twist or pull on the little blue clips.  DELL systems are not cheap,  but they are well designed.  Just my 2¢ worth.  (PS - I don't work for DELL   )


----------



## ralphbsz (Feb 24, 2022)

6502 said:


> You are saying that DELL engineers for "consumer" laptops are not good but for servers are very good?


Your criteria for "good" are all wrong.

The engineers (and other staff) who designed the Dell laptop you're complaining about had certain goals. In consumer laptops, making parts easily replaceable is typically NOT a goal, because most users do not take their laptops apart. Instead, compactness, weight, battery life, and features are a design goal. You can't fault the engineers for building a laptop that most consumers would like, and which you bought in spite of the fact that it is not designed for your particular use (which is disassembling and reassembling it).


----------



## ralphbsz (Feb 24, 2022)

dkline201 said:


> So looking for drivers seems to be mainly a matter of looking for a chipset # rather than the DELL PERC H###.


This is not specific to Dell, but is true industry wide. LSI SCSI chips are used by a wide variety of vendors (including LSI themselves by the way); to use them with Linux or FreeBSD drivers, you have to know what chip model is in which board model number. I know that both Linux and FreeBSD drivers do attempt to create lists of what OEM-branded cards are supported (that includes Dell, IBM, Lenovo, Intel, HP, ...), but those lists tend to be often obsolete.



> LSI and DELL both are supporting the H755 with SAS3916 chipset with drivers for Windows, VMware,  and Linux.


I think most LSI chips can be run with the mrsas driver that is in the released code in FreeBSD; some may use the mfi or mpr drivers. I think those drivers are "provided" by LSI, in the sense that LSI pays someone (might be a consultant to them, might be an employee) to write that code. The same goes for the setup utility (it used to be called megacli, I think it has a new name now). If all this is true, then LSI does "support" FreeBSD.

Usually, LSI also publishes firmware updates (and various versions, like IT versus RAID) on their website. I don't think the firmware files are in general OS dependent, although during beta tests, there may be firmware versions that fix specific OS-dependent bugs. So from a firmware point of view, I think LSI supports FreeBSD as much or as little as any other OS.

I think in earlier days, LSI used to publish updated driver versions for Linux and FreeBSD on their web site; the expectation was that power users would replace the drivers (such as mrsas) that come with the normal source release of things like RedHat or SUSE or FreeBSD with drivers downloaded from them. I don't know whether they still do that, haven't looked in years.

Part of the problem you have here is that you are caught in the chasm between Dell and LSI. That affects both chip numbering and finding help. From this viewpoint, buying Dell gear might be a mistake; you might be better off buying your SAS cards directly from LSI, and then using their technical support.

(When I say "LSI" above, I mean whatever name the corporation has this week, it has recently been Avago and Broadcom.)


----------



## 6502 (Feb 24, 2022)

DELL was top brand and hope still is. But after the case with Laptop HDD and another case with specific fans in DELL desktops, I decided to avoid DELL hardware when possible. The problem with fans is that they were non standard - when stopped working it was hard to find spare part. And they frequently stopped working which means: bad decision for non-standard fan + low quality when faults are not rare = somebody in DELL makes bad design. The problem in this topic shows that RAID card is not excellent.


----------



## richardtoohey2 (Feb 24, 2022)

dkline201 said:


> I downloaded that 'src - FreeBSD source tree', but I don't know how to interpret the 'PERC H755' section. Does this mean it is 'to be done', 'in progress', or 'available'?


The bit I was pointing to looked like it was identification data - so just so that FreeBSD can match a hardware ID to a string e.g. for dmesg.

I don’t know if anyone has done any more than that - I very much hope so but don’t know.


----------



## richardtoohey2 (Feb 24, 2022)

6502 said:


> The problem in this topic shows that RAID card is not excellent.


Ummm, the main problem is that there doesn't yet _seem_ to be a FreeBSD driver for the new generation RAID card.

My experience of cheap low-end Dells (laptop and desktop) has been the same as my experience with cheap low-end Lenovo/HP/Compaq/etc. machines - compromises are made and you get what you pay for.


----------



## dkline201 (Mar 10, 2022)

SOLVED  -  I downloaded FreeBSD 13.1 PRE RELEASE 20220303.  The Trap 12 error on boot has been fixed,  and the PERC H755 RAID setup now works!   I know this is PRE RELEASE,  and NOT FINAL,  but it is a BIG step in the right direction.


----------



## richardtoohey2 (Mar 10, 2022)

That's awesome news, thank you for checking and posting. 

I'm still figuring out my next Dell server purchases - meant to update this thread to say my earlier posts about the R6515 appear to be wrong - looks like they have the H730 generation PERC, not this newer one.


----------



## dkline201 (Mar 10, 2022)

Richard,    The R6515 is an older system with AMD CPUs (nothing wong there,  I just have not used them)  using the LSI SAS 3108 chipset.  You will have no problems with that RAID board,  with FreeBSD 9.0 and forward.   (any RAID board in this series  -->  HW RAID: PERC 9/10 - HBA330, H330, H730P, H740P, H840, 12G SAS HBA)


----------



## richardtoohey2 (Mar 10, 2022)

Yes, thank you - I just haven't decided yet about trying the AMD route for a change or going for the Rx50 range and staying with Intel.

That's why I was so interested in how you were getting on with the H755 support.  If FreeBSD not going to work any time soon with the H755 that was an additional push for the R6515 but it's looking promising for the Rx50/H755 range so I still have options.


----------



## Phishfry (Mar 10, 2022)

dkline201 said:


> use the newer LSI SAS3916 Tri-mode (whatever that is?)


Tri-Mode gives you the ability to run SATA/SAS/NVMe drives from the same controller.
SATA/SAS use the same mixed mode firmware.
For NVMe there is a different firmware you need to flash to the controller. So a 16 port card supports 4 NVMe.
These take very special LSI cables not ordinary NVMe SFF-8643.
Theirs has a extra signal line included. Cables are $100 for two drives.
That feature may have become part of the U.3 standard form factor change (U.2 connector specs upgraded for PCIe4).
So in a nutshell, The SAS3916 card used U.3 cable spec for NVMe before it was a standard.
That allowed them to charge exorbitant cable prices as there were no competitors.
It also meant lots of fail when people tried to use standard U.2 NVMe cables.
LSI did not announce that they were U.3 compatible as they have no reason too.
They have an overpriced part number for you.
I still don't see many U.3 cables on the markets yet.

With the speed hit of NVMe over the controller I really dont see the use case for NVMe on LSI.
Hot swap might be the one case. It is handled different on the controller level.


----------



## dkline201 (Mar 16, 2022)

Thanks for the clarification on Tri-Mode.    A few years back, we bought several DELL PE R630 10 x 2.5" bay systems that had  6 SAS/SATA bays and 4 NVMe bays.  At that time, I don't think hardware RAID on NVMe was available,  so we went with RAIDz1 on the two NVMe drives,  which worked well.  But there was a special board and cabling to install into the system for the NVMe drives.  So the SAS3916 should eliminate that second drive interface?

Moving forward  -  FreeBSD13.0-RELEASE did not work at all on the "newest" DELL PE R450 15th Gen system.  It crashed with a "Trap 12 error"  shortly after boot start.   I just downloaded FBSD 13.1-PRERELEASE (20220303)   ->   (YES,  I KNOW this is not ready for production  -  I am just testing progress of 13.0/1)   This seems to have fixed the Trap12 error,  and also allowing full hardware RAID ability with FBSD13 and the DELL H755 PERC 11 board.

Now to the problem...  We also needed to put FBSD13.0-RELEASE onto a NUC11 for another function.  13.0-RELEASE did not have the correct drivers for Intel i225 network chipset,  booted OK,  but no network access,  did not even see the i225 interface.   I found 13.0-STABLE (20220217) which had the fix in the driver so the network was active and running.  But we cannot use the STABLE version for production system.

So,  going back to the FBSD13.1-PRERELEASE,  I put that on the NUC11 for testing.  13.1 does 'see' the i225 chipset,  an tries to active the network but throws an error and does not retrieve a DHCP IP address.

Sorry for the long-winded description,  but I just wanted to outline my steps to this question(?)   FBSD13.0-STABLE fixed a network issue,  but that fix does not seem to have carried forward to FBSD13.1-PRERELEASE.    As a lowly "user" out here waiting for updates,  is there a mechanism (posting?) to describe errors as we see them during field testing?


----------



## SirDice (Mar 16, 2022)

dkline201 said:


> Sorry for the long-winded description, but I just wanted to outline my steps to this question(?) FBSD13.0-STABLE fixed a network issue, but that fix does not seem to have carried forward to FBSD13.1-PRERELEASE.


PRERELEASE is a moniker that's used on the -STABLE branch (it gets this just before a release branch is made). I suggest you try the BETA1 of 13.1, it was released a couple of days ago.









						FreeBSD 13.1 Release Process
					

FreeBSD is an operating system used to power modern servers, desktops, and embedded platforms.




					www.freebsd.org


----------



## Phishfry (Mar 16, 2022)

dkline201 said:


> did not work at all on the "newest" DELL PE R450 15th Gen system.


My guess here is the PCIID for the Dell versions were not up to date with the driver.
The controller itself is supported for a while now.
FreeBSD imports the LSI driver every once in a while.
The Dell and Intel OEM versions (PCIID) of the cards are not included with LSI driver and must be added.


----------



## dkline201 (Mar 17, 2022)

Phishfry,  where did you find info on DELL H755 RAID controller being supported earlier?  I looked for several months for that info.  Obviously I was looking in the wrong places... 

FBSD 13.1-BETA1 is looking very good for new DELL 15Gen sytems,  and also the latest NUC11 systems.  Basic booting up and hardware functions seem good on both systems.  Lots of testing to go to confirm 'full' hardware and software support though.  The NUC11 network port works fine after I realized that I forgot the set rc.conf to - ifconfig_igc0="DHCP".   Isn't learning NEW stuff so much fun???


----------



## Phishfry (Mar 17, 2022)

dkline201 said:


> Phishfry, where did you find info on DELL H755 RAID controller being supported earlier?


I am just surmising. I had the same experience with Intel Branded SAS TriMode card.
The LSI card was working but my Intel one was not.
A point revision for FreeBSD came along and suddenly the Intel card worked.
So I am guessing here.
LSI driver is imported and someone has to manually add Intel, Dell, HP, SuperMicro OEM version PCI-ID's for the new cards. LSI does not supply that.


----------



## Phishfry (Mar 17, 2022)

PCI-ID is the method for recognizing hardware. If the PCI-ID is unknown it won't work.
I can imagine why manufacturers like Dell uses their own PCI-ID instead of the LSI PCI-ID.
For example mezzanine cards. I would imagine there could be differences between Dell and HP's implementation. Windows detects hardware by PCI ID's too so driver detection is based off PCI-ID's.
pci.ids is an index file with devices much like usbdevs. Vendor ID and Priduct ID are the fields.
pciconf(8)


> PCI vendor    and device information is read from
> /usr/local/share/pciids/pci.ids.  If that file is not present, it is read
> from /usr/share/misc/pci_vendors.    This path can be overridden by setting
> the environment variable PCICONF_VENDOR_DATABASE.


----------



## Phishfry (Mar 17, 2022)

I will technically qualify what I said here. pci.ids are a method for device detection.
But for the LSI cards the PCI-ID's are in the driver. So just guessing here.
Kernel is compiled with LSI support. This means PCI-ID's are in kernel via the driver.
I know the PCI ID is in the driver because I tried and failed to add my Intel SAS9400 card to the FreeBSD driver.



			LSI 9300-16i supported?
		



> Yes, the FreeBSD tree is a bit different.  Avago has out of box drivers that
> are maintained, and we try to keep the FreeBSD tree up to date with the
> in-box changes.  You should be OK to go with the in-box driver.


My experience was fixed by FreeBSD 12.1








						Solved - SAS-12 Gb/s controller card  for FreeBSD?
					

I've traded my 1TB SATA/SSD for 1.2 TB SAS/Seagate-ST 1200MM088 2.5" HD with a person from craigslist. Now, I have...




					forums.freebsd.org


----------



## dkline201 (Mar 22, 2022)

First of all,  THANKS everyone for all of the support above.  We are working on moving OUR software over to FreeBSD 13.1-BETA2 (realizing of course BETA is not final).  The DELL 15Gen 'hardware' portion of the system is booting fine now with the H755 PERC11 controller.  I have been able to load 'smartmontools' and run 'smartctl' and 'smartd' commands to check individual drives in my system.  So far, so good.  But way back (about 5 years ago) when LSI introduced the LSI 9300 series of controllers,  we lost the ability to run 'mfiutils' to monitor the RAID controller itself.  'mfiutils' is loaded and if I just type the command,  it lists out the command options.  But if I try 'mfiutils show drives',  it fails with 'mfi_open: Command not found'.   I have been going around in circles with man pages and posts for  'mfi', 'mrsas', 'megaRAID',  trying to find some combination that works, with no luck.  Any help would be greatly appreciated. 'mfiutil' is great for remotely diagnosing when the RAID controller is causing drive issues.


----------



## richardtoohey2 (Mar 22, 2022)

Look at the MegaCli port - it's not quite the same but should get you on the right path.

EDIT - note I'm on the older generation of PERC so not 100% sure how/if it works with the H755 - but hopefully points you in the right direction.


----------



## richardtoohey2 (Mar 22, 2022)

```
xyz@pqr:~ % pkg info | grep Mega
megacli-8.07.14                SAS MegaRAID FreeBSD MegaCLI
root@pqr:/home/xyz # MegaCli -LDInfo -Lall -aALL
                                     

Adapter 0 -- Virtual Drive Information:
Virtual Drive: 0 (Target Id: 0)
Name                :R430RAID1
RAID Level          : Primary-1, Secondary-0, RAID Level Qualifier-0
Size                : 3.637 TB
Sector Size         : 512
Is VD emulated      : No
Mirror Data         : 3.637 TB
State               : Optimal
Strip Size          : 64 KB
Number Of Drives    : 2
Span Depth          : 1
Default Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU
Current Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU
Default Access Policy: Read/Write
Current Access Policy: Read/Write
Disk Cache Policy   : Disk's Default
Encryption Type     : None
Default Power Savings Policy: Controller Defined
Current Power Savings Policy: None
Can spin up in 1 minute: Yes
LD has drives that support T10 power conditions: Yes
LD's IO profile supports MAX power savings with cached writes: No
Bad Blocks Exist: No
Is VD Cached: No


Virtual Drive: 1 (Target Id: 1)
Name                :
RAID Level          : Primary-1, Secondary-0, RAID Level Qualifier-0
Size                : 7.276 TB
Sector Size         : 512
Is VD emulated      : Yes
Mirror Data         : 7.276 TB
State               : Optimal
Strip Size          : 64 KB
Number Of Drives    : 2
Span Depth          : 1
Default Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU
Current Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU
Default Access Policy: Read/Write
Current Access Policy: Read/Write
Disk Cache Policy   : Disk's Default
Encryption Type     : None
Default Power Savings Policy: Controller Defined
Current Power Savings Policy: None
Can spin up in 1 minute: Yes
LD has drives that support T10 power conditions: Yes
LD's IO profile supports MAX power savings with cached writes: No
Bad Blocks Exist: No
Is VD Cached: No



Exit Code: 0x00
```


----------



## SirDice (Mar 22, 2022)

dkline201 said:


> But way back (about 5 years ago) when LSI introduced the LSI 9300 series of controllers, we lost the ability to run 'mfiutils' to monitor the RAID controller itself. 'mfiutils' is loaded and if I just type the command, it lists out the command options. But if I try 'mfiutils show drives', it fails with 'mfi_open: Command not found'. I have been going around in circles with man pages and posts for 'mfi', 'mrsas', 'megaRAID', trying to find some combination that works, with no luck. Any help would be greatly appreciated. 'mfiutil' is great for remotely diagnosing when the RAID controller is causing drive issues.


mfiutil(8) only works with cards supported by the mfi(4) driver. mpsutil(8) for mps(4), etc. There's no such specific tool for the mrsas(4) driven cards. sysutils/megacli should work on all of them. It's the same tool LSI recommends.


----------



## dkline201 (Mar 23, 2022)

I have tried loading MegaCLi but having problems getting it to run...  Your first three lines may be where I am having problems with the loading and setup.  I don't think I had the latest version either.  Will try that now.  We saw the 'mfiutil' / 'mrsas' conflict back when the LSI9300 series chipsets were released,  but at that time we did not try MegaCLi as a replacement.   There is now a push to get it working again.


----------



## SirDice (Mar 23, 2022)

dkline201 said:


> I have tried loading MegaCLi but having problems getting it to run..


`pkg install megacli`

The syntax of the commands is horrid though. It's going to take some time getting used to it. But there's lots of information to be found. Be careful with that `[E:S]` notation, the shell might 'eat' those brackets. So you may need to add some quotes here and there. 






						MegaCLI cheatsheet
					

Recently I’ve had to do extensive work with Dell PowerEdge servers, and specifically Dell’s that use the LSI MegaRAID controllers. Anyone who has ever had to work with the LSI RAID controllers knows that the MegaCLI provided by LSI is the most cryptic command line utility in existence. MegaCli...



					erikimh.com


----------



## dkline201 (Mar 23, 2022)

SOLVED.     GOT IT!  With the help from comments above I was able to load MegaCLi (not MegaCLi64!) and get it working on the DELL R450 with the H755 controller.  Thank You everyone!


----------



## dkline201 (Mar 23, 2022)

Am I allowed to post a web link from a Cisco link that was a big help with commands,  once I got the MegaCLi loaded properly?


----------



## SirDice (Mar 23, 2022)

Sure, post away.


----------



## dkline201 (Mar 23, 2022)

https://community.cisco.com/t5/secu...i-common-commands-and-procedures/ta-p/3114544


----------



## dkline201 (Apr 15, 2022)

Hi!  It's me again,  with two problems this time.  (these are both in testing stages til 13.1 is fully RELEASED)

1. 'mfsbsd' will not boot on a DELL R450 (15th generation) system.  Crashes with a 'fatal trap 12 error' shortly after POST.  FreeBSD 13.1-PRERELEASE 'installer' does at least boot,  but has problems installing FBSD.  I am guessing we have to wait for a FULL 13.1-RELEASE before mfsbsd13.1 can be released?

2.  DELL PERC9 RAID on a 'PE R450 15th Gen' system has BIG PROBLEMS with RAID compatibility with the previous generations of DELL Servers.  A RAID set from earlier DELL systems are not be recognized as 'foreign' on the PERC9.  I AM in conversation with DELL Tech Support.  I was just wondering if anyone else has seen this problem,  and if any answers are coming from DELL that have not gotten to me yet.

Thanks


----------



## dwaus (Jun 27, 2022)

dkline201 said:


> Hi!  It's me again


Hi dkline201,
Now that FreeBSD 13.1 has now been officially released. Does your Dell R450 with PERC H755 work properly with the official release of FreeBSD 13.1?
I'm looking at getting a Dell R350 with a PERC H755 in the near future, to use with pfSense once they update their version to be based upon FreeBSD 13.1.


----------



## dkline201 (Jun 27, 2022)

dwaus said:


> Hi dkline201,
> Now that FreeBSD 13.1 has now been officially released. Does your Dell R450 with PERC H755 work properly with the official release of FreeBSD 13.1?
> I'm looking at getting a Dell R350 with a PERC H755 in the near future, to use with pfSense once they update their version to be based upon FreeBSD 13.1.


Once I got 'mfsbsd13.1-rel',  I was able to boot from the USB and install FreeBSD 13.1-rel onto my DELL R450 with the PERC H755.   My initial testing for basic functions seem OK.  We are now in the process of loading all of OUR programs,  and seem to be going well.   Hardware wise,  I would say it seem good.  But as for the heavy duty production software,  that will be up to the Programmers.


----------



## dkline201 (Jul 5, 2022)

Wow!  This DELL R450-15th Gen system is still fighting me every step along the way.  Now I am having problems with the Broadcom 10GbE SFP+ and BaseT boards.  According to *FreeBSD 13.1-REL HCL*,  the Broadcom NetExtreme 57412, 57414, and 57416 boards ARE supported, and have an ID of "*bnxt*(x)".  I configured "/etc/rc.conf" with both DHCP and STATIC (not at the same time obviously).  The boards are not configured by FBSD 13.1-rel during boot up.  "pciconf -lv | grep -A1 -B1 network" shows the boards physically *installed* in the system,  but they don't show up on the console.   The 57412 SFP+ board *DID WORK in a DELL R740xd*.  I did not test the 57416  BaseT in the R740xd-14th Gen system, since the first one worked...

_Of course,  DELL only says "We don't support FreeBSD..."   Any suggestions?   Maybe the "bnxt" is wrong?    My Intel 10GbE boards changed from "ix(x)" to "ixl(x)"  when I installed it into the R450 15th Gen...  But DELL is dropping the Intel boards now..._


----------



## richardtoohey2 (Jul 5, 2022)

Just to eliminate the obvious you do have this in loader.conf?






						bnxt
					






					www.freebsd.org
				




     Alternatively, to load the    driver as a module at boot time, place the
     following line in loader.conf(5):

       if_bnxt_load="YES"


----------



## dkline201 (Jul 5, 2022)

Thanks to a "better search" of the Forum for "bnxt",  I found that answer about 10 minutes before you sent your reply.  Adding that line to  "/boot/loader.conf"  worked with the SFP+ board.  Both of the SFP+ got the IP addresses that I put into "/etc/rc.conf".   Now I will try the BaseT board.  THANK YOU MUCH for replying though,  and confirming what I was hoping would fix it.


----------



## richardtoohey2 (Jul 5, 2022)

Got my first 10G NIC machine (Supermicro) this year and learned that not all drivers (especially for these newer NICs) are built-in!

Don’t think I’ll be getting one of this generation of Dells just yet (just because I don’t need them yet) but certainly will at some point so watching your progress with interest - when I do get one I should have some confidence that things will work (and what I need to watch out for).


----------



## dkline201 (Jul 19, 2022)

We have finally gotten to the point where we are moving all of OUR programs over to FreeBSD 13.1-rel.  That conversion is going well.
But I could put together a list of the issues that we had getting to this point,  mainly in the compatibility issues on the new DELL 15th Generation systems with the LSI-Broadcom-MegaRAID-TriMode 3916 chipset series of RAID controllers,  and the 'relatively simple' fix to the Broadcom 5741x series of 10GbE NICs.
It would be a small way to repay the help that I have gotten from the Forum in the past few months...

Could I type that up as an attachment,  and post it here?

And I would 'still' like some feedback from an LSI-Broadcom' engineer as to WHY they decided to change the 'drive ID' numbering system that is causing the boot drive  incompatibility from previous generations of MegaRAID controllers (92xx and 93xx) to the new TriMode 38xx controller?


----------



## DBSystem79 (Sep 12, 2022)

dkline201 said:


> We have finally gotten to the point where we are moving all of OUR programs over to FreeBSD 13.1-rel.  That conversion is going well.
> But I could put together a list of the issues that we had getting to this point,  mainly in the compatibility issues on the new DELL 15th Generation systems with the LSI-Broadcom-MegaRAID-TriMode 3916 chipset series of RAID controllers,  and the 'relatively simple' fix to the Broadcom 5741x series of 10GbE NICs.
> It would be a small way to repay the help that I have gotten from the Forum in the past few months...
> 
> ...


Hi,
We will run in the same configuration as you 
Is it possible to share all your experience/resolution on an attachment ?


----------

