# Can't find /boot/zfsloader



## i.am.the.problem (Jan 28, 2021)

I recently installed FreeBSD 12.2, Root on ZFS, on a DELL R540 with a H330 raid card.









						ZFS Root install with DELL R540 and H330
					

Hi,  I'm trying to install FreeBSD 12.2, Root on ZFS, on a DELL R540 with a H330 raid card.  I've switched the H330 to HBA mode.  I've tried booting into the installer and try to select the Auto (ZFS) - Guided Root-on-ZFS but got a kernel panic.  I setup a VM which the same number of disks, did...




					forums.freebsd.org
				




I just rebooted the machine and was presented with this error.

What can I do to find out what went wrong and can I fix it?


----------



## SirDice (Jan 29, 2021)

Did you add `hw.mfi.mrsas_enable="1"` to /boot/device.hints?


----------



## i.am.the.problem (Jan 29, 2021)

Yes. The machine has been running for a month and has been rebooted several times.
Until yesterday when I rebooted it and got this error.


----------



## SirDice (Jan 29, 2021)

i.am.the.problem said:


> Until yesterday when I rebooted it and got this error.


Changed anything before the reboot? Check the disks too, are they all detected? Is this a one disk pool, or mirror, raidz?


----------



## i.am.the.problem (Jan 29, 2021)

I changed the network configuration to setup link aggregation.

4 disks in total configured as RAID10.

I've booted from the install CD exited to shell and executed zpool import which shows the following...


----------



## i.am.the.problem (Jan 29, 2021)




----------



## Emrion (Jan 29, 2021)

Use `gpart show` instead of `fdisk` since it's a GPT scheme you have.


----------



## i.am.the.problem (Jan 29, 2021)

Emrion said:


> Use `gpart show` instead of `fdisk` since it's a GPT scheme you have.





All 4 drives show the same partitions.


----------



## i.am.the.problem (Jan 29, 2021)

I've imported the pool as follows...

mkdir /tmp/zzz
zpool import -R /tmp/zzz zroot

/tmp/zzz contains tmp, usr, var and zroot directories.

Should boot be in there as well?


----------



## SirDice (Jan 29, 2021)

i.am.the.problem said:


> Should boot be in there as well?


It's just a directory of the root filesystem, not a separate filesystem.


----------



## i.am.the.problem (Jan 29, 2021)

So it looks like all the required files are there.


----------



## SirDice (Jan 29, 2021)

Yes, the pool itself looks fine. But the boot pictures are showing I/O errors, that causes the pool to fail to load. Which in turn means the system can't find the files in /boot/ because it has no access to the pool where these files are stored. The files not being found is a symptom, the cause are the I/O errors.


----------



## Mjölnir (Jan 29, 2021)

Unstable electrical power conditions?  If you have an UPS, go grab it & try again.


----------



## i.am.the.problem (Jan 29, 2021)

OK. I'll see if I can get another machine to boot from the drives.


----------



## i.am.the.problem (Jan 29, 2021)

Mjölnir said:


> Unstable electrical power conditions?  If you have an UPS, go grab it & try again.


Server has 2 power supplies. Each is connected to a separate UPS.


----------



## SirDice (Jan 29, 2021)

Are these disks connected via a SAS port extender? Those things can cause issues too. I've had that happen on several occasions where the port extender was failing and this caused random disks to fail. Yes, even on brand spanking new hardware.


----------



## i.am.the.problem (Jan 29, 2021)

SirDice said:


> Are these disks connected via a SAS port extender? Those things can cause issues too. I've had that happen on several occasions where the port extender was failing and this caused random disks to fail. Yes, even on brand spanking new hardware.


No the drives are plugged in to a a DELL R540.


----------



## SirDice (Jan 29, 2021)

i.am.the.problem said:


> No the drives are plugged in to a a DELL R540.


If it's a chassis with more then 8 disk slots it likely has a port extender on the backplane. The PERC H330 is a 2x4 port card.


----------



## i.am.the.problem (Jan 29, 2021)

SirDice said:


> If it's a chassis with more then 8 disk slots it likely has a port extender on the backplane. The PERC H330 is a 2x4 port card.


That's true. I've just moved the disks from bays 1,2,3,4 to 5,6,7,8. No difference.
If its a hardware fault how am I able to import the zpool from the live cd?


----------



## sko (Jan 29, 2021)

Are those SATA drives by any chance? I've seen some SATA drives with slow spinup/powerup times where the SAS-controller didn't wait for their response and already handed everything over to the BIOS/UEFI before the drives were fully initialized. SAS drives always were perfectly fine.
Booting from those SATA disks then failed, but booting from a flashdisk didn't show any obvious problems as the live-system probes all devices again at a much later time.

Search for some option in the BIOS to allow for a longer spinup-time at boot, I bet Dell has such an option too (supermicro definitely does, even on a per-controller-basis IIRC).
And I don't think this crap found its way into server-BIOS, but if it has, disable all "fast boot" or similar options and let the BIOS do a proper, full POST.


----------



## i.am.the.problem (Jan 29, 2021)

sko said:


> Are those SATA drives by any chance? I've seen some SATA drives with slow spinup/powerup times where the SAS-controller didn't wait for their response and already handed everything over to the BIOS/UEFI before the drives were fully initialized. SAS drives always were perfectly fine.
> Booting from those SATA disks then failed, but booting from a flashdisk didn't show any obvious problems as the live-system probes all devices again at a much later time.
> 
> Search for some option in the BIOS to allow for a longer spinup-time at boot, I bet Dell has such an option too (supermicro definitely does, even on a per-controller-basis IIRC).
> And I don't think this crap found its way into server-BIOS, but if it has, disable all "fast boot" or similar options and let the BIOS do a proper, full POST.


Yes they are SATA drives. I'll have a look in the BIOS.


----------



## i.am.the.problem (Feb 3, 2021)

I couldn't find anything in the BIOS that made this system boot.

I replaced the PERC H330 with a PERC HBA330 and the system booted. 
I've shutdown the system and powered it back on several times and it continues to boot.

Thank you for the assistance.


----------



## Snurg (Feb 3, 2021)

BTW, my impression is that, unlike with for example, AMD/ATI graphics cards, there seems no compatibility matrix table for host adapters.
Only in some release notes one can find a list.
The 12.0 Release list does not contain either of the Perc adapters used by the OP; 12.1 and 12.2 seem not to have updated hardware compatibility lists.
Where to submit success/failure reports so one can know in advance whether a particular hardware can be expected to work with FreeBSD (or not)?


----------



## sko (Feb 3, 2021)

(sorry, this post was meant for another thread - must have mixed up the open tabs. sorry)


----------



## SirDice (Feb 3, 2021)

Snurg said:


> The 12.0 Release list does not contain either of the Perc adapters used by the OP; 12.1 and 12.2 seem not to have updated hardware compatibility lists.


Dell and HP often use rebranded LSI cards. Not all Dell PERC cards are based on LSI (now Avago or Broadcom, I lost track) though. It's not specific cards or brands that are supported, it's the chipsets on those cards that's supported.


----------



## Mjölnir (Feb 3, 2021)

sko said:


> I just checked and all 12.x installs seem to have `vfs.zfs.min_auto_ashift=12` already preset (by the installer?) in /etc/sysctl.conf [...]
> The sysctl.conf (5)hasn't been touched on any of those systems, so those are the defaults set by the installer. I suspect the `ashift` is also set by the installer because on the 12.1-RELEASE host the various security knobs are listed before the ashift.


I didn't look into the installer for quite a while (building from source instead &`make install{world,kernel}` with `DESTDIR` etc. set), but IIRC the partitioning module of the installer asks if it should align partitions on sector borders?  This applies to gpart(8)'s `-a alignment` parameter, so strictly speaking it's OT and doesn't affect ZFS's newly created vdes.  I guess performance suffers on modern large drives  with a blocksize <> 512 when this ZFS tunable isn't set.  `vfs.zfs.min_auto_ashift=12` is the failsafe value for modern large drives with a blocksize of 4 kiB, and on other's it doesn't hurt as long you don't create 1000's of vdevs.  BTW you can file in a bug report: this should be renamed to `vfs.zfs.vdev.min_auto_ashift`


----------

