# Booting Weirdness



## Barney (Dec 9, 2022)

I have an old supermicro system (5017R-M) where when I have a 13.1 disk in drive 0 and either a 9.x or 13.0 disk in drive 1, the system boots from drive 1. It seems to use the boot code from drive 0 as per the bios setting, but then books the kernel and OS from drive 1.This has been a problem with this box for all of time, but I wonder if its FreeBSD related and something that can be fixed or directed. Its a pain to have to pull the drive to reboot particularly when doing driver work.


----------



## SirDice (Dec 9, 2022)

If you disconnect that drive 1 does it boot 13.1 properly from drive 0?

Also double check the connections. Which SATA ports are used? I-SATA or S-SATA? And is drive 0 attached to SATA0?


----------



## Barney (Dec 9, 2022)

Yes of course. As I said, I have to pull the second drive to get drive 0 to boot properly. Note this happened on these boxes using 9.x, so it's not a new thing. The same thing does NOT happen on a newer Supermicro box with the same 2 drives. I've never been able to get them to admit the problem, but if there was some tuning or hack that could force a drive 0 boot it would be helpful


----------



## _martin (Dec 9, 2022)

BIOS should pass proper disk it loaded the first sector from (in your case disk0) to the loader/code from that first sector. I remember very vaguely that next bootloader stages still allowed to press Fx to boot from other disk (F1 - disk one, F2- disk two) but I don't remember if it showed it always or only when user interrupted it.

When the system boots from disk1 how does system see those disks? Which one is ada0?


----------



## Barney (Dec 9, 2022)

It shows the 2nd disk as ada0


----------



## covacat (Dec 9, 2022)

looks more like a bios bug then


----------



## SirDice (Dec 9, 2022)

Barney said:


> It shows the 2nd disk as ada0


I think it might be an order issue. Adding the second disk shifts disk 0 from ada0 to ada1 and that second drive is now ada0. I'm thinking drive 0 is connected to I-SATA0 and drive 1 to S-SATA0, or some variation on that. One controller gets precedence over the other and the ada* disks are numbered as detected.


----------



## _martin (Dec 9, 2022)

How can you tell which disk is being booted by BIOS ? I think prompt at that early stages is the same for many years now (btxloader, bootX). 
Are you using MBR or GPT disk layout ?


----------



## Barney (Dec 9, 2022)

If I put the disks in bays 2 and 3 or 3 and 4 it works fine, so ordering doesnt seem to be the problem. Also it only happens with a new kernel on the disk. So bay 0 is the main disk and bay 1 is an image backup. Boots fine for months. Then  a customer would do an upgrade on ada0, and then the system would boot from bay 1. Almost as if saying "hey, the kernel I booted from last time is in bay 1 so I'll boot from that". It would boot the kernel from bay 1 and then / from bay 0 causing much hair pulling before we figured out what it was doing

Supermicro tests with windows and linux so I always thought it was a freebsd issue. Its been driving me nuts for years.


----------



## Argentum (Dec 10, 2022)

Barney said:


> I have an old supermicro system (5017R-M) where when I have a 13.1 disk in drive 0 and either a 9.x or 13.0 disk in drive 1, the system boots from drive 1. It seems to use the boot code from drive 0 as per the bios setting, but then books the kernel and OS from drive 1.This has been a problem with this box for all of time, but I wonder if its FreeBSD related and something that can be fixed or directed. Its a pain to have to pull the drive to reboot particularly when doing driver work.


Is it EFI or legacy boot you are talking about?

See: efibootmgr(8)


----------



## Barney (Dec 14, 2022)

It's MBR


----------



## PMc (Dec 14, 2022)

Not really sure what's going on (and I don't have a supermicro), but as it is MBR, you could install boot0cfg(8) on the MBR (if not already present). Then you can press F5 on the MBR prompt (before the loader menu), and this should hop to the "next" disk (in BIOS logic) *and* it should store that choice on the respective disk for subsequent boots.
So You might try to put that F5 choice onto your drive-1 and so -hopefully- make it always hop back to drive-0 - unless a different F-key is pressed.

Also beware: the BIOS will try and boot *some* disk (according to CMOS or RAID-controller or whatever configuration), the loader will then load some kernel (according to the loader behaviour), but what you finally get mounted as a running system is whatever happens to be written in that respective /etc/fstab. (Which can create a lot fun *eg* when using a 2nd disk as a reserve copy.)


----------



## Barney (Dec 14, 2022)

Just to note, I updted the bios to the 3.3 which is the latest (the MB is EOL so there will be no moreo) and the problem persists. So anyone with an X9SRI MB take note. I like using these old MBs because I can get 10 core CPUs for like $40. and they're really not that much slower than the newer ones


----------



## Argentum (Dec 15, 2022)

Barney said:


> It's MBR


Can you switch to the EFI boot method?


----------

