# Boot delay



## SirDice (Oct 17, 2013)

I just installed 9.2-RELEASE on one of our new SuperMicro servers and I've noticed a really long delay before it boots. The kernel loads quite quickly, the boot menu is shown but after pressing enter (or waiting the 10 seconds) there's a 5 minute(!) delay before it starts booting.

Does anybody know what delay that is? And more importantly, can I minimize it without having to build a custom kernel?


----------



## jb_fvwm2 (Oct 17, 2013)

It happens here but only one of every fifteen or so boots [I've never waited the five minutes, simply assumed it would not boot, so it may not be the same] , interspersed with "boot but keystrokes do not register", also every fifteen or so boots.  Most of the time there is no problem.  It could be that these two are not caused by nor the same issue as the one starting this thread.


----------



## Beeblebrox (Oct 17, 2013)

What happens if you set in /boot/loader.conf

```
autoboot_delay="-1"
```
My guess is that it won't change anything, which will thus rule-out any bootloader related delays/bugs.

The next item to consider is to play around with BIOS settings and disable possible causes of delay.

Final thought would be to look at the HDDs and file-systems, which you know how to do quite well.


----------



## swirling_vortex (Oct 17, 2013)

Which server is it? Or more specifically, what motherboard is it using? It could be trying to probe for a device and not getting a response. You might be able to reveal more information by playing with some of the boot(8) flags.

My suspicion is that it's a driver issue.


----------



## SirDice (Oct 18, 2013)

Beeblebrox said:
			
		

> What happens if you set in /boot/loader.conf
> 
> ```
> autoboot_delay="-1"
> ...


This only changes the timeout on the menu. The delay I get happens after that.



> The next item to consider is to play around with BIOS settings and disable possible causes of delay.


Already tried that but there isn't much to play with.



> Final thought would be to look at the HDDs and file-systems, which you know how to do quite well.


Filesystems are ok, it's just plain old UFS.


----------



## SirDice (Oct 18, 2013)

swirling_vortex said:
			
		

> Which server is it? Or more specifically, what motherboard is it using? It could be trying to probe for a device and not getting a response.


That's very much possible. The mainboard is a X9DRW.

This same server refused to boot the 9.1-RELEASE memstick too. Booting worked but when it tries to mount the filesystem it can't find the USB. So it may be related to EHCI.


----------



## Beeblebrox (Oct 18, 2013)

> So it may be related to EHCI.


Try disabling "plug-and-play" option in the BIOS and let the O/S handle that. So in BIOS set:

```
Plug-and-Play O/S: YES
```
See if that works...


----------



## phoenix (Oct 18, 2013)

SirDice said:
			
		

> I just installed 9.2-RELEASE on one of our new SuperMicro servers and I've noticed a really long delay before it boots. The kernel loads quite quickly, the boot menu is shown but after pressing enter (or waiting the 10 seconds) there's a 5 minute(!) delay before it starts booting.
> 
> Does anybody know what delay that is? And more importantly, can I minimize it without having to build a custom kernel?



I have this on our storage servers running FreeBSD 9-STABLE (originally 9.0-STABLE, then 9.1-STABLE, now 9.2-STABLE), all with SuperMicro H8DGi-F motherboards.  It's not a 5 minute delay, but it is a long (1-2 minute) delay.  Same place, between the loader and the dmesg output (so after kernel and modules have been loaded).  Just a blinking cursor in the top-left corner of the screen.

Since it doesn't affect the boot other than delaying it, I haven't bothered trying to diagnose it.


----------



## Terry_Kennedy (Oct 19, 2013)

phoenix said:
			
		

> Same place, between the loader and the dmesg output (so after kernel and modules have been loaded).  Just a blinking cursor in the top-left corner of the screen.


I have this on a variety of hardware running 8-STABLE as well. The only thing in common across varying hardware (Dell, Supermicro, etc.) is systems with large amounts of memory (48GB or more). Those systems all use buffered / registered ECC memory of assorted speeds.

I have always assumed that it is the kernel doing a memory scrub when loading, since otherwise any out-of-bounds reference might give a hard ECC error, rather than the expected Trap 12*. I've never been bothered enough to look in the sources to see what it is really doing. A verbose boot might be informative, to see if it happens before or after the kernel prints the memory segment table.

* Of course, if the BIOS does a full memory test, the FreeBSD scrub will be redundant.


----------



## xibo (Oct 19, 2013)

My X9DA7 can boot without delay if it has only 2 memory modules installed per CPU (for a total of 64GB), but delays several minutes if I install more. It's same slow with 4 modules per CPU as it is with 8 though.

I'm not getting a blinking cursor, just a 5 minutes delay.


----------



## SirDice (Oct 22, 2013)

Yep, that's the one. This server does have a huge amount of memory. Oddly enough when it's delaying at that point I can see the OS disk LED blinking like mad. As if it's doing an fsck(8).


----------



## vanessa (Oct 22, 2013)

Good to know! Is this a bug or a kernel feature?  Can it be influenced somehow?
We also always use ECC servers ...


----------



## phoenix (Oct 22, 2013)

Hrm, all of the servers that do this here have 32+ GB (32, 48, 64, 128) of RAM, spread across two CPU sockets.  Now that I think about it, the delay is longer on the two systems with 128 GB.  It might be "size of RAM" related.


----------



## SirDice (Oct 23, 2013)

These servers have a little more than that even. They have 392 GB and the delay is considerable at that point. It feels like it's trying to do both, check memory and check the harddrive. I see no other reason why the HD LEDS would be blinking.


----------



## cforger (May 24, 2015)

This is an old thread, but I have the same problem .. then I came across what may be part of the delay - rc.d scripts.

For me, I have a lot of ZFS volumes, and the rc.d/zvol script was taking it's time.  Most of them run without any diags to the console, so you don't know they are active.


----------



## SirDice (Jun 29, 2015)

Old thread but it seems to be referred a couple of times. Adding this to /boot/loader.conf turns off the memory check before the kernel starts. If you have a lot of memory this check is going to take a considerable amount of time.


```
hw.memtest.tests="0"
```


----------



## usdmatt (Jun 29, 2015)

The memory test shouldn't cause a delay in more recent amd64 systems. The default was changed not long after this thread was originally created.



> Modified _Thu Nov 21 18:37:11 2013 UTC_ (19 months ago) by _emaste_
> File length: 66386 byte(s)
> Diff to previous 258176
> Disable amd64 boot time memory test by default
> ...


----------



## SirDice (Jun 29, 2015)

It doesn't look like it has made its way into 9.x though.

https://reviews.freebsd.org/rS283262


----------



## usdmatt (Jun 29, 2015)

Strange how something that can cause such a huge delay on modern systems and "has little value on contemporary amd64 hardware" wasn't marked for MFC in the first place.

Doesn't seem the sort of thing that would need to go through review to me. The comments on the related review for stable/10 seem to follow a similar opinion



> jhb
> Hmm, I haven't seen a review request for an MFC before. Presumably changes should get reviewed before going into HEAD, so an MFC doesn't need the same level of review. I certainly think it is fine to MFC this change though.





> kib
> I think this is a request for opinion about possibility of MFC, i.e. whether it is suitable to change the behaviour on the stable branch.
> 
> I do not see why not. The memory 'test' is useless, IMO.



Not that I run anything other than 10.1+ on new systems these days (I don't have any existing large memory servers) so unlikely to ever affect me.


----------



## canoodle (Jun 28, 2019)

thanks for this cool OS! 

imho boot delay should kept as short as possible, imho it is important that machines should boot up and shutdown fast, humans should NEVER wait for machines, machines can wait forever...


```
# tested with
uname -a
FreeBSD 12.0-RELEASE FreeBSD 12.0-RELEASE r341666 GENERIC  amd64

# create the file (it does not exist per default)
vi /boot/loader.conf

# insert this
autoboot_delay="1"
```

also it waited very long at: "my unqualified hostname sleeping retry" (many many seconds)


```
# edit first line
vi /etc/rc.conf
# to something like (does not matter if it does not exist)
hostname="www.full-domain.com"
```

test with a reboot: shutdown -r now

and meassure your boot up time (including bios?) with your smart phone.

including bios bootup time is now 30 seconds. (it does ntp sync on boot... so this adds delay but makes time very accurate)


----------

