# What is the minimum boot time of FreeBSD?



## STORP (Feb 17, 2022)

The FreeBSD init system just takes a looooong time and although I am just running it in a VM right now when I actually use FreeBSD for desktop usage the bootup time might be a problem. When windows took too long to boot I would end up doing nothing productive. It breaks the flow. I am gonna use the OpenRC or Runit systems but I was just wondering is it possible to make the time even shorter like clear Linux? (I know this goes against the forum rules but I am just curious)


----------



## tux2bsd (Feb 17, 2022)

It takes a while, it's not that bad though (kind of quaint).  You don't need to reboot as often because there aren't kernel updates every other day.  A lot less often.



STORP said:


> I am gonna use the OpenRC or Runit systems



Ricer gonna rice.


----------



## STORP (Feb 17, 2022)

tux2bsd said:


> It takes a while, it's not that bad though (kind of quaint).  You don't need to reboot as often because there aren't kernel updates every other day.  A lot less often.
> 
> 
> 
> Ricer gonna rice.


I don't see why I would keep my laptop open 24/7 that's power consumption gone to waste


----------



## SirDice (Feb 17, 2022)

My systems boot fairly fast. The POST of those machines takes a lot longer. Machines with extra SAS cards for example take several minutes _before_ that system even starts to boot. The boot itself is just a few seconds.


----------



## eternal_noob (Feb 17, 2022)

To be honest, i don't care if my system needs 10 seconds or 60 seconds to boot. I suggest you try to relax. 

Book recommendation:





						The Discovery of Slowness - Wikipedia
					






					en.wikipedia.org


----------



## ondra_knezour (Feb 17, 2022)

BootTime - FreeBSD Wiki


----------



## tux2bsd (Feb 17, 2022)

STORP said:


> I don't see why I would keep my laptop open 24/7 that's power consumption gone to waste


You should include relevant information like this in your first post.


----------



## SirDice (Feb 17, 2022)

I'm willing to bet the reason it's slow to boot is due to relatively slow I/O on the VM. Picking the right controller can make a world of difference. It also helps to know _which_ VM software you're using. Virtualbox? VMWare? Qemu? And what's the type of the laptop this runs on? Some old laptop from the '90s? Or is it a last generation Core i5/i7? Does the laptop have a slow 5400 RPM 2.5 inch disk or does it have NVMe disks? This can make a world of difference too.


----------



## Deleted member 70481 (Feb 17, 2022)

Building a custom kernel instead of using GENERIC might make it a bit faster


----------



## hbsd (Feb 17, 2022)

Too long like several minutes? please give more information about your system. I used to run FreeBSD on a VM, but boot time was fast. I don't have a SSD but my boot time is less than 20 seconds.


----------



## jmos (Feb 17, 2022)

Here a virtual, blank FreeBSD (official VM-image used with bhyve) boots to the login in less than 8 seconds; My bare metal machine with many, many things takes less than 12 seconds to the X11 login. But does that say anything?


----------



## STORP (Feb 17, 2022)

Mine is i3 4gb ram ssd and ik VMs are not suitable for comparison but the wiki says 25 seconds which is a lil worrying for my PC because a fresh install of windows on my pc took 3 secs to boot up but windows after 3 years the boot time became 20 minutes. I am running qemu. I am not using the vm time as a metric but the wiki time.


SirDice said:


> I'm willing to bet the reason it's slow to boot is due to relatively slow I/O on the VM. Picking the right controller can make a world of difference. It also helps to know _which_ VM software you're using. Virtualbox? VMWare? Qemu? And what's the type of the laptop this runs on? Some old laptop from the '90s? Or is it a last generation Core i5/i7? Does the laptop have a slow 5400 RPM 2.5 inch disk or does it have NVMe disks? This can make a world of difference too.


----------



## STORP (Feb 17, 2022)

jmos said:


> Here a virtual, blank FreeBSD (official VM-image used with bhyve) boots to the login in less than 8 seconds; My bare metal machine with many, many things takes less than 12 seconds to the X11 login. But does that say anything?


My PC on Manjaro or openSUSE took 10 secs from pressing the power button to get to sddm


----------



## STORP (Feb 17, 2022)

What? When I had an hdd the actual OS didn't even run you must have some good specs and about 10 mins in vm.


hbsd said:


> Too long like several minutes? please give more information about your system. I used to run FreeBSD on a VM, but boot time was fast. I don't have a SSD but my boot time is less than 20 seconds.


----------



## STORP (Feb 17, 2022)

eternal_noob said:


> To be honest, i don't care if my system needs 10 seconds or 60 seconds to boot. I suggest you try to relax.
> 
> Book recommendation:
> 
> ...


I am just asking how much it can be trimmed because why not. The boot time most probably will not be a problem


----------



## SirDice (Feb 17, 2022)

STORP said:


> I am just asking how much it can be trimmed because why not.


Simple, don't start services you don't need. This isn't rocket science.


----------



## eternal_noob (Feb 17, 2022)

STORP said:


> because why not


Sorry for not replying seriously, at first i thought you were a troll.


----------



## sko (Feb 17, 2022)

STORP said:


> Mine is i3 4gb ram ssd and ik VMs are not suitable for comparison but the wiki says 25 seconds which is a lil worrying for my PC because a fresh install of windows on my pc took 3 secs to boot up but windows after 3 years the boot time became 20 minutes. I am running qemu. I am not using the vm time as a metric but the wiki time.


Windows 10 isn't shutting down, it is only hibernating. So comparing the time it takes to resume from suspend to a full boot isn't useful. Additionally windows won't start a lot of (essential) services until after login - so you still wait some time after login to actually get a useful system.

Also, running VMs on such a low-spec machine, especially on top of windows (which is by far the slowest and most resource-trashing "hypervisor" you can get) is doomed to fail. Anything below 8GB RAM shouldn't be even considered to run a VM that is supposed to run a desktop environment. But even then - comparing a VM to a bare-metal boot won't get you anywhere either. Most of my Open- and FreeBSD VMs (on bhyve) easily boot up in less than 20 seconds - on bare metal with some HBAs you won't even be at the POST screen in that amount of time...


----------



## SirDice (Feb 17, 2022)

sko said:


> Most of my Open- and FreeBSD VMs (on bhyve) easily boot up in less than 20 seconds


Most of my VMs boot so fast I often wonder if they rebooted at all. I have to check uptime(1) to see that it actually did.


----------



## STORP (Feb 17, 2022)

I am not running VMs on windows that's just impossible this is openSUSE Tumbleweed


sko said:


> Windows 10 isn't shutting down, it is only hibernating. So comparing the time it takes to resume from suspend to a full boot isn't useful. Additionally windows won't start a lot of (essential) services until after login - so you still wait some time after login to actually get a useful system.
> 
> Also, running VMs on such a low-spec machine, especially on top of windows (which is by far the slowest and most resource-trashing "hypervisor" you can get) is doomed to fail. Anything below 8GB RAM shouldn't be even considered to run a VM that is supposed to run a desktop environment. But even then - comparing a VM to a bare-metal boot won't get you anywhere either. Most of my Open- and FreeBSD VMs (on bhyve) easily boot up in less than 20 seconds - on bare metal with some HBAs you won't even be at the POST screen in that amount of time...


----------



## hbsd (Feb 17, 2022)

STORP said:


> What? When I had an hdd the actual OS didn't even run you must have some good specs and about 10 mins in vm.


My CPU is Intel i7-4790K (8 core) 4.00 GHz and RAM 16 GB. When I used a VM (KVM), dedicated nearly half of my hardware to VM but boot time didn't even take a minute.


----------



## STORP (Feb 17, 2022)

Why would a fresh install start services that I don't need? Coming from linux a fresh install of any distro except bloated distros would not start unnessecary services.


SirDice said:


> Simple, don't start services you don't need. This isn't rocket science.


----------



## SirDice (Feb 17, 2022)

STORP said:


> I am not running VMs on windows that's just impossible


Really? So Hyper-V, Virtualbox and VMWare don't work on Windows? 



STORP said:


> Why would a fresh install start services that I don't need? Coming from linux a fresh install of any distro except bloated distros would not start unnessecary services.


A base install of FreeBSD doesn't enable much by default (only devd(8), syslogd(8), sshd(8) and perhaps sendmail(8)). But X and KDE (assuming that because you mentioned sddm) are not part of base OS install. You enabled those yourself.


----------



## sko (Feb 17, 2022)

SirDice said:


> Most of my VMs boot so fast I often wonder if they rebooted at all. I have to check uptime(1) to see that it actually did.


same for the VMs on my workstation - they reside on a pool on a 2x NVMe mirrored vdev. A full reboot from "shutdown -r now" until I can login via ssh again in ~5 seconds or even less for a 'small' (in terms of heavy and slowly stopping services) VM is pretty standard nowadays... Usually on VMs it takes (much) longer to stop the services (especially stuff like databases) than to actually reboot the OS. This gets even more true in jails, where the actual "reboot" is done within 1-2 seconds...

But again: Hardware resources are key when running VMs! You absolutely need enough memory or your host is constantly busy swapping which completely kills the VMs. Same goes for spinning rust when trying to run extremely I/O heavy OSes (windows with its horribly inefficient registry) in a VM. Windows in a VM will just completely trash even bigger ZFS pools on spinning disks due to its horrible I/O patterns at boot...


----------



## sko (Feb 17, 2022)

STORP said:


> I am not running VMs on windows that's just impossible this is openSUSE Tumbleweed


Sorry, I somehow implied by your first message(s) that you are running windows on the host, because you refered to windows boot times...

Still: 4GB is barely enough for even a single OS with a full-blown DE, especially bloated ones like KDE/Gnome, let alone a second OS in a VM that is supposed to run the same amount of bells and whilstles...


----------



## STORP (Feb 17, 2022)

SirDice said:


> Really? So Hyper-V, Virtualbox and VMWare don't work on Windows?
> 
> 
> A base install of FreeBSD doesn't enable much by default (only devd(8), syslogd(8), sshd(8) and perhaps sendmail(8)). But X and KDE (assuming that because you mentioned sddm) are not part of base OS install. You enabled those yourself.


No I mean like how heavy windows already is to be running VMs

And right now in the vm its completely a base install nothing else installed yet 
I mean other linux distributions by sddm


----------



## ralphbsz (Feb 17, 2022)

Let's define "boot time" as: time it takes from power coming on (which may be when the laptop comes out of sleep) to useful functionality, which may be serving network packets, having a shell login prompt, or a desktop environment responding to keyboard and mouse.

Single most important factor: Is the machine actually booting (re-initializing the hardware, OS kernel and all services), or is it just restoring from a hibernate or sleep state. May OSes (in particular MacOS and Windows) do not actually boot when you open a laptop or turn it on. So then the question becomes: How to enable sleep/hibernate when using FreeBSD on that particular hardware.

Second factor: Disk IO. With SSDs (including NVMe) as boot disks, one can get the boot time to around 10 seconds and below. With spinning disks, it's hard to get below 20-30 seconds (can be way more if lots of services are enabled).

Third factor: Hardware initialization, which mostly is BIOS, and (as SirDice already said) initialization of disk controllers. On a large enterprise-grade server with hundreds of (spinning) disks attached, this can take 10-15 minutes, because all those disks have to be spun up.

The OP is talking about using different init systems. My intuition is that this is a fool's errand, and won't make a significant difference. Similarly, creating a custom kernel (for example without modules, and only the needed drivers compiled in) no longer makes a significant difference.


----------



## grahamperrin@ (Feb 18, 2022)

ralphbsz said:


> around 10 seconds and below.



The operating system *plus a desktop environment* within around twelve seconds with FreeBSD 14.0-CURRENT:

FreeBSD 14-CURRENT 12s boot to desktop (sway) ◀ /r/freebsd search for _CURRENT 12s_ in the opening post

A TSLOG-based measurement on the same computer might have found less than twelve.

TSLOG boot profiling


----------



## grahamperrin@ (Feb 18, 2022)

tux2bsd said:


> … You don't need to reboot as often because there aren't kernel updates every other day. …



True, however there can be a reboot with precautionary use of a new ZFS boot environments before a _package_ upgrade. Updates to ports are more frequent than updates to the (RELEASE) kernel.  



sko said:


> … Windows 10 isn't shutting down, it is only hibernating …



Also, Windows 10 can start quickly _without_ hibernation.



eternal_noob said:


> … i don't care if my system needs 10 seconds or 60 seconds …



More than sixty here (see above). For myself, I'm largely carefree about the duration.



sko said:


> … 4GB is barely enough for even a single OS with a full-blown DE, especially bloated ones like KDE/Gnome, …



That's somewhat disputable, however this is a *base system* topic; quote me elsewhere if you'd like to discuss. Thanks.


----------



## richardtoohey2 (Feb 18, 2022)

Colin Percival has been doing a lot of work in making faster boots on AWS - don't think he's _too_ far off ClearLinux.

I think most of his progress reports are on Twitter - this is an old blog post so don't pay too much attention to these numbers (just might give you an idea where to go and dig).



			EC2 boot time benchmarking


----------



## mark_j (Feb 18, 2022)

According to Ed Maste, Colin's got it down to 9.5 seconds. See one of the foundation videos for information (video 2, around 6 minutes in if I recall correctly). This is from 28 seconds.


----------



## grahamperrin@ (Feb 18, 2022)

Talk of boot times begins at 3:30.


----------



## mfjurbala (Feb 18, 2022)

mark_j said:


> According to Ed Maste, Colin's got it down to 9.5 seconds. See one of the foundation videos for information (video 2, around 6 minutes in if I recall correctly). This is from 28 seconds.


Yep, here are links to the Video and Wiki for more information. The wiki also links to Colin's site which has a link to his Patreon if anyone wants to support his work.


----------



## mfjurbala (Feb 18, 2022)

Beat me to it. And someone had my other link posted yesterday. Oh well


----------



## grahamperrin@ (Feb 18, 2022)

CoolHwhipMike said:


> … Colin's site which has a link to his Patreon if anyone wants to support his work.



Thanks, I heard _Patreon_ spoken, but couldn't easily find it linked from any boot time-related area at his site.

Found, relating to FreeBSD on Amazon Elastic Compute Cloud (Amazon EC2): 

<https://www.patreon.com/cperciva> ◀ Faster booting, or support for older instance types? | Colin Percival on Patreon


----------



## hruodr (Feb 18, 2022)

This is also a always from to time recurring theme. Characteristic of it is, that the one that brings it always know that systemd is supposedly faster and that there are alternative init systems like OpenRC and runit that solves the problem that is not a problem. Init seems to be a problem, because due to this perceived knowledge it is made guilty of a slow booting that have other causes. Or perhaps just because the system does not boot at the minimum time, and hence that is too long. I do not find other explanation for these complaints against init. The danger of it is that some developers take these complaints seriously and we get another cool change for the sake change that bring real problems.


----------



## grahamperrin@ (Feb 18, 2022)

hruodr said:


> another cool change for the sake change



What was the _other_ supposed change for the sake of change?


----------



## sko (Feb 18, 2022)

hruodr said:


> Characteristic of it is, that the one that brings it always know that systemd is supposedly faster and that there are alternative init systems like OpenRC and runit that solves the problem that is not a problem.



We were running several TrueOS clients which used OpenRC for nearly 2 years as a test case. For simple systems it might even work - but bring in just *some* e.g. network dependencies and it quickly falls apart on a regular basis. And it wasn't considerably faster than simple init - in fact the vanilla FreeBSD clients usually booted a little bit faster...
Shutdown/Reboot was almost never reliable and kept over 50% of those clients hanging at shutdown. I don't know if this (known!) Problem ever got fixed and I really don't care... I submitted a few patches to rc-scripts to make booting more reliable and most of them were reverted by later updates for the sake of "make booting faster". So the focus seems to be the same as with systemd: they don't want to bring up all systems reliable, but only a small subset of simple hosts in short times so marketing droids have something to brag about...
So OpenRC in the case of TrueOS solved a "problem" that never existed with being not better at what it tried to improve but introducing *real* problems left and right...

(I won't get started with systemd here, this won't end well...)

As it was already stated multiple times: On a cold boot nowadays with NVMes the system often takes longer to finish POST and hand over to the boot loader than the OS itself needs to boot. And if you are constantly in such a hurry that 5, 10 even 30 seconds more or less for a once-per-day task (in case of your laptop/desktop) are absolutely life-critical, you should really prepare for a heart-attack in the near future. Especially if you've traded in reliability at boot with "moh' speed" and thus have to get annoyed over a halfway-booted system once a week.


----------



## hruodr (Feb 18, 2022)

grahamperrin said:


> What was the _other_ supposed change for the sake of change?



For example the change in the boot loader, taking away forth and putting lua.


----------



## sko (Feb 18, 2022)

hruodr said:


> For example the change in the boot loader, taking away forth and putting lua.



IIRC the switch to lua was primarily done to ease maintaining and extending the boot loader while also removing some arcane stuff that has been there for ages and nobody even remembered or could figure out what it was doing but caused problems with EFI on a regular basis.
I think there even was an interview about that (partial) rewrite and the reasons for moving from forth to lua in the old "BSD now" show back when it wasn't just an audio podcast.

I generally find the "current" FreeBSD boot loader very reliable and flexible and never had issues, even when switching between EFI to BIOS booting or moving around disks it 'just works'™ absolutely reliable. So in this regard I think the upgrade was well done and really solved actual problems.


----------



## hruodr (Feb 18, 2022)

sko said:


> I generally find the "current" FreeBSD boot loader very reliable and flexible


Perhaps it is so. I was told in the list that the following problem has nothing to do with the rewrite:









						LUA ERROR: Can not open /boot/lua/loader.lua
					

I tried to install Freebsd 12.0 RC3 and got the same error as mentioned here: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=233098 Also on a amd64 box.  "Startup error in /boot/lua/loader.lua: LUA ERROR: Cannot open /boot/lua/loader.lua: no such file or directory" "can't load kernel"  Is...




					forums.freebsd.org
				




It was very persistent. Was it endly solved?


----------



## mfjurbala (Feb 18, 2022)

Yes, it's hidden a little bit.


----------



## grahamperrin@ (May 16, 2022)

13.1-RELEASE loader in the EFI system partition (ESP)​


tuxador said:


> The release notes say that the bootloader was updated with many improvements and that the boot time is cut to half comparing to 13.0. …



From TSLOG boot profiling : freebsd:






For flame graphs of this type, I'm not sure how much represents the time taken by what's in the ESP (after copying what's required to the partition). 

I'll ask in Reddit.


----------



## Alain De Vos (May 16, 2022)

I have tried to make these kind of graphs but always failed.
Is there a tutor for making these graphs Graham ?
Is there a "minimum kernel version ?"


----------



## sko (May 16, 2022)

Brendan Greggs website and his various articles and blog posts on FlameGraph are probably the best and most thorough starting points you can get.
I was loosely following his articles and presentations about the topic, but never found the time to really dig into it myself.


----------



## grahamperrin@ (May 17, 2022)

grahamperrin said:


> For flame graphs of this type, I'm not sure how much represents the time taken by what's in the ESP (after copying what's required to the partition).



Without reference to any graph, here's a comment from the developer: <https://old.reddit.com/r/freebsd/comments/s67ymv/-/i8utndz/?context=1>



Alain De Vos said:


> … Is there a tutor for making these graphs Graham ?
> Is there a "minimum kernel version ?"





grahamperrin said:


> TSLOG boot profiling : freebsd:



– Alain, uppermost there's a link to the wiki, which links to Colin's repo with mkflame.sh. The wiki includes a graph for 11.1-RELEASE, and so on.


----------



## tux2bsd (May 17, 2022)

grahamperrin said:


> From TSLOG boot profiling : freebsd:​
> View attachment 13927



Disable cups unless you need it, it's a fair chunk.  Start it later if you want, printing is usually infrequent.  (e.g. sudo or doas start cups after your [desktop] session is interactive i.e. start optional stuff late)


----------



## grahamperrin@ (May 17, 2022)

tux2bsd said:


> Disable cups unless you need it, it's a fair chunk. …



Thanks, I'm not bothered by the time taken.


----------



## mer (May 17, 2022)

Reading and rereading this thread, the only thing that comes to mind is:
Do none of you drink coffee?

Power on/boot a system, walk away grab a cup of coffee, drink it and you've taken 2-5 minutes.  Walk back to your system and it should be up for at least 3 minutes.  That is my experience at least.

Some systems have issues (not problems) when probing USB devices.  There are loader options you can set to not wait for these to finish, so the only thing is you may need to wait starting X for the mouse to get mounted.

Init systems have always been:
What depends on what, what can we start "now" what can we start "later".
All of that relies on a person correctly elucidating dependencies.   
OpenRC is not bad, way better than systemd, but my opinion, not really better than current FreeBSD init.

Suspend/hibernate is a different story;  why?  a lot of success depends on the hardware and the drivers talking to it.  Does the HW need to be "cold started" or can it be "warm started"?  That is unclear for a lot of HW devices.

Now for me personally, I don't really care because I don't shut my systems down or suspend/hibernate.  That makes "boot time" uninteresting to me.


----------



## Alain De Vos (May 17, 2022)

I don't care about booting time. But it would nice if measuring it would be part of the handbook.


----------



## mer (May 17, 2022)

I think one would need to define what is being timed.  
Power off to login prompt?  Big difference between console and X/graphical login.
Not sure what is "right" but simply it needs to be defined.


----------



## meine (May 17, 2022)

STORP said:


> The FreeBSD init system just takes a looooong time and although I am just running it in a VM right now when I actually use FreeBSD for desktop usage the bootup time might be a problem. When windows took too long to boot I would end up doing nothing productive. It breaks the flow. I am gonna use the OpenRC or Runit systems but I was just wondering is it possible to make the time even shorter like clear Linux? (I know this goes against the forum rules but I am just curious)



I noticed a probably important difference between booting a Windows 10 box (boss laptop HP EliteBook) and my FreeBSD v 13 (Lenovo ThinkCentre with SSD). Regulary I push the power buttons the same time, just to see who wins the race.

W10 is rather fast in getting to the user login, but after that it takes ages to get an fully functional GUI. The system doesn't give much feedback, but when starting a program often there comes a dialog that the system is busy with things needed for that package. Sometimes the wi-fi connects automatically, sometimes not (it is set not to connect automatically)

FreeBSD boots fast. Searching the wi-fi SSID and syncing time takes some time. Then I'll have to log in to my account and start x (with an alias called 'x'). That happens just as fast as you can type your accountname, password and hitting 'x <enter>'.

In the end the FreeBSD box is always ready for use first !

It seems that FreeBSD is handling all basics before letting anyone login. That means that the system is ready for use. W10 presents itself, but isn't ready for use when showing something graphical. 'Boot philosophy' -- tech optimized (FBSD) or showcase (W10)...

If you want I can stopwatch both and post it here together with hardware specs.

[20220531 edited] I 'stopwatched' both my FreeBSD 13.1 box and W10 boss-box. Below are the times in this speed test:

From Power-button (HW) to login screen primary user
W10 = 35 secs
FBSD = 42 secs

From login (uname, passwd, <Enter>) to a working environment
W10 = 3m 02 secs, fully functional (loading system resources after starting a program) = 4m 11 secs
FBSD = < 1 sec, immediately usable TTY, after typing 'x' (alias for `startx` the same, immediately operational GUI (x11-wm/cwm)

*Conclusion*
The W10 operating system is very good in window dressing -- suggesting it is ready to use, but needs lots of stuff after putting app(e)aling graphics on the screen.

FreeBSD is honestly straigtforward -- when a user is able to login, the box immediately is fully operational, both in TTY as in GUI

Declaration of possible conflicting interests: a started CWM session has nothing to show and therefore is always faster. My preoccupied and verified attitude towards W10 needs no further explanation on this targeted forum.

Disclaimer: measuring time is accurate-ish, because hitting three hardware start buttons on three different devices in the same time is humanly challenging.


----------



## grahamperrin@ (May 18, 2022)

meine said:


> I noticed a probably important difference between booting a Windows 10 box (boss laptop HP EliteBook) and …



HP EliteBook 8570p here. Colleagues previously used this model with Windows 10. I use persistent L2ARC (two thumb drives), which greatly improves things; there's nothing comparable for Windows for this hardware. Comparing would be like apples versus oranges.


----------



## bakul (May 18, 2022)

It takes roughly 18 seconds from invoking bhyve (to run a guest FreeBSD13.1-STABLE running GENERIC) to the first ping and another 7 seconds to logging in via ssh. This includes getting a host address via DHCP. Far too slow


----------



## mer (May 18, 2022)

bakul, that's barely enough time to pour a cup of coffee, forget about drinking it.


----------



## tux2bsd (May 18, 2022)

Some times fast boot is convenient, you may be happy getting a coffee but other people are happy not getting a coffee...

It interests OS devs and they get to do it without malignant systemd cancer.


----------



## mer (May 18, 2022)

I'm not saying a fast boot is a bad thing, but one needs to define "what is being timed".  Most Windows systems boot to a graphical login pretty quickly even with fast boot disabled, but if you log in right away the system is not completely done booting.  I consider "booted" meaning "fully booted", at least to a console, basically what used to be called run level 3 (networking, multiuser).  So all network devices configured and up, all devices in use configured and up.

Heck a fast boot time is not a bad thing for servers because in a failure case, the services are up and running quicker.

If you look at what happens during the init phases, there are some tasks that don't depend on anything and nothing depends on them, other tasks that don't depend on anything but others require them, others that both depend and are required on.  Much like things in the ports tree.
The sequencing of the tasks is the biggest driver (my opinion) of the boot time.  One can do the "thundering herd" where everything starts at the same time, perhaps competing with each other, perhaps completing incorrectly and needing to be restarted (think WiFi and networking).  Or one can try and add the dependencies so tasks are only started when all their prerequisites are satisfied;  this gives each task a greater chance of succeeding the first time.
The thundering herd often "boots faster" but the system may not be completly booted to a usable state.
The sequencing (in theory) always boots completely and correctly, but may be slower.  Of course as soon as one "fixes" something they may break the sequencing.

I think instead of figuring out the last seconds of a boot, proper suspend/resume would be a more interesting path (my opinion).  But there is a lot there that depends on the specific hardware, how it is actually wired and do the device drivers correctly initialize the device on resume (not a trivial task).

Sorry for the too many words.


----------



## sko (May 18, 2022)

mer said:


> Heck a fast boot time is not a bad thing for servers because in a failure case, the services are up and running quicker.



2/3 and more of the boot time on a server are spent for POST of the system and various controllers. The OS is by far the smallest factor when (re)booting a server. And I ABSOLUTELY don't care if the OS takes 10 or even 30 seconds longer to boot, as long as it comes up reliable and predictable/reproducible.


----------



## mer (May 18, 2022)

sko said:


> And I ABSOLUTELY don't care if the OS takes 10 or even 30 seconds longer to boot, as long as it comes up reliable and predictable/reproducible.


This is my position also, even for my home desktops, but I acknowledge others have different needs.


----------



## SirDice (May 18, 2022)

If your 'mission critical' service is running on a single server, you're doing it wrong. Who cares how long it takes for a single node to boot if I have several other nodes running and serving requests?


----------



## tux2bsd (May 19, 2022)

sko said:


> And I ABSOLUTELY don't care if the OS takes 10 or even 30 seconds longer to boot, as long as it comes up reliable and predictable/reproducible.


An OS booting and being reliable and predictable/reproducible, that goes without saying as it's the entire point of an OS booting...

Prove you truly don't care add some sleep in somewhere, random between 0 and 3600 seconds.  You might have time for more than one coffee.


----------



## schorsch_76 (May 27, 2022)

Today i profiled my 13.1-RELEASE on bare metal using the method described here:


			BootTime - FreeBSD Wiki
		


What is loader.efi doing so long?


----------



## VladiBG (May 27, 2022)

It wait 10sec to show you the menu.


----------



## jbo (May 27, 2022)

VladiBG said:


> It wait 10sec to show you the menu.


To add to that: That duration can be adjusted via autoboot_delay="10" in boot/loader.conf where the unit is seconds.

Personally, I find myself leaving it to the standard 10 seconds on servers but setting it to something more sensible like 2 seconds on desktops.


----------



## SirDice (May 27, 2022)

You can turn it off completely if you want. But it's going to make booting to single user mode, or selecting a different kernel or BE a little more challenging.


----------



## jbo (May 27, 2022)

SirDice said:


> You can turn it off completely if you want. But it's going to make booting to single user mode, or selecting a different kernel or BE a little more challenging.


How would one go about doing that if the boot menu is disabled/skipped entirely? Does it involve booting a Live system, mounting the FS and setting some magic flag somewhere before booting? 

I'm asking for a friend.


----------



## schorsch_76 (May 27, 2022)

> kern.geom.label.disk_ident.enable="0"
> kern.geom.label.gptid.enable="0"
> cryptodev_load="YES"
> zfs_load="YES"
> ...


This is my loader.conf. I changed the UEFI bootorder to start directly freebsd. As you can see: The loader.efi is still the biggest chunk. At least 50% of the time is the loader.efi with an autoboot_delay of 1.


----------



## SirDice (May 27, 2022)

jbodenmann said:


> How would one go about doing that if the boot menu is disabled/skipped entirely? Does it involve booting a Live system, mounting the FS and setting some magic flag somewhere before booting?


nextboot(8) is quite useful in that case.


----------



## sko (May 27, 2022)

jbodenmann said:


> How would one go about doing that if the boot menu is disabled/skipped entirely? Does it involve booting a Live system, mounting the FS and setting some magic flag somewhere before booting?
> 
> I'm asking for a friend.



that 'magic flag' is "autoboot_delay" in loader.conf(5) which can be also overridden IIRC by nextboot(8)


----------



## mer (May 27, 2022)

schorsch_76
What if you add the following to your loader.conf?  This means "don't wait for usb probing to finish before continuing the boot".
It does mean that sometimes you may not have things like a mouse cursor immediately but if you are not booting/mounting external USB
drives to boot from, it's reasonably safe or at least is has been for me.

hw.usb.no_boot_wait="1"

EDIT:
Not sure but are you booting from encrypted devices?  That could have an impact as they are decrypted which takes memory and time.


----------



## VladiBG (May 27, 2022)

I'm assuming that loader.efi is searching for zfs to import the zroot pool and then to load the /boot.config and kernel and if you have multiple disks/partitions this can take more time.


----------



## schorsch_76 (May 27, 2022)

mer
I boot from an unencrypted Samsung 870 Evo 1TB SSD device (FreeBSD), 860 Evo 1TB (Linux) and 850 Evo 500GB (Windows) on the machine.

This is my Layout of all online disks:


Spoiler





```
gpart show
=>        40  1953525088  ada0  GPT  (932G)
          40      409600     1  efi  (200M)
      409640  1953115488     2  freebsd-zfs  (931G)

=>        34  1953525101  ada1  GPT  (932G)
          34        2014        - free -  (1.0M)
        2048      204800     1  efi  (100M)
      206848     4194304     2  linux-data  (2.0G)
     4401152  1949123983     3  !ca7d7ccb-63ed-4c53-861c-1742536059cc  (929G) (encrypted luks)

=>       34  976773101  ada2  GPT  (466G)
         34       2014        - free -  (1.0M)
       2048    4194304     1  efi  (2.0G)
    4196352  962221696     2  ms-basic-data  (459G)
  966418048        384        - free -  (192K)
  966418432    1187840     3  ms-recovery  (580M)
  967606272       4096        - free -  (2.0M)
  967610368    1054720     4  ms-recovery  (515M)
  968665088       2048        - free -  (1.0M)
  968667136    1814528     5  ms-recovery  (886M)
  970481664    4194304     6  linux-data  (2.0G)
  974675968     204800     7  efi  (100M)
  974880768      32768     8  ms-reserved  (16M)
  974913536     204800     9  efi  (100M)
  975118336    1654799        - free -  (808M)
```




On the 500G SSD i have an Efi Partition with starts grub. For this freebsd boot time check, I changed the UEFI default boot to the loader.efi from the 1TB FreeBSD SSD.

The system is also reasonable fast i guess. On Linux (including the unlock of the device) its up in under 30 secs. Also Win10 starts very fast.

This is the system info:


Spoiler





```
inxi -F
System:
  Host: Hammerhead Kernel: FreeBSD 13.1-RELEASE amd64 bits: 64
    Console: pty pts/1 OS: FreeBSD 13.1-RELEASE
Machine:
  Type: Desktop System: ASUS product: N/A v: N/A serial: N/A
  Mobo: ASUSTeK model: ROG STRIX B550-F GAMING v: Rev X.0x
    serial: xxxxxxxxxxxxxxxxx UEFI: American Megatrends v: 2201 rev: 5.17
    date: 04/07/2021
CPU:
  Info: 16-core model: AMD Ryzen 7 3700X bits: 64 type: MCP MCM cache:
    L2: 4 MiB note: check
  Speed (MHz): 3593 min/max: N/A cores: No OS support for core speeds.
Graphics:
  Device-1: AMD Curacao PRO [Radeon R7 370 / R9 270/370 OEM] driver: vgapci
  Device-2: HD Pro Webcam C920 type: USB driver: N/A
  Display: server: X.Org 1.20.14 driver: loaded: modesetting unloaded: vesa
    resolution: 1: 1680x1050~60Hz 2: 1680x1050~60Hz
  OpenGL: renderer: AMD Radeon HD 8800 Series (PITCAIRN DRM 3.35.0
    13.1-RELEASE LLVM 13.0.1)
    v: 4.6 Mesa 21.3.8
Audio:
  Device-1: AMD Oland/Hainan/Cape Verde/Pitcairn HDMI Audio [Radeon HD 7000
  Series]
    driver: none
  Device-2: AMD Starship/Matisse HD Audio driver: hdac
  Sound Server-1: OSS v: 2009061500 running: yes
  Sound Server-2: PulseAudio v: 14.2 running: yes
Network:
  Device-1: Intel I350 Gigabit Network driver: igb
  IF: igb0 state: no carrier speed: N/A duplex: N/A mac: a0:36:9f:00:82:3e
  Device-2: Intel I350 Gigabit Network driver: igb
  IF: igb1 state: active speed: 1000baseT duplex: full-duplex
    mac: a0:36:9f:00:82:3f
Bluetooth:
  Device-1: CSR8510 A10 type: USB driver: N/A
  Report: No OS support. Is a comparable bluetooth tool available?
RAID:
  Device-1: zroot type: zfs status: ONLINE level: linear raw: size: 928 GiB
    free: 866 GiB zfs-fs: size: 899.25 GiB free: 807.91 GiB
  Components: Online: 1:
Drives:
  Local Storage: total: raw: 2.27 TiB usable: 4.06 TiB used: 58.29 GiB (1.4%)
  ID-1: /dev/ada0 vendor: Samsung model: SSD 870 EVO 1TB SVT02B6Q
    size: 931.51 GiB scheme: GPT
  ID-2: /dev/ada1 vendor: Samsung model: SSD 860 EVO 1TB RVT03B6Q
    size: 931.51 GiB scheme: GPT
  ID-3: /dev/ada2 vendor: Samsung model: SSD 850 EVO 500GB EMT03B6Q
    size: 465.76 GiB scheme: GPT
Partition:
  ID-1: / size: 821.9 GiB used: 13.99 GiB (1.7%) fs: zfs
    logical: zroot/ROOT/default
  ID-2: /tmp size: 809.32 GiB used: 1.41 GiB (0.2%) fs: zfs
    logical: zroot/tmp
  ID-3: /usr/home size: 815.44 GiB used: 7.53 GiB (0.9%) fs: zfs
    logical: zroot/usr/home
  ID-4: /var/log size: 807.91 GiB used: 965 KiB (0.0%) fs: zfs
    logical: zroot/var/log
  ID-5: /var/tmp size: 807.91 GiB used: 24 KiB (0.0%) fs: zfs
    logical: zroot/var/tmp
Swap:
  ID-1: swap-1 type: partition size: 32 GiB used: 0 KiB (0.0%)
    dev: /dev/zvol/zroot/swap
Sensors:
  System Temperatures: cpu: 42.1 C mobo: N/A
  Fan Speeds (RPM): N/A
Info:
  Processes: 118 Uptime: 53m Memory: 31.88 GiB used: 4.65 GiB (14.6%)
  Shell: Bash inxi: 3.3.11
```


----------



## VladiBG (May 28, 2022)

Can you test with only one disk attached?


----------



## schorsch_76 (May 28, 2022)

This is the Result with just the FreeBSD SSD attached: 70 sec instead of 72 sec.

And this are the 20 top long lasting functions.


Spoiler





```
./stackcollapse-tslog.pl < ts.log | sh supercollapse.sh | head -n20
67470759036 interp_init
57309920784 BIOS
22762042776 317 /sbin/rtsol
14855929380 autoload_font
12667152157 _sleep
12463797132 kvprintf
11232296208 cons_probe
7637027724 1180 sleep
4526271000 hammer_time;DELAY
3826980900 343 sleep
3822667308 344 sleep
3668996087 kernel
2970714456 179 kldload
2337738912 472 /etc/rc.d/netif
2087294112 VFS_MOUNT zfs
1693931652 twiddle
1623312576 DEVICE_ATTACH atkbd;DELAY
1425531960 1294 /etc/rc.d/zfsd
996503509 1030 /etc/rc.d/devmatch
987635592 efipart_readwrite
```


----------



## schorsch_76 (Jun 3, 2022)

I updated my BIOS/UEFI and my boottime got down to 61 sec, The time was "cut off" the BIOS part.

Setting in loader.conf the console to "console=nullconsole" the boottime gets down to 48 sec.

tslog1: new BIOS
tslog2: new BIOS + nullconsole


----------



## l2f (Jun 4, 2022)

STORP said:


> The FreeBSD init system just takes a looooong time and although I am just running it in a VM right now when I actually use FreeBSD for desktop usage the bootup time might be a problem. When windows took too long to boot I would end up doing nothing productive. It breaks the flow. I am gonna use the OpenRC or Runit systems but I was just wondering is it possible to make the time even shorter like clear Linux? (I know this goes against the forum rules but I am just curious)


Do not compare FreeBSD boot time and Ubuntu/Debian (distro based on Linux and systemd) because when you hit the login prompt on systemd (Ubuntu/Debian) nothing is started, but on FreeBSD everything is up and running, ready to work.  Anyway your compute will wait after you 90% of the time ...


----------



## stratact (Jun 4, 2022)

l2f said:


> Do not compare FreeBSD boot time and Ubuntu/Debian (distro based on Linux and systemd) [...]


Huh, yeah, you just reminded me of something. What was that?





Oh yeah! That! 



STORP said:


> I don't see why I would keep my laptop open 24/7 that's power consumption gone to waste


And you don't see anything wrong with the above in regards to said "power consumption"? `(1min 5s / no limit)`


----------



## tux2bsd (Jun 4, 2022)

l2f said:


> Do not compare FreeBSD boot time and Ubuntu/Debian (distro based on Linux and systemd) because when you hit the login prompt on systemd (Ubuntu/Debian) nothing is started, but on FreeBSD everything is up and running, ready to work.  Anyway your compute will wait after you 90% of the time ...


No, they both start things up as they boot.  "Linux" systemd aggressively parallelizes start up routines, while giving the OS cancer at the same time (startup was better, not faster, before systemd).  Yes, ironically systemd can prevent booting. FreeBSD is more sequential in its start up routines.

As for stratact, focus on FreeBSD in FreeBSD forums. There are too many users here that think it is a competition between "Linux" and FreeBSD.


----------

