# FreeBSD re-design as Hybrid Kernel, it it possible?



## tuaris (Jul 5, 2011)

I've recently began to migrate all my server systems from Ubuntu Linux to FreeBSD.  I always liked FreeBSD more and unfortunately got caught up in all the noise Ubuntu was making back in 2007.  

I had a thought... With most modern OS's like NT, OSX, and Haiku having a micro/hybrid kernel design as opposed the old monolithic design that Linux and FreeBSD use... I was wondering how long FreeBSD can continue to stay relevant (let alone compete) with this approach.  Has anyone even considered the possibility of one day re-designing FreeBSD with a hybrid kernel?


----------



## UNIXgod (Jul 5, 2011)

I am not a kernel hacker. Would you mind explaining to me what a micro kernel is as well as its impact on what is currently in place. Is this a performance thing?


----------



## SirDice (Jul 5, 2011)

UNIXgod said:
			
		

> I am not a kernel hacker. Would you mind explaining to me what a micro kernel is as well as it's impact on what is currently in place.


It's a different design of a kernel. FreeBSD (and linux') has a monolithic kernel. This means it's one big chuck of code that contains everything, memory management, process management, drivers etc. A microkernel contains only the bare essentials, memory management, process management and IPC. Everything else, including drivers, is in user space. Both have their advantages and disadvantages.

A hybrid like QNX (OS-X' kernel) tries to take the best of both worlds. It contains a microkernel (the Mach kernel) with some monolithic kernel bits thrown into it (some parts of the FreeBSD kernel).

Torvalds and Tanenbaum had an interesting discussion about micro vs. monolithic.

http://oreilly.com/catalog/opensources/book/appa.html


----------



## vermaden (Jul 5, 2011)

@tuaris

It's just a matter of the amount of cash you want to put for the whole rewrite, of course it's possible, anything is possible with the appropriate amount of resources (cash for developer's time), but will it happen? I do not see it happening it the next 10 years, but who knows later.

Hybrid kernel is not that great, device drivers (the ones that cause most hangs/panics) are still in the kernel: http://en.wikipedia.org/wiki/Hybrid_kernel

The best choice is micro kernel. Check MINIX 3 which uses the BSD license; recently they incorporated *pkgsrc/pkgin* tandem into the system. They are porting NetBSD's userland also, It's closest to the BSD microkernel available currently.

The MINIX also got funds (about $3.3 USD) from EU for the development:
http://www.executivebrief.com/news/eu-funds-research-on-lightweight-stable-open-source-os/



			
				SirDice said:
			
		

> A hybrid like QNX (OS-X' kernel) tries to take the best of both worlds. It contains a microkernel (the Mach kernel) with some monolithic kernel bits thrown into it (some parts of the FreeBSD kernel).



You mislead something here mate, OSX uses XNU kernel which is indeed hybrid, but QNX is a operating system with real-time microkernel and has nothing to do with OSX.


----------



## ChalkBored (Jul 5, 2011)

Possible? Definitely.

I highly doubt it would matter at all in keeping FreeBSD relevant, though.


----------



## SirDice (Jul 5, 2011)

vermaden said:
			
		

> You mislead something here mate, OSX uses XNU kernel which is indeed hybrid, but QNX is a operating system with real-time microkernel and has nothing to do with OSX.


You're right, it's XNU. Damn those TLAs!


----------



## phoenix (Jul 5, 2011)

tuaris said:
			
		

> With most modern OS's like NT, OSX, and Haiku having a micro/hybrid kernel design



Windows NT is a monolithic kernel, just like all of the BSDs, Linux (including Android), HP-UX, Solaris, etc.

Don't know anything about Haiku, but believe it's also monolithic as it is a continuation of BeOS?

Out of your list of "modern OSes", only MacOS X is a hybrid, and just barely.

QNX is pretty much the only successful micro-kernel-based OS on the market today.

IOW, why do you think monolithic kernels are so bad, considering everyone and their dog uses one?


----------



## UNIXgod (Jul 5, 2011)

Been reading about GNU/Hurd. Sound interesting but I imagine there must be a reason it's still never fully made it out of testing.


----------



## kpedersen (Jul 6, 2011)

UNIXgod said:
			
		

> Been reading about GNU/Hurd. Sound interesting but I imagine there must be a reason it's still never fully made it out of testing.



I tend to believe that it is purely due to hardware / driver limitations.

This hurd project is quite cool,
http://www.archhurd.org/

But shows the above limitation when the only video driver available to it is x11-video-vesa

Which is probably one of the main reasons that Windows does so well, in that it has all drivers available to it (though generally in binary forms).

I.e what is the point in a micro-kernel... when I cannot use my nifty phone connectivity suite on it...?  [/troll]


----------



## gordon@ (Jul 6, 2011)

Hurd, operating system of the future. Always has been, always will be.


----------



## SirDice (Jul 6, 2011)

phoenix said:
			
		

> Windows NT is a monolithic kernel, just like all of the BSDs, Linux (including Android), HP-UX, Solaris, etc.


Nope. But it's not exactly a microkernel or a hybrid either.

http://en.wikipedia.org/wiki/Hybrid_kernel#NT_kernel


----------



## vermaden (Jul 6, 2011)

gordon@ said:
			
		

> Hurd, operating system of the future. Always has been, always will be.



Duke Nukem Forever has been just released, maybe some day ... ;p


----------



## Crivens (Jul 6, 2011)

vermaden said:
			
		

> Duke Nukem Forever has been just released, maybe some day ... ;p



Yes, it was rumored to be running on HURD. So they could not release it untill HURD was ready, but it seems they did some porting in the past. 

One point which I think is the most important point which leads to monolithic kernels instead of microkernels is that the design of drivers and subsystems is much easier. It also may be there is a speed penalty involved due to multiple context switches for most calls and the asyncronous working of the drivers which are now seperate processes. It is hard for many developers to get their mind around parallel processings (you can see this in any class about HDLs). Also, the announcement that you can not simply throw a pointer around into the subsystems scares many out of their reserve of clean underwear. This is no problem in a monolithic kernel where that can be done inside the kernel and due to the memory range splitting (3/1 f.e.) even between kernel and user process.

There is a nice video around "inside the MacOS X kernel" which goes into some details about that. Interesting to view.


----------



## phoenix (Jul 6, 2011)

SirDice said:
			
		

> Nope. But it's not exactly a microkernel or a hybrid either.
> 
> http://en.wikipedia.org/wiki/Hybrid_kernel#NT_kernel



Yeah, I really don't buy it, though.  The NT kernel is a monolithic (modular) kernel.  All the major "kernel" subsystems run in kernel space.  Just because it has a bunch of "emulation services" running in userland between the apps and the kernel doesn't make it a hybrid. All the IO, device drivers, video, filesystems, memory management, yadda yadda yadda is running in kernel space.

Next you're going to call Windows 95 a hybrid since it can run 8-bit DOS programs, 16-bit Win16 programs, and 32-bit Win32 programs. :r

Shoot, you may as well classify the FreeBSD kernel as a hybrid as well, since it does the same thing with the Linux subsystem, the IBCS2 (SCO) subsystem, the AOUT subsystem, and the native FreeBSD subsystem, etc.


----------



## randux (Jul 6, 2011)

vermaden said:
			
		

> The MINIX also got funds (about $3.3 USD) from EU for the development:
> http://www.executivebrief.com/news/eu-funds-research-on-lightweight-stable-open-source-os/



I don't know about you guys, but $3.30 will get me to write about half a line of code 

Was that supposed to be thousands, millions, etc? :e


----------



## DutchDaemon (Jul 6, 2011)

It's in the first line of that link ..


----------



## Zare (Jul 6, 2011)

Part of kernel code of Windows 2008 runs in ring1 (Hyper-V). Windows 7 NT kernel runs some stuff split between ring0 and ring3 (sound subsystem).

It would be more beneficial to keep the monolithic architecture, but segmented between ring0, 1 and 2. FreeBSD can already run parts of kernel services in ring3, like FUSE which provides user-mode filesystems implemented through kernel ring0 container subsystem.


----------



## sossego (Jul 6, 2011)

EU funds research on lightweight said:
			
		

> Minix started as a research project funded by Google during one of its Summer of Code programs. Minix 3 runs on computers with Pentium 386 or 486 processors and 16MB or RAM, and a minimum or 50MB disk space. Minix is licensed under the Berkeley Software Distribution (BSD) license.



Someone needs to do their homework.


----------



## wblock@ (Jul 7, 2011)

sossego said:
			
		

> Someone needs to do their homework.



Probably supposed to be "Minix *3* started as a research project funded by Google..."  They do mention that Minix started in 1987 earlier in the article.


----------



## vermaden (Jul 7, 2011)

randux said:
			
		

> I don't know about you guys, but $3.30 will get me to write about half a line of code
> 
> Was that supposed to be thousands, millions, etc? :e


My bad, it supposed to be $3.3 million


----------



## Crivens (Jul 7, 2011)

Zare said:
			
		

> Part of kernel code of Windows 2008 runs in ring1 (Hyper-V). Windows 7 NT kernel runs some stuff split between ring0 and ring3 (sound subsystem).
> 
> It would be more beneficial to keep the monolithic architecture, but segmented between ring0, 1 and 2. FreeBSD can already run parts of kernel services in ring3, like FUSE which provides user-mode filesystems implemented through kernel ring0 container subsystem.



Is this "ring" concept supported outside of x86/amd64? Last time I checked my PPC/MIPS/HPPA/Sparc/... docs I did not come across that concept. Nailing down the kernel on one architecture would be a Bad Idea IMHO.


----------



## SirDice (Jul 7, 2011)

Crivens said:
			
		

> Is this "ring" concept supported outside of x86/amd64? Last time I checked my PPC/MIPS/HPPA/Sparc/... docs I did not come across that concept.


They do, but it's not called rings but cpu modes. Most modern processors support this concept.

http://en.wikipedia.org/wiki/CPU_modes



> Nailing down the kernel on one architecture would be a Bad Idea IMHO.


That I agree with.


----------



## Crivens (Jul 7, 2011)

SirDice said:
			
		

> They do, but it's not called rings but cpu modes. Most modern processors support this concept.
> 
> http://en.wikipedia.org/wiki/CPU_modes


Sorry, I am also some kind of fossil. When I was young, CPUs could only afford two modes, when they were lucky 
The usage of user mode and system/supervisor/kernel/whatevermode is not the point I wanted to make. The point was that depending on more than these two modes being available is a problem. I am used to call it the user mode or kernel mode, not ring. 
But it would be possible to have more levels with a little trick (see here) using part of the PC as the ring number. So it seems to be possible to emulate more than one level without too much trouble, but when it comes to areas with this kind of stability reqirements, I am dead conservative.


----------



## SirDice (Jul 7, 2011)

Crivens said:
			
		

> The usage of user mode and system/supervisor/kernel/whatevermode is not the point I wanted to make. The point was that depending on more than these two modes being available is a problem. I am used to call it the user mode or kernel mode, not ring.


For this same reason most operating systems only use ring0 and ring3 on i386. Even Windows.


----------



## tingo (Jul 7, 2011)

My two points about this discussion:

 "hybrid kernel" is just market speak for "we wanted to use a microkernel but couldn't get it right, so we changed it"
 There is a reason why the only microkernel OS with a certain commercial success (QNX) didn't take off; it is past its heyday in the real world today (QNX was great when all machines had limited memory - 4, 8 or 64 MB and you could load only the drivers you needed for that particular server)
Oh, and I believe FreeBSD will change when it needs to (it has been doing so for a long time already). So far the changes have been good (i.e. concepts or ideas with real value) or needed (for some time at least - I'm thinking about HAL here).


----------



## UNIXgod (Jul 15, 2011)

gordon@ said:
			
		

> Hurd, operating system of the future. Always has been, always will be.





			
				vermaden said:
			
		

> Duke Nukem Forever has been just released, maybe some day ... ;p



hmm. look at that.

http://news.slashdot.org/story/11/07/14/2141229/Watch-Out-Linux-GNU-Hurd-Coming


----------



## vermaden (Jul 15, 2011)

UNIXgod said:
			
		

> hmm. look at that.
> 
> http://news.slashdot.org/story/11/07/14/2141229/Watch-Out-Linux-GNU-Hurd-Coming



Personlly ... I would prefer Debian/kMINIX instead of Debian/kHurd.

There is also ArchHurd ISOs available here: http://www.archhurd.org/


----------



## UNIXgod (Jul 15, 2011)

vermaden said:
			
		

> Personlly ... I would prefer Debian/kMINIX instead of Debian/kHurd.
> 
> There is also ArchHurd ISOs available here: http://www.archhurd.org/



New toys! Funny note. I did a google search for the keywords "Debian/kMINIX" and your post is $1.

I imagine both hurd and minix will have to play driver catch-up for awhile when they hit 'production ready' status.


----------



## ahavatar (Jul 18, 2011)

Personally I don't see how a microkernel based OS is better than a monolithic kernel based OS in practice. The distinctions are mostly academic to me at best.

The concern I have with any OS is if it supports hardware devices, it runs applications I need, etc. As long as it runs applications of my chioce well with the hardware I have or want to have, I'm fine. What concerned me about FreeBSD is a rumor like Gnome will be Linux only, that kind of stuff; not if it's micro-kernel based or not.


----------



## vermaden (Jul 18, 2011)

ahavatar said:
			
		

> Personally I don't see how a microkernel based OS is better than a monolithic kernel based OS in practice. The distinctions are mostly academic to me at best.


Microkernel have all drivers in the userspace, that means if ANY driver fail, the service with that driver can be restarted or disabled, with monolitic kernel you will see kernel panic ...


----------



## ahavatar (Jul 18, 2011)

vermaden said:
			
		

> Microkernel have all drivers in the userspace, that means if ANY driver fail, the service with that driver can be restarted or disabled, with monolitic kernel you will see kernel panic ...



Yes, but I can't remember the last time when I saw a kernel panic. And if a driver is broken, it is broken no matter if the OS is microkernel-based or not, isn't it? In this sense, I can't see the difference in practice. If I do some driver development, a microkenel might be better?


----------



## SirDice (Jul 18, 2011)

ahavatar said:
			
		

> Yes, but I can't remember the last time when I saw a kernel panic. And if a driver is broken, it is broken no matter if the OS is microkernel-based or not, isn't it? In this sense, I can't see the difference in practice.


The difference is a working system with a broken driver as opposed to a non-functional system because of a broken driver.


----------



## ahavatar (Jul 18, 2011)

SirDice said:
			
		

> The difference is a working system with a broken driver as opposed to a non-functional system because of a broken driver.



I don't buy that, for example, a broken disk controller driver makes your computer a working system on a microkernel-based OS? In practice, it is not.


----------



## SirDice (Jul 18, 2011)

ahavatar said:
			
		

> I don't buy that, for example, a broken disk controller driver makes your computer a working system on a microkernel-based OS? In practice, it is not.



The system would still work (i.e. not panic), it just can't boot because of the missing drives that are attached to that controller


----------



## vermaden (Jul 19, 2011)

ahavatar said:
			
		

> Yes, but I can't remember the last time when I saw a kernel panic.



I have had several panics on Lenovo ThinkPad X300 (about 5) and Dell Latitude E6400 (about 2 if I recall correctly).


----------



## kpedersen (Jul 21, 2011)

This is the goal* of JigglypuffBSD (http://code.google.com/p/jigglypuffbsd/)

*Seeing as no actual goals have been defined, lets just assume for now that it aims to redesign FreeBSD with a hybrid kernel...


----------



## ddobson (Feb 23, 2012)

*someone should create hybrid FreeBSD?*

*S*omebody (multiple developers) already has...

See historic fork from freebsd FreeBSD 4.0 "Dragonfly" 
- that could be the hybrid monolithic-microkernel architecture you seek.

I remain to be convinced that the merits of this hybrid are worth its drawbacks - process security/integrity/validation.

Of course to take things to an even greater extreme, an exokernel design would also become stuck as an R&D endeavour - too bogged down in the nitty-gritty - scaring off the why-wait (MTV generation) of developers.

Speed of coding and availability of features and security patches - considered by the majority (commercial or otherwise) to be more important (here and now) - than long-term security by design and stability.



			
				tuaris said:
			
		

> I've recently began to migrate all my server systems from Ubuntu Linux to FreeBSD.  I always liked FreeBSD more and unfortunately got caught up in all the noise Ubuntu was making back in 2007.
> 
> I had a thought... With most modern OS's like NT, OSX, and Haiku having a micro/hybrid kernel design as opposed the old monolithic design that Linux and FreeBSD use... I was wondering how long FreeBSD can continue to stay relevant (let alone compete) with this approach.  Has anyone even considered the possibility of one day re-designing FreeBSD with a hybrid kernel?


----------



## throAU (Feb 23, 2012)

Microkernel vs Monolithic has been done.  Monolithic kernels with loadable modules won.  No credible product based on a Microkernel has emerged.

As I understand it, originally NT was designed as a microkernel, but performance was horrible, hence with NT4 more and more stuff started getting moved into kernel space.


----------



## vermaden (Feb 23, 2012)

throAU said:
			
		

> No credible product based on a Microkernel has emerged.


The only successful ones I know are AMIGA Workbench and QNX.


----------



## fluca1978 (Feb 23, 2012)

I believe all of this is fascinating: from an academic point of view all this OS research is great, but from a practical point of view it does not make sense, at least to me. The only reason to take back the microkernel stuff is if the hardware can provide some kind of "message facilities" that makes running a microkernel easier than it was years ago. Otherwise I don't believe that all these years of experience did not make enough clear that monolithic kernels are the best from an usability, stability, coding, etc. point of view.

For my experience, microkernels are valuable only when aims to target user level applications: a good example in this field is OSGi, which provides a Java framework that can be thought as a microkernel for a set of Java modules. But of course, all the stuff, including the OSGi kernel, is already in userland.
With regard to OSs, I believe that a good quality of drivers and code review will give better results than a microkernel. Not to mention that having a system running even if buggy drivers are present could be much more dangerous than having a system panic.

This is my opinion.


----------



## throAU (Feb 24, 2012)

vermaden said:
			
		

> The only successful ones I know are AMIGA Workbench and QNX.



Am aware of those two.

Now, I say this as a former amiga user, and QNX fan... but...


Look what happened to them.


----------



## vermaden (Feb 24, 2012)

throAU said:
			
		

> Look what happened to them.



That depends what You want value most ... The technical superiority or good marketing.

Both AMIGA and QNX has had terrible marketing, look at Windows for example ...

Besides, QNX serves its role on airports and other mission critical places, they do not put 'Powered by QNX' stickers everywhere


----------



## fluca1978 (Feb 24, 2012)

I believe that microkernels could be a winning architecture for smart devices, where effectively you should have the system running even if a piece of software is buggy.


----------



## Crivens (Feb 24, 2012)

vermaden said:
			
		

> That depends what You want value most ... The technical superiority or good marketing.
> 
> Both AMIGA and QNX has had terrible marketing, look at Windows for example ...
> 
> Besides, QNX serves its role on airports and other mission critical places, they do not put 'Powered by QNX' stickers everywhere



These stickers would be in some highly annoying places. On your nav system, on your dashboard, on your phone/fondleslab/...
And in most other cases the only one ever getting close enough to see them is the guy doing some maintainance.

The marketing of C= was abysmal indeed. Not doing the SUN deal, f.e.
Still I have my A3000T with a build quality unreached in any case I have seen since then.
The OS was great, also. Some concepts are still around, automatic chaining of datatypes for example. And using the A2024 for coding, what an eye relief!
Virtual screen, where are you...


----------



## ring_zero (Apr 1, 2012)

I looked into running a microkernel once.  Then when I saw that the archhurd project was only available in i386, I gave up.  Give me the good 'ol monolithic BSD kernel any day.  Besides with loadable module support, and if you compile the kernel without the drivers you don't need, and most of the drives that you do need as modules, it is not that monolithic anymore.  The thing that got me had to do with server load testing, and Mach fell well below BSD in the benchmarks with even a moderate load.  As the load went up, Mach did even worse.  This is in no small part due to the validity checking that the microkernel HAS to do on every message that is passed to it from user space via the IPC.  

As far as rings are concerned, I cannot speak for the x86_64, but I would assume that it is similar, there are four rings implemented in the Intel x86 processor family, and they are stored internally as numbers 0-3.  The kernel runs in ring zero, and user space programs run in ring three.  I also cannot speak for the FreeBSD kernel, I am still learning how it works, but the Windows and Linux kernels only use rings zero and three.


----------



## fluca1978 (Apr 2, 2012)

ring_zero said:
			
		

> Besides with loadable module support, and if you compile the kernel without the drivers you don't need, and most of the drives that you do need as modules, it is not that monolithic anymore.



Monolithic kernels are nothing to do with loadable modules. It is not a matter of "where" the code is (i.e., into the kernel or loaded aside it), it is a matter of how such code communicate with the other parts. So monolithic means that all loaded modules can speak to each other as if they were developed and linked together, while in microkernel they have to send messages using a message service. Someone with more experience on this subject can comment on.


----------



## ring_zero (Apr 3, 2012)

fluca1978 said:
			
		

> Monolithic kernels are nothing to do with loadable modules. It is not a matter of "where" the code is (i.e., into the kernel or loaded aside it), it is a matter of how such code communicate with the other parts. So monolithic means that all loaded modules can speak to each other as if they were developed and linked together, while in microkernel they have to send messages using a message service. Someone with more experience on this subject can comment on.



My statement "not that monolithic anymore" was only meant as a sort of joke, i.e. the huge chunk of code that represents the monolithic kernel is not so huge when functionality is shifted into modules.  I will admit it was nonsensical, a monolithic kernel is so called due to the fact that the whole kernel runs in the same address space, modules included, not due to the size of the resulting binary when compiled.


----------

