# FOSDEM 2013: The future of X.org on non-Linux systems



## jrm@ (Feb 1, 2013)

The future of X.org on non-Linux systems

I hope there will be videos.


----------



## jrm@ (Feb 7, 2013)

Slides are up.


----------



## YZMSQ (Feb 13, 2013)

Are we lagging behind the Linux guys so much when it comes to X?


----------



## Crivens (Feb 13, 2013)

YZMSQ said:
			
		

> Are we lagging behind the Linux guys so much when it comes to X?



That depends on the one doing the viewing.

"You are lagging behind!" shouts the guy running into the mine field/off the cliff.

And yes, I consider pushing code which is perfectly at home in user space into the kernel doing the first, and binding the GFX stack to one kernel on purpose to be the second.


----------



## wblock@ (Feb 13, 2013)

YZMSQ said:
			
		

> Are we lagging behind the Linux guys so much when it comes to X?



Whether intentional or not, the KMS stuff is written for the Linux kernel.  FreeBSD is emphatically not a Linux kernel, and so it must all be ported.  There are not a lot of people capable of doing that.  We've been fortunate that the Foundation funded porting of the Intel KMS drivers by kib@.


----------



## Crivens (Feb 13, 2013)

wblock@ said:
			
		

> Whether intentional or not, the KMS stuff is written for the Linux kernel.  FreeBSD is emphatically not a Linux kernel, and so it must all be ported.  There are not a lot of people capable of doing that.  We've been fortunate that the Foundation funded porting of the Intel KMS drivers by kib@.


Is the KMS code really portable at all? Apart from the differences in the kernel architecture/implementation, there is the licence issue which would make reusing even some header files a problem, right?


----------



## wblock@ (Feb 13, 2013)

Sorry, I have not heard about license issues.  As far as portable... well, no, it's kind of written for Linux.


----------



## Crivens (Feb 13, 2013)

What I meant is, the implementation inside the Linux kernel will be GPL2. Trying to put code that is derived from that into the FreeBSD kernel source tree would, I suppose and hope, draw some attention. Or am I completely not getting something here?


----------



## kpa (Feb 13, 2013)

You're missing the simple fact that the internals of Linux kernel and FreeBSD kernel have almost nothing in common. The code that implements the KMS system in Linux is 100% Linux specific because it does not have to be portable to other systems. It only needs to implement an agreed API for the X windowing system. Therefor it would be useless as it is for the FreeBSD implementation of KMS.


----------



## Crivens (Feb 13, 2013)

I don't miss these simple facts, I think.
The code is unportable, but what I was aiming at is that even interface header files are a no-no to base your work on. That's what happens when I try to shorten messages before posting :stud

Clean room implementation, for all and everything. And BTW, what will happen to those who do not use intel based GFX hardware once mode setting is dropped from XOrg?


----------



## Anonymous (Feb 13, 2013)

Obviously nothing changed since 1997. Here comes a quote of a post on comp.sys.next.advocacy from Mike Paquette.



> From: mpa...@wco.com (Mike Paquette)
> Subject: Re: X11 Support in Rhapsody (X.  No hardware is safe.)
> Date: 1997/04/02
> Message-ID: <5hua4p$20e@mpaque.mpaque>#1/1
> ...




There is another post on Slashdot in 2003, that seems to come from him, explaining "Why Apple didn't use X for the window system", and it seems that Mike Paquette designed/wrote a good part of Quartz, i.e. the X replacement on Mac OS X.

And another 10 years later,  "The future of X ...". This is a joke, isn't it? Let's add it to Mike's list. Wait, I have another one to add, "X. Back to the past".

Seriously, are there any good reasons, why the *BSD should spend more money on keeping on track with this brain dead technology? Freeze it at a usable stage. Start fund raising and allocate resources on development of a replacement - perhaps along a somewhat modernized path of Quartz - Z Project?

A rhyme in German (from Erich KÃ¤stner): "Es gibt nichts Gutes, auÃŸer man tut es."  in english more or less: "There is no good, unless you do it."


----------



## YZMSQ (Feb 14, 2013)

rolfheinrich said:
			
		

> Obviously nothing changed since 1997. Here come a quote of a post on comp.sys.next.advocacy from Mike Paquette.
> 
> 
> 
> ...


Well, I think you're almost right. However, as wblock@ figured out, "There are not a lot of people capable of doing that", when compared with Linux, our community is rather smaller. Second, if we really implement a very-perfect alternative to your called "brain dead technology", then afterwards we should port all the applications around the world which are only designed for X to our "replacement", that's, I've got to admit, a task requires much time and effort.


----------



## j_m (Feb 25, 2013)

I think using X is not bad.


----------



## throAU (Feb 25, 2013)

Now I am not a graphics developer, but surely the way X "works" is just wrong?

Surely the "proper" way to do this would be to pull GPU drivers into the kernel proper (like every other driver on the system - be it a module or whatever) and provide a standard API (like OpenGL) for whatever Windowing system (or app) to use?  Ditto for found, input devices, etc.  Sort of like (shudder) DirectX?  Or Quartz.

Provide an X compatibility layer, sure (like any of the myriad of X11 servers for Windows or OS X) - but installing a full windowing system to get a GPU driver just seems broken to me.  All the low level driver skullduggery should not be done by an application.

Yes, I'm aware this would require wholesale changes to the way free desktops and the OS work.... and require a committment by kernel people to provide a well defined, stable API. This is a utopia type thing 



The longer we persist with X, IMHO, the further behind the free operating systems will lag.


Sure X11 can be hacked to do what most people want.  But fixing X without providing some sort of stable foundation underneath it, where much of it's functionality should actually be is IMHO traveling further down the wrong path.



			
				YZMSQ said:
			
		

> then afterwards we should port all the applications around the world which are only designed for X to our "replacement", that's, I've got to admit, a task requires much time and effort.



No, you simply write an X11 compatibility shim (or hell, even port one of the many free X11 servers for Windows or OS X) to run X11 apps on top of the replacement platform.


I'm sure the core devs will shudder at the thought of pulling OpenGL into the base system (should bring in OpenCL and OpenAL too for that matter if possible), and having GPU drivers in the kernel either directly or via a stable ABI.

But, this is 2013, not 1993.  Graphics support is kind of a big thing these days.


----------



## Crivens (Feb 25, 2013)

Uh oh. You are wearing asbestos underwear, right? Because some kernel devs will likely want to toast some part of your anatomy.



			
				throAU said:
			
		

> Surely the "proper" way to do this would be to pull GPU drivers into the kernel proper (like every oher driver on the system - be it a module or whatever) and provide a standard API (like OpenGL) for whatever Windowing system (or app) to use?  Ditto for found, input devices, etc.  Sort of like (shudder) DirectX?  Or Quartz.


The proper way would be to add these drivers to the kernel/system source tree but _keep them out of kernel address space at runtime at any cost_.
Check out Minix3 for examples of that, really awesome what can be done these days. Crash your ethernet driver ten times a second? No problem, the downloads get a bit slower, but you would not notice it. Tanenbaum stated that he would gladly sacrifice half the performance of modern hardware only to be sure the thing cannot curl up and core dump on him.


			
				throAU said:
			
		

> Provide an X compatibility layer, sure (like any of the myriad of X11 servers for Windows or OS X) - but installing a full windowing system to get a GPU driver just seems broken to me.  All the low level driver skullduggery should not be done by an application.


This is right, X11 should not bother with it. That should be part of other parts, link libs or shared libs, device drivers, whatever. But keep it out of the kernel space, please.
And no, performance does not provide the silver bullet argument.


			
				throAU said:
			
		

> Yes, I'm aware this would require wholesale changes to the way free desktops and the OS work.... and require a committment by kernel people to provide a well defined, stable API. This is a utopia type thing


The kernel provides mmap of the system address space, any user mode driver may (sufficient credentials provided) map the hardware into his space and do what it wants.


			
				throAU said:
			
		

> No, you simply write an X11 compatibility shim (or hell, even port one of the many free X11 servers for Windows or OS X) to run X11 apps on top of the replacement platform.


Now we only need to specify that platform, some layer between kernel and X.
I once thought about porting f.e. the Haiku desktop to *BSD, maybe that would work out as expected?


			
				throAU said:
			
		

> I'm sure the core devs will shudder at the thought of pulling OpenGL into the base system (should bring in OpenCL and OpenAL too for that matter if possible), and having GPU drivers in the kernel either directly or via a stable ABI.


Do not bring things into the kernel which do not belong there. IMHO file systems also do not really belong there.


			
				throAU said:
			
		

> But, this is 2013, not 1993.  Graphics support is kind of a big thing these days.


Stable systems are also a big thing, yes? The more stuff you push into the kernel the more bugs will end up where they surely do not belong.

I'm all in for a replacement of X11, or better to cut it into layers and keep the "old" X11 as the topmost but optional layer. Pushing mode setting and whatnot into kernel space makes my toenails curl up, but X11 is getting a bit bloated for sure. Pushing that bloat into kernel space is not going to solve it. There need to be other ways.


----------



## Anonymous (Feb 25, 2013)

Crivens said:
			
		

> ...
> Now we only need to specify that platform, some layer between kernel and X. I once thought about porting f.e. the Haiku desktop to *BSD, maybe that would work out as expected?



IMHO, the Simple DirectMedia Layer - SDL, got 90% of what is needed. People are discarding this option only because it does not support multi-windows. IMHO, at this level (equivalent of Apple's CoreGraphics/Quartz) this is not needed. Windows come at the layer above, and in order to have a decent one, it is anyway best to newly implement the Window Manager

FYI, there was a Y project using SDL at the graphics and user-IO layer. This must not be confused with "an April Fools' Day spoof submitted to the LaTeX-L mailing list on April 1, 1992; a copy of the article appears as the fourth message in an archive".


----------



## Crivens (Feb 25, 2013)

Ok, here some sublimal part of me did some typing, it needs to be translated as well.

"Now we only need to specify that platform, some layer between kernel and X."

Since I am not a native speaker, you need to be familiar with "einfach mal eben schnell" in german.
This would mean that "you simply, with a flick of your hand, do this" (closest translation coming to mind) do something that _seems_ so easy but where not only the devil is in the details but he brought all kinds of friends to that party.

SDL seems to be a good start for a layering, indeed. It makes much more sense to me, compared to push it all into kernel space. But time will tell...


----------



## Uniballer (Feb 25, 2013)

Crivens said:
			
		

> "einfach mal eben schnell"



In this context I would translate it as SMOP (Simple matter of Programming).

Believe me, your English is way better than my German.


----------



## throAU (Feb 26, 2013)

Crivens said:
			
		

> Uh oh. You are wearing asbestos underwear, right? Because some kernel devs will likely want to toast some part of your anatomy.



As I said, I'm not a developer - so if I'm wrong, I'm not going to take it personally.  I am an end user.



> The proper way would be to add these drivers to the kernel/system source tree but _keep them out of kernel address space at runtime at any cost_.



Sure.  I agree.  but currently, we have drivers for NICs, USB, sound, etc, etc all in the kernel.  What makes GPUs so different?  If we're doing user space drivers, surely they should be done for pretty much nearly everything?

I agree if you can do it in userspace, do it in userspace.  Windows vista onwards does this, and yes a crashed GPU does not crash the box.  Perhaps I mis-spoke with my comments to put it in the kernel - but if it's going to be in userspace it would certainly need some stable driver ABI?

Perhaps what I MEAN is more tightly integrated into the base system and treated as a first class citizen.  Not some non-standard afterthought that is reimplemented in different ways for everyone who decides to do anything graphics related.

At the moment the practice seems to be to dump every other driver into the kernel, but GPUs - well they're left to their own devices.




> This is right, X11 should not bother with it. That should be part of other parts, link libs or shared libs, device drivers, whatever. But keep it out of the kernel space, please.
> And no, performance does not provide the silver bullet argument.



Yeah, my point was not to do with performance so again perhaps I mis-spoke.  My concern is more to do with having first-class OS support for GPU hardware and drawing operations, like we do with network adapters and their use of IP, etc.  X is a bit of a hack and doesn't really do what anyone wants it to do these days.  IMHO, in 2013, support for drawing operations of some kind really should be an operating system level feature.



> I'm all in for a replacement of X11, or better to cut it into layers and keep the "old" X11 as the topmost but optional layer.



Yeah this is what I mean.  X11 should live on top of some standard library which handles the GPU at the OS level.




			
				rolfheinrich said:
			
		

> IMHO, the Simple DirectMedia Layer - SDL, got 90% of what is needed. People are discarding this option only because it does not support multi-windows. IMHO, at this level (equivalent of Apple's CoreGraphics/Quartz) this is not needed. Windows come at the layer above, and in order to have a decent one, it is anyway best to newly implement the Window Manager



Makes sense to me.  Also, there's heap of stuff already written for SDL, which would continue to work if it was installed as a lower level component below X11 or whatever other windowing system succeeds it.


Anwyay, this is all pie on the sky stuff.  It's how I wish things were.  No system I've ever used prior to Unix had such a mess for a graphics subsystem.

Every X11 window (hell, even the desktop itself) should just be an (accelerated, if appropriate driver is present) OpenGL surface that can be scaled, rendered transparent, rotated off screen, etc. by the OS as required.  How do we get there?


----------



## Crivens (Feb 26, 2013)

throAU said:
			
		

> Sure.  I agree.  but currently, we have drivers for NICs, USB, sound, etc, etc all in the kernel.  What makes GPUs so different?  If we're doing user space drivers, surely they should be done for pretty much nearly everything?


They should. Drivers in user space may produce a speed penalty, but the increase in stability is IMHO worth the cost. Each system is slowest when it has crashed.

GPUs can do some DMA on their own, texture buffers and such. These can be programmed from the user space by GL commands, OpenCL and what not. Other devices can do DMA as well, but they are programmed from kernel space only, it is not so easy to redirect the DMA from the SCSI subsystem, f.e., to overwrite the kernel memory. By accident or by intent, this is not to be allowed. That is why there are dedicated MMUs for these kind of hardware, which needs also to be handled by OS. That, and the update frequency of graphics drivers (a new chip each week, or what is it?) would flood kernel space with hastly hacked drivers.


			
				throAU said:
			
		

> I agree if you can do it in userspace, do it in userspace.  Windows vista onwards does this, and yes a crashed GPU does not crash the box.  Perhaps I mis-spoke with my comments to put it in the kernel - but if it's going to be in userspace it would certainly need some stable driver ABI?


Microsoft started the NT line with graphics drivers in user space and moved them into kernel space to enhance the user experience. That they moved it back should speak volumes.

What I had in mind with the drivers being in user space is this - you need to have the source in the same place. The ABI between kernel and drivers can change at any time, but the interface from the driver to app needs to be stable. On that level you need no stable ABI, but the driver code should not run in kernel context. But with KMS et al, the amount of code running in kernel space, and thus code which _needs to be trustworthy_ is growing.



			
				throAU said:
			
		

> Perhaps what I MEAN is more tightly integrated into the base system and treated as a first class citizen.  Not some non-standard afterthought that is reimplemented in different ways for everyone who decides to do anything graphics related.


Full Ack.


			
				throAU said:
			
		

> Yeah, my point was not to do with performance so again perhaps I mis-spoke.  My concern is more to do with having first-class OS support for GPU hardware and drawing operations, like we do with network adapters and their use of IP, etc.  X is a bit of a hack and doesn't really do what anyone wants it to do these days.  IMHO, in 2013, support for drawing operations of some kind really should be an operating system level feature.


The problem is, there are new chips each monday (so to speak). You do not have the manpower to cover it all, and thus you need some kind of graphics driver framework which is not chained into one kind of kernel. And once some part of that kernel changes where the driver is interfacing directly for some reason, you need to adapt the drivers. Early versions of X had one server for each hardware and gained access to that hardware by mmap()ing the system address space. Clean interface, worked on *BSD, Solaris, Linux, ...
Now with KMS you have to cater for different manufacturers and each new chip generation which behaves differently, this is going to absorb tons of manpower, but what is gained by this? We now have the state that you can not change resolution when running AMD/ATI hardware because KMS in FreeBSD is Intel-only.


			
				throAU said:
			
		

> Anwyay, this is all pie on the sky stuff.  It's how I wish things were.  No system I've ever used prior to Unix had such a mess for a graphics subsystem.


Like DOS? 
But I have to agree, X11 is kind of the saber-tooth rabbit here.


			
				throAU said:
			
		

> Every X11 window (hell, even the desktop itself) should just be an (accelerated, if appropriate driver is present) OpenGL surface that can be scaled, rendered transparent, rotated off screen, etc. by the OS as required.  How do we get there?


IMHO not by requiring the system to support each kind of possible hardware on it's own, that's what got us where we are. Using SDL is a good idea, I think. Enlightment may be a good start for what we want. But in the end, it comes down to Applications Applications Applications (ducks head due to thrown bikeshed).


----------



## adamk (Feb 26, 2013)

throAU said:
			
		

> Every X11 window (hell, even the desktop itself) should just be an (accelerated, if appropriate driver is present) OpenGL surface that can be scaled, rendered transparent, rotated off screen, etc. by the OS as required.  How do we get there?



This is where the developers are heading with Wayland.  

Unfortunately, this switch doesn't improve things for non-Linux systems as the driver requirements are only available on Linux (well, perhaps, the KMS-enabled intel drivers on FreeBSD could support Wayland...  Not really sure about that).

ADam


----------



## throAU (Feb 27, 2013)

Crivens said:
			
		

> Like DOS?
> But I have to agree, X11 is kind of the saber-tooth rabbit here.



Having written my own graphics library for DOS back in the day (inline ASM in Pascal ), at least DOS wasn't a mess because you were on your own.

Any mess was your own making 



			
				adamk said:
			
		

> This is where the developers are heading with Wayland.
> 
> Unfortunately, this switch doesn't improve things for non-Linux systems as the driver requirements are only available on Linux (well, perhaps, the KMS-enabled intel drivers on FreeBSD could support Wayland...  Not really sure about that).
> 
> ADam




Yup.

I've been semi-following wayland and believe it is the way forward.

Whether FreeBSD supports it or not will determine if it gets left behind on the desktop, IMHO.

If it does - so be it.  Every OS doesn't have to be all things to all people.  But I hope it isn't.


However, Wayland looks like it is MIT license, so it's just a matter of implementing the driver support or porting it?  (i.e., it's a technical/resource problem rather than a legal problem?).

In any case, I suspect GPU and GP-GPU support is going to be a "requirement" at some point in the near future for any proper server OS anyway.  Both to do stuff like provide an accelerated VDI platform, and use the GPU for number crunching for other workloads.


----------



## taz (Feb 27, 2013)

It would seem something is starting to happen regarding FreeBSD and Wayland.

http://www.phoronix.com/scan.php?page=news_item&px=MTMwMzE


----------



## throAU (Feb 28, 2013)

Good to hear.  Article is biased as hell though - typical moronix...


----------



## taz (Feb 28, 2013)

Yeah it's good news but I have been wondering something. It's obvious that Xorg is a mess and needs replacement: Windows, Apple (Quartz) and now Linux (with Wayland) are all implementing their own display servers. Why wouldn't FreeBSD do the same and instead of porting Wayland use it as a reference. Hypothetically, what would be necessary to get such a project started.


----------



## fonz (Feb 28, 2013)

taz said:
			
		

> It's obvious that Xorg is a mess and needs replacement


I don't remember exactly when, but at some point FreeBSD switched from XFree86 to X.org. I'd like to think that that decision wasn't taken lightly.


			
				taz said:
			
		

> Hypothetically, what would be necessary to get such a project started.


Sheer manpower goes a long way. But do keep in mind that a *complete* X11 suite is very large and complex, consisting of many components. We're not talking about a CS BSc or even MSc project here.


----------



## kpa (Feb 28, 2013)

fonz said:
			
		

> I don't remember exactly when, but at some point FreeBSD switched from XFree86 to X.org. I'd like to think that that decision wasn't taken lightly.
> 
> Sheer manpower goes a long way. But do keep in mind that a *complete* X11 suite is very large and complex, consisting of many components. We're not talking about a CS BSc or even MSc project here.



I would say that X11 as a whole is a mess and needs a replacement that offers the good stuff from X (network transparency etc.) but also offers a solid and platform for graphics, be it the standard desktop stuff or 3D graphics for games.


----------



## fonz (Feb 28, 2013)

kpa said:
			
		

> I would say that X11 as a whole is a mess and needs a replacement


Now _that_'s a statement that has been true for many years.


----------



## throAU (Mar 1, 2013)

fonz said:
			
		

> But do keep in mind that a *complete* X11 suite is very large and complex, consisting of many components. We're not talking about a CS BSc or even MSc project here.



Don't need a complete X11 suite.

There are plenty of X11 servers which can sit on top of whatever graphics subsystem you have (e.g., XQuartz - open source, could hack it up to have it render direct to SDL or whatever low level rendering engine on FreeBSD instead of Quartz on OS X).


edit:



			
				&quot said:
			
		

> SDL supports Linux, Windows, Windows CE, BeOS, MacOS, Mac OS X, FreeBSD, NetBSD, OpenBSD, BSD/OS, Solaris, IRIX, and QNX. The code contains support for AmigaOS, Dreamcast, Atari, AIX, OSF/Tru64, RISC OS, SymbianOS, and OS/2, but these are not officially supported.



Porting an X11 display server to SDL probably makes a lot of sense anyhow, given the above.

In fact, I'd say its a no brainer.

Treat SDL as a first-class library on FreeBSD, enable GPU drivers to plug into the base system to support SDL properly and we're well on the way...

So in terms of layering from bottom to top (something like)
kernel -> video driver -> openGL <- not sure? -> libSDL -> X11


edit:
Side effect of using SDL:  you could run X11 on top of aalib.  Terminal friendly!


----------



## fonz (Mar 1, 2013)

throAU said:
			
		

> Don't need a complete X11 suite.
> 
> There are plenty of X11 servers which can sit on top of whatever graphics subsystem you have


I was referring to a complete rewrite from scratch but sure, if you're able to use existing components that might make the job considerably easier.


----------



## Crivens (Mar 5, 2013)

Hello incompatible server Nr n+1.
Seems we are not alone with the worries, but let's see what is going to happen to them.


----------



## Carpetsmoker (Mar 5, 2013)

Crivens said:
			
		

> Hello incompatible server Nr n+1.
> Seems we are not alone with the worries, but let's see what is going to happen to them.



Why is it incompatible?

Quoted from the linked page:



> we want to provide a graphics stack that works across different platforms and driver models by limiting our assumptions to a bare minimum





> with our protocol- and platform-agnostic approach, we can make sure that we reach our goal of a consistent and beautiful user experience across platforms and device form factors





> The system should easily be adaptable and portable to different platforms and use-cases





> keeps cross-platform use-cases in mind.



In fact, the word `Linux' in mentioned only twice in the article, both in the introduction.

This, ofcourse, doesn't mean it's compatible, but they do seem to *care* about it, which is an excellent first step and the most you can reasonably expect.


----------



## Crivens (Mar 5, 2013)

There is some heat now between Mir and Wayland, which I can understand from what I have read so far. What I think is going to happen is that now there is another display server, and it will be different in some places.

For application development, this will mean that it is going to be harder to have some "write-once-run-anywhere" apps. Given the current direction of Canonical, which seems to be the tablet and phone. Canonical seems to have the same worries w.r.t. the Xorg direction, and should they succeed - great. If it is portable, or more portable than other solutions, even better. Here, the new focus of tablet/phone may be good, we may be able to participate in this project much easier than getting KMS/GEM/... for all different GPUs FreeBSD can use.
And maybe this was the reason for them not to participate too much in Wayland, which IIRC is also using some (for now) Linux-only APIs.

As I said, let's see what is going to happen. I have seen it just now, still reading up on it, but I'll hope for the best and prepare for the worst.


----------

