# Real time processes



## ogogon (Jan 31, 2020)

Colleagues, tell me please, is it possible now in FreeBSD to get the ability to start processes in a more or less acceptable real time?

Grateful in advance,
Ogogon.


----------



## ralphbsz (Jan 31, 2020)

Do you mean you want a real-time operating system? Define "acceptable". What guarantees do you need? We need more detail.


----------



## ogogon (Jan 31, 2020)

ralphbsz said:


> Do you mean you want a real-time operating system? Define "acceptable". What guarantees do you need? We need more detail.


I have a CNC machine. CNC are have a control machine under Debian. There, a real-time subsystem is included in the kernel.
This Linux solutions use RTAI or the realtime kernel patch set - "Complete Preemption".

I want to understand whether it is possible on the FreeBSD platform to get realtime facilities similar to those listed.


----------



## ralphbsz (Jan 31, 2020)

Ah, I see. I knew a little bit about RTAI, but I don't know how well it actually works in practice. It tries to encapsulate the Linux kernel, and therefore remove the single biggest source of delays, which are hardware device drivers which are not pre-emptable. Actually, I have a funny story to tell about this: in the late 80s, IBM's salespeople told us that AIX version 2.0 was real-time capable, because they had made the whole kernel pre-emptable. We believed them, bought a handful of expensive RS-6000s, and tested it. We found horrible delays, of several ms, in context switch response times, and process interruptions. After much back and forth with IBM, the source was found: It was an interaction between the Ethernet drivers (which in those days were 10 MBit) and the TCP/IP stack, which could lock up the machine easily for a ms or two. After screaming at IBM, they sheepishly admitted that the real-time guarantee would only apply if you turned the network completely off. For a system that is based on a network of workstations, that seemed unrealistic.

Anyway: I know of no equivalent effort for FreeBSD; all I know is that processes can be scheduled at real-time priorities. But if you look at the documentation, you see that there is no way to guarantee that a process is not swapped out, nor can system calls be pre-empted. So it is unlikely that equivalent functionality exists.

The question is: On a modern system, is this really needed?


----------



## Nicola Mingotti (Jan 31, 2020)

i spent some time with the BeagleBone and Freebsd and Debian. Those system are far from real time. If you need something like that, and the capabilities of a ‘reguler’ OS, one way is to offload the hard real time parts to simpler machines, like Arduino.


----------



## ralphbsz (Jan 31, 2020)

Realtime is a complex question. There is super-hard realtime, with ironclad guarantees of sub-millisecond (sometimes microsecond) response times, and guaranteed performance (such as throughput). That's the realm of dedicated hardware. Things like Arduino is the amateur low end; vehicle control, FADEC, industrial control, medical devices, military hardware, and data acquisition is the more high-powered side of this. It uses a mix of full-custom software and dedicated real-time OSes. VXWorks used to be the 400-lbs gorilla in this market, but there are many competitors.

Then there is soft realtime, where response times are more in the millisecond range, and missed response is not catastrophic. In the last two decades, Linux-derived systems have made significant inroads in this application. The biggest one I know of used to be RTLinux. Architecturally it is an interesting hybrid, using a dedicated microkernel for the hard realtime part, and then running Linux in a sort of virtualized environment within that microkernel. I know it had considerable commercial success, but there was also a big scandal involving patents (and I don't remember the details of why it became a scandal). I think it has somewhat died in the last few years.

And then there is a whole slew of intermediate solutions. RTAI is one of them, with a kernel module that shields the (inherently non-real-time) kernel somewhat. There are also research projects that are based around micro-kernels or predictable kernels, which then use a normal userspace for cost saving. One of the big deals in this business is development cost, and having the ability to run a stock Linux userspace makes systems self-hosting, which is very convenient for developers.

But to get back to the OPs question: I have heard that some projects were using *BSD components in their work, because the licensing is easier to deal with, and because of the patent brouhaha. But I've never heard of a canned solution that's based on and compatible with FreeBSD. Which doesn't mean that one doesn't exist, only that I haven't heard.


----------



## obsigna (Feb 1, 2020)

ralphbsz said:


> Realtime is a complex question. There is super-hard realtime, ... and data acquisition is the more high-powered side of this ...



I use FreeBSD for data aquisition of electrochemical laboratory equipment, and I didn’t miss RT capabilities of the OS. For very fast measurements (ADC conversion rate 1 MSamples/s and DAC update rate 3 MSamples/s), I utilize the DMA channels together with the 100 MHz sample clock of the DAQ board for spooling the measurement data into huge memory buffers (500 MB), and the DAC reads its values from another buffer (50 MB). Once started, such measurement cycles are not dependent on defined response times of the OS.

Recently, I implemented automatic range switching of the shunts for current measurement, and with that, accurate currents can be measured over 12 orders of magnitude. This feature is for various reasons limited to slow measurements, and here, I cannot use the sample clock and DMA channels of the DAQ board anyway. So, I do on demand conversion of 2 multiplexed ADC channels and also on demand DAC updates. The on demand sample rate on a 4 GHz quad core i7 running FreeBSD 12.1-CURRENT is still 200 kSample/s. I know that the duration between each of the samples is not exact, however, in this case this is not crucial, because the samples are used for building a moving average and tiny lapses hardly affect the result.

IMHO, for DAQ purposes, the lack of RT capabilities of the OS, which would otherwise be crucial for fast measurements, can be easily overcome by external ADC/DAC timing and spooling of the data into/from huge DMA buffers. Modern systems are anyway much more than quick enough for getting the bulk of the data out of the buffer and doing even complex math on it in no time. For slow measurements < 10kSample/s, RT is not crucial anyway.


----------



## tingo (Feb 1, 2020)

ogogon said:


> I have a CNC machine. CNC are have a control machine under Debian. There, a real-time subsystem is included in the kernel.
> This Linux solutions use RTAI or the realtime kernel patch set - "Complete Preemption".


Ah, this is LinuxCNC (or something like it). That is one way to control a CNC machine. Another way is to use a small microcontroller (Arduino) and grbl (https://github.com/grbl/grbl).
Both work equally well (at least for machine with up to 3 axes) but from what I have read LinuxCNC is much harder to set up.


----------



## ogogon (Feb 1, 2020)

Nicola Mingotti said:


> i spent some time with the BeagleBone and Freebsd and Debian. Those system are far from real time. If you need something like that, and the capabilities of a ‘reguler’ OS, one way is to offload the hard real time parts to simpler machines, like Arduino.


But this means a completely different solution.
I wanted to understand if it is possible to run the EMC2/LinuxCNC package on the FreeBSD platform instead of Linux.

Ogogon.


----------



## ogogon (Feb 1, 2020)

tingo said:


> Ah, this is LinuxCNC (or something like it). That is one way to control a CNC machine. Another way is to use a small microcontroller (Arduino) and grbl (https://github.com/grbl/grbl).
> Both work equally well (at least for machine with up to 3 axes) but from what I have read LinuxCNC is much harder to set up.


I believe that you are not quite right.
All such decisions on the Arduino designed to control not full CNC machines, but for 3D printers or laser engravers. This is a much simpler class of solutions. A 3D printer is a sandwich of 2D layers, and a laser engraver is generally one 2D layer.
This is a much simpler level of interpolation calculations.

Ogogon.


----------



## Nicola Mingotti (Feb 1, 2020)

ogogon said:


> But this means a completely different solution.
> I wanted to understand if it is possible to run the EMC2/LinuxCNC package on the FreeBSD platform instead of Linux.



Oh, i have no idea about that. It is far too much specific. I never tried to build a CNC on FreeBSD. My toy project was (~2 years ago) to build a Phone-remote-controlled Rover. 

Then, If you are a bit into robotics you know it is all about motor, sensors and switches. Be it a Drone, a CNC, a Rover an IoT solution. The basics are the same AFAIK.

The mixed solution: Regualr OS + Microcontroller(s) is very popular. If you want to go that way I can tell you wich book you should read. If instead you want to modify the Regular FreeBSD to make it realtime I don't know what to suggest. Nobody was going in that diraction 2 years ago. Not even in NetBSD, a community more hardware focused. That was the result of my search.

If you want something that you can not trust (not true realtime) than go for RT linux. This is not for me. My motto is: if it works it works. 

I did not try Xenomai. I had no time for it. It seemed complex.
RTAI : I never tried that.

HTH
n.


----------



## ogogon (Feb 2, 2020)

Nicola Mingotti said:


> If instead you want to modify the Regular FreeBSD to make it realtime I don't know what to suggest. Nobody was going in that diraction 2 years ago. Not even in NetBSD, a community more hardware focused. That was the result of my search.


Alas, I do not have such plans. They would be too global.
But I remember, that around 20-25 years ago there were several projects related to this. And I figured they brought some kind of success.
It turns out that no. It's a pity.



Nicola Mingotti said:


> The mixed solution: Regualr OS + Microcontroller(s) is very popular. If you want to go that way I can tell you wich book you should read.


I understand perfectly how to properly build such a system.
It is necessary to simplify and minimize the control functions of real time as much as possible and transfer them to the hardware controller. And all the “hard” math — the language interpreter, multidimensional interpolation calculations — should be left on the desktop host without guaranteed real time.

But, unfortunately, we have what we have. And there is no such correct free solution yet.
Or I do not know it.

Ogogon.


----------



## Nicola Mingotti (Feb 2, 2020)

ogogon said:


> .... And all the “hard” math — the language interpreter, multidimensional interpolation calculations — should be left on the desktop host without guaranteed real time.



Exactly, you leave all the ordinary computer stuff: calculations, services, internet/networking to a usual compter with a usual OS. The rest is discharged to simpler machines.



ogogon said:


> But, unfortunately, we have what we have. And there is no such correct free solution yet.
> Or I do not know it.



Why are you not happy with a mixed solution? It seems to me it makes a lot of sense. It is simple to manage and also ensures more reliability IMHO.


----------



## ogogon (Feb 2, 2020)

Nicola Mingotti said:


> Why are you not happy with a mixed solution? It seems to me it makes a lot of sense. It is simple to manage and also ensures more reliability IMHO.


Not at all, my friend, I did not mean at all to say that I was not satisfied with the architecture of such a solution.
I am not happy that it is not in the /usr/ports directory.

Ogogon.


----------



## tingo (Feb 2, 2020)

ogogon said:


> I believe that you are not quite right.
> All such decisions on the Arduino designed to control not full CNC machines, but for 3D printers or laser engravers. This is a much simpler class of solutions. A 3D printer is a sandwich of 2D layers, and a laser engraver is generally one 2D layer.
> This is a much simpler level of interpolation calculations.
> 
> Ogogon.


You may believe (of course) what you wish. Still, I have used Arduino-controlled CNC mills quite a few times, and I see others use them every week.
Here is a video of one in operation: 



_View: https://www.youtube.com/watch?v=iexEcJaL6aU_

and here is how you can build one: https://github.com/fellesverkstedet/fabricatable-machines/wiki


----------



## trasz@ (Feb 5, 2020)

Apart from using a separate microcontroller, I think the most feasible way would be to just dedicate one of CPU cores to run realtime code and tell the kernel to keep away from it.  Might already be possible by using cpuset(1).  I never got so far, tbh - it was just much easier to use GRBL.


----------

