# What is sysctl kern.sched.preempt_thresh



## larshenrikoern (Jun 22, 2022)

Hi

I have wonder what sysctl kern.sched.preempt_thresh is doing. My guess is something with system throughput vs interactivity. In all guides I see this:

sysctl kern.sched.preempt_thresh=224
Default: 80

But it accepts other values, and the number people is writing is a bit odd 224 (my guess=7*32). Why not 256. For me higher numbers gives a even more responsive desktop system. I have tried with different numbers And it accepts numbers from 1 to ? very high.

I am worried about everyone copying the same number for ages without questioning. I have for now set it to sysctl kern.sched.preempt_thresh=384 (256*1,5). And the system is even more responsive with that

Have an nice day


----------



## Alain De Vos (Jun 22, 2022)

Most of those values are a trade-off.
Like set it low when you are running a database server.
Set it high when you want to play youtube files.
Set it in the middle when you want to do both.

A think with a higher value your CPU will do more context switching. And each context switch wastes useful
CPU time.


----------



## larshenrikoern (Jun 22, 2022)

Thank you

It is the same I see. But what does it do. Timeslices, values for max time per thread ??.

The most strange thing is that every guide since a loooong time suggest the value 224 without questioning. It means it is not optimal for most systems. I am using my system for sound production, so a high value is most likely good.


----------



## thindil (Jun 22, 2022)

If you want some technical details, perhaps this conversation will be interesting for you: https://forums.freebsd.org/threads/...-kern-sched-preempt_thresh.63388/#post-366995 and two posts below. 

Generally, it is recommended to set it high on a desktop and left unchanged on a server. But as usual, YMMV.


----------



## Alain De Vos (Jun 22, 2022)

Note, you can give as root individual processes also RT-characteristics.





						rtprio(1)
					






					www.freebsd.org


----------



## larshenrikoern (Jun 22, 2022)

thindil said:


> If you want some technical details, perhaps this conversation will be interesting for you: https://forums.freebsd.org/threads/...-kern-sched-preempt_thresh.63388/#post-366995 and two posts below.
> 
> Generally, it is recommended to set it high on a desktop and left unchanged on a server. But as usual, YMMV.


So it is timeslicing. How are the values calculated ?? And what are meaningful values. Probably not extreme (but it accepts 1 and values over 12000).


----------



## larshenrikoern (Jun 22, 2022)

Alain De Vos said:


> Note, you can give as root individual processes also RT-characteristics.
> 
> 
> 
> ...


I would use rtprio rarely. Mostly because very often I will just move problems to another place on a well optimized system. But giving the main program a little higher value than default, and stopping  unneeded services might be beneficial.


----------



## thindil (Jun 22, 2022)

larshenrikoern said:


> So it is timeslicing. How are the values calculated ?? And what are meaningful values. Probably not extreme (but it accepts 1 and values over 12000).


Unfortunately, I'm not familiar with FreeBSD code, so I can't answer 100% accurate. Command `sysctl -d kern.sched.preempt_thresh` gives: `kern.sched.preempt_thresh: Maximal (lowest) priority for preemption`.  My guess is: the lower value you are comfortable, the better. If you set it too high, switching between tasks probably will be rarer, and can have an opposite effect than expected. I would prefer the value below 500. Maybe the value depends on the amount of processes either?


----------



## mer (Jun 22, 2022)

Not really time slicing;  preemption.  As in "process A is running, process B needs to run, scheduler does some math, some checking and says Yes, B can start running so I'll stop A and start B"
Timeslicing in general/simple terms is more of "every process gets at least X amount of time to run".
The ULE scheduler code is the definitive source for how things are calculated and exactly what different sysctls do, but scheduler code is typically non trivial to understand.

I agree with thindil and his "guess".  I don't have a feel for what values are good or bad, but I think they also depend greatly on your specific use case/workload.  A system running a desktop so you can browse the web is a lot different than a server system  spewing the content for the YouTube videos you are watching.

Another thing to keep in mind is the hardware.  Capability of the hardware runs ahead of the software (typically).  Faster CPUs, more/faster RAM, sw has to catch up to fully utilize, so the defaults in sw maybe good for last gen/a few generations ago hardware but are merely ok for current gen.  The value of 224 has been around for a while as a recommendation for desktop use, so it's likely from 3 or 4 generations of hardware ago and maybe bumping it is appropriate.  If you look at the mailing lists there have been discussions as to "is 0 a better default value".


----------



## larshenrikoern (Jun 22, 2022)

mer said:


> Not really time slicing;  preemption.  As in "process A is running, process B needs to run, scheduler does some math, some checking and says Yes, B can start running so I'll stop A and start B"
> Timeslicing in general/simple terms is more of "every process gets at least X amount of time to run".
> The ULE scheduler code is the definitive source for how things are calculated and exactly what different sysctls do, but scheduler code is typically non trivial to understand.
> 
> ...


Would 0 not mean the scheduler is almost out of the way. Does not sound healthy. But for sure responsive. I tried it for a few minutes.


----------



## larshenrikoern (Jun 22, 2022)

thindil said:


> Unfortunately, I'm not familiar with FreeBSD code, so I can't answer 100% accurate. Command `sysctl -d kern.sched.preempt_thresh` gives: `kern.sched.preempt_thresh: Maximal (lowest) priority for preemption`.  My guess is: the lower value you are comfortable, the better. If you set it too high, switching between tasks probably will be rarer, and can have an opposite effect than expected. I would prefer the value below 500. Maybe the value depends on the amount of processes either?


That is why I want someone to explain how it is calculated. Because otherwise it is just guessing, and of course to test the result with different values. The tryout with 0 was interesting.


----------



## larshenrikoern (Jun 22, 2022)

It somehow reminds me of swappiness in linux. For desktop use they recommend a very low swappiness (10 or less). But the system is just as responsive with more healthy values like 25, 40 and even 80. The worst for desktop is the default 60. Right now I try with kern.sched.preempt_thresh=40 and the system seems to behave well.


----------



## mer (Jun 22, 2022)

A link to one of the emails about a value of 0.  Gets a bit technical, but the folks involved in the email are pretty knowledgable about the kernel.



			Is kern.sched.preempt_thresh=0 a sensible default?
		

To me this is a key paragraph on the value and what it does.
`>[I] Is PRI_MIN_KERN=80 really a good default value for the preemption threshold?[/I]

Yeah, a good question...
I am not really sure about this.  In my opinion it would be better to set
preempt_thresh to at least PRI_MAX_KERN, so that all threads running in kernel
are allowed to preempt userland threads.  But that would also allow kernel
threads (with priorities between PRI_MIN_KERN and PRI_MAX_KERN) to preempt other
kernel threads as well, not sure if that's always okay.  The same argument
applies to higher values for preempt_thresh as well.`

To my knowledge this type of discussion has been around forever, on all systems I've ever worked with.
Too much depends on a specific use case (hardware, how many processes, what the user expects) to say "This is the best number".


----------



## larshenrikoern (Jun 22, 2022)

mer said:


> A link to one of the emails about a value of 0.  Gets a bit technical, but the folks involved in the email are pretty knowledgable about the kernel.
> 
> 
> 
> ...


I agree fully with the last part. Different systems needs different configurations. That is why i started this thread, questioning the 224. And maybe the extremes (0 or 255) are maybe OK in some cases (they seems to make a fast desktop), but I would like to use less extreme values until I fully understand what is going on. And many servers are probably well with 80.


----------



## mer (Jun 22, 2022)

If you are using a USB mouse, take a look at your Xorg.0.log (over in /var/log).  Check it for messages like:
[3079719.510] (EE) event4  - SIGMACHIP Usb Mouse, class 0/0, rev 1.10/1.10, addr 32: client bug: event processing lagging behind by 18ms, your system is too slow

and occasional mouse glitches.
A lot of people report them, not sure if it's an X server issue, USB infrastructure or sysmouse issue, not sure if it's even "real", but it would be interesting to see if frequency of the messages or the "lagging behind by..." value changes.


----------



## larshenrikoern (Jun 22, 2022)

mer said:


> If you are using a USB mouse, take a look at your Xorg.0.log (over in /var/log).  Check it for messages like:
> [3079719.510] (EE) event4  - SIGMACHIP Usb Mouse, class 0/0, rev 1.10/1.10, addr 32: client bug: event processing lagging behind by 18ms, your system is too slow
> 
> and occasional mouse glitches.
> A lot of people report them, not sure if it's an X server issue, USB infrastructure or sysmouse issue, not sure if it's even "real", but it would be interesting to see if frequency of the messages or the "lagging behind by..." value changes.


I have no such problems. I have looked  My mouse is a bit secial as it is wireless with the same reciever as the keybord (a nice Logitch). All this is not because my machine is slow. But I like it to be running so I feel it is fast. And im am always trying to understand what my configuration is doing. I do not like black magic .


----------



## larshenrikoern (Jun 22, 2022)

I think I have found the answer to why 223 is the suggested value. 80 is PRI_MIN_KERN. Look here https://wiki.freebsd.org/AndriyGapon/AvgThreadPriorityRanges . What to choose ?????

A short test seems like 171 is a good compromise on my system


----------



## mer (Jun 22, 2022)

larshenrikoern said:


> I think I have found the answer to why 223 is the suggested value. 80 is PRI_MIN_KERN. Look here https://wiki.freebsd.org/AndriyGapon/AvgThreadPriorityRanges . What to choose ?????
> 
> A short test seems like 171 is a good compromise on my system


That's a good find on that wiki page.  Thanks.


----------



## larshenrikoern (Jun 22, 2022)

After a little bit more testing I think a good value for kern.sched.preempt_thresh is 151. It is also the lowest value the author suggested for interactivity. Suggest you try it out. It also makes sure that not all processes runs with high priority. So reduces general load if the previous value was 223 




151PRI_MAX_INTERACTTime-share threads in interactive category.
ULE puts threads with these priorities onto a real-time queue.
Distinguished value: PUSER (120,  PRI_MIN_TIMESHARE)

​


----------



## mer (Jun 22, 2022)

I think it's going to wind up being a case of looking at the chart, thinking about it and trying different values. I'm going to set mine to 171 and leave it for a bit (week or so) and use the systems, then set it to 151 and try again.

Thanks.


----------



## larshenrikoern (Jun 22, 2022)

mer said:


> I think it's going to wind up being a case of looking at the chart, thinking about it and trying different values. I'm going to set mine to 171 and leave it for a bit (week or so) and use the systems, then set it to 151 and try again.
> 
> Thanks.


Lets write and share how it goes. The values must be tried on different systems. Maybe they will behave differently:

My system:
I7-8700 non K, 48GB ram and gtx-1080.

On other systems  a value 171 might be necessary.


----------



## PMc (Jun 22, 2022)

In order to make sense of these values, you need first to understand how timesharing was originally designed. There was only one core, and a HZ value (the HZ value is the timer interrupt frequency).
When the kernel starts a process, that process gets into control and owns the cpu. There is no way to take away the cpu from the process, until either

there is an interrupt occuring from somewhere, or
the process does a system call and thereby gives control back to the kernel.
When an interrupt occurs, the kernel will put the process on hold and service the interrupt. Thats why the timer interrupt is important, because otherwise, in the absence of device i/o, processes could run forever.
Occasionally the scheduler would look at the acitivity patterns of the processes and calculate a priority for each of them (visible in `ps axlH`). This priority is then used to decide on the next process to run, so that smooth interactive processing can go along with compute-intensive tasks.

The mechanism was not so well-suited for multiprocessor systems, and therefore preemption was introduced. Preemption allows the scheduler to interrupt and switch processes much more often, without having to wait for the (rather expensive) timer interrupt, and even while they execute in kernel mode.

Then there are three special cases: the `idprio` and `rtprio` processes and the kernel processes. All of these have a fixed priority. Together, the user processes with their ad-hoc calculated priority, plus these three kinds, form a contiguous scale of priorities from somewhere -99 to +155 (lower number means higher priority), or normalized from 0 to 255 (you never know which one you're currently dealing with), as follows (give or take one or two - if you need the exact numbers, figure them out yourself):

```
-100  ..  -52   interrupt processing
   -51  ..  -21   [CMD]rtprio[/CMD] tasks
   -20  ..  +19   kernel tasks
   +20  .. +120   user processes (dynamic assigned
  +124  .. +154   [CMD]idprio[/CMD] tasks
           +155   the idle process
```

The kern.sched.preempt_thresh is then the cutoff value in that sequence beyond with you do not allow preemption (but in the internal 0..255 scale).

So, 224 is basically the starting point of the idprio scale. The code itself has three default values: normally 80, with FULL_PREEMTION 255 (that does afaik not work well), and wihout PREEMPTION 0 (that didn't work at all for me). Somewhere it came out that 162 might be a good value, so I am running with this (but forgot the detailed reason).

Now for the logic (for the non-native inglisch speakers): to preempt somebody means to queue-jump them. 
So, if _our own_ prioritiy number is _smaller_ than this threshold (meaning we have _more_ priority), then we are allowed to preempt others.

If this then does any good - well, that depends....



larshenrikoern said:


> am worried about everyone copying the same number for ages without questioning.


And this is exactly the problem with the scheduler. From what I figured, the ULE scheduler was written because the old one was just unsatisfying for SMP. It was written by a guy who had this great design idea (which it is), and it was then put into the system, it solved the imminent problems, and end of story. It was never really honed to the system, some of the tuneables are rather lab-testing instrumentation, and there are cornercases where it behaves just bad (which are solveable). 
I once talked to the author, and when he figured that I really wanted to go into depth of the stuff, he went immediately U-boot. Certainly he has found other interesting things to put his attention to.


----------



## larshenrikoern (Jun 22, 2022)

PMc said:


> In order to make sense of these values, you need first to understand how timesharing was originally designed. There was only one core, and a HZ value (the HZ value is the timer interrupt frequency).
> When the kernel starts a process, that process gets into control and owns the cpu. There is no way to take away the cpu from the process, until either
> 
> there is an interrupt occuring from somewhere, or
> ...


Thank you for the detailed explanation. It seems you are using a value in the middle of the two 151 and 171. That makes a sort of sense. I hope all of us knows a bit more after this conversation


----------



## Deleted member 70435 (Jun 22, 2022)

Thread Scheduling | FreeBSD Process Management | InformIT
					

This chapter explains process management in FreeBSD in detail, covering topics like context and thread switching, signals, and jails. Included are some exercises to help you put your new knowledge into practice.




					www.informit.com


----------



## larshenrikoern (Jun 23, 2022)

Vadim Alexandrov said:


> Thread Scheduling | FreeBSD Process Management | InformIT
> 
> 
> This chapter explains process management in FreeBSD in detail, covering topics like context and thread switching, signals, and jails. Included are some exercises to help you put your new knowledge into practice.
> ...


Thank you for this overviev, both historical and current. I also did find this https://people.freebsd.org/~jeff/FBSD_14_10ULE_aj.jm_cx10_09.pdf witch explains almost the same things.


----------



## larshenrikoern (Jun 23, 2022)

To sum up the first day of discussions in this thread.

The value suggested in most guides 223 is to high. My suspicion is that it is a value suggested quite early in the development of the ULE scheduler. On my system lowering the value has actually lowered the cpu temperature quite a bit. So using the resources more efficient, and reducing cpu load (and thereby improving interactivity more than a higher value). We are currently considering the following values for a interactive desktop system:

kern.sched.preempt_thresh=151 Time-share threads in interactive category

kern.sched.preempt_thresh=171 Time-share threads in batch category with nice priority between -1 and -20

kern.sched.preempt_thresh=162 as a middle ground between the two

On servers these values is not recommended. Mostly for desktops.

I might suggest that one or more of these values will be placed in the handbook as a part of tuning with sysctl.conf

Comments much appreciated. 
Have a nice day


----------



## thindil (Jun 23, 2022)

larshenrikoern said:


> To sum up the first day of discussions in this thread.
> 
> The value suggested in most guides 223 is to high. My suspicion is that it is a value suggested quite early in the development of the ULE scheduler. On my system lowering the value has actually lowered the cpu temperature quite a bit. So using the resources more efficient, and reducing cpu load (and thereby improving interactivity more than a higher value). We are currently considering the following values for a interactive desktop system:
> 
> ...


While I agree that value 162 looks as the best recommendation for the typical desktop, I would wait with calling 223 too high. 
I have done some tests with values above 255, if I understand correctly, it should lead to almost disable preemption. The daily desktop tasks didn't get any benefit from it, but extremely heavy tasks, like playing modern AAA Windows games with almost constant full usage of the resources, I had some better interactivity. But to be honest, I need more tests about it.


----------



## larshenrikoern (Jun 23, 2022)

thindil said:


> While I agree that value 162 looks as the best recommendation for the typical desktop, I would wait with calling 223 too high.
> I have done some tests with values above 255, if I understand correctly, it should lead to almost disable preemption. The daily desktop tasks didn't get any benefit from it, but extremely heavy tasks, like playing modern AAA Windows games with almost constant full usage of the resources, I had some better interactivity. But to be honest, I need more tests about it.


Values over 256 has no effect. The max value is 256. To raise one single programs priority over the others use rtprio


----------



## thindil (Jun 23, 2022)

larshenrikoern said:


> Values over 256 has no effect. The max value is 256. To raise one single programs priority over the others use rtprio


Yes, now I know that they don't have effect. But what happens when you enter too high value? It is reduced to 0 or the max allowed value used?


----------



## larshenrikoern (Jun 23, 2022)

thindil said:


> While I agree that value 162 looks as the best recommendation for the typical desktop, I would wait with calling 223 too high.
> I have done some tests with values above 255, if I understand correctly, it should lead to almost disable preemption. The daily desktop tasks didn't get any benefit from it, but extremely heavy tasks, like playing modern AAA Windows games with almost constant full usage of the resources, I had some better interactivity. But to be honest, I need more tests about it.


What we are doing when moving around with these values are moving the bottlenecks on a system. If you have sufficient memory the cpu, diskd, or gpu might be your limiting factor. So the optimal optimizations will be a different from system to system and after its usage. If you are using just one program that might be the one thing that matters. 

To me interactivity means that the system is made not to do one single task, but a lot of different tasks after my need. And make sure the system responds quickly and efficient. My current value of kern.sched.preempt_thresh=151 is doing this perfectly right now on machine. Of course I have other optimizations as well, and right now they are playing together very well.


----------



## PMc (Jun 23, 2022)

thindil said:


> Yes, now I know that they don't have effect. But what happens when you enter too high value? It is reduced to 0 or the max allowed value used?


Being at a point where you know what to look for and what it is about, I believe you can now start reading the source.  I't not so difficult:


```
$ find /usr/src/sys -type f | xargs grep -l preempt_thresh
/usr/src/sys/kern/sched_ule.c
/usr/src/sys/amd64/compile/C6R12V1/kernel.full
/usr/src/sys/amd64/compile/C6R12V1/sched_ule.o
/usr/src/sys/amd64/compile/C6R12V1/kernel.debug

$ grep preempt_thresh /usr/src/sys/kern/sched_ule.c
 * preempt_thresh:      Priority threshold for preemption and remote IPIs.
static int __read_mostly preempt_thresh = PRI_MAX_IDLE;
static int __read_mostly preempt_thresh = PRI_MIN_KERN;
static int __read_mostly preempt_thresh = 0;
        if (preempt_thresh == 0)
        if (pri <= preempt_thresh)
SYSCTL_INT(_kern_sched, OID_AUTO, preempt_thresh, CTLFLAG_RW,
    &preempt_thresh, 0,
```

Besides variable initialization, there isn't much there. Have a look at it:





						sched_ule.c « kern « sys - src - FreeBSD source tree
					






					cgit.freebsd.org
				




That isn't too difficult to read, or is it?


----------



## mer (Jun 23, 2022)

PMc said:


> That isn't too difficult to read, or is it?


Reading source code is often never very difficult, once you understand the language.
Understanding the behavior is usually the difficult part


----------



## thindil (Jun 23, 2022)

PMc said:


> Being at a point where you know what to look for and what it is about, I believe you can now start reading the source.  I't not so difficult:
> 
> 
> ```
> ...


Isn't too difficult, thank you for pointing me there.


----------



## Alain De Vos (Jun 24, 2022)

I've put:

```
kern.sched.preempt_thresh=120
```


----------



## larshenrikoern (Jun 24, 2022)

Alain De Vos said:


> I've put:
> 
> ```
> kern.sched.preempt_thresh=120
> ```


Is that for a desktop ??? And for general use. I am sure that it is running well for you   My pick was the other end of the same 151


----------



## larshenrikoern (Jun 24, 2022)

Hi

I am still a bit unsure what some of the terms means for desktop use. For instance I am a bit unsure about "Time-share threads in batch category". Some practical examples would be nice. Because I want to put the number as low as possible, to spare system resources.

I do know about the kernel threads and in the other end idle threads. And giving the latter realtime priority will most of the time be silly. But the ones in between. Where to put the line for kern.sched.preempt_thresh ?? Lets argue a bit over that, just to making everyone a bit more knowledgeable 
120
151
152
162
171
172
203
204
223


----------



## mer (Jun 24, 2022)

batch category, maybe compilation?  Like doing a build world, would those threads wind up in batch?
The part about "nice priority" does that imply the process/thread was subject to the "nice" command?
Don't threads start off with a default priority?  Running top looking at things running under my username
they are all at the same priority (minor changes as they are used)  A user can nice something to lower the priority (yes root can increase the priority), but otherwise when the thread is created the priority is assigned.

I could be wrong about how a thread gets it's default priority, but that's my understanding.

EDIT:
It been reported/stated that priorities in top are "value from the kernel - 100" so most of the threads owned by my user are running at 20:  typical desktop use, firefox, terminal windows, ssh sessions.  firefox has a few different threads running (even with one tap open), sometimes the priority for a firefox thread goes to 21-27.  This implies my user threads run at about "120" normally, maybe get to 127.  That winds up crossing categories a little.  I'm not nicing anything so I'd guess these are default priority for user processes.


----------



## PMc (Jun 24, 2022)

Just run an endless loop and watch the priority change:

```
$ while true
do
:
done &
$ ps axl | head -1 ; ps axl | grep sh
  UID  PID PPID  C PRI NI    VSZ    RSS MWCHAN   STAT TT        TIME COMMAND
    2 4981 3650  0  20  0  12176   3244 wait     S     3     0:00.01 sh
    2 4987 4981  2  97  0  12176   3244 -        R     3     0:17.58 sh
```


----------



## larshenrikoern (Jun 24, 2022)

PMc said:


> Just run an endless loop and watch the priority change:
> 
> ```
> $ while true
> ...


I do not understand why I should try doing that. And it stops at 103 when running as normal user. So what does it prove ??


----------



## _al (Jun 24, 2022)

The Battle of the Schedulers: FreeBSD ULE vs. Linux CFS


----------



## larshenrikoern (Jun 24, 2022)

Looks quite interesting. And again FreeBSDs approach is simpler to understand than the linux one (I did use deadline as my scheduler while running linux, but that is another matter). I am beginning to understand how FreeBSD calculates interactive threads, vs batch interactive processes. What I am still not fully aware of is what those interactive batch processes are, and to what degree I will have to think about it. Compiling ports for instance is to me not interactive, but the computer might consider differently. So in the following scheme everything above 151, as I see it will, be of lesser importance to desktop users. But for me as an audio editor ??? Correct me if I am wrong 




MinMaxDescription0PRI_MIN
PRI_MIN_ITHD47PRI_MAX_ITHDInterrupt thread real-time priorities.
Distinguished values: PI_REALTIME, PI_AV, PI_NET, PI_DISK, PI_TTY, PI_DULL, PI_SOFT, PI_SWI48PRI_MIN_REALTIME79PRI_MAX_REALTIMErtprio real-time priorities80PRI_MIN_KERN119PRI_MAX_KERNKernel thread real-time priorities.
Distinguished values: PSWP, PVM, PINOD, PRIBIO, PVFS, PZERO, PSOCK, PWAIT, PLOCK, PPAUSE120PRI_MIN_TIMESHARE
PRI_MIN_INTERACT151PRI_MAX_INTERACTTime-share threads in interactive category.
ULE puts threads with these priorities onto a real-time queue.
Distinguished value: PUSER (120,  PRI_MIN_TIMESHARE)152PRI_MIN_BATCH171Time-share threads in batch category with nice priority between -1 and -20172SCHED_PRI_MIN203SCHED_PRI_MAXTime-share threads in batch category204223PRI_MAX_TIMESHARE
PRI_MAX_BATCHTime-share threads in batch category with nice priority between 1 and 20224PRI_MIN_IDLE255PRI_MAX_IDLE
PRI_MAXidprio idle threads and the system idle threads


The scheme is from here: https://wiki.freebsd.org/AndriyGapon/AvgThreadPriorityRanges


----------



## Deleted member 70435 (Jun 24, 2022)

what good, you put this, really I ask you what is the behavior, kern.sched, how does it behave in FreeBSD?. which implementation you will be able to use with it, well that's it.


----------



## Erichans (Jun 24, 2022)

larshenrikoern said:


> [...] What I am still not fully aware of is what those interactive batch processes are, and to what degree I will have to think about it. Compiling ports for instance is to me not interactive, but the computer might consider differently.


"interactive batch processes" sounds like a a contradictio in terminis. Where did you get that term from? A link and a precise quote would be helpful. Perhaps you are referring to a timeshare thread (a "normal" user thread) that started as an interactive one, that means that at that time it is listed on the _real-time queue_, and over time the thread has landed in the _batch queue_. For each processor core a thread can be listed in one of three run queues: the _real-time queue, _the _batch queue_ or the _idle queue._ The scheduler decides what thread is to run next (=gets CPU time) and if it stays in the same queue.

The decision on whether a thread is considered interactive or not is based on the relation between sleep time and run time. The interactivity score is based on that and calculated accordingly. The ULE article (e.g. #4, just after equation 2):


> The scaling factor is the maximum interactivity
> score divided by two. Threads that score
> below the interactivity threshold are considered
> to be interactive; all others are noninteractive.



The earlier referenced part (4.4. Thread Scheduling) of _Chapter 4 Process Management_ in this thread is of the first edition of _The Design and Implementation of the FreeBSD Operating System_. The same chapter—#6 in the list below—is also freely available for the second edition. Perhaps also helpful:

_An Overview of Scheduling in the FreeBSD Kernel by Marshall Kirk McKusick - EuroBSDcon 2021; video & slides_
Kirk McKusick briefly discusses the article _The Battle of the Schedulers: FreeBSD ULE vs. Linux CFS_ in this presentation.
_An Overview of Scheduling in the FreeBSD Kernel_ by Marshall Kirk McKusick - BSD Can 2020; video & slides
This is basically the same talk as in #4 but, there are interesting differences*
_ULE: A Modern Scheduler For FreeBSD_ by Jeff Roberson - BSDCon 2003
_The FreeBSD ULE scheduler_ by Marshall Kirk McKusick and Jeff Roberson - Science, Systems and FreeBSD, September / October 2014 - The FreeBSD Journal, Vol 1, Issue No. 5, page 20-26
_The FreeBSD ULE scheduler_ - _just the article as mentioned above,_ from Jeff Roberson's web page.
_The Battle of the Schedulers: FreeBSD ULE vs. Linux CFS_ by Justinien Bouron, Sebastien Chevalley, Baptiste Lepers, and Willy Zwaenepoel - USENIX ATC 2018; pdf - slides - audio of presentation
Chapter 4: Process Management or pdf of _The Design and Implementation of the FreeBSD Operating System, 2nd Edition_
While both articles #3 & #4 describe the ULE scheduler and #3 might look more of an extensive (academic-like) article, I personally find it less cramped (for the lack of a better word) than the article in the FreeBSD Journal.

___
* The 4BSD scheduler has long been the default scheduler of FreeBSD. As of FreeBSD 8, the new ULE scheduler took its place as the default scheduler. The 4BSD scheduler is still part of FreeBSD and you can use it instead of the ULE scheduler! You might consider using it on a small (embedded) system, like the Raspberry Pi; see the (part of the) Q&A session following the presentation in #1.


----------



## larshenrikoern (Jun 25, 2022)

Erichans said:


> "interactive batch processes" sounds like a a contradictio in terminis. Where did you get that term from? A link and a precise quote would be helpful. Perhaps you are referring to a timeshare thread (a "normal" user thread) that started as an interactive one, that means that at that time it is listed on the _real-time queue_, and over time the thread has landed in the _batch queue_. For each processor core a thread can be listed in one of three run queues: the _real-time queue, _the _batch queue_ or the _idle queue._ The scheduler decides what thread is to run next (=gets CPU time) and if it stays in the same queue.
> 
> The decision on whether a thread is considered interactive or not is based on the relation between sleep time and run time. The interactivity score is based on that and calculated accordingly. The ULE article (e.g. #4, just after equation 2):
> 
> ...


Thank you for a detailed reaction I will look into the few of the sources that i did not already know

I got the term "Time-share threads in batch category from the this link" https://wiki.freebsd.org/AndriyGapon/AvgThreadPriorityRanges as referred above. And yes it sounds confusing. So that is why I am confused. But on my system there is a to me important difference between kern.sched.preempt_thresh=151 and 152. on the one value sound in multi-track recordings and sequencing are shuttering. At 152 not at all. 

All this might come down to native English vs. non native (like myself). Other languages does have different terms than Engllsh. And translating those back to English may give confusing results


----------



## larshenrikoern (Jun 25, 2022)

larshenrikoern said:


> Thank you for a detailed reaction I will look into the few of the sources that i did not already know
> 
> I got the term "Time-share threads in batch category from the this link" https://wiki.freebsd.org/AndriyGapon/AvgThreadPriorityRanges as referred above. And yes it sounds confusing. So that is why I am confused. But on my system there is a to me important difference between kern.sched.preempt_thresh=151 and 152. on the one value sound in multi-track recordings and sequencing are shuttering. At 152 not at all.
> 
> All this might come down to native English vs. non native (like myself). Other languages does have different terms than Engllsh. And translating those back to English may give confusing results


The values in equtation on this page from

Chapter 4: Process Management or pdf of _The Design and Implementation of the FreeBSD Operating System, 2nd Edition_
might tell what he means.


----------



## larshenrikoern (Jun 25, 2022)

Erichans said:


> "interactive batch processes" sounds like a a contradictio in terminis. Where did you get that term from? A link and a precise quote would be helpful. Perhaps you are referring to a timeshare thread (a "normal" user thread) that started as an interactive one, that means that at that time it is listed on the _real-time queue_, and over time the thread has landed in the _batch queue_. For each processor core a thread can be listed in one of three run queues: the _real-time queue, _the _batch queue_ or the _idle queue._ The scheduler decides what thread is to run next (=gets CPU time) and if it stays in the same queue.
> 
> The decision on whether a thread is considered interactive or not is based on the relation between sleep time and run time. The interactivity score is based on that and calculated accordingly. The ULE article (e.g. #4, just after equation 2):
> 
> ...


I can now see that in the slides to _An Overview of Scheduling in the FreeBSD Kernel _McKursick also uses these terms_. _So the term is used by the developers_._


----------



## PMc (Jun 25, 2022)

larshenrikoern said:


> Thank you for a detailed reaction I will look into the few of the sources that i did not already know
> 
> I got the term "Time-share threads in batch category from the this link"


Ah, yes. I didn't understand that at first, too. Then I figured that "time-share" is the older term for "multitasking". When I started with unix, it was "multitasking/multiuser".
So, all processes on the machine are time-sharing processes, and the discussion is about batch processes (that run in the background and use cpu until they have completed their task) and interactive processes (that wait for user input, react on it, and then wait again).

It is commonly said that an interactive action should not need more than 50ms to react, to give the user a smooth experience.
But this does not concern things like audio/video processing, where 50ms is far too slow. In such cases rtprio may help.

There is another issue that is not considered in the docs: there is a difference between batch processes that do regularly access the disk, and those that do not. 
Under certain conditions I observed two otherwise equal processes run with 95% and 5% of cpu usage - one would almost always compute, and the other would not make progress.

I found that unacceptable, so I fixed it. You can try that fix if it makes any difference for your workload (to the better or worse).


```
diff --git a/sys/kern/sched_ule.c b/sys/kern/sched_ule.c
index 1f859b28684..c5ef566c998 100644
--- a/sys/kern/sched_ule.c
+++ b/sys/kern/sched_ule.c
@@ -38,7 +38,7 @@
  */
 
 #include <sys/cdefs.h>
-__FBSDID("$FreeBSD$");
+__FBSDID("$FreeBSD: releng/12.2/sys/kern/sched_ule.c 355610 2019-12-11 15:15:21Z mav $");
 
 #include "opt_hwpmc_hooks.h"
 #include "opt_sched.h"
@@ -224,6 +224,7 @@ static int __read_mostly preempt_thresh = 0;
 static int __read_mostly static_boost = PRI_MIN_BATCH;
 static int __read_mostly sched_idlespins = 10000;
 static int __read_mostly sched_idlespinthresh = -1;
+static int __read_mostly resume_preempted = 1;
 
 /*
  * tdq - per processor runqs and statistics.  All fields are protected by the
@@ -485,7 +486,10 @@ tdq_runq_add(struct tdq *tdq, struct thread *td, int flags)
                 * This queue contains only priorities between MIN and MAX
                 * realtime.  Use the whole queue to represent these values.
                 */
-               if ((flags & (SRQ_BORROWING|SRQ_PREEMPTED)) == 0) {
+               if (((flags & SRQ_PREEMPTED) && resume_preempted) ||
+                               (flags & SRQ_BORROWING))
+                       pri = tdq->tdq_ridx;
+               else {
                        pri = RQ_NQS * (pri - PRI_MIN_BATCH) / PRI_BATCH_RANGE;
                        pri = (pri + tdq->tdq_idx) % RQ_NQS;
                        /*
@@ -496,8 +500,7 @@ tdq_runq_add(struct tdq *tdq, struct thread *td, int flags)
                        if (tdq->tdq_ridx != tdq->tdq_idx &&
                            pri == tdq->tdq_ridx)
                                pri = (unsigned char)(pri - 1) % RQ_NQS;
-               } else
-                       pri = tdq->tdq_ridx;
+               }
                runq_add_pri(ts->ts_runq, td, pri, flags);
                return;
        } else
@@ -3193,6 +3196,9 @@ SYSCTL_UINT(_kern_sched, OID_AUTO, interact, CTLFLAG_RW, &sched_interact, 0,
 SYSCTL_INT(_kern_sched, OID_AUTO, preempt_thresh, CTLFLAG_RW,
     &preempt_thresh, 0,
     "Maximal (lowest) priority for preemption");
+SYSCTL_INT(_kern_sched, OID_AUTO, resume_preempted, CTLFLAG_RW,
+    &resume_preempted, 0,
+    "Reinsert preemted threads at queue-head");
 SYSCTL_INT(_kern_sched, OID_AUTO, static_boost, CTLFLAG_RW, &static_boost, 0,
     "Assign static kernel priorities to sleeping threads");
 SYSCTL_INT(_kern_sched, OID_AUTO, idlespins, CTLFLAG_RW, &sched_idlespins, 0,
```

After installing + starting the new kernel, the fix is disabled by default. To activate it (at any time), set `sysctl kern.sched.resume_preempted=0`. Switch it off again with `sysctl kern.sched.resume_preempted=1` and see if this makes a difference.


----------



## mer (Jun 25, 2022)

PMc have you tried to push that upstream to a committer?


----------



## larshenrikoern (Jun 25, 2022)

PMc said:


> Ah, yes. I didn't understand that at first, too. Then I figured that "time-share" is the older term for "multitasking". When I started with unix, it was "multitasking/multiuser".
> So, all processes on the machine are time-sharing processes, and the discussion is about batch processes (that run in the background and use cpu until they have completed their task) and interactive processes (that wait for user input, react on it, and then wait again).
> 
> It is commonly said that an interactive action should not need more than 50ms to react, to give the user a smooth experience.
> ...


To me it come down to how are the machine reacting for my workload, and currently it is OK.

About video and audio. Yes FreeBSD is not optimal but woks for my not to big demands with audio. I have had some writing with Hans Peter Selasky (programming on parts af FreeBSD audio system). He might look into a patch like this one, and get some ideas from it, and come up with a practical solution.


----------



## PMc (Jun 25, 2022)

mer said:


> PMc have you tried to push that upstream to a committer?


I don't have a sponsor.


----------



## larshenrikoern (Jun 25, 2022)

PMc said:


> I don't have a sponsor.


 Me neither


----------



## mer (Jun 25, 2022)

PMc said:


> I don't have a sponsor.


File a bug report, attach as a patch.


----------



## PMc (Jun 26, 2022)

mer said:


> File a bug report, attach as a patch.


Well, I do that usually when I get onto something that is specific to my installation, or might be difficult to notice. But this here is not difficult to notice. And it is not that I keep the matter fror me - I tell it publicly, here and on the lists.

There were recurring complaints on the lists, that there are certain problems with the scheduler, over years. There are generally responded with the flat answer: if you don't like the new scheduler, you are free to use the old one.
Then you have, further up, lists of literature, from people who claim that they have not only used the scheduler, but have done research on it. If these would have done a slight bit of their homework, instead of just writing nicely formatted papers, they would have noticed this.

So, if there is just no interest in serious, professional work, then I am not the guy who tries to push it by all means, for no own benefit.


----------



## Alain De Vos (Jun 26, 2022)

The ULE scheduler is better then the previous one. But still not "perfect".
The following simple command can block my desktop interactivity which should not happen. I have the impression niceness is not honoured enough.

```
/usr/bin/nice -n 15 /usr/sbin/idprio 23 /usr/local/bin/poudriere bulk
```
I'm lacking psychological knowledge but devs should be open to constructive substantiated critics.


----------



## larshenrikoern (Jun 26, 2022)

PMc said:


> Well, I do that usually when I get onto something that is specific to my installation, or might be difficult to notice. But this here is not difficult to notice. And it is not that I keep the matter fror me - I tell it publicly, here and on the lists.
> 
> There were recurring complaints on the lists, that there are certain problems with the scheduler, over years. There are generally responded with the flat answer: if you don't like the new scheduler, you are free to use the old one.
> Then you have, further up, lists of literature, from people who claim that they have not only used the scheduler, but have done research on it. If these would have done a slight bit of their homework, instead of just writing nicely formatted papers, they would have noticed this.
> ...


Sorry to hear your experience with the developers. No software is perfect and will always need improvements and bug fixing. If anyone says something else they are plain wrong. Even if they are well known. And suggesting replacing something imperfect with something they know is worse is simply stupid and ignorance.

That people have done research and academic testing is fine, but again not anything but ignorance of problems there is and always will be with any software. Again all software can be improved upon and should be over time. I can see from the changelog in main that there is less things happening than in most other areas. And there are a couple of important ones (load balancer, search for load, refactor/optimize cpu search). And then a handful of other things.

But I do miss (and that is what I could see your patch doing) a possibility to set the timer to a higher value than 1000 for realtime multimedia work. I would still suggest to contact Hans Peter Selasky. He is a nerdy person, musician and in one of his presentations on FreeBSD and sound told it was not quite good enough yet for realtime audio.

I plan in my next computer to have a lot of cores, so many that there most of the time simply will be one free. This will for sure reduce latency.


----------



## larshenrikoern (Jun 26, 2022)

Alain De Vos said:


> The ULE scheduler is better then the previous one. But still not "perfect".
> The following simple command can block my desktop interactivity which should not happen. I have the impression niceness is not honoured enough.
> 
> ```
> ...


I am currently testing your suggestion for kern.sched.preempt_thresh=120. And after removing a pair of other "optimizations" it seems to work just fine. And fulfilling my wish for the lowest possible value


----------



## mer (Jun 26, 2022)

larshenrikoern said:


> I am currently testing your suggestion for kern.sched.preempt_thresh=120. And after removing a pair of other "optimizations" it seems to work just fine. And fulfilling my wish for the lowest possible value


What optimizations did you remove?


----------



## larshenrikoern (Jun 26, 2022)

#kern.sched.slice=3
#kern.sched.interact=5
#vm.overcommit=2
#kern.ipc.shm_use_phys=1
#vfs.hirunningspace=33554432

in order to let the scheduler do its thing


----------



## larshenrikoern (Jun 28, 2022)

Hi again

Still running with kern.sched.preempt_thresh=120. And the desktop is responsive and fast.

Reenabled vm.overcommit=2 and vfs.hirunningspace=33554432 in sysctl.conf again. no issues.

Is everyone else happy with this. Or what have you experieced.


----------



## mer (Jun 28, 2022)

I've been running with preempt_thresh=121 for a little bit now (maybe a week or so) working fine for my desktop/daily usage.  If watching a youtube in firefox on one tab and bring up another tab to watch, not autoplay, but basically select and download another youtube video, the first one playing has occasional pauses, similar to a dropped frame or two, but that's it.  Audio stays fine in the first one, keyboard, mouse seem normal.  I spent a day with it set to 120, but based on what I was seeing in top with most of my user processes at PRI 20, it caused a lot more glitches than the value of 121.


----------



## larshenrikoern (Jun 28, 2022)

mer said:


> I've been running with preempt_thresh=121 for a little bit now (maybe a week or so) working fine for my desktop/daily usage.  If watching a youtube in firefox on one tab and bring up another tab to watch, not autoplay, but basically select and download another youtube video, the first one playing has occasional pauses, similar to a dropped frame or two, but that's it.  Audio stays fine in the first one, keyboard, mouse seem normal.  I spent a day with it set to 120, but based on what I was seeing in top with most of my user processes at PRI 20, it caused a lot more glitches than the value of 121.


What is your hardware. What to use depends a lot on that. On my system no glitches what so ever.

Are you sure you do not have any other modifiactions to kern.sched. On my sytem my interactive foreground processes get a higher priority. Not everything I do, needs to have a high priority. But my foreground process (what I call interactive) is getting a higher priority by itself. When I stop having them they get lower priority after a few seconds. So the scheduler does what it should do.


----------



## mer (Jun 28, 2022)

Haven't tried disabling everything else yet.
`kern.sched.steal_thresh=1
kern.sched.balance=0
kern.sched.balance_interval=1000
kern.sched.affinity=10000
kern.sched.preempt_thresh=121`

Hardware is an older NUC, CPU is i3-4010U CPU @ 1.70GHz, dual core 16GB ram.


----------



## larshenrikoern (Jun 28, 2022)

mer said:


> Haven't tried disabling everything else yet.
> `kern.sched.steal_thresh=1
> kern.sched.balance=0
> kern.sched.balance_interval=1000
> ...


Disabling the other tunables was the first thing I did. Otherwise I cannot test the scheduler in its own right. 

So try it out. It will probably make a difference


----------



## mer (Jun 28, 2022)

So I'm running with kern.sched defaults except for preempt_thresh, have that set to 120.  We'll see.


----------



## larshenrikoern (Jun 28, 2022)

mer said:


> So I'm running with kern.sched defaults except for preempt_thresh, have that set to 120.  We'll see


And maybe try with both 120 and 121 to see if that makes a difference. Your machine has limited CPU resources, so maximizing use of those might be your goal


----------



## mer (Jun 28, 2022)

Actually my goal is usually "Did I wake up above ground today?"   For computers, "does it run fast enough, does it not crash, does it run the applications I need".
Basically, I'm a simple man, coffee, dogs, fishing.


----------



## mer (Jun 28, 2022)

So running with preempt_thresh at 120 and the rest of the kern.sched seems to be "good".  An interesting thing of note is I've always gotten messages in Xorg.0.log about "event processing lagging" messages about the USB mouse.  I'm still getting them, but at a lot slower rate


----------



## Erichans (Jun 29, 2022)

mer said:


> `kern.sched.steal_thresh=1
> kern.sched.balance=0
> kern.sched.balance_interval=1000
> kern.sched.affinity=10000
> ...


What is the presumed effect of `kern.sched.balance_interval=1000`* (127 by default on my 12.3-RELEASE) when having _deactivated_ the long term load balancer with `kern.sched.balance=0` ?

What is the intended effect of `kern.sched.balance=0` on your system with regard to your system's problematic behaviour?

___
* integer kern.sched.balance_interval: Average period in stathz ticks to run the long-term balancer


----------



## mer (Jun 29, 2022)

Erichans said:


> What is the presumed effect of `kern.sched.balance_interval=1000`* (127 by default on my 12.3-RELEASE) when having _deactivated_ the long term load balancer with `kern.sched.balance=0` ?
> 
> What is the intended effect of `kern.sched.balance=0` on your system with regard to your system's problematic behaviour?
> 
> ...


Don't know exactly.  These values came from somewhere on a mailing list or somewhere here.
My systems behavior isn't problematic for me.


----------



## larshenrikoern (Jun 29, 2022)

mer said:


> So running with preempt_thresh at 120 and the rest of the kern.sched seems to be "good".  An interesting thing of note is I've always gotten messages in Xorg.0.log about "event processing lagging" messages about the USB mouse.  I'm still getting them, but at a lot slower rate


As I suggested you could try 121 again as well to see if there are even less problems with that. If so I might try it out as well. But the difference is more easy to feel on a slower system like yours. But the "recomended" value of 223 is probably to much in most places.


----------



## larshenrikoern (Jun 29, 2022)

mer said:


> Don't know exactly.  These values came from somewhere on a mailing list or somewhere here.
> My systems behavior isn't problematic for me.


kern.sched.balance_interval = Average period in stathz ticks to run the long-term balancer. Source https://gist.github.com/dch/e2ccb70254fdf401679ab5e936fd6d00#integer-kernschedbalance_interval


----------



## mer (Jun 29, 2022)

larshenrikoern said:


> As I suggested you could try 121 again as well to see if there are even less problems with that. If so I might try it out as well. But the difference is more easy to feel on a slower system like yours. But the "recomended" value of 223 is probably to much in most places.


seems to be very little difference between 120 and 121 with my standard use.  I'll probably test at some point with it at the default 80.
ETA
Agree that the 223/224 is probably too high.


----------



## larshenrikoern (Jun 29, 2022)

mer said:


> seems to be very little difference between 120 and 121 with my standard use.  I'll probably test at some point with it at the default 80.
> ETA
> Agree that the 223/224 is probably too high.


Go with the lowest setting that gives good and fluent speed. So Alain De Vos was right with his suggestion for his desktop. And I think kern.sched.preempt_thresh=120 will be my recommendation for the future.


----------



## mer (Jul 2, 2022)

So running with defaults but kern.sched.preempt_thresh=120 for about a while, seems reasonable for my nominal use.  Occasional pauses on mouse events when starting up 3 youtube videos (plus adblocker), but not objectionable.


----------



## paulw (Jul 3, 2022)

There's some relevant discussion here (an old thread, but still pretty interesting) that I found at one point when I was googling this stuff:






						SCHED_ULE should not be the default
					






					fa.freebsd.stable.narkive.com
				




I used to use kern.sched.preempt_thresh=224 as frequently recommended, but reverted to the default quite a while ago without really noticing any detrimental effects. For the last few days I've been trying 121 which (also) seems to work well on my system where I only use a window manager (cwm).

EDIT: Just for some additional context since ymmv depending on specs:

paul@zoo-FreeBSD ~ $ neofetch

...
CPU: Intel i5-9400F (6) @ 2.900GHz
GPU: GK208B [GeForce GT 710] 
Memory: 3226MiB / 16274MiB

(Re-edited for bad formatting).


----------



## PMc (Jul 3, 2022)

paulw said:


> There's some relevant discussion here (an old thread, but still pretty interesting) that I found at one point when I was googling this stuff:
> 
> 
> 
> ...


Quoting from there:


> If I recall
> correctly the most loud supporters of the notion that SCHED_BSD is faster
> than SCHED_ULE are using more threads than there are cores, causing CPU core
> contention and more importantly unevenly distributed runtimes among threads,
> resulting in suboptimal execution times for their programs.



Okay, so the really serious issue is indeed the one that I did accidentially run into and consequentially have fixed. 
Cool, the number of known and discussed issues where only I have a working fix (and nobody wants it) is increasing (IPv6 inflight refragmentation would be the latest addition to that list).


----------



## larshenrikoern (Jul 3, 2022)

PMc said:


> Quoting from there:
> 
> 
> Okay, so the really serious issue is indeed the one that I did accidentially run into and consequentially have fixed.
> Cool, the number of known and discussed issues where only I have a working fix (and nobody wants it) is increasing (IPv6 inflight refragmentation would be the latest addition to that list).


I understand your frustration. Ignorance is not a good thing. And the argument that someone has researched about this and therefore you cannot suggest changes.

If one of the developers of the ULE sees this please take a look on the patch PMc did provide way earlier in this thread.


----------



## larshenrikoern (Jul 3, 2022)

paulw said:


> There's some relevant discussion here (an old thread, but still pretty interesting) that I found at one point when I was googling this stuff:
> 
> 
> 
> ...


As you said that thread is old. And yes there are most likely workloads where the old scheduler is faster. And if you want you can use it. But I still believe that there is a reason the ULE scheduler is the default.


----------



## PMc (Jul 3, 2022)

larshenrikoern said:


> I understand your frustration.


No, I'm absolutely not frustrated. You know, if that thing would be committed, then some committer would claim the merits for it.

Furthermore, getting something committed appears to be an infinite amount of buerocracy, where you are forced onto your knees (just like Alfred tries to do with Uthred in TheLastKingdom), as if you would receive a gratefulness by being allowed to fix the bugs other complain about.


----------



## larshenrikoern (Jul 3, 2022)

PMc said:


> No, I'm absolutely not frustrated. You know, if that thing would be committed, then some committer would claim the merits for it.
> 
> Furthermore, getting something committed appears to be an infinite amount of buerocracy, where you are forced onto your knees (just like Alfred tries to do with Uthred in TheLastKingdom), as if you would receive a gratefulness by being allowed to fix the bugs other complain about.


Hi, good to know you are not frustrated 

That is the one bad thing about the BSD license, that they can take your code and claim it without any mentioning. At least you should acknowledged for your contribution. On the other hand, If they used it a problem you experience is solved. although you probably uses the patch yourself. So solved for you anyway


----------



## mer (Jul 3, 2022)

I'm guessing the patch applies cleanly to a lot of kernel versions.  Looks like it was against 12.2, I looked t 13-stable git repo earlier today and it looked like it would apply cleanly.  Did not look at 14.

BSD License:  they can take your code, use it, they cannot blame you if something goes wrong, but I don't think they can claim it as theirs.  At one point I think the BSD license actually listed all the copyright holders, but current version does not.  So PMc would own the copyright to his patch, a good committer would probably reference the source of the patch, who found the bug, etc.

My opinion on the "why ULE is the default":  The hardware.  If you look at the history, the BSD scheduler was really written for uniprocessor systems, with a different workload in mind.  Take the lowest end consumer laptop from say Dell, it has multiple cores, mutiple threads so a lot more CPU than used to be.  ULE was designed from the ground up for modern CPU, so SMP, multi cores, HTT, all the fun stuff that you would only find on an old SGI machine or some of the higher end SPARCs.  Instead of trying to backfit all that into the BSD scheduler, it got a rewrite, which is a good thing.
But there has been a lot of time, a lot of machines, a lot of different workloads that maybe the default values and corner cases in ULE need to be revisted.
I believe there are still uses for the BSD scheduler, a lot  of documentation talk about embedded, smaller footprint, etc.

Anyway, this has actually been one of the best scheduler threads I've read/participated in.  Just discussion, suggestions, no finger pointing.


----------



## PMc (Jul 3, 2022)

mer said:


> BSD License:  they can take your code, use it, they cannot blame you if something goes wrong, but I don't think they can claim it as theirs.  At one point I think the BSD license actually listed all the copyright holders, but current version does not.  So PMc would own the copyright to his patch, a good committer would probably reference the source of the patch, who found the bug, etc.


That was always the case with BSD. As this was originally created by UCB, it had to be considered a common public effort that belongs to all people. And that is quite cool, because it implicitely means "for the greater good of mankind". It is what _science_ originally was meant to be for. (But a lot of things have changed in the last 20 years regarding that.)

Now with a software of that size it is obvious that a lot of people will come along like "i have coded this and that cool feature and I want it included into the source." So it is obvious that thorough scrutiny is necessary in deciding what should go into the product and what shouldn't.
But, bugfixes were always something different, because it is not a question that bugs should be fixed. So there was always a fast-lane for bugfixes - and with some packages (e.g. sendmail) it might take less than 24 hours from sending in a bugfix to having it appear in HEAD.

This has all changed with the advent of so-called "security". Now the fast-lane is entirely reserved for "security". And all the people want security - no matter if it does them any good or not or not even concern them at all - they always want the newest security. So first and foremost of all there is security, then come new feature discussions, and finally fixing bugs (the classic QA, with the word "quality" in it) has apparently been abandoned entirely. In the best case bugfixes are treated like feature request, that is, the bugfixer must legitimate and explain why it would be desireable to fix bugs.

So, I think, anybody from the 80's or 90's, when looking at what we are doing today (and this is *not* limited to software development only), would be reminded of Dr. Tarr and Prof. Fether.


----------



## larshenrikoern (Jul 4, 2022)

PMc said:


> That was always the case with BSD. As this was originally created by UCB, it had to be considered a common public effort that belongs to all people. And that is quite cool, because it implicitely means "for the greater good of mankind". It is what _science_ originally was meant to be for. (But a lot of things have changed in the last 20 years regarding that.)
> 
> Now with a software of that size it is obvious that a lot of people will come along like "i have coded this and that cool feature and I want it included into the source." So it is obvious that thorough scrutiny is necessary in deciding what should go into the product and what shouldn't.
> But, bugfixes were always something different, because it is not a question that bugs should be fixed. So there was always a fast-lane for bugfixes - and with some packages (e.g. sendmail) it might take less than 24 hours from sending in a bugfix to having it appear in HEAD.
> ...


As in other places of human life and society's there are always room for improvement. But if no one mentions it it is less likely to happen.

I know the bsd license quite well. And yes it in principle a gift to humankind. And I like that thougth, and is the reason for me to use open suurce software in the first place.. But it should also mean that those who administer it should be open to others and let improvements happen from other people as well. But at the same time be aware of security among other things. If that is not happening it is a problem for the project (in this case FreeBSD).

But lets keep the thread on the subject: My recommendation for sysctl kern.sched.preempt_thresh is sysctl kern.sched.preempt_thresh=120 for desktop use.

Thanks to everyone who has contributed this far


----------



## bsduck (Jul 13, 2022)

That's interesting. Too technical for me to understand the details, but I've always wondered about this sysctl everyone seems to use with that same value of 224 while it's poorly documented and isn't even mentioned in the handbook. It definitely deserves better documentation.


----------



## rootbert (Jul 13, 2022)

my recommendation is kern.sched.preempt_thresh=224 ... otherwise mouseclicks are behind on my system (AMD Ryzen 7 3700X, 64GB ECC RAM, Nvidia GeForce GTX 1650, NVME)


----------



## facedebouc (Jul 13, 2022)

rootbert said:


> my recommendation is kern.sched.preempt_thresh=224 ... otherwise mouseclicks are behind on my system (AMD Ryzen 7 3700X, 64GB ECC RAM, Nvidia GeForce GTX 1650, NVME)


Following recommendation upper I have set kern.sched.preempt_thresh=120. My system is an AMD FX-6300, 8Gb RAM, Nvidia GT610, SSD sata and I don't have any issue like yours. I am using FVWM. All is fast and responsive.
Before this thread I used kern.sched.preempt_thresh=224 for years and I haven't seen any difference for now for my desktop use.


----------



## mer (Jul 13, 2022)

I think a lot of what you see is based on what you do.  Someone doing stuff with real time video/audio is more sensitive to stutters in a player, drops in a stream that someone writing code or simply browsing with Firefox.  I also think that your specific hardware plays into "what is a good value".

For me, this thread has been a good test of "value X has been recommended for years, but value Y works a lot better for me.  Why?"  
Understanding the meaning/impact of the value on the scheduler and how it manifests in your typical workload is a key towards "what is the best value for me".


----------



## larshenrikoern (Jul 14, 2022)

rootbert said:


> my recommendation is kern.sched.preempt_thresh=224 ... otherwise mouseclicks are behind on my system (AMD Ryzen 7 3700X, 64GB ECC RAM, Nvidia GeForce GTX 1650, NVME)


Do you have other kern.sched**** values in your syctl.conf ?? If so please disable them and test again


----------



## bob2112 (Jul 17, 2022)

I posted something and then changed my mind, but couldn't find a way to delete it.


----------

