# Tweaking FreeBSD for Desktop Use - kern.sched.preempt_thresh



## poorandunlucky (Nov 23, 2017)

When looking at how to tweak FreeBSD for desktop use, the most common tweak is this:


```
sysctl kern.sched.preempt_thresh=224
Default: 80
```

This, supposedly, does something with X Windows, allowing it to run more smoothly when CPU use is very high... or something like that...

My question is:  What does it mean, and what does it do?

Also, I can't set it in /boot/loader.conf... how do I set it?

Also, any things you'd recommend doing, before a custom kernel, to optimize FreeBSD for desktop use?  Anything you did you want to share?

I know of tuning(7), but it's not exactly a stimulating read...  : <


----------



## sidetone (Nov 23, 2017)

See sysctl.conf(5), and you would use something like 
	
	



```
kern.sched.preempt_thresh=224
```
 in /etc/sysctl.conf.


----------



## tobik@ (Nov 23, 2017)

poorandunlucky said:


> Also, any things you'd recommend doing, before a custom kernel, to optimize FreeBSD for desktop use?


Use the defaults first to find out where your pain points with them really are. Then look for solutions for each one in a directed manner iff you have any problems.


----------



## poorandunlucky (Nov 23, 2017)

sidetone said:


> See sysctl.conf(5), and you would use something like
> 
> 
> 
> ...



Ohh...  I should've checked...  : \  Thanks...

How come some get set in /boot/loader.conf, though?  Why don't they all get set at the same place?



tobik@ said:


> Use the defaults first to find out where your pain points with them really are. Then look for solutions for each one in a directed manner iff you have any problems.



My only thing is that sometimes when the CPU is really being drained, like when building ports on all threads or something like that, X isn't really responsive, and I think that's what *kern.sched.preempt_thresh* was supposed to do, but I'm not sure exactly what it means, nor what value to set it at...

I'm not even sure if it does help, or if it's just coincidental...


----------



## tobik@ (Nov 23, 2017)

poorandunlucky said:


> My only thing is that sometimes when the CPU is really being drained, like when building ports on all threads or something like that, X isn't really responsive, and I think that's what kern.sched.preempt_thresh was supposed to do, but I'm not sure exactly what it means, nor what value to set it at...


I wish I could tell you but I don't know.  However it might suffice if you just reduce the number of builders or make jobs that run in parallel. If you use Poudriere that's configurable in /usr/local/etc/poudriere.conf via PARALLEL_JOBS. It's a good idea anyway if you use the build machine for something other than building packages.


----------



## sidetone (Nov 23, 2017)

poorandunlucky said:


> Ohh...  I should've checked...  : \  Thanks...


You didn't know where to look, but it became obvious in hindsight. I use `man -k`, with `grep` to look for commands: many are based on their filenames.


> How come some get set in /boot/loader.conf, though?  Why don't they all get set at the same place?


Loader is mostly for what's needed to start basic services, and needed before everything else, such as what's needed for single user mode. /boot is on its own partition, or the first parts of your drive, and that's needed first. If it is needed to boot everything else, it won't work in /etc/rc.conf. In the words from loader.conf(5), it's a bootstrap. Your window manager isn't a lower level service, so stuff relating to it can go into /etc/rc.conf. Some services that seem they would fit into loader.conf actually go into rc.conf, because they can work like that.



> My only thing is that sometimes when the CPU is really being drained, like when building ports on all threads or something like that, X isn't really responsive, and I think that's what kern.sched.preempt_thresh was supposed to do, but I'm not sure exactly what it means, nor what value to set it at...
> 
> I'm not even sure if it does help, or if it's just coincidental...


IDK if this applies, but if you're using a transparent terminal, it will take up more resources. It can also be because of a heavyweight window manager. Long compile times are typical on FreeBSD, but it's at least more efficient than other operating systems. You can use tmpfs(5) (not md(4) with tmpmfs which is also a RAM filesystem, but inefficient for this purpose) in /etc/rc.conf and /etc/fstab, to use RAM for faster compiling. If that is not enough, perhaps you can upgrade your CPU.

I use this for mounting a RAM compiling directory
/etc/fstab

```
tmpfs           /usr/obj        tmpfs   rw,mode=1777    0 0 # for kernel & world
tmpfs           /tmp            tmpfs   rw,mode=1777    0 0 # need config below for ports
```
You can add /var/tmp to the above, or you can softlink it to /tmp. A swap file system will also be needed, if there is not enough RAM for port compiles.

In /etc/make.conf

```
WRKDIRPREFIX=/tmp/bport
```
or any chosen filename is needed for ports.


----------



## poorandunlucky (Nov 23, 2017)

sidetone said:


> You didn't know where to look, but it became obvious in hindsight. I use `man -k`, with `grep` to look for commands: many are based on their filenames.
> 
> Loader is mostly for what's needed to start basic services, and needed before everything else, such as what's needed for single user mode. /boot is on its own partition, or the first parts of your drive, and that's needed first. If it is needed to boot everything else, it won't work in /etc/rc.conf. In the words from loader.conf(5), it's a bootstrap. Your window manager isn't a lower level service, so stuff relating to it can go into /etc/rc.conf. Some services that seem they would fit into loader.conf actually go into rc.conf, because they can work like that.
> 
> ...



Poudriere does that, compile in RAM, I mean...  The window manager being unresponsive is just what other people were saying that setting helped, I'm not sure if it really changes anything, I'd have to try twice the same thing, which I haven't...

I actually have another problem semi-related to compiling in RAM with Poudriere... it runs out of memory, and it creates a kernel panic, and everything just stops working...

I set Poudriere to use 6 builders, and 2 GB of RAM for the work dir + 1 GB of RAM for jails instead of 8 + unlimited to prevent that happening, but I'm not sure if that's optimal...

I'd rather just properly manage swap and resources than set limits for a particular program... like this is a big problem, IMO...

Here's the thread I'm talking about: https://forums.freebsd.org/threads/63384/


----------



## phoenix (Nov 24, 2017)

/boot/loader.conf is for setting things *before* the kernel is loaded.  This is mainly for things that can't be changed once the kernel is loaded.

/etc/sysctl.conf is for setting things *after* the kernel is loaded but *before* any users have logged in.

sysctl(8) is for settings things *after* the kernel is loaded and *after* a user has logged in.  Mostly for changing/testing things on the fly before making the setting permanent by adding it to /etc/sysctl.conf.

The easy way to find out where the setting goes:  try to change it form a shell using sysctl(8).  If you get an error about the tunable being read-only, then it goes into loader.conf.  If the change takes effect, then it goes in sysctl.conf.


----------



## rigoletto@ (Nov 25, 2017)

> Poudriere does that, compile in RAM, I mean...



Just in case, you need to set it in poudriere.conf


```
# Use tmpfs(5)
# This can be a space-separated list of options:
# wrkdir    - Use tmpfs(5) for port building WRKDIRPREFIX
# data      - Use tmpfs(5) for poudriere cache/temp build data
# localbase - Use tmpfs(5) for LOCALBASE (installing ports for packaging/testing)
# all       - Run the entire build in memory, including builder jails.
# yes       - Enables tmpfs(5) for wrkdir and data
# no        - Disable use of tmpfs(5)
# EXAMPLE: USE_TMPFS="wrkdir data"
USE_TMPFS="wrkdir data"
```



> I actually have another problem semi-related to compiling in RAM with Poudriere... it runs out of memory, and it creates a kernel panic, and everything just stops working...



You probably are running without SWAP, otherwise it will be pretty slow but not actually break. Or, you have a faulty memory.


----------



## poorandunlucky (Nov 25, 2017)

lebarondemerde said:


> Just in case, you need to set it in poudriere.conf
> 
> 
> ```
> ...




```
vm.swap_enabled: 1
vm.disable_swapspace_pageouts: 0
vm.swap_idle_enabled: 1
vm.stats.vm.v_swappgsout: 0
vm.stats.vm.v_swappgsin: 0
vm.stats.vm.v_swapout: 0
vm.stats.vm.v_swapin: 0
vm.swap_idle_threshold2: 10
vm.swap_idle_threshold1: 2
vm.nswapdev: 1
vm.swap_async_max: 4
vm.swap_maxpages: 64926752
vm.swap_reserved: 10847272960
vm.swap_total: 5368709120
```

I think swap is enabled, but I'm not sure if it's enabled properly...  I never see anything in it...

Also there was the ZFS ARC that was set to take up to 14 GB of my 16... and it didn't feel nimble...  I just let the system run all night after a reboot, and there it was this morning, using all it could...  : \

I'll see if that helps... hopefully I don't have any bad RAM...


----------



## sidetone (Nov 25, 2017)

poorandunlucky said:


> I think swap is enabled, but I'm not sure if it's enabled properly...  I never see anything in it...
> 
> Also there was the ZFS ARC that was set to take up to 14 GB of my 16... and it didn't feel nimble...  I just let the system run all night after a reboot, and there it was this morning, using all it could...  : \
> 
> I'll see if that helps... hopefully I don't have any bad RAM...



`swapinfo -m`

Swap is enabled through fstab(5). There are a few examples in that manpage, some with encryption. If you don't have a dedicated hard drive partition for swap, you can use a filename for it, but that will wear out parts of your hard drive faster.

A book on FreeBSD would help you a lot, but you can also learn a lot without it.


----------



## rigoletto@ (Nov 25, 2017)

> If you don't have a dedicated hard drive partition for swap, you can use a filename for it, but that will wear out parts of your hard drive faster.



This is specially bad when using ZFS, it will work but will bring more memory issues.



> A book on FreeBSD would help you a lot, but you can also learn a lot without it.



Absolute FreeBSD.


----------



## poorandunlucky (Nov 25, 2017)

sidetone said:


> `swapinfo -m`
> 
> Swap is enabled through fstab(5). There are a few examples in that manpage, some with encryption. If you don't have a dedicated hard drive partition for swap, you can use a filename for it, but that will wear out parts of your hard drive faster.
> 
> A book on FreeBSD would help you a lot, but you can also learn a lot without it.



Yeah, swap is enabled... idk why it's never used, though... probably because it waits to be out of RAM, as I'd expect...  I have 16 GB of RAM, which is plenty for a single user, I only set 5 GB aside in case of overflow...  Though I set it to swap out idling processes, though... though maybe there's just none...


```
Device          1M-blocks     Used    Avail Capacity
/dev/ada0p2.eli      5120        0     5120     0%
```


```
################################################################################
# DogeOS 0.0.1
# Filesystems Table
# ------------------------------------------------------------------------------
# Configures filesystems to mount at boot time, and to automount when available.
# ==============================================================================

# ====================================================================================================
# Device                Mountpoint              FStype          Options                 Dump    Pass
# ----------------------------------------------------------------------------------------------------
# NTFS Partition from old Windows installation (mounts read-only only).
# To be made into a more compatible filesystem as soon as possible.
#
#/dev/ada1s1            /mnt                    fuse.ntfs       ro,noauto               0       0
#
# Swap space.
#
/dev/ada0p2.eli         none                    swap            sw                      0       0
#
# procfs
#
#proc                   /var/proc               procfs          rw                      0       0
#
# ====================================================================================================

# ==============================================================================
# DogeOS 0.0.1 - Yoonix | poorandunlucky <poorandunlucky@boardermail.com>
################################################################################
```

Yes, I have very neat configuration files, I know


----------



## poorandunlucky (Nov 27, 2017)

I still don't know what preemption scheduling is or means...


----------



## Beastie (Nov 27, 2017)

poorandunlucky said:


> Yeah, swap is enabled... idk why it's never used, though... probably because it waits to be out of RAM, as I'd expect...  I have 16 GB of RAM, which is plenty for a single user, I only set 5 GB aside in case of overflow...


Yes, swap is not used when it's not needed. My machine has 8GB and it rarely swaps even when compiling or copying hundreds of GBs of data.




poorandunlucky said:


> I still don't know what preemption scheduling is or means...


Preemptive scheduling is a type of process management algorithm. It simply means that the kernel allocates execution time to each process and interrupts the execution of the currently-running process when its time's up. How that is done depends on the kernel type and the specific algorithm chosen for its process manager.
This sysctl tunable is just a parameter that tweaks the behavior of the scheduler.


----------



## Snurg (Nov 27, 2017)

Back to topic...


poorandunlucky said:


> ```
> sysctl kern.sched.preempt_thresh=224
> ```
> My question is:  What does it mean, and what does it do?


It obviously determines the threshold preemptive multitasking sets in, in case a process doesn't yield back cpu in time.
This reminds me of the old kernel option HZ=100, which many tuned to HZ=1000 or higher to reduce the latency.
I used it myself in FreeBSD 5, but nowadays it is no longer in GENERIC and only mentioned indirectly in NOTES. So I guess it is obsolete.


----------



## poorandunlucky (Nov 28, 2017)

Beastie, Snurg:

So this value determines how much time a job can run on the CPU before it's suspended, and other tasks in the queue can run (all depending on their niceness, and affinity, of course), and if they don't complete in the time that's given to them, they're also suspended, and back to the first suspended job, until all jobs complete... right?


```
kern.sched.topology_spec: <groups>
kern.sched.steal_thresh: 2
kern.sched.steal_idle: 1
kern.sched.balance_interval: 127
kern.sched.balance: 1
kern.sched.affinity: 1
kern.sched.idlespinthresh: 157
kern.sched.idlespins: 10000
kern.sched.static_boost: 152
kern.sched.preempt_thresh: 224
kern.sched.interact: 30
kern.sched.slice: 12
kern.sched.quantum: 94488
kern.sched.name: ULE
kern.sched.preemption: 1
kern.sched.cpusetsize: 32
```

These are the other tunables, for reference, if you guys wonder what else there is...

SirDice, (and others): Do you know if there's a document somewhere that lists and explains all the system variables and tunables (better than sysctl -ad)... something I could understand?


----------



## rigoletto@ (Nov 28, 2017)

Trying to tweak too much always and up in slowness and mess.

This is my entire /etc/sysctl.conf

```
net.inet.ip.random_id=1
kern.coredump=0
hw.snd.default_unit=5
hw.snd.feeder_rate_quality=4
hw.snd.vpc_0db=100
hw.syscons.bell=0
kern.sched.preempt_thresh=224
kern.maxfiles=450000
kern.maxvnodes=250000
```


----------



## poorandunlucky (Nov 28, 2017)

lebarondemerde said:


> Trying to tweak too much always and up in slowness and mess.
> 
> This is my entire /etc/sysctl.conf
> 
> ...



What are vnodes?


----------



## rigoletto@ (Nov 28, 2017)

See: Tuning Kernel Limits & ZFSTuningGuide.

EDIT: I actually don't need `kern.maxvnodes=250000` anymore. With my current hardware the default is `kern.maxvnodes: 338007`.


----------



## poorandunlucky (Nov 28, 2017)

lebarondemerde said:


> See: Tuning Kernel Limits & ZFSTuningGuide.



Looks like my question was on point...


```
M6500# sysctl vfs.numvnodes
vfs.numvnodes: 348964
M6500# sysctl kern.maxvnodes
kern.maxvnodes: 348995
```

I set it to 350,000... says to increase by 1,000 in the guide...  *shrugs*


----------



## rigoletto@ (Nov 28, 2017)

If you run off vnodes it will appear in /var/log/messages.


----------



## PacketMan (Nov 30, 2017)

poorandunlucky said:


> My only thing is that sometimes when the CPU is really being drained, like when building ports on all threads or something like that, X isn't really responsive,.......





poorandunlucky said:


> I actually have another problem semi-related to compiling in RAM with Poudriere... it runs out of memory, and it creates a kernel panic, and everything just stops working...



When I go to 'rebuild' any of my machines, I tell my family I am doing maintence. I edit /etc/rc.conf and comment out all the stuff that starts up including desktop. Reboot. Now I got a machine with not much running, just the OS and terminal sessions.  I run ports-mgmt/synth, the compiling eats ram like crazy, still get some swapping, but the machine plows through it like a heavy-duty purring diesel engine.  After I am done with the rebuilt I uncomment the stuff in edit /etc/rc.conf and reboot.  Always ask yourself and think through: "What's the problem you are trying to fix?".

Just a thought.


----------



## azathoth (Nov 30, 2017)

poorandunlucky said:


> When looking at how to tweak FreeBSD for desktop use, the most common tweak is this:
> 
> 
> ```
> ...




Who is the guy in your profile pic?


----------



## wolffnx (Nov 30, 2017)

poorandunlucky said:


> Beastie, Snurg:
> 
> So this value determines how much time a job can run on the CPU before it's suspended, and other tasks in the queue can run (all depending on their niceness, and affinity, of course), and if they don't complete in the time that's given to them, they're also suspended, and back to the first suspended job, until all jobs complete... right?
> 
> ...



I' changed


```
kern.sched.slice to 3
and kern.sched.interact to 5
```

and a notice a little speedup when starting applications


----------



## wolffnx (Nov 30, 2017)

poorandunlucky said:


> When looking at how to tweak FreeBSD for desktop use, the most common tweak is this:
> 
> 
> ```
> ...




`rtprio` is your friend when you are compiling ports and doing others task

from the man page:

*To    make depend while not disturbing other machine usage:
      idprio 31 make depend*

rtprio()


----------



## poorandunlucky (Dec 14, 2017)

PacketMan said:


> When I go to 'rebuild' any of my machines, I tell my family I am doing maintence. I edit /etc/rc.conf and comment out all the stuff that starts up including desktop. Reboot. Now I got a machine with not much running, just the OS and terminal sessions.  I run ports-mgmt/synth, the compiling eats ram like crazy, still get some swapping, but the machine plows through it like a heavy-duty purring diesel engine.  After I am done with the rebuilt I uncomment the stuff in edit /etc/rc.conf and reboot.  Always ask yourself and think through: "What's the problem you are trying to fix?".
> 
> Just a thought.



I fixed those things, they were unrelated to this...



azathoth said:


> Who is the guy in your profile pic?



Who?  Me?  ... me?  ... Who else could it be?

... D ;



juan9182 said:


> I' changed
> 
> 
> ```
> ...



Thanks, I'll try that...


----------

