# Limiting resources (CPU, memory) for Jails - rctl



## ikevinjp (Jan 13, 2012)

Is rctl the tool to use limit CPU and memory resources for jails? Is it stable and working reliably?

http://wiki.freebsd.org/Hierarchical_Resource_Limits

I've also seen other methods:

http://wiki.freebsd.org/JailResourceLimits
http://www.tomjudge.com/index.php/FreeBSD/Jails/MemoryLimits

Could anyone advise what is the best way to limit CPU and memory for each individual jails?

Thanks!

Kevin.


----------



## jake (Jan 20, 2012)

I'd say rctl(8) is the way to go, the other methods require kernel/userland patch and recompile, that will most likely break when you upgrade and the patches will probably only work on the revision of the code they were created for.

This quarterly report suggests the code is basically considered to be stable, http://www.freebsd.org/news/status/report-2011-01-2011-03.html#RCTL,-aka-Resource-Containers I don't think %cpu time made it in time for 9.0-RELEASE, but maybe soon?



> Most of the code has already been merged into CURRENT. There are two remaining problems I would like to solve before 9.0-RELEASE - see below - but otherwise, the code is stable; please test and report any problems. You will need to rebuild the kernel with "options RACCT" and "options RCTL". The rctl(8) manual page should be a good introduction on how to use it.



Also have a look at cpuset(1), it will let you assign one or more cpu core to a jail. For example, to assign CPU0 and CPU2 to jailid 3:

`# cpuset -l 0,2 -j 3`

More examples in the man page, let us know how you get on!


----------



## ikevinjp (Feb 4, 2012)

Hi, 

I've just installed FreeBSD 9.0 and noticed that when I did a [cmd=]which rctl[/cmd] it replies with the path 
	
	



```
/usr/bin/rctl
```

Please clarify if I really need to rebuild the kern*e*l to use rctl. 

Thanks!

Kevin.


----------



## jake (Feb 5, 2012)

Yes, you will need to rebuild your kernel with the relevant options. There doesn't seem to be a way to load it as a module via kldloadyet.

The userland tool /usr/bin/rctl is included in the default install, but it will not work with out modification to GENERIC kernel. If you do attempt to use it you will get an error with an exit code of 1:
`# /usr/bin/rctl ; echo $?`

```
rctl: rctl_get_rules: Function not implemented
1
```
For completeness the steps needed to do this:

Create a new kernel config file containing something like this.
`# cat /usr/src/sys/amd64/conf/RCTL`

```
include         GENERIC
ident           RCTL

options         RACCT
options         RCTL
```
`# cd /usr/src`
`# make buildkernel KERNCONF=RCTL`
`# make installkernel KERNCONF=RCTL`
`# reboot`

After reboot /usr/bin/rctl will function as intended.


----------



## ikevinjp (Oct 8, 2012)

As we are nearing the release of FreeBSD version 9.1, are there any improvements over rctl in this release?


----------



## trasz@ (Oct 27, 2012)

Yes, there were some important fixes.  No new functionality, though.  And 9-STABLE should get CPU percentage limits in a month or so (they are already there in 10-CURRENT).


----------



## ikevinjp (Oct 28, 2012)

trasz@:

Thanks for the update, you said "9-STABLE should get CPU percentage limits in a month or so"... do you mean I can expect this is the 9.1 release expected in December (or soon)? I'll happily rejoice if so... 

Thanks!

Kevin.


----------



## trasz@ (Oct 28, 2012)

No, unfortunately those changes won't be in 9.1-RELEASE.  9.1 was frozen for a month or so now.

Note that Rudo Tomori - the guy who implemented %CPU limits - has prepared a patch against 9-STABLE; see http://wiki.freebsd.org/SummerOfCode2012/CPULimits


----------



## ikevinjp (Jan 14, 2013)

*Does not work in 9.1*



			
				jake said:
			
		

> Yes, you will need to rebuild your kernel with the relevant options. There doesn't seem to be a way to load it as a module via kldloadyet.
> 
> The userland tool /usr/bin/rctl is included in the default install, but it will not work with out modification to GENERIC kernel. If you do attempt to use it you will get an error with an exit code of 1:
> `# /usr/bin/rctl ; echo $?`
> ...



Well, I tried to do the same thing in FreeBSD 9.1 and it failed:


```
/usr/src/sys/amd64/acpica/acpi_switch.S: Assembler messages: 
/usr/src/sys/amd64/acpica/acpi_switch.S:146: Error: no such instruction: 'xsetbv'
/usr/src/sys/amd64/acpica/acpi_switch.S:147: Error: no such instruction: 'xrstor (%rbx)'
*** [acpi_switch.o] Error code 1

Stop in /usr/obj/usr/src/sys/RCTL.
*** [buildkernel] Error code 1

Stop in /usr/src
*** [buildkernel] Error code 1

Stop in /usr/src.
```

Note: I was trying to test it by installing under VMware Fusion on OS X.


----------



## trasz@ (Jan 14, 2013)

This doesn't seem to be related to RACCT/RCTL, but rather to some newer ACPI code being incompatible with older (9.0?) compiler.  Please rebuild the world and then the kernel.


----------



## zennybsd (Feb 23, 2013)

*any update RCTL patch to FreeBSD 9.1?*

@trasz@: Great work! 

Is there any update on the RCTL patch to FreeBSD 9.1? 

Resource allocation is the main reason that prevented me to adopt jails from OpenVZ. BTW, is there someone using RCTL+jails in a production? Any use-case?

Thanks!


----------



## Ikinoki (Sep 26, 2013)

@trasz@,

One question, the pcpu limit works only on one core. Will there be a way to limit CPU cores also? Or maybe make the pcpu limit with a sum of all cores (8x100 - 800%)?


----------



## trasz@ (Oct 1, 2013)

The method you suggest - e.g. 800% - is supposed to be working just fine.  Doesn't it?


----------



## Ikinoki (Oct 2, 2013)

I will test it and reply here.


----------



## Ikinoki (Oct 22, 2013)

@trasz@,

I found one problem with rctl and I don't know how to go around it.

See, I have sshd with chroot, and of course it hits the limit that is set for service users (pcpulimit deny=10/proc). This heavily deteriorates the performance of sshd. So what I wanted to know if either you could match a specific process name or better to just ignore rules for particular process names?

I really don't know how to figure this out: if I set racct limit only for the user - it breaks SSH performance because. if I don't set the limits, users go havoc in Apache chroot jails...


----------



## Ikinoki (Oct 22, 2013)

One way is to run the processes in jails.


----------



## ikevin8me (Jan 22, 2014)

How is the status of rctl in FreeBSD 10? Is it built in, or do we need to rebuild the kernel?

This page http://www.freebsd.org/doc/handbook/security-resourcelimits.html seems to be describing that it can limit memory usage for jails. Anyone using this (or plan to use it) for production mode?


----------



## User23 (Feb 25, 2014)

ikevin8me said:
			
		

> How is the status of rctl in FreeBSD 10? Is it built in, or do we need to rebuild the kernel?
> 
> This page http://www.freebsd.org/doc/handbook/security-resourcelimits.html seems to be describing that it can limit memory usage for jails. Anyone using this (or plan to use it) for production mode?



A custom kernelbuild with


```
options         RACCT
options         RCTL
```
is still needed to use `rctl`.

We already using `cpuset` to limit number of cpu cores per jail and we'll try to limit the memory usage by using `rctl` soon.


----------



## ikevin8me (Nov 5, 2020)

What is currently the best way to limit CPU and memory for jails for the latest version of FreeBSD (12.2)?


----------



## SirDice (Nov 5, 2020)

The same thing that was posted 8 years ago.


----------

