# Swap warnings after upgrade to 11.2



## mickey (Jul 7, 2018)

I am getting these warnings about swap space since I upgraded an old i386 machine from FreeBSD 11.1-p10 to 11.2 from source tree:

```
warning: total configured swap (524288 pages) exceeds maximum recommended amount (509152 pages).
warning: increase kern.maxswzone or reduce amount of swap.
warning: total configured swap (1048576 pages) exceeds maximum recommended amount (509152 pages).
warning: increase kern.maxswzone or reduce amount of swap.
```
The machine has 1G RAM installed and has two GPT partitioned hard drives, each with a 2G swap area. A ratio of 1:4 is probably more than sufficient but I would not call it excessive. And as you can see even one 2G partition already exceeds the recommended amount. The setting of kern.maxswzone also does not appear too small and I don't think I've ever messed with the default value:

```
kern.maxswzone: 36175872
```
Any idea why I now get those warnings? Did I maybe miss to upgrade something in my cutom kernel config or elsewhere?


----------



## rigoletto@ (Jul 7, 2018)

I am also getting this message on my home server (2GB of RAM, 4B of SWAP) with 11.2; however I did not invested time to investigate it yet. This is my default in there:


```
kern.maxswzone: 0
```

(EDIT)

So, I set mine to `kern.maxswzone="15728640"` and that shut the message. Why, I don't know.


----------



## chrbr (Jul 7, 2018)

I have got that warnings as well, may be it has been after upgrade from FreeBSD10.3 to FreeBSD11.0. At that time I have had 2G of RAM and 8G of swap space. I have searched in the internet and found that too much swap space can be contra productive. I have reduced the size of the swap to 6G and never run out of swap space working with the system.


----------



## trev (Jul 7, 2018)

From the arm mailing list via Mark Millard:



> "man 8 loader" and its kern.maxswzone material,
> 
> Note that swap metadata can be fragmented, which means that
> the system can run out of space before it reaches the
> ...


----------



## mickey (Jul 7, 2018)

The more interesting part is this:

```
Limits the amount of KVM to be used to hold swap metadata,
                   which directly governs the maximum amount of swap the
                   system can support, at the rate of approximately 200 MB of
                   swap space per 1 MB of metadata.  This value is specified
                   in bytes of KVA space.  If no value is provided, the system
                   allocates enough memory to handle an amount of swap that
                   corresponds to eight times the amount of physical memory
                   present in the system.
```
Considering above, 2G of swap or even 4G is hardly anywhere near _eight times the amount of physical memory present in the system_. Also when my kern.maxswzone reads 36175872 it should roughly support 36175872 / 1048576 * 200 = 6900 Mb of swap space, according to above mentioned math. But none of this explains why a system with 1G ram complains with 2G swap, which is just ridiculous.


----------



## trev (Jul 8, 2018)

```
sysctl kern.maxswzone
kern.maxswzone: 0
```

The default _is_ zero, so:



> If no value is provided, the system allocates enough memory to handle an amount of swap that
> corresponds to eight times the amount of physical memory present in the system.



The above statement now appears incorrect (as of FreeBSD 9.2 - _change to 0 was merged into 9-stable in September 2012_).


----------



## mickey (Jul 8, 2018)

trev said:


> ```
> sysctl kern.maxswzone
> kern.maxswzone: 0
> ```
> ...


I have absolutely no idea, where the system gets the value of 36175872 from, which is the value sysctl reports for kern.maxswzone when the system is running. Maybe i386 has a different default value than amd64? I double checked /boot/loader.conf as well as /etc/sysctl.conf it's not in there. I even grepped through all the files in /boot and /etc and found nothing. My custom kernel config also contains nothing swap related, the only VM related stuff in there is some ZFS tuning, namely:

```
options         KVA_PAGES=512
options         KSTACK_PAGES=4
```
But that aside, the value should still be large enough to allow for roughly 6900 Mb swap space. A single 2G swap partition should not even exceed half of that maximum, yet the system complains once for the first 2G swap partition and once more for the second.


----------



## PMc (Jul 8, 2018)

Same here after upgrading 11.1-p10 to 11.2. i386, 2G ram, 8G swap.
`warning: total configured swap (2097152 pages) exceeds maximum recommended amount (1022784 pages).
warning: increase kern.maxswzone or reduce amount of swap.`

Increasing `kern.maxswzone` does not change anything. Default is:
`kern.maxswzone: 36175872`
(on amd64 the default appears to be 0)

This is hardcoded in sys/i386/include/param.h, independent of installed ram:

```
* 276 is sizeof(struct swblock), but we do not always have a definition
* in scope for struct swblock, so we have to hardcode it.  Each struct
* swblock holds metadata for 32 pages, so in theory, this is enough for
* 16 GB of swap.  In practice, however, the usable amount is considerably
* lower due to fragmentation.
*/
#ifndef VM_SWZONE_SIZE_MAX
#define VM_SWZONE_SIZE_MAX      (276 * 128 * 1024)
#endif
```

Various posts (and the code, and my experience) appear to say that kern.maxswzone can only be set to reduce the value, not increase it. So it seems at least the warning message is bogus.
Anyway, supporting a maximum swap of 2*mem seems too small. So something else is also wrong here.

Glancing through the code shows more severe changes - and the math triggering the warning seems to be changed so that it now triggers at a 1/4 of the former value. I might tend to assume that the error is on the side of the warning, not the behaviour.

Anyway, it seems possible to watch the actual usage:

```
# vmstat -z | grep swblk
ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP
swblk:                  136, 492826,   36464,    6978,  337344,   0,   0
```


----------



## mickey (Jul 11, 2018)

PMc said:


> Same here after upgrading 11.1-p10 to 11.2. i386, 2G ram, 8G swap.
> `warning: total configured swap (2097152 pages) exceeds maximum recommended amount (1022784 pages).
> warning: increase kern.maxswzone or reduce amount of swap.`
> 
> ...



In the meantime I also upgraded my desktop machine to RELENG-11.2 and can confirm that _kern.maxswzone_ is indeed 0 on an amd64 system.



PMc said:


> This is hardcoded in sys/i386/include/param.h, independent of installed ram:
> 
> ```
> * 276 is sizeof(struct swblock), but we do not always have a definition
> ...


I remember having read something similar somewhere, that the value can only be decreased but not increased, but haven't tested it myself, mainly because I don't even think the value is too small in the first place. I agree that this must be the (probably erroneous) result of code changes to /sys/vm/swap_pager.c and friends.



PMc said:


> Anyway, it seems possible to watch the actual usage:
> 
> ```
> # vmstat -z | grep swblk
> ...


Do I need to be concerned about the low number of free blocks?

```
ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP
swblk:                   72, 127288,    2489,      31,    2752,   0,   0
```


----------



## trev (Jul 11, 2018)

It is probably worth lodging a PR (https://bugs.freebsd.org/bugzilla/) at least against the man page for loader.

BTW, the default value for kern.maxswzone is also 0 for ARM64 (RPi3).


----------



## PMc (Jul 11, 2018)

mickey said:


> Do I need to be concerned about the low number of free blocks?
> 
> ```
> ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP
> ...



One would need to read about the working of the uma thing (but I have to catch a flight just now) - I would suppose (with no certainty) that the FREE column is space that was already used and stays allocated until mem shortage. I would suppose that reaching the LIMIT is what would give a problem.

But yesterday my desktop (amd64, 4g ram, 10g swap, no warning) has slowly run into complete unresponsiveness after opening some 50 tabs in Firefox. Such has never happened before, but Firefox ESR was upgraded to 60 which now consumes immense mem - and just now I see the these (lots of) errors in the log:
`Jul 10 16:34:26 <kern.crit> disp kernel: swap_pager: I/O error - pageout failed; blkno 796755,size 32768, error 12`
If that were an actual disk I/O error, and reset was pushed, how could these errors have made it to /var/log/messages ? And error12 is ENOMEM. So there seems to be more crap upcoming...


----------



## Datapanic (Jul 11, 2018)

Related, I filed a bug report for FreeBSD 11.2 amd64 VM's failing to boot because if the VM was configured with 256MB of ram, /etc/rc.conf would not be written to disk during the install. 

I also noted that the swap size would be different if I configured a VM with 384MB of ram and either a 12GB or 32GB disk.  if I set a 12GB disk, then swap was set at 628736K but if I set a 32GB disk, then swap was set at 1677312K. 

To reproduce...

ESXi 5.5
VM with 1 CPU
12GB DISK
256MB RAM
GPT default partitions
DOC, SCR and PORTS

reboot after install and there is no /etc/rc.conf.

Works when setting ram to 384MB

:fishy:


----------



## Lamia (Jul 11, 2018)

Welcome to the world of FreeBSD 11.2. I was welcomed by the "get_swap_......." messages too after an upgrade. It seems to be a  nicer substitute for "swap_pager_getswapspace". Then, I was completely locked out via ssh. All fixes for the SSH failed; so I decided [and needed] to build kernel/world.

I have successfully built 11.1 world/kernel on a local machine running the same version without any hacks, but not 11.2 on a VM [already upgraded to 11.2]. The most problem is with one or two header files - xmmintrin.h for 11.1 & wmmintrin.h for 11.2 on the VM. Most suggestions  from the FreeBSD bugzilla here failed. Basically, copying the clang files/folders  to /usr/include did not work. I am now trying to use the lang/clang-devel [which comes with llvm-devel as a dependency]. Of course, that looks more promising as a devel/header package.


----------



## mickey (Jul 12, 2018)

PMc said:


> One would need to read about the working of the uma thing (but I have to catch a flight just now) - I would suppose (with no certainty) that the FREE column is space that was already used and stays allocated until mem shortage. I would suppose that reaching the LIMIT is what would give a problem.


Yes, it appears to be that way indeed. After a fresh boot I get 0 free blocks:

```
ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP
swblk:                   72, 256200,       0,       0,       0,   0,   0
```
I just upgraded the machine to 2G ram for the sake of testing, but it's still complaining about the size of the swap partitions. Well at least now I only get one such warning. Man this thing is picky:

```
warning: total configured swap (1048576 pages) exceeds maximum recommended amount (1024800 pages).
warning: increase kern.maxswzone or reduce amount of swap.
```
Interestingly now the swap size reported by `swapinfo` and `top` seems to match. I could have sworn that with only 1G ram `top` displayed a slightly smaller amount than `swapinfo`:

```
swapinfo -h:

Device          1K-blocks     Used    Avail Capacity
/dev/ada0p2       2097152       0B     2.0G     0%
/dev/ada1p2       2097152       0B     2.0G     0%
Total             4194304       0B     4.0G     0%

top:

Swap: 4096M Total, 4096M Free
```



PMc said:


> But yesterday my desktop (amd64, 4g ram, 10g swap, no warning) has slowly run into complete unresponsiveness after opening some 50 tabs in Firefox. Such has never happened before, but Firefox ESR was upgraded to 60 which now consumes immense mem - and just now I see the these (lots of) errors in the log:
> `Jul 10 16:34:26 <kern.crit> disp kernel: swap_pager: I/O error - pageout failed; blkno 796755,size 32768, error 12`
> If that were an actual disk I/O error, and reset was pushed, how could these errors have made it to /var/log/messages ? And error12 is ENOMEM. So there seems to be more crap upcoming...


I hope not, but I guess only time will tell. How did you manage to open 50(!) tabs in firefox anyways? I guess my personal garbage collector would have kicked in long before doing so :-D


----------



## PMc (Jul 17, 2018)

mickey said:


> I hope not, but I guess only time will tell. How did you manage to open 50(!) tabs in firefox anyways? I guess my personal garbage collector would have kicked in long before doing so :-D



I dont know how many there are actually - it's just how I am doing things. Say, I search for some matter, find a bunch of interesting links, and just open them to new tabs for later evaluation. Then I might get a mail and be pushed to some other matter, grab a free screen (icewm) and dive into that. So, at any time there may be a couple of browser windows on different screens with various tabs where I might continue reading later, and while these are not actively used, they are supposed to move into paging space.


----------



## Lamia (Jul 17, 2018)

It is so easy to have firefox with over 50tabs running. Using multiple profiles of firefox (by running "firefox --no-remote -P") is one example of doing so. A user could install several categories of add-ons and themes on the various profiles according to their needs. With that, they can open tonnes of websites on each one of them. 

A typical situation is when one is trying to troubleshoot a failing service, package, etc and needs to catch up with the other distractions of life - Sports (NBA/Tour de France/AFL/Wimbledon/FiFa@Russia/etc).


----------



## PMc (Jul 19, 2018)

Didn't find a solution to the swap issue yet, but found that the memory greediness of Firefox can be tuned. 
There's a switch in preferences, how many content worker processes it should run, and these are not threads, but full-sized processes, and they get *BIG*. Since the workload seems to be distributed among them, there is no point in having more than real mem can fit. As soon as the amount of tabs leads to paging, one is better off with only one worker process.


----------

