# sysctl -a hanging



## man0v (Feb 6, 2017)

Dear Forumers,

I have been battling all day with a problem I can find very little information about and I was hoping you could give me a hand. I'm putting this issues in this forum only because I only did a fresh new install today and have been trying to resolve the issue basically the better part of the day.

My machine(workstation):
CPU: Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz
MB: ASUS PRIME Z270-P
RAM: 16GB(2x8GB Kingston Hyperx)(not sure which ones, can check if necessary)
HDD: 1TB Samsung 850 Evo
Graphics Card: Intel built-in(with SCFB driver).

I did a fresh FreeBSD(11 stable) installation this morning from USB bootonly image. All the software I installed I installed from ports with the exception of `ffmpeg`(as it wouldn't install from ports).

eventually(before I brought up the X server) I did a `sysctl -a | grep something` and the shell just froze. I could see things I write, Ctrl+C or Ctrl+Z but I couldn't kill the process. I tried to do `kill -9` from a different shell but no luck there.

I started debugging and the only way I could test whether `sysctl -a` works is by putting it inside `tmux`. That way round I could kill the pane if it froze and run it again.

The output of `sysctl -a` seemed normal to the part it just stops working(printing out hundreds of values).

I am new to `dtrace`but decided to debug with it. I followed the handbook manual, enabled `dtrace`:
`kldload dtraceall`
and used `/usr/share/dtrace/blocking sysctl` to monitor what's happening with `sysctl -a`

This is the ouput of it once I run sysctl -a:

```
sysctl(874) is blocking...

              kernel`_sleep+0x1fd
              kernel`AcpiOsAcquireMutex+0xe8
              kernel`AcpiUtAcquireMutex+0x3a
              kernel`AcpiExEnterInterpreter+0xb
              kernel`AcpiNsEvaluate+0x1e8
              kernel`AcpiUtEvaluateObject+0x4d
              kernel`AcpiUtExecute_HID+0x27
              kernel`AcpiGetObjectInfo+0xd9
              kernel`acpi_child_pnpinfo_str_method+0x24
              kernel`device_sysctl_handler+0x11b
              kernel`sysctl_root_handler_locked+0xbf
              kernel`sysctl_root+0x1f6
              kernel`userland_sysctl+0x1c4
              kernel`sys___sysctl+0x74
              kernel`amd64_syscall+0x4ce
              kernel`0xffffffff80f8442b

              libc.so.7`___sysctl+0xa
              sysctl`0x402b7d
              sysctl`0x401c61
              sysctl`0x401aff
              sysctl`0x40159f
              ld-elf.so.1`0x800626000
```

And this is repeating restlessly...

Also, `dmesg` started piping out this:


```
ACPI Exception: AE_ERROR, Thread 100635 could not acquire Mutex [0x0] (20160527/utmutex-345)
ACPI Error: Could not acquire AML Interpreter mutex (20160527/exutils-104)
ACPI Error: Mutex [0x0] is not acquired, cannot release (20160527/utmutex-386)
ACPI Error: Could not release AML Interpreter mutex (20160527/exutils-147)
ACPI Exception: AE_ERROR, Thread 100635 could not acquire Mutex [0x0] (20160527/utmutex-345)
ACPI Error: Could not acquire AML Interpreter mutex (20160527/exutils-104)
ACPI Error: Mutex [0x0] is not acquired, cannot release (20160527/utmutex-386)
ACPI Error: Could not release AML Interpreter mutex (20160527/exutils-147)
ACPI Exception: AE_ERROR, Thread 100616 could not acquire Mutex [0x0] (20160527/utmutex-345)
ACPI Error: Could not acquire AML Interpreter mutex (20160527/exutils-104)
ACPI Error: Mutex [0x0] is not acquired, cannot release (20160527/utmutex-386)
ACPI Error: Could not release AML Interpreter mutex (20160527/exutils-147)
ACPI Exception: AE_ERROR, Thread 100616 could not acquire Mutex [0x0] (20160527/utmutex-345)
ACPI Error: Could not acquire AML Interpreter mutex (20160527/exutils-104)
ACPI Error: Mutex [0x0] is not acquired, cannot release (20160527/utmutex-386)
ACPI Error: Could not release AML Interpreter mutex (20160527/exutils-147)
```

And the part I don't have is where `sysctl -a` cuts off(I'm going to post it first thing in the morning) but the last row had to do with acpi too.

Now the reason I am posting to the forums is because the only information about this issue I can find has to do with some bug reports mainly on laptops and also very very old(with the exception of one from August).

https://lists.freebsd.org/pipermail/freebsd-usb/2016-August/014377.html
https://lists.freebsd.org/pipermail/freebsd-acpi/2007-September/004015.html

Now I am sure I am missing something, but this is as far as I can go. Do let me know if you would like more information.

Thank you in advance!


----------



## k.jacker (Feb 6, 2017)

Hei.
From the debugging and `dmesg` output and the shell freezing it seems like an ACPI/Bios problem to me.
AML means ACPI Machine Language and thats the compiled Bios code.
Sadly crapy Bios code and ACPI problems are very common, especially in consumer grade mainboards.

As `sysctl -a` reads every available oid, including ACPI related ones, I would first try `sysctl kern.ostype` (or any other oid not ACPI related) just to see if `sysctl` works.
If that works, reboot and than boot into FreeBSD "with ACPI disabled". Run `sysctl -a` again.
When everything works fine with ACPI disabled, you should continue reading the related chapter in the Handbook:

https://docs.freebsd.org/doc/5.5-RELEASE/usr/share/doc/handbook/acpi-debug.html

Hope that will give you a start.


----------



## k.jacker (Feb 6, 2017)

https://www.freebsd.org/doc/en/books/handbook/acpi-overview.html

Sorry, the link to the Handbook I posted first is outdated, here is the right one!


----------



## GilGalaad (Feb 17, 2017)

Hi, I am experiencing the same issue on FreeBSD 11.0 Release.

My harware configuration is:
CPU: Intel(R) Core(TM) i3-7320 CPU @ 4.10GHz
MB: Asrock H270 Pro4
RAM: 32GB Corsair
HD: I tried on a OCZ Agility 2 SSD and two different USB sticks

Actually, before I noticed the Mutex errors in dmesg, the first symptom was that `poweroff` does work, but `reboot` does not, it hangs after "Uptime X min..." message.
So I was quite sure it was an ACPI issue and started blaming Asrock. I tried some of the suggestions in the linked page (_hw.acpi.disable_*_*_on_*_*_reboot_, _hw.acpi.handle_*_*_reboot_, then I tried to recompile the AML file), but nothing worked.

But considering that

the issue is the same on an Asus motherboard (Kaby Lake series)
the issue is *not *present on FreeBSD 10.3 on the same hardware
could it be a regression in kernel code?
Should we report it?

Thanks for any answer.


----------



## k.jacker (Feb 18, 2017)

Hei,
I'm sure that, especially with UEFI and newest hardware like yours, it's a work in progress to make ACPI work well on every piece of new hardware. 

If I where you, I would just wait and stay up to date with FreeBSD and keep track with Bios updates eventually.
Keep in mind that a lot Motherboard's Bios/UEFI problems never get fixed by the manufacturer because they do not care as long as everything works ok with "the other OS".

In the 17 years I have worked with FreeBSD I have learned that, besides trying to fix problems, in most cases it saves you a lot of headache if you just change the motherboard.

But in your case, since your motherboard is pretty new.... be patient 


Greetings
Matthias


----------



## GilGalaad (Feb 18, 2017)

I will be, considering that the problem does not compromise the server's stability, and Kaby Lake has been released not more than one month ago.
I was just wondering if, considering that *may* be a regression in ACPICA rev. 20160527, filling a problem report could be helpful or not, other than writing here on the forum.

Thank you for your answers


----------



## Bleo (Mar 19, 2017)

I'm having the exact same problem on similar hardware.

Processor: i3-7350k
Mobo: Gigabyte Z270N-WiFi

Kind of at a loss on what to do as well. I just purchased all this hardware and returning most of it is going to be impossible. I suppose i'll roll the dice and pick up one of these motherboards and see how it goes: https://www.supermicro.com/products/motherboard/Core/Q170/X11SSV-Q.cfm


----------



## GilGalaad (Mar 19, 2017)

Bleo said:


> I'm having the exact same problem on similar hardware.
> Kind of at a loss on what to do as well. I just purchased all this hardware and returning most of it is going to be impossible. I suppose i'll roll the dice and pick up one of these motherboards and see how it goes: https://www.supermicro.com/products/motherboard/Core/Q170/X11SSV-Q.cfm



To be honest, I have run my box for over a month of uptime, without any other issue.
I suggest to stick with your hardware and wait for a fix.


----------



## Bleo (Mar 20, 2017)

GilGalaad said:


> To be honest, I have run my box for over a month of uptime, without any other issue.
> I suggest to stick with your hardware and wait for a fix.



Good to know. I'm actually running pfSense 2.4 which is based on FreeBSD 11 and while it may be unrelated the web interface is locking up on this hardware. No issues on my other (older) hardware. Everything memtested/prime95/stressapptested fine so I don't believe it's bad hardware.


----------



## nicos.ka (Apr 4, 2017)

Fyi, the same issue here, with Dell XPS 9360 (Kaby Lake) and FreeBSD 11. No issues with older one (9333, Haswell) though.


----------



## joconcepts (Apr 21, 2017)

Same problem here with Kabylake processor on mainboard with Intel B150 Express Chipset


```
# sysctl hw.model hw.machine hw.ncpu
hw.model: Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz
hw.machine: amd64
hw.ncpu: 8
```

`sysctl -a` hangs, dmesg has the following output:


```
cpu0: <ACPI CPU> on acpi0
ACPI Error: Mutex [0x0] is not acquired, cannot release (20160527/utmutex-386)
ACPI Error: Could not release AML Interpreter mutex (20160527/exutils-147)
ACPI Error: Mutex [0x0] is not acquired, cannot release (20160527/utmutex-386)
ACPI Error: Could not release AML Interpreter mutex (20160527/exutils-147)
cpu1: <ACPI CPU> on acpi0
ACPI Error: Mutex [0x0] is not acquired, cannot release (20160527/utmutex-386)
ACPI Error: Could not release AML Interpreter mutex (20160527/exutils-147)
ACPI Error: Mutex [0x0] is not acquired, cannot release (20160527/utmutex-386)
ACPI Error: Could not release AML Interpreter mutex (20160527/exutils-147)
```

Also the system stops on rebooting. It prints the uptime and hangs then forever.

Is there any bugreport according to that?


----------



## Jeckt (May 5, 2017)

I've been migrating a machine to a Dell Precision, with a Xeon E3-1225 (kaby lake-ish CPU). Everything was fine but when I copied over sysctl.conf and rebooted, I encountered this problem. Thankfully I was able to work around it by commenting out  dev.em.0.wake=1 in sysctl.conf.  I highly doubt anyone would have that specifically enabled, but you may be able to pin the problem to something set in sysctl.conf , and bypass the issue for now.


----------



## Qoo (May 7, 2017)

Same problem here with a G4560 (Kaby Lake) but on a Z170 (Skylake-gen) board. Haven't tried `sysctl -a` with the 7700k yet, but there are no issues with my i3-6100 (Skylake) on an H110 so this may be specific to Kaby Lake CPU's.


----------



## darcagn (May 21, 2017)

I am having the same issue with an ASUS Z170-A motherboard and i5-7600 CPU. 

When I first installed the system, the BIOS was revision 2202, and the reboot functionality worked just fine. However, I realized after installing everything that the BIOS wasn't detecting my memory correctly; it showed all 16GB at the POST screen but only 8GB within BIOS config and within FreeBSD. After updating to 3401, the system will no longer reboot (but will poweroff) and the console resolution defaults to 1024x768 instead of my monitor's native resolution like it did before (I'm going to be running headless so that's not a big deal though). 

I need to manage this box remotely so hopefully there is some resolution to this. :-\


----------



## mbevand (Jun 10, 2017)

I have the same problem as everyone here. *sysctl -a* hangs (in the middle of printing a few hundreds lines, last one is "dev.fpupnp.0.%parent: acpi0", if I kill my ssh session I see "ACPI Error: Could not acquire AML Interpreter mutex" in dmesg) and *reboot* also hangs (after printing "All buffers synced.\nUptime: ...").

FreeBSD 11.0-RELEASE-p1
Motherboard: ASRockRack C236 WSI (latest BIOS P2.10)
CPU: Intel Pentium G4600 Kaby Lake

I found *one workaround for reboot hanging*: run instead *shutdown -r now* and the system reboots flawlessly.

Not sure what's the difference between reboot and shutdown -r now.


----------



## mbevand (Jun 10, 2017)

GilGalaad said:


> I will be, considering that the problem does not compromise the server's stability, and Kaby Lake has been released not more than one month ago.
> I was just wondering if, considering that *may* be a regression in ACPICA rev. 20160527, filling a problem report could be helpful or not, other than writing here on the forum.



Yes I do think this sysctl -a and reboot hangs may be caused by regressions in ACPICA 20160527. In fact subsequent versions seem to fix mutex issues:

https://acpica.org/sites/acpica/files/changes_41.txt

Fixed in 20170224:

```
Debugger: Fixed an AML interpreter mutex issue during the single stepping
of control methods. If certain debugger commands are executed during
stepping, a mutex aquire/release error could occur. Lv Zheng.
```

Fixed in 20160831:

```
Fixed a possible mutex error during control method auto-serialization. Lv
Zheng.
```


----------



## GilGalaad (Jun 10, 2017)

mbevand said:


> I found *one workaround for reboot hanging*: run instead *shutdown -r now* and the system reboots flawlessly.



I noticed the same thing, but I forgot to write it here.
*shutdown -r now *works in place of *reboot* and
*shutdown -p now *works in place of *poweroff*.

Thank you for sharing it


----------



## mbevand (Jun 11, 2017)

Yesss, FreeBSD 11.1-Beta1 was released today and appears to fix all my issues: "reboot" and "sysctl -a" no longer hang.
This version of FreeBSD comes with ACPICA 20170303, and is the only ACPI-related change between 11.0 and 11.1-Beta1 (see https://www.freebsd.org/relnotes/11-STABLE/relnotes/article.html), so this is more evidence that the bug was caused by a regression in ACPICA.


----------



## darcagn (Jul 28, 2017)

Sorry for the month and a half old bump, but I just wanted to confirm that the release of FreeBSD 11.1-RELEASE fixed this issue on my system. sysctl -a no longer hangs, nor does using the *reboot* command. Thanks devs!


----------

