# C states not used



## Crivens (Jun 8, 2018)

I noticed that some time ago my laptops did stop using C states, and looking it up with sysctrl shows that f.e. C2 is unused in 30 seconds idle time. Yes, powerd is running, est is loaded, Cmax is set in tuneables to C3. It happens on a core2duo as well as core1solo. I can supply more info when I am back on my machines.

Did I miss some migration action on 11.1?


----------



## Crivens (Jun 9, 2018)

Okay, here as promised the data:
Machine is running 
	
	



```
FreeBSD Wanderer 11.1-RELEASE-p10 FreeBSD 11.1-RELEASE-p10 #26 r333924
```

This is loader.conf

```
kern.geom.label.gptid.enable="0"
zfs_load="YES"
acpi_video_load="YES"
kern.vty="vt"

#hw.pci.enable_io_modes="0"
hint.p4tcc.0.disabled=1
hint.apic.0.clock=0
hint.atrtc.0.clock=0

#acpi Override
hw.acpi.osname="Linux"
hw.usb.timings.extra_power_up_time="700"
vm.pmap.pti=0
```

Whats in /etc/sysctrl.conf

```
hw.acpi.video.lcd0.brightness=7
compat.linux.osrelease=2.6.18
kern.module_path=/boot/kernel;/boot/modules;/usr/local/modules

#hw.usb.no_suspend_wait=1
#hw.pci.do_power_suspend=0
#hw.pci.do_power_resume=0
hw.acpi.reset_video=1

vfs.usermount=1
```

And what is set in /etc/rc.conf

```
keymap="german.iso.acc.kbd"
ifconfig_re0="DHCP"
sshd_enable="YES"
moused_enable="YES"
powerd_enable="YES"

# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"
zfs_enable="YES"

battmond_enable="YES"

inetd_enable="YES"
linux_enable="YES"

anacron_enable="YES"
background_dhclient="YES"
clear_tmp_enable="YES"
cron_enable="NO"
cupsd_enable="YES"
lpd_enable="YES"

firewall_enable="YES"
firewall_type="workstation"
# ssh and syncthing
firewall_myservices="22 22000/tcp 21027/udp"
firewall_allowservices="A B C"

fusefs_enable="YES"

powerd_flags="-a adaptive -b adaptive -n adaptive"

devfs_system_ruleset="system"

economy_cx_lowest="Cmax"
performance_cx_lowest="Cmax"

#wlan
wlans_iwn0="wlan0"
ifconfig_wlan0="WPA SYNCDHCP"

keymap=de.noacc.kbd

tmpmfs="YES"
tmpsize="1024m"

wpa_supplicant="YES"

# List of kernel modules to load
kld_list="fuse iwn4965fw if_iwn"
```

What I see in sysctrl is this: `sysctrl -a |grep cx`

```
hw.acpi.cpu.cx_lowest: C8
dev.cpu.1.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc/bma
dev.cpu.1.cx_usage_counters: 228378 0 0
dev.cpu.1.cx_usage: 100.00% 0.00% 0.00% last 284us
dev.cpu.1.cx_lowest: C8
dev.cpu.1.cx_supported: C1/1/1 C2/2/1 C3/3/162
dev.cpu.0.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc/bma
dev.cpu.0.cx_usage_counters: 666229 0 0
dev.cpu.0.cx_usage: 100.00% 0.00% 0.00% last 2us
dev.cpu.0.cx_lowest: C8
dev.cpu.0.cx_supported: C1/1/1 C2/2/1 C3/3/162
```

Anyone having an idea what's going on?


----------



## shepper (Jun 10, 2018)

This may not apply to your hardware but some recent Intel SoC's require firmware for the display to wake from deep C-states:
My  OpenBSD dmesg AsRock J3355m
Here is a listing of i915 firmware Intel i915 firmware


----------



## Crivens (Jun 10, 2018)

The problem is not the waking from C state but the not-using. This is from a laptop core2duo about 10 years of age. It can sit idle for minutes and not use any C state, only about close to 30 watts of power 

The old T60 core1solo is the same...

Edit: that is close to 160% of the previous power intake.


----------



## Crivens (Jun 11, 2018)

I am bisecting my way trough the memstick images, booting them into live media and checking the sysctrl settings. With 10.4 it works, with 11.0 it does not. Investigation continues.


----------



## shepper (Jun 11, 2018)

I was thinking in terms of newer code having to accomodate the loading/implemention/utilization of firmware.  Perhaps it is not ready/disabled or needs more testing/debugging.


----------



## Crivens (Jun 11, 2018)

It is not about sleep states of the system (a.k.a. S3, S4, ...) but power saving in the CPU, and in older ones that is. The acpi code of 10.4 did the right thing, and somewhat after that there came a regression. Maybe not only I should check this again, who in the forum has the same old hardware and the same effect? Am I the only one (looking)?


----------



## Crivens (Jun 13, 2018)

The old code sadly does not build, so I need to dig deeper. Maybe until I find Jimmy Hoffa or the S3 which worked on 10.4 but not now. I call this a pretty heavy regression.


----------



## PMc (Jun 13, 2018)

Whats going on: for some reason it doesn't switch to C2 (or further). That might happen if the board does not know these states (obviousely not the case) or if hw.acpi.cpu.cx_lowest=C1 is set (obviousely not the case).

The C2/C3 states should work independent of powerd and without any laptop power saving modes. They exist on genuine desktop boards as well, and there it should be enough to put this into /etc/sysctl.conf:
`hw.acpi.cpu.cx_lowest=C3`

Sadly my vintage machine has now a workstation/server board that does not know any C states. It did work well on a pentium-2 (but that was retired with Rel 9 or 10).

That C8 in your output looks strange (but might work nevertheless). What does happen if You set it manually to hw.acpi.cpu.cx_lowest=C3  (or C1 or C2)?


----------



## Crivens (Jun 13, 2018)

I think the code tries to enumerate the states and gets something wrong. The list of C states comes from the acpi defs in ROM, so it does not need to match the found states. Now when the code only detects C1 and keeps that as maximum, you can overwrite that with anything - it will not matter but lead people into thinking all is well. So whoever uses a laptop, please check this. It only takes 10 seconds or so.


----------



## Maxnix (Jun 13, 2018)

Seem the same behaviour with my Celeron 530.

```
$ uname -rm
11.1-RELEASE-p10 amd64
$ sysctl -a | fgrep cx
has_pipe_cxsr: no
hw.acpi.cpu.cx_lowest: C8
dev.cpu.0.cx_method: C1/mwait/hwc C2/mwait/hwc/bma
dev.cpu.0.cx_usage_counters: 1969022 0
dev.cpu.0.cx_usage: 100.00% 0.00% last 1578us
dev.cpu.0.cx_lowest: C8
dev.cpu.0.cx_supported: C1/1/1 C2/2/17
```

/etc/rc.conf

```
#--- General settings -------------------------------------------------
cron_flags="-m ''"
devfs_system_ruleset="system"
dumpdev="AUTO"
economy_cx_lowest="Cmax"
gptboot_enable="NO"
kern_securelevel_enable="YES"
kern_securelevel="1"
kld_list="dtraceall.ko i915kms.ko vboxdrv.ko"
openntpd_enable="YES"
openntpd_flags="-sv"
performance_cx_lowest="Cmax"
savecore_enable="NO"
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
syslogd_flags="-ss"
update_motd="NO"
virecover_enable="NO"
vnstat_enable="YES"
xdm_enable="YES"
zfs_enable="YES"

#--- Network settings -------------------------------------------------
cloned_interfaces="lo1"
icmp_log_redirect="YES"
ifconfig_lo1="inet 10.0.0.0/29"
ifconfig_ue0="SYNCDHCP group egress"
#ifconfig_wlan0="WPA SYNCDHCP group egress"
ip6addrctl_enable="NO"
log_in_vain="1"
pf_enable="YES"
wlans_ath0="wlan0"
```
/etc/sysctl.conf

```
#--- Hardware --------------------------
dev.cpu.0.freq=216

hw.snd.vpc_autoreset=0

#--- Kernel ----------------------------
kern.coredump=0
kern.ipc.shm_use_phys=1
kern.maxprocperuid=150
kern.randompid=376
kern.vt.enable_bell=0

#--- Network ---------------------------
net.inet.icmp.drop_redirect=1
net.inet.ip.check_interface=1
net.inet.ip.process_options=0
net.inet.ip.random_id=1
net.inet.ip.redirect=0
net.inet.tcp.blackhole=2
net.inet.tcp.delayed_ack=0
net.inet.tcp.drop_synfin=1
net.inet.tcp.icmp_may_rst=0
net.inet.tcp.rfc1323=0
net.inet.tcp.sack.enable=0
net.inet.udp.blackhole=1
net.inet6.icmp6.rediraccept=0
net.inet6.ip6.redirect=0

#net.inet.sctp.blackhole=2

#--- Security --------------------------
security.bsd.hardlink_check_gid=1
security.bsd.hardlink_check_uid=1
security.bsd.see_other_uids=0
security.bsd.see_other_gids=0
security.bsd.stack_guard_page=1
security.bsd.unprivileged_proc_debug=0
security.bsd.unprivileged_read_msgbuf=0
security.jail.allow_raw_sockets=0
security.jail.chflags_allowed=0
security.jail.set_hostname_allowed=0

#--- VFS -------------------------------
vfs.usermount=1
```
/boot/loader.conf

```
autoboot_delay="2"

hint.acpi_throttle.0.disabled="0"
hint.p4tcc.0.disabled="0"

hw.pci.do_power_nodriver="3"
hw.psm.tap_enabled=1

vfs.root.mountfrom="zfs:zroot/ROOT/default"
vfs.zfs.arc_max="700M"
vfs.zfs.arc_min="300M"
vfs.zfs.resilver_delay=0
vfs.zfs.resilver_min_time_ms=5000
vfs.zfs.scrub_delay=0
vfs.zfs.top_maxinflight=128

zfs_load="YES"
```

Setting *hw.acpi.cpu.cx_lowest* to C2 makes no difference...


----------



## Crivens (Jun 14, 2018)

Well, that makes two. Any more?


----------



## Maxnix (Jun 14, 2018)

I don't know if this can be useful, but booting an old 10.2-RELEASE (i386 however, while actually using AMD64) 3 C states are detected, instead of 2, and they are all used.


----------



## Crivens (Jun 22, 2018)

Just checking -current after I saw commit r314211. Continuing with fingers crossed.


----------



## Crivens (Jun 24, 2018)

Now I have cx_max being C8. Otherwise no change.


----------



## Phishfry (Jun 24, 2018)

What about cpufreq(4)
I have to use it for powerd to work right on some platforms.


----------



## Crivens (Jun 24, 2018)

That is loaded and working. Sad thing is the laptop now draws (about) two times the power it used to. Suspend is borked also (used to work with 10.4). So I guess I will roll back to 10.4 and stay there. Since I not only bought one of these little road warriors but a trunk full, I am a little disapointed.


----------



## MYXOMOP (Feb 23, 2019)

*Crivens*, did you get any chance to find out what happened there? Any fix? I'm experiencing the same issue on a w530 laptop...


----------



## Crivens (Feb 25, 2019)

Sorry, no idea. Never mounted enough anger to file a PR. I would supply ACPI dumps, however, and gladly.


----------



## serjsk8 (May 2, 2019)

Hello *Crivens*,

Do you have something new about C3?
I have the same problem: lenovo-t530-power-saving

Best regards,


----------



## Crivens (May 2, 2019)

I just today moved the SSD to a refurbished X230 and it works very well there. But it also lists C8 as max, but only knows about C1-C3. I will check what I can do about the T60 I still have lurking around.


----------



## serjsk8 (May 2, 2019)

When I put:
`# cat /boot/loader.conf | grep acpi_throttle
hint.acpi_throttle.0.disabled="0"`

My CPU only know about C1 and C2, but the current frequency is low
`dev.cpu.0.cx_method: C1/mwait/hwc C2/mwait/hwc
dev.cpu.0.cx_usage_counters: 2667 79626
dev.cpu.0.cx_usage: 3.24% 96.75% last 1899us
dev.cpu.0.cx_lowest: C8
dev.cpu.0.cx_supported: C1/1/1 C2/2/80
dev.cpu.0.freq_levels: 2601/35000 2600/35000 2500/33218 2400/31470 2300/29755 2200/28074 2100/26426 2000/24816 1900/23556 1800/22002 1700/20480 1600/18989 1500/17534 1400/16106 1300/15009 1200/13638 1050/11933 900/10228 750/8523 600/6819 450/5114 300/3409 150/1704
dev.cpu.0.freq: 150`

When I changed to:
`# cat /boot/loader.conf | grep acpi_throttle
hint.acpi_throttle.0.disabled="1"`

My CPU detect C1, C2 and C3 but the current frequency is higt
`dev.cpu.0.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.0.cx_usage_counters: 5377 1366 309651
dev.cpu.0.cx_usage: 1.69% 0.43% 97.86% last 588us
dev.cpu.0.cx_lowest: C8
dev.cpu.0.cx_supported: C1/1/1 C2/2/59 C3/3/87
dev.cpu.0.freq_levels: 2601/35000 2600/35000 2500/33218 2400/31470 2300/29755 2200/28074 2100/26426 2000/24816 1900/23556 1800/22002 1700/20480 1600/18989 1500/17534 1400/16106 1300/15009 1200/13638
dev.cpu.0.freq: 1200
dev.cpu.0.temperature: 48.0C`


----------



## serjsk8 (May 2, 2019)

Do you think it's better leave at 0 ?
`hint.acpi_throttle.0.disabled="0"`


----------



## PMc (May 2, 2019)

`# cat /boot/loader.conf | grep acpi_throttle
hint.acpi_throttle.0.disabled="1"`

This setting was regularly done during upgrade to Rel. 10. It was accompanied by a paper, which explained that the very low frequencies which were enabled by setting this to "0" would not contribute to actual power savings.

I would not trust in any power influx readings. I would just compare temperature, under otherwise equal circumstances (as the power can go nowhere else).


----------



## Crivens (May 2, 2019)

Frequencies below 1200, for your setup, are emulated by halting the cpu for some percentage of a duty cycle, if memory serves me right. As long as nothing is to be done, it is not important if the CPU sleeps at 1200 for the complete slice or only a tiny part of it while being halted elsewhile.

If the C states do not work, however, that makes a huuuge difference.


----------



## mickey (Nov 17, 2020)

Crivens said:


> Well, that makes two. Any more?


Just stumbled upon this thread while checking C-state usage on my different machines, which are all running RELENG-12.2. It seems my Compaq notebook (Core2Duo T6600) shows the same behaviour of neither C2 nor C3 being ever used:

```
dev.cpu.1.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc/bma
dev.cpu.1.cx_usage_counters: 2152134 0 0
dev.cpu.1.cx_usage: 100.00% 0.00% 0.00% last 2634us
dev.cpu.1.cx_lowest: C3
dev.cpu.1.cx_supported: C1/1/1 C2/2/1 C3/3/57
dev.cpu.1.temperature: 43,0C
dev.cpu.1.coretemp.throttle_log: 0
dev.cpu.1.coretemp.tjmax: 100,0C
dev.cpu.1.coretemp.resolution: 1
dev.cpu.1.coretemp.delta: 57
dev.cpu.1.%parent: acpi0
dev.cpu.1.%pnpinfo: _HID=none _UID=0
dev.cpu.1.%location: handle=\_PR_.CPU1
dev.cpu.1.%driver: cpu
dev.cpu.1.%desc: ACPI CPU
dev.cpu.0.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc/bma
dev.cpu.0.cx_usage_counters: 2105107 0 0
dev.cpu.0.cx_usage: 100.00% 0.00% 0.00% last 1056us
dev.cpu.0.cx_lowest: C3
dev.cpu.0.cx_supported: C1/1/1 C2/2/1 C3/3/57
dev.cpu.0.freq_levels: 2200/35000 1600/23000 1200/15000
dev.cpu.0.freq: 1200
dev.cpu.0.temperature: 43,0C
dev.cpu.0.coretemp.throttle_log: 0
dev.cpu.0.coretemp.tjmax: 100,0C
dev.cpu.0.coretemp.resolution: 1
dev.cpu.0.coretemp.delta: 57
dev.cpu.0.%parent: acpi0
dev.cpu.0.%pnpinfo: _HID=none _UID=0
dev.cpu.0.%location: handle=\_PR_.CPU0
dev.cpu.0.%driver: cpu
dev.cpu.0.%desc: ACPI CPU
dev.cpu.%parent:
```
Another machine with a Core i3 550 CPU doesn't really like to spend much time in C3 although it's probably being idle most of it's life:

```
dev.cpu.3.cx_usage: 2.20% 94.62% 3.16% last 3272us
dev.cpu.2.cx_usage: 4.91% 92.32% 2.75% last 911us
dev.cpu.1.cx_usage: 10.80% 87.59% 1.60% last 940us
dev.cpu.0.cx_usage: 6.44% 93.37% 0.18% last 180us
```
Whereas my desktop machine running a Core i5 6600 paints an entirely different picture:

```
dev.cpu.3.cx_usage: 33.36% 43.27% 23.35% last 436us
dev.cpu.2.cx_usage: 32.77% 35.33% 31.89% last 376us
dev.cpu.1.cx_usage: 31.43% 32.92% 35.64% last 586us
dev.cpu.0.cx_usage: 34.09% 36.13% 29.76% last 661us
```
As I haven't been running anything older than FreeBSD 12 on this particular notebook, I cannot say whether or not it has been working (differently/at all) in 10.x. Still, the fact that it doesn't seem to be using C2/3 at all doesn't look right.


----------



## FrostKiwi (May 27, 2021)

Crivens said:


> I noticed that some time ago my laptops did stop using C states, and looking it up with sysctrl shows that f.e. C2 is unused in 30 seconds idle time. Yes, powerd is running, est is loaded, Cmax is set in tuneables to C3. It happens on a core2duo as well as core1solo. I can supply more info when I am back on my machines.
> 
> Did I miss some migration action on 11.1?


Necromancer FrostKiwi reporting in for 3 year old Necroposting:
Same deal for me stuck at C1, FBSD 13, Thinkpad x200, Core2Duo. I had the same issue on DragonFlyBSD, but IIRC there it was solved by switching Hardware timer, to i8254 as per doc. This does not apply to FBSD and it's realtime kernel anymore if I understand it correctly.
Here my sysctl dev.cpu

```
dev.cpu.1.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.1.cx_usage_counters: 14390 0 0
dev.cpu.1.cx_usage: 100.00% 0.00% 0.00% last 1393us
dev.cpu.1.cx_lowest: C8
dev.cpu.1.cx_supported: C1/1/1 C2/2/1 C3/3/55
dev.cpu.1.freq_levels: 2667/35000 2666/35000 2133/25000 1600/15000 800/12000
dev.cpu.1.freq: 1600
dev.cpu.1.%parent: acpi0
dev.cpu.1.%pnpinfo: _HID=none _UID=0 _CID=none
dev.cpu.1.%location: handle=\_PR_.CP01
dev.cpu.1.%driver: cpu
dev.cpu.1.%desc: ACPI CPU
dev.cpu.0.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc
dev.cpu.0.cx_usage_counters: 14522 0 0
dev.cpu.0.cx_usage: 100.00% 0.00% 0.00% last 4018us
dev.cpu.0.cx_lowest: C8
dev.cpu.0.cx_supported: C1/1/1 C2/2/1 C3/3/55
dev.cpu.0.freq_levels: 2667/35000 2666/35000 2133/25000 1600/15000 800/12000
dev.cpu.0.freq: 1600
dev.cpu.0.%parent: acpi0
dev.cpu.0.%pnpinfo: _HID=none _UID=0 _CID=none
dev.cpu.0.%location: handle=\_PR_.CP00
dev.cpu.0.%driver: cpu
dev.cpu.0.%desc: ACPI CPU
dev.cpu.%parent:
```
For good measure also "dmesg | grep cpu"

```
cpu0: <ACPI CPU> on acpi0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
[drm ERROR :intel_cpu_fifo_underrun_irq_handler] CPU pipe A FIFO underrun
```
Something curious here and in every other user's post:
mwait comanded C-states are detected, but ACPI commanded c states are not. There should be two sets of C-states listed, but only one is present. Normally it looked like this when it worked IIRC:


> dev.cpu.0.cx_method: C1/acpi C1/mwait C2/acpi C2/mwait C3/acpi C3/mwait



Either way, if memory does not betray, these CPUs can get you to C4 (even though the OS only sees C3). And the powersavings going from C1 to C2 were huge. I'm hitting myself for not properly writing the data down, but on DragonFlyBSD I split a Notebook cable and measured a drop from ~30W to 16W going from C1 to C4. (On a Core2Quad Thinkpad with fullbright Monitor).








Thus I would love to get those c-states going. If there is anything I could do (creating dumps with a debug Kernel or what have you), I'm here.


----------



## FrostKiwi (May 30, 2021)

Hell yeah it's fixed 
The problem was indeed the same as with DragonFlyBSD.

kern.timecounter.hardware is by default TSC-low, even when the KENV kern.timecounter.hardware was set to something different. So first of all, the KENV is not being honored. (Though this can luckily be set after boot as well)

Next, as long as TSC-low is enabled, the kernel just refuses to go into lower C-states. Simply setting kern.timecounter.hardware to something different, even after boot fixes everything.
kern.timecounter.hardware=HPET works for my Thinkpad x200 with an Intel Core2Duo P8800


----------



## Crivens (May 30, 2021)

I'll give that a try


----------



## Crivens (May 30, 2021)

Now C3 is used about 22% when totally idle. This seems to be the way.


----------



## FrostKiwi (Jun 1, 2021)

Crivens said:


> Now C3 is used about 22% when totally idle.


Confusingly, the percentages are cumulative.  As long as the percentage rises, it is being used 100% of the time. If you have had a CPU load for an hour and the CPU enters complete idle, it will still show 90-99% usage of C1 for the next 10 minutes, even when only C3 is being used.


----------

