# Can't find cores per packages or threads per core.



## Freakbeat (May 23, 2019)

Hi all, I'm trying to see what's my cores per package or threads per core but I can't see them, I'm running Bhyve in FreeBSD 12.0 and I got the following output:


```
FreeBSD 12.0-RELEASE-p4 GENERIC amd64
FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on LLVM 6.0.1)
VT(efifb): resolution 800x600
CPU: Intel(R) Core(TM) i5-5250U CPU @ 1.60GHz (1596.34-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x306d4  Family=0x6  Model=0x3d  Stepping=4
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x7ffafbbf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,A
ESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x121<LAHF,ABM,Prefetch>
  Structured Extended Features=0x21c27ab<FSGSBASE,TSCADJ,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,NFPUSG,RDSEED,ADX,SMAP,PROCTRACE>
  Structured Extended Features3=0xc000000<IBPB,STIBP>
  XSAVE Features=0x1<XSAVEOPT>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)
avail memory = 8141471744 (7764 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <INTEL NUC5i5RY>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 hardware threads

root@freebsd:/root/ # sysctl -a hw.vmm.topology.cores_per_package=1
sysctl: unknown oid 'hw.vmm.topology.cores_per_package'
root@freebsd:/root/ # sysctl -a hw.vmm.topology.threads_per_core=4
sysctl: unknown oid 'hw.vmm.topology.threads_per_core'

root@freebsd:/root/ # sysctl -a | grep hw.vmm.
hw.vmm.amdvi.domain_id: 0
hw.vmm.amdvi.disable_io_fault: 0
hw.vmm.amdvi.ptp_level: 4
hw.vmm.amdvi.host_ptp: 1
hw.vmm.amdvi.enable: 0
hw.vmm.amdvi.count: 0
hw.vmm.npt.pmap_flags: 0
hw.vmm.svm.num_asids: 0
hw.vmm.svm.disable_npf_assist: 0
hw.vmm.svm.features: 4294967295
hw.vmm.svm.vmcb_clean: 1023
hw.vmm.vmx.l1d_flush_sw: 1
hw.vmm.vmx.l1d_flush: 1
hw.vmm.vmx.vpid_alloc_failed: 0
hw.vmm.vmx.posted_interrupt_vector: -1
hw.vmm.vmx.cap.posted_interrupts: 0
hw.vmm.vmx.cap.virtual_interrupt_delivery: 0
hw.vmm.vmx.cap.invpcid: 1
hw.vmm.vmx.cap.monitor_trap: 1
hw.vmm.vmx.cap.unrestricted_guest: 1
hw.vmm.vmx.cap.pause_exit: 1
hw.vmm.vmx.cap.halt_exit: 1
hw.vmm.vmx.initialized: 1
hw.vmm.vmx.cr4_zeros_mask: 18446744073705936896
hw.vmm.vmx.cr4_ones_mask: 8192
hw.vmm.vmx.cr0_zeros_mask: 18446744071025197056
hw.vmm.vmx.cr0_ones_mask: 32
hw.vmm.vmx.no_flush_rsb: 0
hw.vmm.ept.pmap_flags: 1531
hw.vmm.vrtc.flag_broken_time: 1
hw.vmm.ppt.devices: 0
hw.vmm.iommu.enable: 1
hw.vmm.iommu.initialized: 0
hw.vmm.bhyve_xcpuids: 0
hw.vmm.topology.cpuid_leaf_b: 1
hw.vmm.create: beavis
hw.vmm.destroy: beavis
hw.vmm.trace_guest_exceptions: 0
hw.vmm.ipinum: 251
hw.vmm.halt_detection: 1
```

Thanks


----------



## Phishfry (May 23, 2019)

Freakbeat said:


> FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 hardware threads


`sysctl hw.ncpu`


----------



## malavon (May 23, 2019)

Phishfry said:


> sysctl hw.ncpu


This will show 8 for a 4-core hyperthreading CPU. There is no simple sysctl for getting the actual cpus and cores as far as I know.
There is however the following:
(edited, had the wrong sysctl!)
`sysctl kern.sched.topology_spec` which in my case (4-core HT) returns this:
`<groups>
<group level="1" cache-level="3">
  <cpu count="8" mask="ff,0,0,0">0, 1, 2, 3, 4, 5, 6, 7</cpu>
  <children>
   <group level="2" cache-level="2">
    <cpu count="2" mask="3,0,0,0">0, 1</cpu>
    <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
   </group>
   <group level="2" cache-level="2">
    <cpu count="2" mask="c,0,0,0">2, 3</cpu>
    <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
   </group>
   <group level="2" cache-level="2">
    <cpu count="2" mask="30,0,0,0">4, 5</cpu>
    <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
   </group>
   <group level="2" cache-level="2">
    <cpu count="2" mask="c0,0,0,0">6, 7</cpu>
    <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
   </group>
  </children>
</group>
</groups>`
It's something annoying that I also struggled with when porting UE4. I still don't have a good solution for this, so unless there's a new sysctl added in 12.0, this is the best there is.
You might need powerd for this sysctl to appear, if I remember correctly.


----------



## Freakbeat (May 23, 2019)

Hi Phishfry this is my output:


```
root@freebsd:/root/ # sysctl hw.ncpu
hw.ncpu: 4
root@freebsd:/root/ # sysctl kern.ccpu
kern.ccpu: 0
```


----------



## PMc (May 23, 2019)

malavon said:


> There is however the following:
> `sysctl kern.ccpu` which in my case (4-core HT) returns this:
> `<cpu count="8" mask="ff,0,0,0">0, 1, 2, 3, 4, 5, 6, 7</cpu>
> <cpu count="2" mask="3,0,0,0">0, 1</cpu>
> ...



Slight correction: the value for this would be `sysctl kern.sched.topology_spec`


----------



## malavon (May 23, 2019)

PMc said:


> Slight correction: the value for this would be  sysctl kern.sched.topology_spec


You are entirely correct, I've edited my original post. I was too quick after grepping and copied only the filtered values.
Prepping next UE4 release, prepping poudriere for some builds and answering forum posts apparently isn't a good combination.


----------



## Freakbeat (May 23, 2019)

So anybody knows if hw.vmm.topology.cores_per_package and hw.vmm.topology.threads_per_core are gone on FreeBSD 12? I have another server 11.2 and I can see both settings with sysctl.


thanks


```
root@freebsd:/root/ # sysctl kern.sched.top
sysctl: unknown oid 'kern.sched.top'
root@quasar:/home/freakbeat # sysctl kern.sched.topology_spec
kern.sched.topology_spec: <groups>
 <group level="1" cache-level="3">
  <cpu count="4" mask="f,0,0,0">0, 1, 2, 3</cpu>
  <children>
   <group level="2" cache-level="2">
    <cpu count="2" mask="3,0,0,0">0, 1</cpu>
    <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
   </group>
   <group level="2" cache-level="2">
    <cpu count="2" mask="c,0,0,0">2, 3</cpu>
    <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
   </group>
  </children>
 </group>
</groups>
```


----------



## malavon (May 23, 2019)

Freakbeat said:


> So anybody knows if hw.vmm.topology.cores_per_package and hw.vmm.topology.threads_per_core are gone on FreeBSD 12? I have another server 11.2 and I can see both settings with sysctl.


No, I'm on 12.0. Like I said, I think you need to be running powerd. Try it out with `service powerd onestart`.


----------



## Freakbeat (May 23, 2019)

Hi malavon  I have powerd running:


```
in rc.conf
powerd_enable="YES"
powerd_flags="-a hiadaptive"

root@freebsd:/etc # service powerd status
powerd is running as pid 57103.

root@freebsd:/etc # powerd -v
powerd: unable to determine AC line status
load   8%, current freq  500 MHz (12), wanted freq  500 MHz
load   4%, current freq  500 MHz (12), wanted freq  500 MHz
load  12%, current freq  500 MHz (12), wanted freq  500 MHz
```


----------



## malavon (May 23, 2019)

I now see I read your message too hastily. I haven't used the hw.vmm* sysctls and have only 12.0 running so I can't verify if I had these on 11.2.
I'm sorry about the confusion. To compensate for that I dug a bit deeper.
The FreeBSD sourcecode `sys/amd64/vmm/x86.c` shows the following:

```
#if __FreeBSD_version < 1200060    /* Remove after 11 EOL helps MFCing */
extern u_int threads_per_core;
SYSCTL_UINT(_hw_vmm_topology, OID_AUTO, threads_per_core, CTLFLAG_RDTUN,
    &threads_per_core, 0, NULL);

extern u_int cores_per_package;
SYSCTL_UINT(_hw_vmm_topology, OID_AUTO, cores_per_package, CTLFLAG_RDTUN,
    &cores_per_package, 0, NULL);
#endif
```
and then in `sys/sys/param.h (line 63)`

```
#define __FreeBSD_version 1200086    /* Master, propagated to newvers */
```
The commit message read this:


> *Add the ability to control the CPU topology of created VMs*
> 
> from userland without the need to use sysctls, it allows the old
> sysctls to continue to function, but deprecates them at
> ...



Bhyvectl does indeed have that option now, but I don't really know if you can use this to get the topology from inside a VM.


----------

