# Number of CPUs and cores



## SirDice (Aug 9, 2013)

I'm looking for one or more sysctl(8) that can get me this information:


```
FreeBSD/SMP: Multiprocessor System Detected: 24 CPUs
FreeBSD/SMP: 2 package(s) x 6 core(s) x 2 SMT threads
```

The number 24 is probably either hw.ncpu (the hardware) or kern.smp.cpus (what the kernel actually uses). But I need to figure out how many physical processors and how many cores each processor for each machine has. I also can't find the sysctl(8) that tells me if Hyper-Threading is turned on (that's the '2 SMT threads').

Surely I could just grep dmesg(8) output but that seems to be unreliable as sometimes the info already got flushed out by other messages.


----------



## DutchDaemon (Aug 9, 2013)

dmesg.boot won't get flushed


----------



## SirDice (Aug 9, 2013)

DutchDaemon said:
			
		

> dmesg.boot won't get flushed


That file doesn't seem to exist on some of the systems I looked at :\

But I think I found a way, kern.smp.active and kern.smp.disabled will tell me if HT is turned on. And I think I can figure out the number of packages by looking at the dev.cpu.*.%location strings (*.SCK0.* and *.SCK1.*). So the number of cores will be hw.ncpu divided by 2 (if HT is turned on) divided by the number of packages I've found.

The rational behind this is that I'm looking for a way to add this info automatically into Zabbix's inventory by crafting a neat UserParameter :e


----------



## Anonymous (Aug 9, 2013)

Perhaps kern.sched.topology_spec could give you all the information. Mine says:


```
kern.sched.topology_spec: <groups>
 <group level="1" cache-level="0">
  <cpu count="4" mask="f">0, 1, 2, 3</cpu>
  <children>
   <group level="2" cache-level="2">
    <cpu count="4" mask="f">0, 1, 2, 3</cpu>
    <children>
     <group level="3" cache-level="1">
      <cpu count="2" mask="3">0, 1</cpu>
      <flags><flag name="HTT">HTT group</flag><flag name="THREAD">THREAD group</flag></flags>
     </group>
     <group level="3" cache-level="1">
      <cpu count="2" mask="c">2, 3</cpu>
      <flags><flag name="HTT">HTT group</flag><flag name="THREAD">THREAD group</flag></flags>
     </group>
    </children>
   </group>
  </children>
 </group>
</groups>
```


```
[CMD]dmesg | grep FreeBSD/SMP[/CMD]
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 HTT threads
```


----------



## tingo (Aug 10, 2013)

On my new workstation (with AMD cpu) it looks like this: `tingo@kg-core1$ dmesg | grep SMP`

```
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
SMP: AP CPU #3 Launched!
SMP: AP CPU #1 Launched!
SMP: AP CPU #2 Launched!
```
and `tingo@kg-core1$ sysctl dev.cpu | grep location`

```
dev.cpu.0.%location: handle=\_PR_.P001
dev.cpu.1.%location: handle=\_PR_.P002
dev.cpu.2.%location: handle=\_PR_.P003
dev.cpu.3.%location: handle=\_PR_.P004
```
and `tingo@kg-core1$ sysctl kern.sched.topology_spec`

```
kern.sched.topology_spec: <groups>
 <group level="1" cache-level="0">
  <cpu count="4" mask="0xf">0, 1, 2, 3</cpu>
  <children>
   <group level="2" cache-level="2">
    <cpu count="4" mask="0xf">0, 1, 2, 3</cpu>
   </group>
  </children>
 </group>
</groups>
```
Not sure if it helps.


----------



## kbw (Sep 3, 2020)

... many years later.

It's best to use devel/hwloc.


----------



## Mjölnir (Sep 3, 2020)

SirDice said:


> I'm looking for one or more sysctl(8) that can get me this information:
> `FreeBSD/SMP: 2 package(s) x 6 core(s) x 2 SMT threads` [...]


I guess you want to get these informations in a script and not use a C library?  IMHO it's better to have sysctls than to compute it from XML provided by `kern.sched.topology_spec`, because that's error-prone.
Consider to file in a bug report and kindly request to add new read-only sysctl()s:
`kern.smp.cpu_packages`
`dev.cpu_package.*.cores`
`dev.cpu.*.threads`
`hw.cpu.packages`
`hw.cpu.*.cores`
`hw.cpu.*.threads`
Hopefully these are self-explanatory.  The `hw.*` is what the hardware has, the other's are what is in use by the kernel.  Maybe it should be `hw.acpi.cpu` instead?
Keep in mind that the `dev.cpu*` is dynamic, since it can be changed at runtime on some systems, and in general for jails on any system.  I'm not aware of systems with non-uniform CPU packages, but... IIRC there have been very strange combinations in the past by Unisys?  AFAIK SMT/HyperThreading on/off is not dynamic, but who knows?


----------



## kbw (Sep 3, 2020)

I revived this thread because I wanted to share a method that works reliably. The methods discussed in this thread are not reliable. Plus, it's portable. It is a set of executables and a library.

For example, kern.smp.cpu_packages does not exist on my systems; 12.1-RELEASE.


----------



## olli@ (Sep 3, 2020)

Well, answering this question 7 years later is probably a waste of time …

… but I think it’s worth mentioning that, nowadays, SMT does _not_ imply 2 threads per core anymore. I've recently read about new processors that implement 4 threads per core, and even more might be possible in the future.

And second, it also needs to be mentioned that modern CPUs can be more complicated than just “packages / cores / threads”. For example, CPUs can have one or more levels of cache groups, and NUMA architectures are becoming more and more common. Here’s a rather simple example (by today’s standards):

```
FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs
FreeBSD/SMP: 1 package(s) x 2 cache groups x 4 core(s) x 2 hardware threads
```
That’s not an unusual processor – it’s an AMD Ryzen 2700 that consists of two so-called core complexes (CCX) with localized L3 caches. FreeBSD is NUMA-aware and respects cache groups, which can make a measurable difference in performance.

There are AMD Epyc servers that have two packages, each package is a “Naples” processor with four “Zeppelin” chips (these are AMD codenames), each chip consists of two core complexes, each of theses has four cores, and every core supports two SMT threads. So, such a system has a 2 x 4 x 2 x 4 x 2 topology.

IMHO, the best way to read the CPU topology is using the `kern.sched.topology_spec`, because that’s why it exists.  
All common programming and scripting languages provide functions to parse XML into a native data structure. There are even tools that assist in parsing XML from within shell scripts, for example textproc/xmlstarlet, even though I strongly recommend to use a language like Python for such tasks.

For the above CPU, the XML looks like this:

```
<groups>
  <group level="1" cache-level="0">
    <cpu count="16" mask="ffff,0,0,0">0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15</cpu>
    <children>
      <group level="2" cache-level="3">
        <cpu count="8" mask="ff,0,0,0">0, 1, 2, 3, 4, 5, 6, 7</cpu>
        <children>
          <group level="3" cache-level="2">
            <cpu count="2" mask="3,0,0,0">0, 1</cpu>
            <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
          </group>
          <group level="3" cache-level="2">
            <cpu count="2" mask="c,0,0,0">2, 3</cpu>
            <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
          </group>
          <group level="3" cache-level="2">
            <cpu count="2" mask="30,0,0,0">4, 5</cpu>
            <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
          </group>
          <group level="3" cache-level="2">
            <cpu count="2" mask="c0,0,0,0">6, 7</cpu>
            <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
          </group>
        </children>
      </group>
      <group level="2" cache-level="3">
        <cpu count="8" mask="ff00,0,0,0">8, 9, 10, 11, 12, 13, 14, 15</cpu>
        <children>
          <group level="3" cache-level="2">
            <cpu count="2" mask="300,0,0,0">8, 9</cpu>
            <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
          </group>
          <group level="3" cache-level="2">
            <cpu count="2" mask="c00,0,0,0">10, 11</cpu>
            <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
          </group>
          <group level="3" cache-level="2">
            <cpu count="2" mask="3000,0,0,0">12, 13</cpu>
            <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
          </group>
          <group level="3" cache-level="2">
            <cpu count="2" mask="c000,0,0,0">14, 15</cpu>
            <flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
          </group>
        </children>
      </group>
    </children>
  </group>
</groups>
```

If you really want to dissect that with a shell script (_not_ recommended!), a command like this could be used as a starting point:

```
$ sysctl -n kern.sched.topology_spec | xml el -v | grep count
groups/group/cpu[@count='16' and @mask='ffff,0,0,0']
groups/group/children/group/cpu[@count='8' and @mask='ff,0,0,0']
groups/group/children/group/children/group/cpu[@count='2' and @mask='3,0,0,0']
groups/group/children/group/children/group/cpu[@count='2' and @mask='c,0,0,0']
groups/group/children/group/children/group/cpu[@count='2' and @mask='30,0,0,0']
groups/group/children/group/children/group/cpu[@count='2' and @mask='c0,0,0,0']
groups/group/children/group/cpu[@count='8' and @mask='ff00,0,0,0']
groups/group/children/group/children/group/cpu[@count='2' and @mask='300,0,0,0']
groups/group/children/group/children/group/cpu[@count='2' and @mask='c00,0,0,0']
groups/group/children/group/children/group/cpu[@count='2' and @mask='3000,0,0,0']
groups/group/children/group/children/group/cpu[@count='2' and @mask='c000,0,0,0']
```

Alternatively, awk(1) might be helpful, too:

```
$ sysctl -n kern.sched.topology_spec | xml el -v | awk -F/ '/count/ {print int(NF / 2), $NF}' | sort -n | tr -dc '\n 0-9'
1 16  000
2 8  000
2 8  00000
3 2  3000
3 2  30000
3 2  300000
3 2  3000000
3 2  000
3 2  0000
3 2  00000
3 2  000000
```
Here, the first column gives the level (1 = package, 2 = CCX cache group, 3 = core), and the second column specifies the number of SMT threads on that level. This example (a Ryzen 2700) has 3 levels. As mentioned above, there are systems with 4 or even 5 levels.


----------



## Mjölnir (Sep 3, 2020)

IIRC the SPARC T-series CPU's had 4 threads/core?  And modern ARM have mixed CPUs in a package (A72 & A5x). EDIT And as noted above, IIRC _Unisys_ built mainboards with mixed packages (different CPU classes) in the 90ies?


----------



## kbw (Sep 3, 2020)

Interesting, thanks.

Any idea how those cores can be identified in user space? For example, say I wanted to bind two application threads to different cpu threads on the same core, or different cores?


----------



## Mjölnir (Sep 3, 2020)

kbw said:


> Any idea how those cores can be identified in user space? For example, say I wanted to bind two application threads to different cpu threads on the same core, or different cores?


You have to figure out the CPU numbers from the CPU topology, then use cpuset(8) and/or rctl(8) to assign the tasks/threads to the respective CPUs. EDIT You can use rctl.conf(5) to permanently apply resource allocation rules.


----------



## kbw (Sep 3, 2020)

You have to figure out the CPU numbers from the CPU topology...

That was kinda the question.


----------



## a6h (Sep 3, 2020)

mjollnir said:


> then use cpuset(8)


cpuset(1)


----------



## Mjölnir (Sep 3, 2020)

kbw said:


> You have to figure out the CPU numbers from the CPU topology...
> 
> That was kinda the question.


`sysctl kern.sched.topology_spec`, read it's output & apply the _mask_ entry... E.g.
kern.sched.topology_spec: <groups>
 <group level="1" cache-level="3">
<cpu count="4" mask="f,0,0,0">0, 1, 2, 3</cpu>
<children>
<group level="2" cache-level="2">
<cpu count="2" mask="3,0,0,0">`0, 1`</cpu>
<flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
</group>
<group level="2" cache-level="2">
<cpu count="2" mask="c,0,0,0">`2, 3`</cpu>
<flags><flag name="THREAD">THREAD group</flag><flag name="SMT">SMT group</flag></flags>
</group>
</children>
</group>
</groups>
Thus, cpu0 & cpu1 are sharing the same core via SMT, as well as cpu2 & cpu3.


----------



## Mjölnir (Sep 3, 2020)

Please reload, I edited my previous post.


----------



## olli@ (Sep 3, 2020)

kbw said:


> You have to figure out the CPU numbers from the CPU topology...


That can be found in the XML from the `kern.sched.topology_spec` sysctl. For examle, this line:

```
<cpu count="2" mask="c000,0,0,0">14, 15</cpu>
```
specifies that the threads 14 and 15 are on the same core (and therefore also in the same cache group). Alternatively, depending on programming language, you can also use the `mask` attribute which is a hexadecimal representation of the bitmap: 0xc000 is binary 1100 0000 0000 0000, i.e. bits 14 and 15 are set. This might be useful when calling certain systemcalls like cpuset_setaffinity(2) directly.


----------

