# Influence (new) hardware on strenuous tasks



## Erichans (Sep 12, 2021)

I intend to use FreeBSD also on a desktop/server CPU, NVME SSD & DDR4  (and graphics card). Currently using FreeBSD REL-12.2 (and XFCE) (binaries, packages) on an old Samsung laptop: Intel Core2 Duo P7350 @ 2.00GHz, 3GB DDR3, SATA SSD.

What is the effect of the amount of memory,  Intel (amd64 architecture) CPU (because I have some experience with those) and core count on heavy tasks such as more well defined tasks:  (cross) compiling world or Firefox as an upper bound.

At what amount of ram will I get (vastly) diminishing returns on more RAM?
What is the influence of core count (i.e. parallelization) on compiling?
I'm looking for ballpark influence.

I have no idea what amount of time compiling world on a modern CPU takes; if you have a (ballpark) figure, perhaps with a specific CPU and ram, I would appreciate that.


----------



## Geezer (Sep 12, 2021)

In my opinion, ram is most important, not cores. Apache, zfs, DEs and browsers. All these can really do with _loads _of ram.



Erichans said:


> At what amount of ram will I get (vastly) diminishing returns on more RAM?



When you run out of money.


----------



## Erichans (Sep 12, 2021)

Thanks, perhaps my old laptop gave the wrong impression as to the desktop/server hardware intended. Thinking along the lines of an Intel Xeon E3-1280 v5 @ 3.70GHz and up (or perhaps even something like an Intel i7-6700HQ in my P50).

As to "_loads_ of ram", this is a very broad ballpark amount; a little more detailed amount would be helpfull. Thinking along the lines of 32, 64, 96 & 128GB.


----------



## ct85711 (Sep 12, 2021)

From my experience ram is going to give you a significant boost up to a point.  I'd recommend at least 16-32GB of ram to get the best performance, 8GB minimum.  Once you get past 16-32GB of ram, the returns you gain drops off.  The reasoning that I've seen on this is that, frankly outside of ZFS, and DB's; few packages will use that much ram ever.  As far as compiling, more core/thread counts will speed up compiling for most packages (some packages are not designed to be compiled in parallel and cause troubles).  I don't have personal experience with core counts from like the Ryzen/Threadripper amounts of cores to ever see if there is a drop on core amounts.  Though the part you will see, is that few packages will use too many of the cores outside of compiling.

The one rule of thumb I've encountered quite commonly on Gentoo (and I am assuming it would be similar to FreeBSD), is expect on average about 2GB per thread when compiling.

Now one thing, that is bonus on compiling when you have plenty of spare ram, is to use the unused ram for as a ram disk.  While the performance gain isn't as high as when compared to just using the spinning rust buckets; the ram disk also gives a side bonus in that you get less writes onto the SSD (as you setup the temporary compiled stuff can be set to compile onto the ram disk and not the SSD).  The downside on RAM disk, is that it's hogs the portion of ram that you gave it.  The other thing to keep in mind (commonly encountered on the browsers), is some packages can be too big to fit in the ram disk and cause it to fail to compile.


----------



## grahamperrin@ (Sep 18, 2021)

Erichans said:


> … compiling world …





Erichans said:


> … Thinking along the lines of 32, 64, 96 & 128GB.



32 would be ample.

<https://forums.FreeBSD.org/threads/8877/post-524752> – for a while, I used an HP ZBook with an eight-thread CPU <https://bsd-hardware.info/?probe=f2d911563a#cpu:intel-6-60-3-core-i7-4710mq> | <https://ark.intel.com/content/www/u...4710mq-processor-6m-cache-up-to-3-50-ghz.html>.

*Much* faster, for building, than the four <https://bsd-hardware.info/?probe=646148fc25#cpu:intel-6-58-9-core-i7-3520m> | <https://ark.intel.com/content/www/u...-3520m-processor-4m-cache-up-to-3-60-ghz.html> in an HP EliteBook 8570p.



ct85711 said:


> … (commonly encountered on the browsers) …



Also FYI <https://cgit.freebsd.org/ports/commit/?id=b1670e2c3d42a2aeacff843ef0ccea21c0929d03>


----------



## SirDice (Sep 23, 2021)

ct85711 said:


> As far as compiling, more core/thread counts will speed up compiling for most packages (some packages are not designed to be compiled in parallel and cause troubles). I don't have personal experience with core counts from like the Ryzen/Threadripper amounts of cores to ever see if there is a drop on core amounts. Though the part you will see, is that few packages will use too many of the cores outside of compiling.


Don't have any exact figures but at some point you're going to hit a bottleneck on your I/O. In order for all those compiler process to work they need to be able to quickly read and write various (intermediate) files. So you're going to need some fast storage too. Poudriere can be configured to use RAM (tmpfs(5) [1]) for this, which is the fastest option you could get it to work. But this obviously consumes copious amounts of memory besides the amount of memory all those compilers are going to need.

[1]:

```
# Use tmpfs(5)
# This can be a space-separated list of options:
# wrkdir    - Use tmpfs(5) for port building WRKDIRPREFIX
# data      - Use tmpfs(5) for poudriere cache/temp build data
# localbase - Use tmpfs(5) for LOCALBASE (installing ports for packaging/testing)
# all       - Run the entire build in memory, including builder jails.
# yes       - Enables tmpfs(5) for wrkdir and data
# no        - Disable use of tmpfs(5)
# EXAMPLE: USE_TMPFS="wrkdir data"
```


----------



## astyle (Sep 24, 2021)

Might as well weigh in here... My rig has 32 GB of RAM, and a Ryzen 5 1400 (quad-core, 3.4 GHz). And I'm noticing that only one core is really used for compiling - unless I specify the -j4 flag for `make`. However, specifying the -j4 flag seems to be useful only when recompiling the kernel or compiling a port that has ALL deps (both build-time and run-time) completely satisfied. So basically, very limited variety of scenarios. However, without that flag, my Ryzen just churns without complaining (and it does have a stock Wraith cooler it came with, which is absolutely essential), it can go for hours, even overnight.

Some lessons I learned: be very careful with the make flags that enable parallellism - make is not smart enough to manage different cores, so it might start a job (on another core) that is actually not ready to start yet.

Without the -j4 flag, my Ryzen 5 would take about 3 hours to compile Firefox (after all deps have been satisfied for www/firefox).


----------



## SirDice (Sep 24, 2021)

astyle said:


> However, specifying the -j4 flag seems to be useful only when recompiling the kernel


You can use it for buildworld too. Speeds things up tremendously. Only the installkernel/installworld phases should be done without.


----------



## Alain De Vos (Sep 24, 2021)

When you build everything from source, number of cores is important.


----------



## astyle (Sep 24, 2021)

Alain De Vos said:


> When you build everything from source, number of cores is important.











						Trying out plasma-wayland
					

First I have no plasma-wayland .sh script. But i have a compiled /usr/local/bin/startplasma-wayland. But when i try it gives did you start qdbus ?




					forums.freebsd.org


----------



## Alain De Vos (Sep 24, 2021)

Offcourse number of cores for the same cpu-architecture.
I think its a good idea to assign a few cores for compiling and a few cores for realtime stuff like playing youtube.


----------



## Sevendogsbsd (Sep 24, 2021)

When I built packages on my old HP z800 (96 GB ram), I used as much as 60GB building rust, chromium, Firefox and llvm (?), libreoffice, etc. I had my settings as far as concurrency set pretty high to take advantage of the machine's capabilities. I had 2 hexacore xeon 5550 (I think). I may be misstating some of the terms here because it has been a long time and I no longer have that machine. 100% of my builds were using tmpfs because of the amount of ram I had so that may account for the high ram usage in my case.


----------



## grahamperrin@ (Sep 25, 2021)

SirDice said:


> … Poudriere can be configured to use RAM (tmpfs(5) [1]) for this, which is the fastest option you could get it to work. But this obviously consumes copious amounts of memory besides …



Thanks, I stepped mine down from `USE_TMPFS=all` to `USE_TMPFS=yes`. 16 G real memory here. I'll see how things go.

Previously – with `all` – just _rarely_, I'd find swap space (16.0 G) nearly exhausted, as in the first of these four shots:




Incidentally, there was `gmake -f Makefile -j4 all` (four jobs) whilst, if I recall correctly, poudriere was set to `PARALLEL_JOBS=3`.


----------



## grahamperrin@ (Sep 26, 2021)

grahamperrin said:


> I stepped mine down from `USE_TMPFS=all` to `USE_TMPFS=yes`. 16 G real memory here. I'll see how things go.



On second thoughts, I might stick with `all` for a while.

I've been using the system with 16.0 of 16.0 G swap used for a few hours, whilst building electron12. Performance has been fine.



Postscript (note to self): not too _strenuous_, but maybe too adventurous for electron12 build purposes.


```
…
[ 69% 26723/38463] python "../../build/toolchain/gcc_link_wrapper.py" --output="./chromedriver" -- c++ -fuse-ld=lld -Wl,--build-id=sha1 -fPIC -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,--icf=all -Wl,--color-diagnostics -Wl,--no-call-graph-profile-sort -m64 -Wl,-O2 -Wl,--gc-sections -rdynamic -pie -Wl,--disable-new-dtags -L/usr/local/lib  -fstack-protector-strong -L/usr/local/lib  -o "./chromedriver" -Wl,--start-group @"./chromedriver.rsp"  -Wl,--end-group  -lpthread -lgmodule-2.0 -lglib-2.0 -lgobject-2.0 -lgthread-2.0 -lintl -lnss3 -lsmime3 -lnssutil3 -lplds4 -lplc4 -lnspr4 -ldl -lexecinfo -lkvm -lutil -lgio-2.0 -lrt -ljpeg -lm -lopus -lavcodec -lavformat -lavutil -lopenh264 -lX11 -lXcomposite -lXdamage -lXext -lXfixes -lXrender -lXrandr -lXtst -lfontconfig -lpng16 -lz -lwebpdemux -lwebpmux -lwebp -lfreetype -lexpat -lharfbuzz-subset -lharfbuzz -lxcb -ldrm -lxkbcommon -ldbus-1
FAILED: chromedriver
python "../../build/toolchain/gcc_link_wrapper.py" --output="./chromedriver" -- c++ -fuse-ld=lld -Wl,--build-id=sha1 -fPIC -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,--icf=all -Wl,--color-diagnostics -Wl,--no-call-graph-profile-sort -m64 -Wl,-O2 -Wl,--gc-sections -rdynamic -pie -Wl,--disable-new-dtags -L/usr/local/lib  -fstack-protector-strong -L/usr/local/lib  -o "./chromedriver" -Wl,--start-group @"./chromedriver.rsp"  -Wl,--end-group  -lpthread -lgmodule-2.0 -lglib-2.0 -lgobject-2.0 -lgthread-2.0 -lintl -lnss3 -lsmime3 -lnssutil3 -lplds4 -lplc4 -lnspr4 -ldl -lexecinfo -lkvm -lutil -lgio-2.0 -lrt -ljpeg -lm -lopus -lavcodec -lavformat -lavutil -lopenh264 -lX11 -lXcomposite -lXdamage -lXext -lXfixes -lXrender -lXrandr -lXtst -lfontconfig -lpng16 -lz -lwebpdemux -lwebpmux -lwebp -lfreetype -lexpat -lharfbuzz-subset -lharfbuzz -lxcb -ldrm -lxkbcommon -ldbus-1
ld.lld: error: failed to open ./chromedriver: No space left on device
c++: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
*** Error code 1

Stop.
make: stopped in /usr/ports/devel/electron12
=>> Cleaning up wrkdir
===>  Cleaning for electron12-12.0.9_2
build of devel/electron12 | electron12-12.0.9_2 ended at Mon Sep 27 03:11:50 BST 2021
build time: 16:19:48
!!! build failure encountered !!!
```


----------

