# make with few threads or wher is the '-j' option



## blind0ne (Jan 5, 2022)

Every single port that I'm trying to compile takes ages, on linux there is '-j' option to specify the number of threads to compile with. I think that already questioned this issue, but really can't find this thread anymore (probably it was my dream =) ), so here is the example that i'm bumping in in every single port compilation attempt whenever need to run something from sources. 

```
# make -j 24
--- config-conditional ---
--- do-config ---
*** [do-config] Stopped -- signal 22
```

So my question is - give me a hint where I can read 'handbook' on 'mastering' ports, or another links on it? How did you learn this skill in advance?


----------



## meaw229a (Jan 6, 2022)

Maybe make -j24


----------



## diizzy (Jan 6, 2022)

The ports framework will set that appropriately to your system by default


----------



## blind0ne (Jan 6, 2022)

meaw229a said:


> Maybe make -j24




```
root@ihor:/usr/ports/games/openarena # make -j24
--- stage ---
--- /usr/ports/games/openarena/work/.build_done.openarena._usr_local ---
--- stage-message ---
===>  Staging for openarena-0.8.8.s1910_17,2
--- stage-dir ---
--- run-depends ---
===>   openarena-0.8.8.s1910_17,2 depends on file: /usr/local/share/openarena/baseoa/pak0.pk3 - not found
--- check-deprecated ---
--- fake-pkg ---
--- /usr/ports/games/openarena-data/work/.PLIST.mktmp ---
===>   NOTICE:

The data port currently does not have a maintainer. As a result, it is
more likely to have unresolved issues, not be up-to-date, or even be removed in
the future. To volunteer to maintain this port, please create an issue at:
--- check-deprecated ---
https://bugs.freebsd.org/bugzilla
More information about port maintainership is available at:
https://docs.freebsd.org/en/articles/contributing/#ports-contributing
--- check-license ---
===>   Registering installation for openarena-data-0.8.8 as automatic
===>  License GPLv2 accepted by the user
--- pkg-depends ---
--- fake-pkg ---
pkg-static: open(/usr/ports/games/openarena-data/work/.metadir): No such file or directory
*** [fake-pkg] Error code 1

make[2]: stopped in /usr/ports/games/openarena-data
--- pkg-depends ---
===>   openarena-data-0.8.8 depends on file: /usr/local/sbin/pkg - found
--- /usr/ports/games/openarena-data/work/.PLIST.mktmp ---
*** [/usr/ports/games/openarena-data/work/.PLIST.mktmp] Error code 6

make[2]: stopped in /usr/ports/games/openarena-data
2 errors

make[2]: stopped in /usr/ports/games/openarena-data
*** [run-depends] Error code 2

make[1]: stopped in /usr/ports/games/openarena
1 error

make[1]: stopped in /usr/ports/games/openarena
*** [stage] Error code 2

make: stopped in /usr/ports/games/openarena
1 error

make: stopped in /usr/ports/games/openarena
root@ihor:/usr/ports/games/openarena #
```


----------



## blind0ne (Jan 6, 2022)

blind0ne said:


> `root@ihor:/usr/ports/games/openarena # make -j24
> --- stage ---
> --- /usr/ports/games/openarena/work/.build_done.openarena._usr_local ---
> --- stage-message ---
> ...


An if I type just make it start's fetching! So what steps do I need to perform before typing make -j24?


----------



## mer (Jan 6, 2022)

Going by memory, "make" in a port directory does:
make fetch
make config
make    <<<<<<  This is the point where you want to add the -j

To actually install a port, 
make install

there may be other targets for make if you want create a package, clean directories, etc.


----------



## astyle (Jan 6, 2022)

blind0ne : try reading some of my profile posts: 



			https://forums.freebsd.org/profile-posts/3201/
		

Make is not very smart about thread management, you're gonna have race conditions. And what's up with using 24 threads? That only works if you have a 12-core Threadripper available.


----------



## zirias@ (Jan 6, 2022)

There's a fundamental misunderstanding here. *Of course*, FreeBSD's `make` supports the `-j` option. It just won't do anything useful for building ports.

The reason is that a Port is kind of a _wrapper_ around all steps necessary to build some software (fetching the source, extracting it, building, installing to some "staging" dir, building a binary package from the results), and this process is automated using `make`. For the actual "build" step, the ports framework _calls_ the tool necessary for it (which might be `make` as well, but could be anything else). You want your `-j` option in _that_ invocation!

Fortunately, the ports framework already detects the number of your CPUs and passes appropriate flags (e.g. `-j` for `make`) to the build. Yes, the default is to have as many make jobs as you have CPUs (cores, threads, ...).

This _should_ already give best results, except for ports with a broken build system that would just fail with parallelization (in that case, `MAKE_JOBS_UNSAFE` is set in the port Makefile and the build always only uses a single job).

Still, you can override the number of jobs using `MAKE_JOBS_NUMBER`, e.g. in make.conf(5), see also bsd.port.mk.


----------



## astyle (Jan 6, 2022)

Zirias : I once tried to compile devel/llvm10 with and without the `-j` flag.  Without the flag, it took my processor 2.5 hours to compile that port. With the flag (`-j4`), it was 50 minutes for the same thing.  But other than that, I have to agree with your assessment - there is a fundamental mis-understanding about processor threads and compilation jobs here.


----------



## blind0ne (Jun 5, 2022)

-j option somehow affects the dependencies installation. After -j2 at make step it gives error 22 or something like that. And complaining about dependencies


----------



## astyle (Jun 5, 2022)

blind0ne said:


> -j option somehow affects the dependencies installation. After -j2 at make step it gives error 22 or something like that. And complaining about dependencies


That's because you introduce race conditions. 

One thread compiles lib.so.1, 
another thread compiles lib.so.2,  and finishes earlier than lib.so.1.
Thread finishes, tries to start the linker as the next task, and fails - why?
lib.so.1 is not done compiling, so it's not available to the linker yet!

Make is not smart enough to manage race conditions like that. *Don't use the -j option for ports*. Only use it when ALL the deps have been satisfied.


----------

