# Poudriere spawns only one make process per building-port.



## Alain De Vos (May 10, 2021)

# By default MAKE_JOBS is disabled to allow only one process per cpu
# Use the following to allow it anyway
ALLOW_MAKE_JOBS=YES

Does not seem to work.


----------



## zirias@ (May 10, 2021)

Sure it does. Most likely, you're trying to build a port that has `MAKE_JOBS_UNSAFE` set. This is only set for good reasons (the upstream build system is broken and can't correctly handle parallel make). When it's set, this port will _never_ use parallel make.

A bit less likely, but also possible: Upstream uses a build system that just doesn't support parallel make.


----------



## Alain De Vos (May 10, 2021)

So if I try:

Poudriere.conf:
PARALLEL_JOBS=1000
PREPARE_PARALLEL_JOBS=1000
ALLOW_MAKE_JOBS=YES


make.conf:
MAKE_JOBS_NUMBER=1000

Then 1000 make jobs are spawn when I build, let's say , LLVM11 , with poudriere ?
----------------------
I just performed the test, only 1 make job is running.
ps aux | grep make
root       45692   0.0  0.0   12960   3536  0  IJ   08:45      0:00.06 /usr/bin/make -C /usr/ports/devel/llvm11 build
root       25825   0.0  0.0   12948   2052  6  S+   08:47      0:00.00 grep make
root@aladin:/home/x #


----------



## zirias@ (May 10, 2021)

poudriere doesn't use the system's make.conf but constructs one from the files you place in /usr/local/etc/poudriere.d. Then, if you put `MAKE_JOBS_NUMBER` there, that's what's passed to `make -j`, so, in theory, yes, although it's extremely unlikely `make` will ever find 1000 jobs it _can_ execute in parallel. There are dependencies during builds!

Your settings in poudriere.conf decide how many ports are built in parallel, with these settings, `poudriere` would attempt to create 1000 builder jails and use them in parallel (and each of them trying to use up to 1000 jobs).

All in all, these settings are extremely insane and will probably just crash your machine.


----------



## Alain De Vos (May 10, 2021)

No, I just tested with the settings listed. It creates just one make process.
The setting ALLOW_MAKE_JOBS=YES does not work.


----------



## zirias@ (May 10, 2021)

I wouldn't be surprised if there was a sanity check preventing such insane configurations.

Apart from that, check the port you're trying to build. If it has `MAKE_JOBS_UNSAFE`, there's nothing you can do.


----------



## Alain De Vos (May 10, 2021)

When I try with 100 it does not work. When I try with 10 it does not work.
There is no unsafe.
            https://cgit.freebsd.org/ports/plain/devel/llvm11/Makefile


----------



## zirias@ (May 10, 2021)

All I can tell you is that this option works as advertized, I use it quite often and there was never a problem. You should find enough pointers in my answers, there's something wrong with your configuration for sure. And stop trying insane stuff.

The default configuration (_ncpu_ builders, no make jobs) is somewhat conservative but will max out CPUs as long as all builders are in the "build" phase. Allowing make jobs for some selected packages will help if they typically end up building alone. If your goal is to always max out CPUs, something like reducing(!) the number of builders to 2 or 3, while allowing make-jobs for all packages, will do the trick. There's almost never a point in increasing either the number of builders OR the number of make jobs, both default to _ncpu_.


----------



## Alain De Vos (May 10, 2021)

My experience is that with my 8=4x2-core CPU poudriere allows me to build 8 different ports in parallel.
But not one port using the 8-cores.


----------



## zirias@ (May 10, 2021)

Alain De Vos said:


> My experience is that with my 8=4x2-core CPU poudriere allows me to build 8 different ports in parallel.
> But not one port using the 8-cores.


That's the default configuration. Setting `ALLOW_MAKE_JOBS=yes` will use 8 jobs for every port where that's possible in addition (ending up with up to 8*8=64 jobs when all 8 builders are in the build phase). If that doesn't happen on your machine, check all your configs. You will for example see everything poudriere puts in make.conf in the build log of a port.


----------



## mtu (May 10, 2021)

I just looked it up on my build host. Try setting `MAKE_JOBS_NUMBER=` in /usr/local/etc/poudriere.d/make.conf.

I remember being confused as hell about poudriere's configuration, and the question of what poudriere.conf does in relation to /usr/local/etc/poudriere.d/make.conf was one of the points that annoyed me.

Still can't tell you what's happening there, but I think that's how it works on my machine.


----------



## SirDice (May 10, 2021)

mtu said:


> and the question of what poudriere.conf does in relation to /usr/local/etc/poudriere.d/make.conf was one of the points that annoyed me.


These two have nothing to do with each other[*]. poudriere.conf is for configuring poudriere, it defines how the application itself operates. make.conf(5) is for setting generic build time options.

[*] Some options in poudriere.conf to define _how_ ports are built, but only from a high level view. By default it will spawn as much jobs as you have cores (_ncpu_), each job will run on one core. ALLOW_MAKE_JOBS=YES removes that limitation and allow a job to use all cores. But as you normally have jobs running on the other cores this may just slow everything down. ALLOW_MAKE_JOBS_PACKAGES allows you to be more selective when to turn this on, you can add `llvm*` for example. If you have a lot of jobs _waiting_ on a single core build of llvm10 for example and all the other jobs are idling.

Some ports ignore this setting completely and will only use one core regardless if you enabled ALLOW_MAKE_JOBS_PACKAGES or ALLOW_MAKE_JOBS.


----------



## zirias@ (May 10, 2021)

mtu said:


> I remember being confused as hell about poudriere's configuration, and the question of what poudriere.conf does in relation to /usr/local/etc/poudriere.d/make.conf was one of the points that annoyed me.


It's a feature. You can have for example jail- and ports-tree-specific make.conf entries with poudriere(8) (it's all documented in the manpage).


SirDice said:


> These two have nothing to do with each other. poudriere.conf is for configuring poudriere, it defines how the application itself operates. make.conf(5) is for setting generic build time options.


Sure, but `MAKE_JOBS_NUMBER` is a make.conf configuration. I see no reason to change it though…


----------



## mtu (May 10, 2021)

So, /usr/local/etc/poudriere.d/make.conf (and its jail-/tree-specific cousins) configure what `make` does when buildings ports in poudriere's jails, except in the case of `MAKE_JOBS_NUMBER=`, which is _ignored_, _unless_ `ALLOW_MAKE_JOBS=yes` is also set in /usr/local/etc/poudriere.conf,_ or_ the package being built is matched in `ALLOW_MAKE_JOBS_PACKAGES=""` in /usr/local/etc/poudriere.conf.

This behavior may be documented, and it may make sense after years of practice, but I call that a mess.


----------



## SirDice (May 10, 2021)

Some usage tips then perhaps. 

Use ZFS, this is a no-brainer, if you have ZFS configure poudriere to use it. Creating a clean jail for a job is MUCH faster if it can just clone a jail.  
Enable TMPFS. This will also speed up building as the intermediate files are stored on a RAM disk. My lowly Core i5 only has 16 GB and has no problems with this turned on. I can build even the biggest ports without issues.
Look at your build runs, are there ports that hold up the queue? Then add those to ALLOW_MAKE_JOBS_PACKAGES
Leave PARALLEL_JOBS and PREPARE_PARALLEL_JOBS alone, you usually don't need to change them.

Don't put any weird compiler options or other "optimizations" in /usr/local/etc/poudriere.d/make.conf. Use that file to set some defaults (DEFAULT_VERSIONS) or port options (OPTIONS_SET or OPTIONS_UNSET).


----------



## zirias@ (May 10, 2021)

mtu said:


> but I call that a mess.


And nobody cares.

(btw, the only thing that you could _maybe_ call a "mess" is the fact that `MAKE_JOBS_NUMBER` isn't documented in any manpage. Of course it _is_ documented in /usr/ports/Mk/bsd.port.mk, but that's not the most obvious place. OTOH, it's pretty obvious that overriding the _number_ of jobs is an entirely different thing than enabling/disabling parallel builds.)


----------

