# Does anyone here use poudriere-devel?



## Jose (Apr 21, 2020)

I'm thinking about trying it because I want to try out the overlays feature:





						FreeBSD Mail Archives
					






					docs.freebsd.org


----------



## SirDice (Apr 22, 2020)

Yes, but I haven't been able to try that overlay feature. It looks like it could be useful for me but I just haven't gotten around to it yet.


----------



## getopt (Apr 22, 2020)

Jose what exactly are you trying to achieve? Just experimenting?
Remember you can manage the ports tree with a version control system.


----------



## Lamia (Apr 22, 2020)

Poudriere-devel is used here. No knowledge of the overlay though.


----------



## Jose (Apr 23, 2020)

I want to have a few selected ports that I actually work on in my home directory, owned by me. I want everything else to live in the usual /usr/ports location and to be updated in the usual ways. This is so I can edit files in the ports in my home dir, and then kick off poudriere(8) builds that will incorporate my latest changes.  I'm trying to make my build/run/debug cycle simple and idiot-resistant.


----------



## Lamia (Apr 28, 2020)

Would it be good to continue using these settings for a Quad-core 2GHz 32GBRAM?
PARALLEL_JOBS=24
PREPARE_PARALLEL_JOBS=36

So far, Poudriere rocks well; some services continue to go down during each build though e.g. mysql.


----------



## zirias@ (Apr 28, 2020)

Lamia it depends a lot on what you want to accomplish. In general, it never makes sense to give a system more _runnable_ jobs than it can run in parallel. Of course, with jobs waiting for I/O, they can't run, so it might be beneficial to have more jobs than cores/threads. Poudriere can make use of tmpfs which reduces the amount of I/O needed for building a lot, so if you have enough spare RAM, that's what I would recommend (and with 32GB, I assume it would be applicable for you). Then you can comment out the `PARALLEL_JOBS`, so poudriere will run exactly as many jobs as there are cores/threads. The prepare phase will always be I/O-bound, so it might speed up things to keep the parallel jobs higher for that.

If your system is running other services at the same time and you want theses services to stay "responsive", a good way is to run poudriere on idle priority, so the build jobs only occupy CPU when there's nothing more important to do. But then, I/O will often require some kernel-code to run, and this code will run on higher priority anyways, see my recent thread (and some very insightful answers there): https://forums.freebsd.org/threads/...ices-when-a-normal-prio-task-could-run.75040/ -- therefore, if your goal is to run poudriere as fast as possible while still not affecting performance of other services on the same machine, the best way is to keep `PREPARE_PARALLEL_JOBS` at default as well.


----------



## Lamia (Apr 28, 2020)

Thanks Zirias, running poudriere on idle priority would be great if it can just be done by adjusting the settings in poudriere.conf.

The System goes into "out of swap" mode when Firefox, Chromium and Webkit2-gtk3 are concurrently being built. We always have to power cycle it to get going again.

We'll now try enter same values for "PRIORITY_BOOST & ALLOW_MAKE_JOBS_PACKAGES".


----------



## zirias@ (Apr 28, 2020)

Lamia said:


> Thanks Zirias, running poudriere on idle priority would be great if it can just be done by adjusting the settings in poudriere.conf.


No need to configure anything for that, just run e.g. `idprio 16 poudriere [...]` (or whatever idle priority you want), it is automatically inherited by any process poudriere starts.


Lamia said:


> The System goes into "out of swap" mode when Firefox, Chromium and Webkit2-gtk3 are concurrently being built. We always have to power cycle it to get going again.


Ok, this sounds more like you are already short of RAM, so I retract my advice to use tmpfs. Of course, the "best" fix would be to put more RAM into that machine


----------



## Lamia (Apr 28, 2020)

Zirias said:


> No need to configure anything for that, just run e.g. `idprio 16 poudriere [...]` (or whatever idle priority you want), it is automatically inherited by any process poudriere starts.
> 
> Ok, this sounds more like you are already short of RAM, so I retract my advice to use tmpfs. Of course, the "best" fix would be to put more RAM into that machine


We shall try the  idprio.

I thought we had more than enough. Only 60% of the RAM has been used at peak time. I must add that we have a 16GB md99 swap file on the system too. 70% of it is used at build peak. Buying another 15GB RAM won't help, we think. At peak, we still have about 10G RAM available for use over tmux cpu-mem and MMonit readings. 
MySQL service continually restarts at this time.


----------



## Lamia (Apr 28, 2020)

rtprio throws an error in a jail - idprio: RTP_SET: Operation not permitted.

See https://forums.freebsd.org/threads/...edparam-failed-operation-not-permitted.57657/

Any suggestions?


----------



## Lamia (Apr 29, 2020)

Setting security.bsd.unprivileged_idprio=1 isn't the best option. Until we further increase the RAM (though we ain't convinced we need more), it might be our only option.


----------



## zirias@ (Apr 29, 2020)

Well, priority won't solve issues with RAM anyways, this is just about not stealing CPU time from more important tasks. I never ran into this issue as I have poudriere in a jail and the whole jail runs on idle priority  But I don't see a problem with allowing ordinary users to use idle priority -- maybe I'm overlooking something here.


----------

