# Managing sets of patches (through profiles) using build managers



## Mjölnir (Mar 8, 2021)

This is a continuation of a discussion that went off-topic.  See also this post ff. Snurg & me then had a PM discussion (in german) on the topic, drop a note to get invited (PMs are not visible for other's), we can switch to english.  We welcome any valuable feedback & ideas.  Let's start to brainstorm...


Snurg said:


> Hmm Mjölnir ...
> I just got an idea when I read that
> You know, I asked for poudriere/synth's properties...
> it is actually intended for bugfix patches that are not (yet) integrated.
> ...


Quickly hacking in a patch is one thing, writing a set of tests - ideally before that - it (the patched software) has to pass, is another topic...

So you want to write a connection shim layer, plus interface (e.g. web and/or CLI dialog(1) and/or Qt's QML).  This layer sits between a build manager & the FreeBSD Bugzilla, to pull in patches that are not yet commited, apply these, build & run tests in a jail or VM, all on a mouse or keyboard click.  This could help the commiters, because you can provide reports on test results, success, or failure of the patch in a controlled environment, e.g. a clean jail or bhyve VM with only this one patch applied.  Or a bunch of patches; however it is a well defined environment, and the test results of the tests you run within are your protocol to file back upstream.

That's a very good idea, thus it deserves a well thought out & clean architecture, i.e. clean separation of backend logic from config & UI, as well as the connection shims to both sides.  Add to that list: choosing an appropriate model (OOP, functional, declarative, ...) & a suitable programming language for each part, i.e. it's common to mix models & languages in a project, instead of writing all in that one favourite model & language of yours.

How to find pitfalls before you start hacking in erroneous scripts?  I mean: how to avoid mistakes before doing them, and not reinvent the wheel but make it quadratic?  Managing configurations & patches is a well known domain for decades, and open source solutions to help managing it already exist.  Did you search the ports for anything similar?  If not there, I'd strongly guess we can find something similar in the habitat of a certain open source OS, because the folks living there have managed to survive in a djungle of patches, whose shere amount & complexity is unsurpassed by any BSD, current and those to come.  If there's already s/th similar, it might be easier to add the features you're missing, than to start from scratch.

If there's nothing that's near to fulfill your requirements, let's start with writing tests, apply KISS whenever possible & follow vigole's advice (_"When in Rome, do as Romans do")_: we have atf(7) which uses kyua(1), so let's just stick with that.


----------



## Snurg (Mar 8, 2021)

Mjölnir, you are thinking much farther than I.

*I just imagined sort of easy-to-use "patch management tool".
Via a web browser, controlled by a cgi on the PC.* (note below *)

Let me explain my problem.
If I want to add a number of patches for

personal modifications, tailoring of applications or
testing bugfixes and modding,
this is much hard and tedious manual work for a GUI degenerate like me.

*What I miss is an easy way to do this.*
Through the clouds in my glass orb, I see a page on the browser, which shows me

the patches the community works on currently
patches/fixes for the most important bugs which, aren't getting fixed upstream or in the ports/packages
as well as for my personal favorite build etc options
Each entry similar like a Freshports port entry, with description and links.
But, in addition to this there are also links (bugzilla, phabricator, github, ...), checkboxes and buttons.
So I can view/edit, install, update, remove, etc a patch or set of patches with a few clicks.

Ideally such would be well community-brainstormed.
So that it makes collaborate developing and testing easier, thus accessible for more people.
Both FreeBSD developers and users could profit.

Imagine just click and let the computer work.
Without the boring and error-prone hassle of doing that "manually".

*How would you like such a patch manager tool to look and work?
Which functionalities would you like to see/have?*


(*) Of course, behind the scenes a set of scripts works, which can be used perfectly without GUI...


----------



## Jose (Mar 8, 2021)

Mjölnir said:


> So you want to write a connection shim layer, plus interface (e.g. web and/or CLI dialog(1) and/or Qt's QML).  This layer sits between a build manager & the FreeBSD Bugzilla, to pull in patches that are not yet commited, apply these, build & run tests in a jail or VM, all on a mouse or keyboard click.  This could help the commiters, because you can provide reports on test results, success, or failure of the patch in a controlled environment, e.g. a clean jail or bhyve VM with only this one patch applied.  Or a bunch of patches; however it is a well defined environment, and the test results of the tests you run within are your protocol to file back upstream.


This sort of functionality is built in to many code-review tools. I'm surprised to see that it's "very rough" in Phabricator:





						◉ Harbormaster User Guide
					






					secure.phabricator.com
				




The one I've used and liked in the past is Gerrit:


			gerrit-ci-scripts - Git at Google
		


Github and Gitlab have similar functionality:





						CI/CD pipelines | GitLab
					

Documentation for GitLab Community Edition, GitLab Enterprise Edition, Omnibus GitLab, and GitLab Runner.




					docs.gitlab.com
				




The idea is too add the build system and automated tests as code reviewers. No need to bug human reviewers until and unless your broken crap compiles and passes the tests.


----------



## Snurg (Mar 8, 2021)

Sadly these professional tools like Harbormaster are way too complicated for retards like me. 

What I think of is a user-selectable choice, of maybe the top ten or twenty patches/fixes/modified builds, but also things like configuration changes. Extremely easy to install/remove etc, PC-BSD style.

Examples:
Remove libinput (when it breaks too much functionality or makes mouse unusable in other ways)
Add/Activate Zap keys
etc etc

So one can do these things that would each cost a more or less substantial amount of time again and again for every installation very quickly and easily, with just a few clicks.


----------



## Mjölnir (Mar 8, 2021)

Snurg said:


> Sadly these professional tools like Harbormaster are way too complicated for retards like me.


Well, that's reasonable because it's a complex domain.  Usually complex problems require complex solutions.  It's extremely rare that an easy solution fits, let's say 98% of the problem's incarnations.


Snurg said:


> Remove libinput (when it breaks too much functionality or makes mouse unusable in other ways)


This could serve as a good example why it's not as easy as providing a knob: "[x] remove libinput"
You could easily screw up your system, because some vital ports(7) you need for your profile (set of ports) depend on that library.


----------



## Jose (Mar 8, 2021)

Mjölnir said:


> This could serve as a good example why it's not as easy as providing a knob: "[x] remove libinput"
> You could easily screw up your system, because some vital ports(7) you need for your profile (set of ports) depend on that library.


Exactly. Some ports you didn't even build (because you mixed locally-built ports with standard upstream ports) might break because they were built with libinput=ON and you just yanked the rug out from under them.

The only safe way to do things like remove libinput is to walk the entire dependency tree and figure out what needs to be re-built. This is what Poudriere does, perhaps not always efficiently, but always safely. I value stability over speed so I'll spend the extra time.

Having said that, I have zero experience with Synth. Perhaps it's safe and efficient. I don't know or care to find out.


----------



## Snurg (Mar 9, 2021)

Maybe I have a misconception that for example, setting flags like libinput=OFF in, say,  /etc/make.conf would cause rebuilding all packages that have different libinput configuration settings or depend on anything that is being pulled in by libinput.

I thought this would result in a big initial (re-)build of a lot of packages on the very first pkg calls, always when packages are to be installed which in their default configuration depend on libinput.

The idea is to shield the user from the related build work, and to reduce the build/install times to the bare necessary minimum by building on-demand.
If possible, I particularly would like to avoid the need to tell the build manager in advance which ports/packages to build.
Ideally the user only needs to talk to pkg, and not with both pkg and build manager.


----------



## Jose (Mar 9, 2021)

Snurg said:


> The idea is to shield the user from the related build work, and to reduce the build/install times to the bare necessary minimum by building on-demand.


I would optimize this by not doing it at all. Point the system at a package repository with all the necessary packages pre-built with all the custom patches and options.


----------

