# My exciting journey with poudriere on a Raspberry Pi



## eternal_noob (Nov 5, 2021)

Hi,

i played with poudriere and now i want to remove the files i created.
I removed the jail via `poudriere jails -d -j 13aarch64` and now want to remove the data directory in the poudriere directory i configured.

Unfortunately, i can't delete the files, even as root. What's wrong?

```
root@xxx:/home/xxx/poudriere # rm -rf data/
rm: data/.m/13aarch64-local-workstation/ref/lib/libcrypt.so.5: Operation not permitted
rm: data/.m/13aarch64-local-workstation/ref/lib/libthr.so.3: Operation not permitted
rm: data/.m/13aarch64-local-workstation/ref/lib/libc.so.7: Operation not permitted
rm: data/.m/13aarch64-local-workstation/ref/lib: Directory not empty
rm: data/.m/13aarch64-local-workstation/ref/usr/lib/librt.so.1: Operation not permitted
rm: data/.m/13aarch64-local-workstation/ref/usr/lib: Directory not empty
rm: data/.m/13aarch64-local-workstation/ref/usr/bin/chpass: Operation not permitted
rm: data/.m/13aarch64-local-workstation/ref/usr/bin/login: Operation not permitted
rm: data/.m/13aarch64-local-workstation/ref/usr/bin/opiepasswd: Operation not permitted
rm: data/.m/13aarch64-local-workstation/ref/usr/bin/opieinfo: Operation not permitted
rm: data/.m/13aarch64-local-workstation/ref/usr/bin/su: Operation not permitted
rm: data/.m/13aarch64-local-workstation/ref/usr/bin/crontab: Operation not permitted
rm: data/.m/13aarch64-local-workstation/ref/usr/bin/passwd: Operation not permitted
rm: data/.m/13aarch64-local-workstation/ref/usr/bin: Directory not empty
rm: data/.m/13aarch64-local-workstation/ref/usr: Directory not empty
rm: data/.m/13aarch64-local-workstation/ref/var/empty: Operation not permitted
rm: data/.m/13aarch64-local-workstation/ref/var: Directory not empty
rm: data/.m/13aarch64-local-workstation/ref/sbin/init: Operation not permitted
rm: data/.m/13aarch64-local-workstation/ref/sbin: Directory not empty
rm: data/.m/13aarch64-local-workstation/ref/libexec/ld-elf.so.1: Operation not permitted
rm: data/.m/13aarch64-local-workstation/ref/libexec: Directory not empty
rm: data/.m/13aarch64-local-workstation/ref: Directory not empty
rm: data/.m/13aarch64-local-workstation/02/libexec/ld-elf.so.1: Operation not permitted
rm: data/.m/13aarch64-local-workstation/02/libexec: Directory not empty
rm: data/.m/13aarch64-local-workstation/02/sbin/init: Operation not permitted
rm: data/.m/13aarch64-local-workstation/02/sbin: Directory not empty
rm: data/.m/13aarch64-local-workstation/02/var/empty: Operation not permitted
rm: data/.m/13aarch64-local-workstation/02/var: Directory not empty
rm: data/.m/13aarch64-local-workstation/02/usr/bin/passwd: Operation not permitted
rm: data/.m/13aarch64-local-workstation/02/usr/bin/crontab: Operation not permitted
rm: data/.m/13aarch64-local-workstation/02/usr/bin/su: Operation not permitted
rm: data/.m/13aarch64-local-workstation/02/usr/bin/opieinfo: Operation not permitted
rm: data/.m/13aarch64-local-workstation/02/usr/bin/opiepasswd: Operation not permitted
rm: data/.m/13aarch64-local-workstation/02/usr/bin/login: Operation not permitted
rm: data/.m/13aarch64-local-workstation/02/usr/bin/chpass: Operation not permitted
rm: data/.m/13aarch64-local-workstation/02/usr/bin: Directory not empty
rm: data/.m/13aarch64-local-workstation/02/usr: Directory not empty
rm: data/.m/13aarch64-local-workstation/02: Directory not empty
rm: data/.m/13aarch64-local-workstation/03/libexec/ld-elf.so.1: Operation not permitted
rm: data/.m/13aarch64-local-workstation/03/libexec: Directory not empty
rm: data/.m/13aarch64-local-workstation/03/sbin/init: Operation not permitted
rm: data/.m/13aarch64-local-workstation/03/sbin: Directory not empty
rm: data/.m/13aarch64-local-workstation/03/var/empty: Operation not permitted
rm: data/.m/13aarch64-local-workstation/03/var: Directory not empty
rm: data/.m/13aarch64-local-workstation/03/usr/bin/passwd: Operation not permitted
rm: data/.m/13aarch64-local-workstation/03/usr/bin/crontab: Operation not permitted
rm: data/.m/13aarch64-local-workstation/03/usr/bin/su: Operation not permitted
rm: data/.m/13aarch64-local-workstation/03/usr/bin/opieinfo: Operation not permitted
rm: data/.m/13aarch64-local-workstation/03/usr/bin/opiepasswd: Operation not permitted
rm: data/.m/13aarch64-local-workstation/03/usr/bin/login: Operation not permitted
rm: data/.m/13aarch64-local-workstation/03/usr/bin/chpass: Operation not permitted
rm: data/.m/13aarch64-local-workstation/03/usr/bin: Directory not empty
rm: data/.m/13aarch64-local-workstation/03/usr: Directory not empty
rm: data/.m/13aarch64-local-workstation/03: Directory not empty
rm: data/.m/13aarch64-local-workstation/04/libexec/ld-elf.so.1: Operation not permitted
rm: data/.m/13aarch64-local-workstation/04/libexec: Directory not empty
rm: data/.m/13aarch64-local-workstation/04/sbin/init: Operation not permitted
rm: data/.m/13aarch64-local-workstation/04/sbin: Directory not empty
rm: data/.m/13aarch64-local-workstation/04/var/empty: Operation not permitted
rm: data/.m/13aarch64-local-workstation/04/var: Directory not empty
rm: data/.m/13aarch64-local-workstation/04/usr/bin/passwd: Operation not permitted
rm: data/.m/13aarch64-local-workstation/04/usr/bin/crontab: Operation not permitted
rm: data/.m/13aarch64-local-workstation/04/usr/bin/su: Operation not permitted
rm: data/.m/13aarch64-local-workstation/04/usr/bin/opieinfo: Operation not permitted
rm: data/.m/13aarch64-local-workstation/04/usr/bin/opiepasswd: Operation not permitted
rm: data/.m/13aarch64-local-workstation/04/usr/bin/login: Operation not permitted
rm: data/.m/13aarch64-local-workstation/04/usr/bin/chpass: Operation not permitted
rm: data/.m/13aarch64-local-workstation/04/usr/bin: Directory not empty
rm: data/.m/13aarch64-local-workstation/04/usr: Directory not empty
rm: data/.m/13aarch64-local-workstation/04: Directory not empty
rm: data/.m/13aarch64-local-workstation/01/libexec/ld-elf.so.1: Operation not permitted
rm: data/.m/13aarch64-local-workstation/01/libexec: Directory not empty
rm: data/.m/13aarch64-local-workstation/01/sbin/init: Operation not permitted
rm: data/.m/13aarch64-local-workstation/01/sbin: Directory not empty
rm: data/.m/13aarch64-local-workstation/01/var/empty: Operation not permitted
rm: data/.m/13aarch64-local-workstation/01/var: Directory not empty
rm: data/.m/13aarch64-local-workstation/01/usr/bin/passwd: Operation not permitted
rm: data/.m/13aarch64-local-workstation/01/usr/bin/crontab: Operation not permitted
rm: data/.m/13aarch64-local-workstation/01/usr/bin/su: Operation not permitted
rm: data/.m/13aarch64-local-workstation/01/usr/bin/opieinfo: Operation not permitted
rm: data/.m/13aarch64-local-workstation/01/usr/bin/opiepasswd: Operation not permitted
rm: data/.m/13aarch64-local-workstation/01/usr/bin/login: Operation not permitted
rm: data/.m/13aarch64-local-workstation/01/usr/bin/chpass: Operation not permitted
rm: data/.m/13aarch64-local-workstation/01/usr/bin: Directory not empty
rm: data/.m/13aarch64-local-workstation/01/usr: Directory not empty
rm: data/.m/13aarch64-local-workstation/01: Directory not empty
rm: data/.m/13aarch64-local-workstation: Directory not empty
rm: data/.m: Directory not empty
rm: data/: Directory not empty
```


----------



## SirDice (Nov 5, 2021)

You need to remove the schg flag from those files/directories before you can remove them. `chflags -R noschg data/` should do the trick.


```
schg, schange, simmutable
                       set the system immutable flag (super-user only)
```
See chflags(1).


----------



## Geezer (Nov 5, 2021)

eternal_noob If you are not going to use poudriere, what ports manager will you use?


----------



## eternal_noob (Nov 5, 2021)

None. I use packages. Just wanted to play a bit with it.



SirDice said:


> You need to remove the schg flag from those files/directories before you can remove them. `chflags -R noschg data/` should do the trick.


That worked. Thank you.


----------



## astyle (Nov 6, 2021)

Sorry, eternal_noob !  Looks like you played with Poudriere on my suggestion, but that didn't work for you. Poudriere can be an unwieldy beast...


----------



## eternal_noob (Nov 6, 2021)

astyle said:


> Poudriere can be an unwieldy beast...


It all worked fine until i realized that compiling a single port would pull in 128 another dependencies. I currently only have a Raspberry Pi 400 available which has limited resources and an SD card.
I don't want to torment that card and wait until next year for the compilation to finish.

I guess i postpone being a ports maintainer.


----------



## astyle (Nov 6, 2021)

eternal_noob said:


> It all worked fine until i realized that compiling a single port would pull in 128 another dependencies. I currently only have a Raspberry Pi 400 available which has limited resources and an SD card.
> I don't want to torment that card and wait until next year for the compilation to finish.
> 
> I guess i postpone being a ports maintainer.


I'm trying to set up Poudriere for myself, but not with the goal of being a port maintainer. My goal is to compile KDE properly (with as many makefile knobs turned on as possible), and then keep upgrading that setup. Using pkg - I don't like the provided defaults, I want to do my own, which ports makes possible. Poudriere offers a reasonably complete avenue for automating the building I'd need to do.


----------



## cmoerz (Nov 6, 2021)

eternal_noob said:


> It all worked fine until i realized that compiling a single port would pull in 128 another dependencies. I currently only have a Raspberry Pi 400 available which has limited resources and an SD card.
> I don't want to torment that card and wait until next year for the compilation to finish.
> 
> I guess i postpone being a ports maintainer.


If I may hand out some unsolicited advice: if you can spare the money, get yourself a USB harddisk or SSD (preferably with a dedicated power line or splice additional power in because the Pi might not be able to serve the disk with sufficient power) and run your OS from that - the pi 400 can boot directly from USB. It's a totally different experience, a ton faster and you don't need to mind poudriere writing a ton of files and swapping - and yes, you will need swap.

I've been running poudriere on a Pi4/4GB and for some ports, it needed 16GB swap and multiple restarts to complete because it attempted to compile those large ports in parallel. It won't take forever to finish, but a 3-4 day run isn't too unusual, admittedly.

On the other hand, unless you update your ports tree all the time, you should be fine after a one-time compilation run of all your dependencies.


----------



## cmoerz (Nov 6, 2021)

I guess, there could be a bunch more "would like this too" on this poudriere issue








						Try to fetch dependencies instead of building them locally · Issue #319 · freebsd/poudriere
					

Testing ports before committing changes is expensive. So, is building overlay of a few ports with non-default options. /head is a fast moving target, ports -u often renders many out-of-date Tier1 s...




					github.com


----------



## astyle (Nov 6, 2021)

From /usr/local/etc/poudriere.conf.sample, lines 317 - 321:

```
# Only build what is requested. Do not rebuild build deps if nothing requested
# depends on them. This can create an inconsistent repository if you often
# build one-off packages but expect the repository to stay consistent.
# Defaut: yes
#TRIM_ORPHANED_BUILD_DEPS=yes
```
My Poudriere port was installed back in September. I'm still trying to set it up so that it works for my scenario. In my case, I'm trying to avoid rebuilding the QT ports.


----------



## eternal_noob (Nov 6, 2021)

Yeah, would be nice if you could tell Poudriere to only build the port requested and use installed packages for dependencies. I have all that stuff installed already but unfortunately, Poudriere doesn't give a damn.


----------



## astyle (Nov 6, 2021)

eternal_noob said:


> Yeah, would be nice if you could tell Poudriere to only build the port requested and use installed packages for dependencies. I have all that stuff installed already but unfortunately, Poudriere doesn't give a damn.


There should be a way to make Poudriere cooperate with installed stuff. Sometimes, that means studying the ports system, the /etc/make.conf file, and not depending on Poudriere to be smart enough to handle EVERYTHING. This is why I've been working on Poudriere setup since July.


----------



## chrbr (Nov 6, 2021)

As far as I remember it is now possible to fetch at least build dependencies. I have just checked the sample configuration file on githup. The interesting part is at the bottom of the file.

```
# Set to always attempt to fetch packages or dependencies before building.
# XXX: This is subject to change
# Default: off; requires -b <branch> for bulk or testport.
#PACKAGE_FETCH_BRANCH=latest
# The branch will be appended to the URL:
#PACKAGE_FETCH_URL=pkg+http://pkg.FreeBSD.org/\${ABI}
# Packages which should never be fetched.  This is useful for ports that
# you have local patches for as otherwise the patches would be ignored if
# a remote package is used instead.
#PACKAGE_FETCH_BLACKLIST=""
# Alternatively a whitelist can be created to only allow specific packages to
# be fetched.
# Default: everything
#PACKAGE_FETCH_WHITELIST="gcc* rust llvm*"
```
Currently I am back to packages. Therefore I have not tested that option. I have tried poudriere to see how it works some time ago. But lazy as I am I can live with the defaults of the package repository.


----------



## linux->bsd (Nov 6, 2021)

chrbr said:


> As far as I remember it is now possible to fetch at least build dependencies. I have just checked the sample configuration file on githup. The interesting part is at the bottom of the file.



Glad to see that feature finally made it over from ports-mgmt/poudriere-devel.

poudriere-bulk():


```
-b name  Specify the name of the binary package branch to use to prefetch
              packages.  Should be "latest", "quarterly", "release_*", or url.
              With this option poudriere will first try to fetch from the
              binary package repository specified the binary package prior to
              do the sanity check if the package does not already exist.

              poudriere will only use packages that:
              •   come from a repository having the same or older version of
                  pkg.
              •   do not have a locally fetched package already.
              •   are not IGNORED.
              •   match the expected local version.
              •   match the expected ABI.
              •   match the expected runtime and library dependencies.
              •   match the expected OPTIONS when CHECK_CHANGED_OPTIONS is
                  enabled (default: on).
              •   The package is NOT listed in PACKAGE_FETCH_BLACKLIST
              •   The package is NOT listed with -C, or -c, when -t is used.
              The -v flag can be used to show these decisions during build.
              Specifing twice will show more details on why some are skipped.

              !!

              poudriere has no way of determining differences outside of the
              above list.  That is, if the local ports framework, or port, has
              custom patches or special WITH_FOO knobs (not OPTIONS) then it
              is required to add its name into PACKAGE_FETCH_BLACKLIST.
              Otherwise a package may be fetched and used that lacks the
              custom patch or knob.

              !!

              See PACKAGE_FETCH_BRANCH, PACKAGE_FETCH_URL,
              PACKAGE_FETCH_BLACKLIST, and PACKAGE_FETCH_WHITELIST in
              poudriere.conf.sample.  The entries in the lists will be matched
              against package names without versions.
```


----------



## grahamperrin@ (Nov 7, 2021)

eternal_noob said:


> … games/stonesoup …



Yeah, with poudriere-devel (for example): 


```
root@mowa219-gjp4-8570p-freebsd:~ # poudriere bulk -j main games/stonesoup
[00:00:00] Creating the reference jail... done
[00:00:26] Mounting system devices for main-default
[00:00:26] Stashing existing package repository
[00:00:26] Mounting ccache from: /var/cache/ccache
[00:00:26] Mounting ports from: /usr/local/poudriere/ports/default
[00:00:26] Mounting packages from: /usr/local/poudriere/data/packages/main-default
[00:00:26] Mounting distfiles from: /usr/ports/distfiles
[00:00:26] Copying /var/db/ports from: /usr/local/etc/poudriere.d/main-options
[00:00:26] Appending to make.conf: /usr/local/etc/poudriere.d/make.conf
/etc/resolv.conf -> /usr/local/poudriere/data/.m/main-default/ref/etc/resolv.conf
[00:00:26] Starting jail main-default
[00:00:27] Will build as nobody:nobody (65534:65534)
[00:00:27] Logs: /usr/local/poudriere/data/logs/bulk/main-default/2021-11-07_15h36m07s
[00:00:27] Loading MOVED for /usr/local/poudriere/data/.m/main-default/ref/usr/ports
[00:00:28] Ports supports: FLAVORS SELECTED_OPTIONS
[00:00:30] Inspecting ports tree for modifications to git checkout... yes
[00:00:53] Ports top-level git hash: 5f8277c28b (dirty)
[00:00:53] Gathering ports metadata
[00:00:54] Calculating ports order and dependencies
[00:00:55] Trimming IGNORED and blacklisted ports
[00:00:55] Package fetch: Looking for missing packages to fetch from pkg+http://pkg.FreeBSD.org/${ABI}/latest
Updating FreeBSD repository catalogue...
[main-default] Fetching meta.conf: 100%    163 B   0.2kB/s    00:01   
[main-default] Fetching packagesite.pkg: 100%    6 MiB   3.3MB/s    00:02   
Processing entries: 100%
FreeBSD repository update completed. 31180 packages processed.
All repositories are up to date.
[00:01:07] Package fetch: Will fetch 4 packages from remote or local pkg cache
The following packages will be fetched:

New packages to be FETCHED:
        libyaml: 0.2.5 (71 KiB: 1.15% of the 6 MiB to download)
        lua51: 5.1.5_9 (166 KiB: 2.68% of the 6 MiB to download)
        py38-yaml: 5.4.1 (152 KiB: 2.45% of the 6 MiB to download)
        stonesoup: 0.23.2_1 (6 MiB: 93.72% of the 6 MiB to download)

Number of packages to be fetched: 4

The process will require 6 MiB more space.
6 MiB to be downloaded.
[main-default] Fetching lua51-5.1.5_9.pkg: 100%  166 KiB 170.1kB/s    00:01   
[main-default] Fetching py38-yaml-5.4.1.pkg: 100%  152 KiB 155.8kB/s    00:01   
[main-default] Fetching stonesoup-0.23.2_1.pkg: 100%    6 MiB   6.0MB/s    00:01   
[main-default] Fetching libyaml-0.2.5.pkg: 100%   71 KiB  73.1kB/s    00:01   
[00:01:09] Sanity checking the repository
[00:01:09] Checking packages for incremental rebuild needs
[00:01:10] Deleting stale symlinks... done
[00:01:10] Deleting empty directories... done
[00:01:10] Package fetch: Generating logs for fetched packages
[00:01:15] Unqueueing existing packages
[00:01:15] Unqueueing orphaned build dependencies
[00:01:15] Sanity checking build queue
[00:01:15] Processing PRIORITY_BOOST
[00:01:15] Balancing pool
[main-default] [2021-11-07_15h36m07s] [load_priorities:] Queued: 4  Built: 0  Failed: 0  Skipped: 0  Ignored: 0  Fetched: 4  Tobuild: 0   Time: 00:00:48
[00:01:15] Recording filesystem state for prepkg... done
[00:01:15] Creating pkg repository
Creating repository in /tmp/packages: 100%
Packing files for repository: 100%
[00:01:31] Committing packages to repository: /usr/local/poudriere/data/packages/main-default/.real_1636299458 via .latest symlink
[00:01:31] Removing old packages
[00:01:31] Fetched ports: lang/lua51 games/stonesoup textproc/libyaml devel/py-yaml@py38
[main-default] [2021-11-07_15h36m07s] [committing:] Queued: 4  Built: 0  Failed: 0  Skipped: 0  Ignored: 0  Fetched: 4  Tobuild: 0   Time: 00:01:03
[00:01:31] Logs: /usr/local/poudriere/data/logs/bulk/main-default/2021-11-07_15h36m07s
[00:01:31] Cleaning up
main-default: removed
main-default-n: removed
[00:01:31] Unmounting file systems
root@mowa219-gjp4-8570p-freebsd:~ #
```

Forcing a build of the port: 


```
root@mowa219-gjp4-8570p-freebsd:~ # poudriere bulk -j main -Ct games/stonesoup
[00:00:00] Creating the reference jail... done
[00:00:19] Mounting system devices for main-default
[00:00:19] Stashing existing package repository
[00:00:19] Mounting ccache from: /var/cache/ccache
[00:00:19] Mounting ports from: /usr/local/poudriere/ports/default
[00:00:19] Mounting packages from: /usr/local/poudriere/data/packages/main-default
[00:00:19] Mounting distfiles from: /usr/ports/distfiles
[00:00:19] Copying /var/db/ports from: /usr/local/etc/poudriere.d/main-options
[00:00:19] Appending to make.conf: /usr/local/etc/poudriere.d/make.conf
/etc/resolv.conf -> /usr/local/poudriere/data/.m/main-default/ref/etc/resolv.conf
[00:00:19] Starting jail main-default
[00:00:19] Will build as nobody:nobody (65534:65534)
[00:00:19] Logs: /usr/local/poudriere/data/logs/bulk/main-default/2021-11-07_15h37m47s
[00:00:19] Loading MOVED for /usr/local/poudriere/data/.m/main-default/ref/usr/ports
[00:00:20] Ports supports: FLAVORS SELECTED_OPTIONS
[00:00:20] Inspecting ports tree for modifications to git checkout... yes
[00:00:25] Ports top-level git hash: 5f8277c28b (dirty)
[00:00:25] Gathering ports metadata
[00:00:26] Calculating ports order and dependencies
[00:00:26] -C specified, cleaning listed packages
[00:00:26] (-C) Will delete existing package: stonesoup-0.23.2_1.pkg
[00:00:26] (-C) Flushing package deletions
[00:00:27] Trimming IGNORED and blacklisted ports
[00:00:27] Package fetch: Looking for missing packages to fetch from pkg+http://pkg.FreeBSD.org/${ABI}/latest
[00:00:27] Package fetch: No eligible missing packages to fetch
[00:00:27] Sanity checking the repository
[00:00:27] Checking packages for incremental rebuild needs
[00:00:30] Deleting stale symlinks... done
[00:00:30] Deleting empty directories... done
[00:00:30] Unqueueing existing packages
[00:00:30] Unqueueing orphaned build dependencies
[00:00:30] Sanity checking build queue
[00:00:30] Processing PRIORITY_BOOST
[00:00:31] Balancing pool
[main-default] [2021-11-07_15h37m47s] [balancing_pool:] Queued: 1  Built: 0  Failed: 0  Skipped: 0  Ignored: 0  Fetched: 0  Tobuild: 1   Time: 00:00:11
[00:00:31] Recording filesystem state for prepkg... done
[00:00:31] Building 1 packages using 1 builders
[00:00:31] Starting/Cloning builders
[00:00:31] Hit CTRL+t at any time to see build progress and stats
[00:00:31] [01] [00:00:00] Building games/stonesoup | stonesoup-0.23.2_1
…
```


----------



## eternal_noob (Nov 19, 2021)

grahamperrin said:


> Yeah, with poudriere-devel (for example):


I wanted to try your method with poudriere-devel but it seems like `poudriere bulk -j main games/stonesoup` won't prefetch the packages but wants to build them.
What am i missing?

```
root@xxx:~ # poudriere bulk -j main games/stonesoup 
[00:00:00] Creating the reference jail... done
[00:05:32] Mounting system devices for main-default
[00:05:32] Warning: Using packages from previously failed, or uncommitted, build: /usr/local/poudriere/data/packages/main-default/.building
[00:05:32] Mounting ports from: /usr/local/poudriere/ports/default
[00:05:32] Mounting packages from: /usr/local/poudriere/data/packages/main-default
[00:05:33] Mounting distfiles from: /usr/ports/distfiles
/etc/resolv.conf -> /usr/local/poudriere/data/.m/main-default/ref/etc/resolv.conf
[00:05:34] Starting jail main-default
[00:05:34] Will build as nobody:nobody (65534:65534)
[00:05:36] Logs: /usr/local/poudriere/data/logs/bulk/main-default/2021-11-19_19h27m09s
[00:05:36] Loading MOVED for /usr/local/poudriere/data/.m/main-default/ref/usr/ports
[00:05:38] Ports supports: FLAVORS SELECTED_OPTIONS
[00:05:38] Inspecting ports tree for modifications to git checkout... no
[00:06:45] Ports top-level git hash: e449f292c 
[00:06:45] Gathering ports metadata
[00:06:46] Calculating ports order and dependencies
[00:06:46] pkg package missing, cleaning all packages... done
[00:06:46] Trimming IGNORED and blacklisted ports
[00:06:46] Sanity checking the repository
[00:06:46] Checking packages for incremental rebuild needs
[00:06:46] Deleting stale symlinks... done
[00:06:46] Deleting empty directories... done
[00:06:46] Unqueueing existing packages
[00:06:46] Unqueueing orphaned build dependencies
[00:06:47] Sanity checking build queue
[00:06:47] Processing PRIORITY_BOOST
[00:06:47] Balancing pool
[main-default] [2021-11-19_19h27m09s] [balancing_pool:] Queued: 30 Built: 0  Failed: 0  Skipped: 0  Ignored: 0  Fetched: 0  Tobuild: 30  Time: 00:01:10
[00:06:47] Recording filesystem state for prepkg... done
[00:06:47] Building 30 packages using 4 builders
[00:06:47] Starting/Cloning builders
```


----------



## astyle (Nov 19, 2021)

Could there be a difference between the *in-jail* /var/db/pkg and *bare-metal* /var/db/pkg ? My thinking goes, if you don't null-mount /var/db/pkg in the jail, that might make things messy.


----------



## eternal_noob (Nov 19, 2021)

astyle said:


> Could there be a difference between the *in-jail* /var/db/pkg and *bare-metal* /var/db/pkg ?


/var/db/pkg on the host contains some files while the directory in the jail is empty.


```
root@xxx:~ # ls -ltr /var/db/pkg
total 67748
-r--r--r--  1 root  wheel   6805387 Sep 16 03:48 vuln.xml
-rw-r--r--  1 root  wheel  49852416 Nov 19 04:10 repo-FreeBSD.sqlite
-rw-r--r--  1 root  wheel       158 Nov 19 04:10 FreeBSD.meta
-rw-r--r--  1 root  wheel  12582912 Nov 19 17:57 local.sqlite

root@xxx:~ # ls -ltr /usr/local/poudriere/jails/main/var/db/pkg/
total 0
```



astyle said:


> My thinking goes, if you don't null-mount /var/db/pkg in the jail, that might make things messy.


I didn't configure anything, i just use the default configuration file of poudriere-devel.


----------



## astyle (Nov 19, 2021)

eternal_noob said:


> /var/db/pkg on the host contains some files while the directory in the jail is empty.


This is probably why Poudriere in the jail is trying to build the deps, even though you have them on the host.


----------



## eternal_noob (Nov 19, 2021)

I don't have the deps on the host because i thought the command i used will automagically fetch them.
I install the packages on the host and try again.

Edit: Nope, same result. Do i need to install the packages in the jail? Or install pkg in the jail in general?


----------



## astyle (Nov 19, 2021)

eternal_noob said:


> I don't have the deps on the host because i thought the command i used will automagically fetch them.
> I install the packages on the host and try again.


I thought the point of compiling in the jail was to avoid doing that? if the deps are getting pulled in, I'd be OK if it were happening in the jail, rather than on the host.


----------



## eternal_noob (Nov 19, 2021)

Edit: I totally misunderstood your last comment.

<redacted> 

So how do i install the dependencies as a package in the jail? Using the _-j_ parameter of `pkg` doesn't work:

```
root@xxx:~ # pkg -j main install stonesoup
pkg: jail "main" not found
```

 Sorry, i am a jail noob.


----------



## rigoletto@ (Nov 19, 2021)

TL;DR
If you are using zfs(8) poudriere create some zfs datasets you can't delete except using `zfs destroy`.

Run `zfs list` to have a look on the existing datasets.


----------



## eternal_noob (Nov 19, 2021)

The problem with the undeletable directory is solved. Maybe i should update the title of the thread.


----------



## astyle (Nov 19, 2021)

eternal_noob :  To keep track of whether you're in a jail or not, try launching a separate Xterm (or Konsole) window.  It sounds like you need to have a jail up and running, not just installed. Just having separate directories under the folder named main (the name of your jail) is not gonna help. For all intents and purposes, you can treat a jail as  VM.


----------



## Alain De Vos (Nov 19, 2021)

A jail should be treated as a separate "namespace" from the host.
For the sake of simplicity i ssh into my jails.
And all jails have "ip4 = inherit;  ip6 = inherit;"

/root/.zshrc in the jail has:
PROMPT='a:$fg[green]%n: $fg[default]%d #'
So it is clear i'm in the jail.


----------



## eternal_noob (Nov 19, 2021)

astyle said:


> It sounds like you need to have a jail up and running, not just installed.


Yeah, this is the problem here. I never worked with jails and only used `poudriere` to create the reference jail. I want poudriere to take care of all this stuff (for simplicity) and don't want to use jails regularly.

I think i let it build all the packages and see what the result is. For now it builds pkg in the jail:

```
[main-default] [2021-11-19_22h33m59s] [balancing_pool:] Queued: 30 Built: 0  Failed: 0  Skipped: 0  Ignored: 0  Fetched: 0  Tobuild: 30  Time: 00:00:36
[00:04:10] Recording filesystem state for prepkg... done
[00:04:10] Building 30 packages using 1 builders
[00:04:10] Starting/Cloning builders
[00:08:12] Hit CTRL+t at any time to see build progress and stats
[00:08:12] [01] [00:00:00] Building ports-mgmt/pkg | pkg-1.17.5
[00:12:57] [01] [00:04:45] Finished ports-mgmt/pkg | pkg-1.17.5: Success
[00:12:57] [01] [00:00:00] Building print/indexinfo | indexinfo-0.3.1
[00:17:45] [01] [00:04:48] Finished print/indexinfo | indexinfo-0.3.1: Success
[00:17:45] [01] [00:00:00] Building devel/gettext-runtime | gettext-runtime-0.21
```

This will take a while.


----------



## Alain De Vos (Nov 19, 2021)

Is cross-compiling from a speedy host not an option ?


----------



## eternal_noob (Nov 19, 2021)

Nope, i only have the Raspberry available at the moment..


----------



## SirDice (Nov 19, 2021)

eternal_noob said:


> ```
> [00:04:10] Building 30 packages using 1 builders
> ```


Unless this build contains 6 different compilers (LLVM and GCC) it should finish fairly quickly. You might want to increase the number of jobs though, so it can build several things in parallel.



eternal_noob said:


> I only have the Raspberry available at the moment..


Right. That's going to take some time then. The Pi isn't known for its speed.


----------



## eternal_noob (Nov 19, 2021)

SirDice said:


> Unless this build contains 6 different compilers (LLVM and GCC) it should finish fairly quickly.


Nope, it's been ~50 minutes and it only built 5 packages so far

```
[00:08:12] [01] [00:00:00] Building ports-mgmt/pkg | pkg-1.17.5
[00:12:57] [01] [00:04:45] Finished ports-mgmt/pkg | pkg-1.17.5: Success
[00:12:57] [01] [00:00:00] Building print/indexinfo | indexinfo-0.3.1
[00:17:45] [01] [00:04:48] Finished print/indexinfo | indexinfo-0.3.1: Success
[00:17:45] [01] [00:00:00] Building devel/gettext-runtime | gettext-runtime-0.21
[00:25:06] [01] [00:07:21] Finished devel/gettext-runtime | gettext-runtime-0.21: Success
[00:25:07] [01] [00:00:00] Building devel/libtextstyle | libtextstyle-0.21
[00:33:53] [01] [00:08:46] Finished devel/libtextstyle | libtextstyle-0.21: Success
[00:33:54] [01] [00:00:00] Building devel/gettext-tools | gettext-tools-0.21
[00:49:48] [01] [00:15:54] Finished devel/gettext-tools | gettext-tools-0.21: Success
[00:49:49] [01] [00:00:00] Building lang/perl5.32 | perl5-5.32.1_1
```



SirDice said:


> You might want to increase the number of jobs though, so it can build several things in parallel.


Nope, otherwise i can't surf the forums in parallel. Leaving the default of 4 build jobs just makes the Pi unresponsive.


----------



## Alain De Vos (Nov 19, 2021)

For raspberry pi i would put in make.conf

```
MAKE_JOBS_NUMBER=2
```


----------



## eternal_noob (Nov 19, 2021)

I used `poudriere bulk -j main games/stonesoup` to start the build, can i interrupt the build process via `Ctrl + c` and resume the build afterwards or will it create the reference jail again and start from scratch?


----------



## Alain De Vos (Nov 19, 2021)

You can stop it. The packages which are completely build will not be needed to rebuild when you restart.
Note when you stop it, the builds are not added to repository (which is mostly what you want)


----------



## eternal_noob (Nov 19, 2021)

Alain De Vos said:


> For raspberry pi i would put in make.conf
> 
> ```
> MAKE_JOBS_NUMBER=2
> ```


I've increased the number of builders to 2 but not in make.conf but in poudriere.conf

```
# parallel build support.
#
# By default poudriere uses hw.ncpu to determine the number of builders.
# You can override this default by changing PARALLEL_JOBS here, or
# by specifying the -J flag to bulk/testport.
#
# Example to define PARALLEL_JOBS to one single job
PARALLEL_JOBS=2
```

Multitasking is slow but acceptable.


----------



## eternal_noob (Nov 20, 2021)

The port built fine (after 5 hours for 30 packages) and this was only the console version.

It has an SDL flavor and i don't know if i should try to build it.
But out of interest: How do i enable another flavor with poudriere?

From the Makefile

```
FLAVORS=    console sdl
```

Do i use `poudriere options` or is it another command?


----------



## grahamperrin@ (Nov 20, 2021)

eternal_noob said:


> … `poudriere bulk -j main games/stonesoup` won't prefetch the packages but wants to build them.
> What am i missing?



Relevant lines may be missing from the configuration file. From the sample configuration:


```
root@mowa219-gjp4-8570p-freebsd:~ # tail -n 15 /usr/local/etc/poudriere.conf.sample  


# Set to always attempt to fetch packages or dependencies before building.
# XXX: This is subject to change
# Default: off; requires -b <branch> for bulk or testport.
#PACKAGE_FETCH_BRANCH=latest
# The branch will be appended to the URL:
#PACKAGE_FETCH_URL=pkg+http://pkg.FreeBSD.org/\${ABI}
# Packages which should never be fetched.  This is useful for ports that
# you have local patches for as otherwise the patches would be ignored if
# a remote package is used instead.
#PACKAGE_FETCH_BLACKLIST=""
# Alternatively a whitelist can be created to only allow specific packages to
# be fetched.
# Default: everything
#PACKAGE_FETCH_WHITELIST="gcc* rust llvm*"
root@mowa219-gjp4-8570p-freebsd:~ #
```


----------



## Alain De Vos (Nov 20, 2021)

PS for flavors,
In make.conf:

```
.if ${.CURDIR:M*/games/stonesoup}
FLAVOR=SDL
.endif
```
or

```
.if ${.CURDIR:M*/games/stonesoup}
FLAVOR=CONSOLE
.endif
```


----------



## astyle (Nov 20, 2021)

eternal_noob said:


> The port built fine (after 5 hours for 30 packages) and this was only the console version.
> 
> It has an SDL flavor and i don't know if i should try to build it.
> But out of interest: How do i enable another flavor with poudriere?
> ...


Off the top of my head, I think there's something in poudriere.conf to enable building a different flavor, as well. I'd caution you to educate yourself on usage of word 'flavor', because that can mean a different architecture like amd64 or aarch64. But this looks more like a Makefile option. You can tell Poudriere to look in custom options directory and follow that.


----------



## eternal_noob (Nov 20, 2021)

grahamperrin said:


> Relevant lines may be missing from the configuration file.


Oops. Thanks. I totally missed chrbr s post.


Alain De Vos said:


> In make.conf





astyle said:


> Off the top of my head, I think there's something in poudriere.conf to enable building a different flavor, as well.


Thanks to both of you, i will look into this.

Edit: I think i found the answer in the man page (poudriere(8)):


> A FLAVOR of *bar* for port _devel/foo_ is specified as *devel/foo@bar*



So the command should be `poudriere bulk -j main games/stonesoup@sdl`


----------



## eternal_noob (Nov 20, 2021)

For the record:

1. The @ to specify the flavor works.

2. PACKAGE_FETCH_BRANCH=latest doesn't work for aarch64, there are a lot of missing packages, e.g.

```
[00:08:53] Warning: libmodplug-0.8.9.0.pkg not found. Remote PKG_SUFX likely differs temporarily
[00:08:53] Warning: zopfli-1.0.3_1.pkg not found. Remote PKG_SUFX likely differs temporarily
[00:08:53] Warning: gperf-3.1.pkg not found. Remote PKG_SUFX likely differs temporarily
[00:08:53] Warning: evdev-proto-5.8.pkg not found. Remote PKG_SUFX likely differs temporarily
```

2. PACKAGE_FETCH_BRANCH=quarterly does work, but it too does not fetch all packages due to missing dependencies:

```
[00:07:58] Deleting dbus-glib-0.112.pkg: missing dependency: glib-2.70.1,2
[00:07:58] Deleting libxcb-1.14_1.pkg: missing dependency: libXdmcp-1.1.3
[00:07:59] Deleting minixmlto-0.0.3.pkg: missing dependency: libxslt-1.1.34_2
[00:07:59] Deleting libX11-1.7.2,1.pkg: missing dependency: libxcb-1.14_1
```

Out of 143 packages, it fetched only 56 and wants to build 87.

```
[main-default] [2021-11-20_05h18m42s] [balancing_pool:] Queued: 143 Built: 0   Failed: 0   Skipped: 0   Ignored: 0   Fetched: 56  Tobuild: 87   Time: 00:03:23
```

Not today.


----------



## grahamperrin@ (Nov 20, 2021)

eternal_noob said:


> Edit:



Crossed paths  with <https://forums.freebsd.org/threads/82848/post-542861>

Incidentally, following the example there: FreeBSD bug 259942 – sysutils/fusefs-ext2 and flavoured sysutils/e2fsprogs@nobootfsck



eternal_noob said:


> PACKAGE_FETCH_BRANCH=latest doesn't work for aarch64, there are a lot of missing packages,



For the packages that are not fetched, does it not proceed to build packages?


Hint: maybe change the title here, it's a broad discussion. Thanks.


----------



## eternal_noob (Nov 20, 2021)

grahamperrin said:


> For the packages that are not fetched, does it not proceed to build packages?


I expressed myself in a misleading way. Poudriere would proceed but won't fetch a single package but build them all.



grahamperrin said:


> Hint: maybe change the title here, it's a broad discussion.


Will do.


----------



## grahamperrin@ (Nov 20, 2021)

linux->bsd said:


> … Glad to see that feature finally made it over from ports-mgmt/poudriere-devel.
> 
> poudriere-bulk():
> 
> ...



As far as I know, it's *not* yet a feature of the non -devel port <https://www.freshports.org/ports-mgmt/poudriere/#history> at 3.3.7_1.

Option *-b*:

does not exist at <https://github.com/freebsd/poudriere/wiki/poudriere-bulk.8#OPTIONS>
does exist at <https://github.com/freebsd/poudriere/wiki/poudriere-bulk.8-devel#OPTIONS>
FreeBSD bug 256676 – ports-mgmt/poudriere manpage of poudriere-bulk(8) claims features which aren't available in 3.3.6



> … please delete the Poudriere manpages. It's giving people the wrong information.


----------



## astyle (Dec 4, 2021)

eternal_noob : I re-read this thread. Comments on Page 1 of this thread seem to touch on the same issues I'm having (with Poudriere not 'seeing' stuff that it should be able to 'see' from a jail). In my case, I was able to isolate the issue to null-mounting. A workaround that I'm thinking of is to fake a remote repo (call it repo A), tell Git and Poudriere about repo A being a default 'remote repo', then `poudriere ports` should use git to pull stuff from remote (repo A) to local, and then it should be visible, no problem. I'm still in the process of setting it up and seeing if that way will even work.


----------

