# Ports manpage: Need for clarification of make target function and behavior



## Snurg (Jan 14, 2021)

Reading the ports(7) manpage, I am having difficulties to find out how I can do an unattended scripted install.

For example, when I do `make extract`, I get a build options dialog(1), and after exiting the dialog, the port was apparently built.
What I expected, however, was make just extracting the archived files, like with any packer.
Nothing more. _In particular, no passing through the subsequent steps._

Side note, what worried me somewhat also was that I didn't see the screen notices I expected to see from the '_patch_' step. I don't know whether it is intended behavior If the '_patch_' step apparently gets skipped if one does a `make fetch; make extract; make install` sequence?!?

Anyway, what I need is to find how I can do a *completely non-interactive* scripted install of Apache, about this way:
`make fetch
make extract [without interactive dialog]
make patch
<some source file editing, either via perl or sed>
<edit the make configuration (change from '-suexec' to '+suexec' or the like) via script>
make build
make install`

I also failed to find out where the build configuration I entered in the dialog I got after `make extract` is stored. 

Thank you for any assistance


----------



## SirDice (Jan 14, 2021)

Snurg said:


> Side note, what worried me somewhat also was that I didn't see the screen notices I expected to see from the '_patch_' step. I don't know whether it is intended behavior If the '_patch_' step apparently gets skipped if one does a `make fetch; make extract; make install` sequence?!?


The step isn't skipped because it's also part of the install target. In order to 'install' it has to have done 'build', in order for 'build' to complete it has to complete 'configure', 'configure' needs 'patch', 'patch' needs 'extract' etc. So each step is eventually executed. It keeps track of these steps by setting 'flag' files in WRKDIRPREFIX:

```
root@molly:/usr/ports/security/sshguard # ll work/.[^.]*
-rw-r--r--  1 root  wheel    0 Jan 14 22:48 work/.build_done.sshguard._usr_local
-rw-r--r--  1 root  wheel    0 Jan 14 22:48 work/.configure_done.sshguard._usr_local
-rw-r--r--  1 root  wheel    0 Jan 14 22:48 work/.extract_done.sshguard._usr_local
-rw-r--r--  1 root  wheel  209 Jan 14 22:48 work/.license-catalog.mk
-rw-r--r--  1 root  wheel   81 Jan 14 22:48 work/.license-report
-rw-r--r--  1 root  wheel    0 Jan 14 22:48 work/.license_done.sshguard._usr_local
-rw-r--r--  1 root  wheel    0 Jan 14 22:48 work/.patch_done.sshguard._usr_local
```
(I picked security/sshguard because it was easy, did all the steps and didn't require any dependencies)



Snurg said:


> I also failed to find out where the build configuration I entered in the dialog I got after `make extract` is stored.


The configuration options (`make config`) are stored in PORT_DBDIR which is /var/db/ports/ by default. The build itself happens in WRKDIRPREFIX, which defaults to the work directory in the port's directory.

As for the specific question to enable SUEXEC on www/apache24, add this to your /etc/make.conf:

```
.if ${.CURDIR:M*/www/apache24}
  OPTIONS_SET+=SUEXEC
.endif
```
Or

```
SUEXEC_SET
```

You can also use it on the command line: `make -DWITH_SUEXEC install`

Have a look through /usr/ports/Mk/bsd.options.mk.


----------



## Snurg (Jan 14, 2021)

Wow this answer points at just the right points!
Thank you so much again!

I probably missed the patch messages, as the following `dialog` quickly overwrote the screen. My mistake then.

*Is there a way to prevent  make pass/fall through to the next target(s)?*
Reason is, I would like to have my script call `make fetch; make extract; make patch` one-by-one to do these steps non-interactively.
Then I would like to skip the `make config` step, letting my script do the changes itself, and `touch`ing the appropriate 'flag file'.
After that I would like my script to call `make build; make install`.

This whole thing confuses me so much because, after `make fetch` make did *not* pass/fall through to the next target '_extract_'.
This was the behaviour I expected the other targets to run, too.

So I feel very confused and stupid, as I do not understand why the other targets seem to pass/fall through.
*How can I call a non-interactive  make extract; make patch that does just this, without doing '*_config_*', '*_build_*' and '*_install_*' afterwards?*


----------



## ljboiler (Jan 14, 2021)

From list of targets in bsd.port.mk:

```
# config                - Configure options for this port (using ${DIALOG}).
#                                 Automatically run prior to extract, patch, configure, build,
#                                 install, and package.
```
Defining the make variable *BATCH* turns off the automatic running of the '_config_' target, and make will run using the
default options for the port).  The '_config_' is also skipped if options have already been saved for the port.

The following should do what you're looking for:

```
make -DBATCH extract
make -DBATCH patch
```


----------



## Snurg (Jan 14, 2021)

Somehow I missed the part about suexec the first time I read your answer...
Another 
Thank you again 

In particular I am glad to learn I can avoid modifying /etc/make.conf by just invoking with option `make -DWITH_SUEXEC install`!
Reading /usr/ports/Mk/bsd.options.mk, I see not only WITH, but also things like WITHOUT, _SET, _UNSET and more 
So no need to touch things in /etc just to build a port!

I'll test tomorrow whether I just can prepend `env ASSUME_ALWAYS_YES=YES` and call the apache24 make with `-DWITH_SUEXEC -DBATCH`, and report back


----------



## SirDice (Jan 15, 2021)

Snurg said:


> I'll test tomorrow whether I just can prepend `env ASSUME_ALWAYS_YES=YES`


That's for pkg-install(8), it doesn't do anything when building ports, `make install` never asks if you're sure.


----------



## Snurg (Jan 19, 2021)

*Finding 1:*
For some reason the presence of the -j option breaks the make run.
It apparently only works well in the `make build` step.


*Finding 2:*
Furthermore, I found out that `make extract` failed when trying to install the build and library dependencies (see the freshports info).
To me, it seems sensible to install these separately before doing the work on the port.
So it is made sure that packages get installed and "no more ports are being opened".
These build and library dependencies can be easily looked up at freshports.org.


Now only the last step, starting Apache, still fails.

I seem unable to `service start` or `onestart` Apache.


```
httpd does not exist in /etc/rc.d or the local startup
directories (/usr/local/etc/rc.d), or is not executable
executer: Got nonzero result from 'service httpd onestart' at ./bootie.pl
root@wurst:~ #
```

When I check, I find `apache24_enable="YES"` in /etc/rc.conf.
In /usr/local/etc/rc.d I find two scripts, apache24 and htcacheclean.
In /etc/rc.d there is no script of apache.
In /usr/local/etc/apache24 there is the updated httpd.conf.

So I do not understand what is wrong/missing.

Below you can see what happened until the failing last step `service httpd onestart`.


```
zfs snapshot -r wurstpool@bootie_snap_before_install_2021-01-19-16:57:20
env ASSUME_ALWAYS_YES=YES PAGER=cat freebsd-update fetch
[0 1 2]env ASSUME_ALWAYS_YES=YES PAGER=cat freebsd-update install
env ASSUME_ALWAYS_YES=YES pkg update
env ASSUME_ALWAYS_YES=YES pkg upgrade
env ASSUME_ALWAYS_YES=YES pkg install devel/autoconf
env ASSUME_ALWAYS_YES=YES pkg install devel/automake
env ASSUME_ALWAYS_YES=YES pkg install devel/libtool
env ASSUME_ALWAYS_YES=YES pkg install textproc/expat2
env ASSUME_ALWAYS_YES=YES pkg install devel/apr1
env ASSUME_ALWAYS_YES=YES pkg install devel/pcre
env ASSUME_ALWAYS_YES=YES pkg install www/libnghttp2
env ASSUME_ALWAYS_YES=YES pkg install ftp/curl
env ASSUME_ALWAYS_YES=YES pkg install devel/jansson
env ASSUME_ALWAYS_YES=YES pkg install www/libnghttp2
env ASSUME_ALWAYS_YES=YES pkg install converters/libiconv
env ASSUME_ALWAYS_YES=YES pkg install textproc/libxml2
cd /usr/ports/www/apache24/
make -DWITH=SUEXEC -DBATCH fetch
make -DWITH=SUEXEC -DBATCH extract
make -DWITH=SUEXEC -DBATCH patch
_back_up_and_patch_suexec_c_
make -j12 -DWITH=SUEXEC -DBATCH build
make -DWITH=SUEXEC -DBATCH install
env ASSUME_ALWAYS_YES=YES pkg install ap24-mod_perl2
cp /usr/local/etc/apache24/httpd.conf /usr/local/etc/apache24/httpd.conf.orig
_write_httpd_conf_
mkdir -p /cgi-bin
_write_the_helloworld_cgi_
_update_rc_conf_for_apache_
env ASSUME_ALWAYS_YES=YES pkg autoremove
service httpd onestart
```

Any idea which steps are still missing so Apache can be started using `service start` or `onestart`?


----------



## SirDice (Jan 19, 2021)

Snurg said:


> For some reason the presence of the -j option breaks the make run.
> It apparently only works well in the `make build` step.


Most of the other steps need to be run sequentially and can't work concurrently. 


```
env ASSUME_ALWAYS_YES=YES pkg install devel/autoconf
env ASSUME_ALWAYS_YES=YES pkg install devel/automake
env ASSUME_ALWAYS_YES=YES pkg install devel/libtool
env ASSUME_ALWAYS_YES=YES pkg install textproc/expat2
env ASSUME_ALWAYS_YES=YES pkg install devel/apr1
env ASSUME_ALWAYS_YES=YES pkg install devel/pcre
env ASSUME_ALWAYS_YES=YES pkg install www/libnghttp2
env ASSUME_ALWAYS_YES=YES pkg install ftp/curl
env ASSUME_ALWAYS_YES=YES pkg install devel/jansson
env ASSUME_ALWAYS_YES=YES pkg install www/libnghttp2
env ASSUME_ALWAYS_YES=YES pkg install converters/libiconv
env ASSUME_ALWAYS_YES=YES pkg install textproc/libxml2
```
You can replace all of this with `env ASSUME_ALWAYS_YES= pkg install -y -A $(make -C /usr/ports/www/apache24 missing)`. That will make it more resilient to changes in the dependency chain. You also want to add the `-A` (automatic) option so things are cleaned up with a pkg-autoremove(8). No sense in keeping a bunch of build dependencies around or when you decide to remove Apache later on. 


```
make -DWITH=SUEXEC -DBATCH fetch
{and a bunch of other invocations}
```
It's `WITH_SUEXEC`, not `WITH=SUEXEC`

And to not have to add the variables with every single line, define them once:

```
export ASSUME_AWAYS_YES=
export WITH_SUEXEC=
export BATCH=
export PAGER=/bin/cat
```
The variables just have to exist, they don't need to have anything assigned to them. Except the PAGER variable, that does need a value.


----------



## SirDice (Jan 19, 2021)

Snurg said:


> ```
> mkdir -p /cgi-bin
> ```


That will create a cgi-bin directory in your root, probably not what you intended here. For websites create a /usr/local/www/mysite/ for example. Put everything in that.


----------



## SirDice (Jan 19, 2021)

Snurg said:


> _back_up_and_patch_suexec_c_


If you create a proper patch file you can just drop it in the files/ directory of the port. It'll be automatically applied during the patch stage. Then you only have to run `make build` and `make install`.









						Chapter 4. Slow Porting
					

Description about creating a FreeBSD Port when the program need some modifications




					www.freebsd.org


----------



## Snurg (Jan 20, 2021)

Thank you very very much, SirDice !!!

Your snippet to find and install the dependencies is great!  
I instantly added it, it works great 
Your thoughts and tips regarding proper clean-up after deinstallation are extremely helpful too 

Regarding the filepaths (their sources and destinations) all is still variable.
Atm I just have lazy short paths for dry testing. mostly /tmp/...
All file/pathnames are stored in (constant) variables so they can be easily changed.
As soon as CGI basic function is there, clean-up and adaptation to conventions will follow.

Now I am no longer battling with the make process, but with AHxxxxx.
So  progress is going on 
As soon testing shows all works as intended, I'll make a patchfile.
So no more wild patching. 
Thank you so much


----------



## Snurg (Jan 20, 2021)

Regarding export, can I really use that?
The "batch" is being processed by a system() call, one for (almost) each line).
I know too little about shell 
So I'd like to ask, will the exports remain between the perl system() calls?
This would be very useful tidying the resulting installation "batch script".


----------



## SirDice (Jan 20, 2021)

Snurg said:


> Regarding export, can I really use that?


If it's a POSIX/Bourne shell script, yes, that's how you pass variables to child shells (which is basically what make(1) does). 


Snurg said:


> The "batch" is being processed by a system() call, one for (almost) each line).


Why do it like that? In that case an export isn't going to work. The variables are passed to child shells (a system(3) call spawns a child shell), the export is executed there but as the system call returns to the parent those environment variables are forgotten. In that case a execve(2) is a better option as you can pass environment variables with it. 



Snurg said:


> I know too little about shell


You have most of it already right there. Just put all the commands in a file and execute the whole lot with `sh <script file>`. Or add the `#!/bin/sh` as the first line, make the file executable and run it directly. Shell scripts don't have to be complex.





__





						POSIX Shell Tutorial
					





					www.grymoire.com


----------



## Snurg (Jan 21, 2021)

Now that's a coincidence... when searching about shell and export I found and bookmarked that Posix Shell Tutorial 
It's great and full of tricks!
But there are a few reasons why for now I stick with Perl, even without execve (which afaik isn't available in Perl).

I think I just have to sweep a rug over the ugly calls with many many environment parameters. 
Maybe just cover with splash screen or progress bar and all is good  [*]

Thank you again, SirDice ! 

([*]I just ask myself, are there maybe hooks in FreeBSD make system for updating dialog() progress bars while doing builds?)


----------

