# How to specify kernel configuration when compiling out-of-tree kernel modules?



## Bobi B. (Jul 22, 2019)

Or, having an out-of-tree (i.e. not in /usr/src/sys) custom FreeBSD kernel module, how one can specify a kernel configuration, like `GENERIC`, the module shall be built against?

Here is a very small kernel module that can be compiled, but cannot be loaded on a stock FreeBSD 12.

Makefile:

```
KMOD=    showifn
SRCS=    showifn.c
.include <bsd.kmod.mk>
```

showifn.c:

```
#include <sys/param.h>
#include <sys/module.h>
#include <sys/kernel.h>
#include <sys/systm.h>

#include <sys/types.h>
#include <sys/socket.h>
#include <net/if.h>
#include <net/if_var.h>

static void
show_ifn(void)
{
        struct ifnet *ifp;

        IFNET_RLOCK();
        CK_STAILQ_FOREACH(ifp, &V_ifnet, if_link) {
                printf(" %s via %s\n", ifp->if_xname, ifp->if_dname);
        }
        IFNET_RUNLOCK();
}

static int
kmod_main(struct module *module, int event, void *arg)
{
        int res = 0;

        switch (event) {
        case MOD_LOAD:
                printf("showifn loading\n");
        show_ifn();
                break;

        case MOD_UNLOAD:
                printf("showifn unloading\n");
                break;

        default:
                res = EOPNOTSUPP;
                break;
        }

        return res;
}

static moduledata_t showifn_conf = {
    "showifn",
    kmod_main,
    NULL
};

DECLARE_MODULE(showifn, showifn_conf, SI_SUB_DRIVERS, SI_ORDER_MIDDLE);
```

Module should compile cleanly on FreeBSD 12, but cannot be loaded (reason below):

```
# make load
/sbin/kldload -v /root/proj/custkmod/showifn.ko
kldload: an error occurred while loading module /root/proj/custkmod/showifn.ko. Please check dmesg(8) for more details.
*** Error code 1

Stop.
make: stopped in /root/proj/custkmod

# dmesg | tail -2
link_elf_obj: symbol ifnet undefined
linker_load_file: /root/proj/custkmod/showifn.ko - unsupported file type
```

The issue with this particular example is, that since FreeBSD 12 `VIMAGE` is now enabled in `GENERIC`, by default. When `VIMAGE` is enabled a lots of pre-processor magic happens, behind the scenes, and the names of some symbols, related to networking, are changed (probably much more than just names). BTW the module can be loaded on a system with custom-built kernel, where `VIMAGE` is disabled.

How come configuration for `GENERIC` kernel is not taken into account when building this module? Module can be built, but not loaded.
How to specify a custom kernel configuration (or even `GENERIC` one) when compiling out-of-tree kernel modules?
Thanks in advance!


----------



## Barney (Jul 22, 2019)

Are you running the GENERIC kernel? The module has an external reference to ifnet, so it won't load unless the running kernel has the symbol. 

External modules should be independent of the kernel, but they do need the references to exist to load.


----------



## Bobi B. (Jul 22, 2019)

Barney said:


> Are you running the GENERIC kernel?




```
# uname -a
FreeBSD x.y.z 12.0-RELEASE-p7 FreeBSD 12.0-RELEASE-p7 GENERIC  amd64
```




Barney said:


> The module has an external reference to ifnet, so it won't load unless the running kernel has the symbol.


Yes and No.

```
GENERIC-12# nm /boot/kernel/kernel
ffffffff81b91570 D vnet_entry_ifnet
ffffffff81b99de0 D vnet_entry_ipstat

CUSTOM-12# nm /boot/kernel/kernel
ffffffff81dc66d8 B ifnet
ffffffff81dc6fe0 B ipstat
```
Having `VIMAGE` enabled in `GENERIC` will change symbol names as it will virtualise some kernel objects.



Barney said:


> External modules should be independent of the kernel, but they do need the references to exist to load.


Changing the kernel build options, namely `VIMAGE` switch, will change kernel making your statement incorrect. You have all the source code in front of you. Give it a try.


----------



## Barney (Jul 22, 2019)

I'm confused here. You're the one who doesn't know what he's doing. I know exactly how it works


----------



## Bobi B. (Jul 23, 2019)

Barney said:


> I'm confused here. You're the one who doesn't know what he's doing. I know exactly how it works


Did you succeed in compiling and loading kernel module I gave above? How?


----------



## Barney (Jul 23, 2019)

You asked the question, dunderhead. Why would I build a module for you? I don't need to test things to convince you. 

If you build a module that doesnt support  VIMAGE, it won't load on a kernel that requires it. It has nothing to do with the name of the kernel. 

For a module to load, the running kernel must support the same things that the module requires. It's not as restrictive as linux, which will complain if the module wasn't built for the exact kernel. If you want the module to load on a VIMAGE kernel, then you likely have to define VIMAGE when you build the module. I don't have a half a day to figure it out for you. Try -DVIMAGE in your module makefile and maybe it will work.


----------



## Bobi B. (Jul 23, 2019)

Barney said:


> You asked the question, dunderhead. Why would I build a module for you? I don't need to test things to convince you.
> 
> If you build a module that doesnt support  VIMAGE, it won't load on a kernel that requires it. It has nothing to do with the name of the kernel.
> 
> For a module to load, the running kernel must support the same things that the module requires. It's not as restrictive as linux, which will complain if the module wasn't built for the exact kernel. If you want the module to load on a VIMAGE kernel, then you likely have to define VIMAGE when you build the module. I don't have a half a day to figure it out for you. Try -DVIMAGE in your module makefile and maybe it will work.


Dear Barney, frankly I fail to see how your posts contribute to this thread. I'm aware that adding `-DVIMAGE` to my Makefile would help, but my question is how to make bsd.kmod.mk to define whatever necessary from a GENERIC or a custom kernel configuration file.

PS: I write C code for 25 years.


----------



## Barney (Jul 23, 2019)

Sorry, I thought you were someone trying to make something work. Not someone hoping the sky would turn the color you prefer.


----------



## shkhln (Jul 24, 2019)

Let's see if I understand the question correctly.

Normally, VIMAGE will be defined (or not) in $KERNBUILDDIR/opt_general.h file, generated on the start of a kernel build by config utility. Then it will be read back into KERN_OPTS makefile variable. For out-of-tree builds KERNBUILDDIR variable is empty, so you'll either have to set it to something or specify some sensible defaults.


----------



## Bobi B. (Jul 25, 2019)

Thank you, shkhln! For future reference, when Makefile is something like

```
KMOD=    showifn
SRCS=    showifn.c

CLEANDIRS+= .kconf

# Remember to do `make cleandepend` if changing KERNCONF!
KERNCONF?=    GENERIC
KERNCONFDIR=  /usr/src/sys/${MACHINE_ARCH}/conf
.if !empty(KERNCONF)
SRCS+=        .kconf/opt_global.h
CFLAGS+=      -include .kconf/opt_global.h
.endif

.kconf/opt_global.h: ${KERNCONFDIR}/${KERNCONF}
        cd ${KERNCONFDIR} && config -d ${.CURDIR}/.kconf ${KERNCONF}

.include <bsd.kmod.mk>
```
... magic will happen automatically.


----------



## shkhln (Jul 25, 2019)

Bobi B. said:


> ```
> .if !empty(KERNCONF)
> SRCS+=        .kconf/opt_global.h
> CFLAGS+=      -include .kconf/opt_global.h
> ...



One nitpick: without an actual fallback value this part is redundant.


----------



## Bobi B. (Jul 26, 2019)

shkhln said:


> One nitpick: without an actual fallback value this part is redundant.


In fact you can disable opt_global.h inclusion if you compile with `make KERNCONF=`.


----------



## shkhln (Jul 26, 2019)

Bobi B. said:


> you can disable opt_global.h inclusion



I picked somewhat unfortunate example with that "-include" line. See this code instead. The whole point of that idiom is to generate _opt_whatever.h_ files in case the kernel build directory is not specified, otherwise _#include "opt_whatever.h"_ statements in C code would fail.

(By the way, looks like the kernel build directory is unspecified each time you build modules starting make from their directory, regardless of whether they are placed in-tree or out-of-tree. In fact, I never had a compelling enough reason to build a complete FreeBSD kernel from source, so I'm not entirely sure when modules are actually compiled with kernel configuration rather than with their defaults.)


----------

