# How I built a module



## Incnis Mrsi (Jul 3, 2020)

Hello.
Due to non-standard operational requirements I attempted to modify a module, namely /usr/src/sys/net/if_stf.c (instead of trying more conventional workarounds). This has been a “FreeBSD 12.1-RELEASE r354233 GENERIC amd64” system without any prior attempts to build kernel, without changes to the kernel and its source files. Of course, documentation first, so I found myself studying the /books/arch-handbook/driverbasics-kld.html page. Then I followed the handbook.

Went root;
Changed to /usr/src/sys/net/;
`mv if_stf.c if_stf.c.orig` and wrote the new if_stf.c;
Created the Makefile;
Ran `make` which made its job.
Now the fun started. Where did `make` place if_stf.ko, to the working directory? Failed to guess – both .o and .ko went to /usr/obj/usr/src/amd64.amd64/sys/net/. Moreover,
  netto kernel: link_elf_obj: symbol `in_ifaddrhashtbl` undefined 
  netto kernel: linker_load_file: /usr/obj/usr/src/amd64.amd64/sys/net/if_stf.ko - unsupported file type
The original module was, of course, already `kldunload`ed. Inspection with nm(1) revealed that the two modules (new vs original, out-of-the-box FreeBSD) have somewhat different sets of U symbols. In other words, some symbols in that new .ko were spurious.
Ī̲ traced the problematical stuff to /usr/src/sys/netinet/in_var.h which defines:
	
	



```
VNET_DECLARE(struct in_ifaddrhashhead *, in_ifaddrhashtbl);
VNET_DECLARE(struct in_ifaddrhead, in_ifaddrhead);
VNET_DECLARE(u_long, in_ifaddrhmask);                /* mask for hash table */

#define      V_in_ifaddrhashtbl      VNET(in_ifaddrhashtbl)
#define      V_in_ifaddrhead         VNET(in_ifaddrhead)
#define      V_in_ifaddrhmask        VNET(in_ifaddrhmask)
```
where the obscure macros are defined in /usr/src/sys/net/vnet.h, and both header files can be found in /usr/obj/usr/src/amd64.amd64/sys/net/.depend.if_stf.o which suggests that the C compiler included these definitions. As there were no “vnet_entry_in_” prefix in my new if_stf.ko, I inspected vnet.h, was almost lost in it and started to type a question to Stackoverflow (the first in my life), but stopped short of posting when by chance spotted the `#ifdef VIMAGE` conditional.

Hence I finally realized that my Makefile should contain

```
SRCS=if_stf.c
KMOD=if_stf
# “VIMAGE” was not mentioned in the handbook, but it must be defined!
CFLAGS = -DVIMAGE

.include <bsd.kmod.mk>
```

TL;DR: *BSD systems have reputation of a stuff limited to IT professionals and geeks for a reason. Documentation at freebsd.org is stubby and partially obsolete (not a single word about “VIMAGE”). Wiki.freebsd.org, in general, has some details and shows edit history, whereas authorship of the “handbook” is anonymous (for a general reader, not a fan of versioning systems) and the book neglects to specify on which versions of the software were its conceits tested. Without my experience in C programming I would be powerless.


----------



## George (Jul 3, 2020)

I would have suggested to download the kernel sources,  modify your .c files, and then run "make buildkernel" with MODULES_OVERRIDE=..

With this, you wouldn't need to edit any Makefiles. Also, you avoid trouble with files that only exist during a kernel build (some if_...h files).

But you got it working, right?


----------



## Incnis Mrsi (Jul 6, 2020)

Elazar said:


> But you got it working, right?


At the end I got. Positive impressions: an extensive collection of man pages in the Section 9. Negative impressions: paranoid programming practices, namely `in_stf_input()` scrabbles and quibbles where `stf_encapcheck()` already performed. It IMHO defeats the intent of verification as a separate routine.

By the way, the “VIMAGE” stuff was already discussed at Thread how-to-specify-kernel-configuration-when-compiling-out-of-tree-kernel-modules.71576, but did not find its way into the handbook.


----------

