# 11.2-RELEASE boot -> bootpool/boot issues



## da1 (Oct 2, 2018)

So I need to manually compile and install a few kernel modules and also manually load them via loader.conf. But when I want to install them, I get:

```
[root@xxx 1:33pm] /usr/src/sys/modules/mlx4/>make install
install -T release -o root -g wheel -m 555   mlx4.ko /boot/modules/
install: /boot/modules/: No such file or directory
*** Error code 71

Stop.
make: stopped in /usr/src/sys/modules/mlx4
```

I then manually create /bootpool/boot/modules and install the modules. All good so far.

Now I want to load them via loader.conf so I create the file and add the necessary parts. Problem is, it's ignored and the modules don't get loaded.

What change am I missing here?

PS: this is a fresh 11.2-RELEASE install.


----------



## SirDice (Oct 2, 2018)

da1 said:


> So I need to manually compile and install a few kernel modules and also manually load them via loader.conf.


Why? Everything is already built. Even stuff that's not in the kernel config will get built as a module. As a matter of fact, everything is built as a module. The kernel config only collects everything and links them together.


----------



## da1 (Oct 2, 2018)

SirDice said:


> Why? Everything is already built.


WHAT?!?!?!? 
please explain then where I can find the .ko file for the mlx4 kernel module.



SirDice said:


> Even stuff that's not in the kernel config will get built as a module. As a matter of fact, everything is built as a module. The kernel config only collects everything and links them together.


Am I missing something here? I didn't mention a kernel config file and I'm NOT recompiling the whole kernel; I don't see any point in doing so.


----------



## SirDice (Oct 2, 2018)

da1 said:


> WHAT?!?!?!?
> please explain then where I can find the .ko file for the mlx4 kernel module.


A buildkernel builds everything, including devices you didn't put in the kernel config. 


```
dice@armitage:~ % ll /boot/kernel/mlx*
-r-xr-xr-x  2 root  wheel   67456 Jul  7 20:14 /boot/kernel/mlx.ko*
-r-xr-xr-x  2 root  wheel  467424 Jul  7 20:14 /boot/kernel/mlx4.ko*
-r-xr-xr-x  2 root  wheel  180568 Jul  7 20:14 /boot/kernel/mlx4en.ko*
-r-xr-xr-x  2 root  wheel  432616 Jul  7 20:14 /boot/kernel/mlx5.ko*
-r-xr-xr-x  2 root  wheel  175800 Jul  7 20:14 /boot/kernel/mlx5en.ko*
```
The mlx device is typically already built into GENERIC, but it's still available as a module.


```
dice@armitage:~ % uname -a
FreeBSD armitage.sirdice.nl 11.2-RELEASE-p3 FreeBSD 11.2-RELEASE-p3 #0: Thu Sep  6 07:14:16 UTC 2018     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64
```

(note to self, need to update this machine)


----------



## da1 (Oct 2, 2018)

Dude, I did not mention rebuilding the whole kernel   I do not want to do that. I just need a handful of modules that I load at boot.

Back to the point now. Sadly mlx4 is not included in GENERIC so yeah, there's that. But to my initial question, something changed between 11.1-RELEASE and 11.2-RELEASE regarding the loader.conf. Do we need to do create zfs boot env's now or something? I mean, I'm very surprised that 1) loader.conf doesn't exist anymore and 2) it's ignored if I create it manually.


----------



## SirDice (Oct 2, 2018)

da1 said:


> Sadly mlx4 is not included in GENERIC so yeah, there's that.


Again, this doesn't matter. It's a module, it always is, regardless if it's included in the kernel or not. 



da1 said:


> But to my initial question, something changed between 11.1-RELEASE and 11.2-RELEASE regarding the loader.conf.


Negative. It's still the same process as it was on FreeBSD 3.0.



da1 said:


> loader.conf doesn't exist anymore


The file doesn't exist by default. It never did.


----------



## da1 (Oct 2, 2018)

ok so let me rephrase: I create loader.conf, compile and install a module from /usr/src, reboot the machine and see no module loaded.


----------



## SirDice (Oct 2, 2018)

da1 said:


> compile and install a module from /usr/src


And I'm trying to tell you that you do not need to do this because _everything_ is already built and installed as a module. 



da1 said:


> see no module loaded.


What happens if you load it by hand? `kldload mlx4en`


----------



## da1 (Oct 2, 2018)

```
kldload mlx4
kldload: can't load mlx4: No such file or directory
```


----------



## SirDice (Oct 2, 2018)

Yeah, sorry. I had to actually read the manual page, it's mlx4en(4), so `kldload mlx4en`.


----------



## da1 (Oct 2, 2018)

```
kldload mlx4en
kldload: can't load mlx4en: No such file or directory
```


----------



## SirDice (Oct 2, 2018)

It should be right here:

```
root@armitage:~ # ll /boot/kernel/mlx4*
-r-xr-xr-x  2 root  wheel  467424 Jul  7 20:14 /boot/kernel/mlx4.ko*
-r-xr-xr-x  2 root  wheel  180568 Jul  7 20:14 /boot/kernel/mlx4en.ko*
```


----------



## da1 (Oct 2, 2018)

```
[root@xxx 3:11pm] ~/>ls -lahd /boot
lrwxr-xr-x  1 root  wheel    13B Jun 22 07:37 /boot -> bootpool/boot
[root@xxx 3:11pm] ~/>ls -lahd /bootpool/
drwxr-xr-x  2 root  wheel     2B Oct  2 13:38 /bootpool/
[root@xxx 3:11pm] ~/>ls -lahd /bootpool/boot
ls: /bootpool/boot: No such file or directory
```


----------



## da1 (Oct 3, 2018)

So SirDice? How's that module coming along?


----------



## SirDice (Oct 3, 2018)

All kernel modules are in /boot/kernel/. At least the ones from FreeBSD itself. Additional kernel modules can be installed through ports but are saved in a different location. There's no need to specifically build or install _any_ module from FreeBSD, it's all there already.


----------



## da1 (Oct 3, 2018)

so again, can you please tell me where I can find the mlx4en pre-compiled module?


----------



## bds (Oct 3, 2018)

It looks like you have an encrypted root with a separate ZFS boot pool, but the bootpool filesystem is not mounted. Try zfs mount bootpool? Otherwise use zfs/zpool to diagnose where it has gone.


----------

