# Moving the kernel and modules after a 'nextboot -k' hides modules



## DutchDaemon (Jun 27, 2013)

I wonder if I should consider this a bug, a feature, or a 'duh'. 

Whenever I build a new world and kernel, I routinely invoke `/sbin/nextboot -k kernel.NEW/boot/kernel` to make sure I can actually boot with the newly compiled kernel. If that works, I move /boot/kernel to /boot/kernel.old, and /boot/kernel.NEW/boot/kernel to /boot/. However, I do not reboot after that, I just leave everything running. It's working.

I found an interesting snag with this. I wanted to test something using amd, and it needs a couple of NFS things I do not compile into the kernel (as I don't use any NFS anywhere). Normally, the invocation of amd will simply load the NFS necessities as modules, and everything works. However, I got some interesting errors about modules that could not be found. Of course they exist, they're in /boot/kernel/, as usual.

After a bit of brain crunching, an _aha_ moment overcame me, and I ran `kldstat -v`. It showed that every module that was listed there originated from /boot/kernel.NEW/boot/kernel; a location that no longer existed. Up until now, I had assumed that modules would _always_ be searched in /boot/kernel/, whether one booted from there or not, but apparently this is not the case. And yes, a reboot was all it took.

I'm wondering whether I should make any report of this, or that this is simply a case of something that's entirely logical that I simply missed because I never ran into it. In that case I will make it my business to always reboot one more time after moving the working kernel and modules to their final location. But I _do_ feel that /boot/kernel should always be _tried_, at least.


----------



## kpa (Jun 27, 2013)

It may be that the path to the kernel file that was used for booting sets the search path for modules automatically to the containing folder.


----------



## vermaden (Jun 27, 2013)

@DutchDaemon

I do similar, but not quite.

```
# cd /usr/src
# make buildworld kernel KODIR=/boot/test
# beadm create backup
# nextboot -k test
```

This way _y_ou have _the_ previous kernel at /boot/kernel and _the_ new one as /boot/test, if everything works, then _y_ou do these:


```
# cd /boot
# rm -rf old
# mv kernel old
# mv test kernel
```

_A_lso check this: *http://man.freebsd.org/kldxref*


----------



## DutchDaemon (Jun 27, 2013)

@vermaden, sure, there are ways around this, of course. But I was wondering about the general practice of setting the module search path to the one where the actual running kernel is located, and 'forgetting' the default path of /boot/kernel when calling a module. 

By the way, what you describe above is exactly what I do. So you should run into the same problem when running on a kernel booted from /boot/test after moving kernel/modules over to /boot/kernel _and not rebooting after that_. Note: this is not about booting into that new system (because /boot/kernel and /boot/test will _both_ exist at that point), but loading an additional module on a system that was booted from /boot/test _after (re)moving_ that /boot/test directory.

In other words: if `kldstat -v` shows your module directory path as /boot/test, you won't be able to load a new module after removing /boot/test. And I'm saying: would be nice if /boot/kernel was at least _attempted_.


----------



## vermaden (Jun 27, 2013)

DutchDaemon said:
			
		

> In other words: if `kldstat -v` shows your module directory path as /boot/test, you won't be able to load a new module after removing /boot/test. And I'm saying: would be nice if /boot/kernel was at least _attempted_.



kldxref does not fix that?


----------



## DutchDaemon (Jun 27, 2013)

Like I said, there are ways around it, it's just the default behavior (i.e. not trying /boot/kernel and /boot/modules when not being able to find a module) that I question


----------



## kpa (Jun 27, 2013)

I think this would be an easy fix. Add the custom module paths to the kern.module_path loader(8) variable before the standard paths /boot/kernel;/boot/module instead of replacing them. It's all set in the loader and what nextboot(8) does is just add a file with extra settings that are read at boot.


----------



## DutchDaemon (Jun 27, 2013)

Ok, I must have formulated the issue wrong, because people keep suggesting fixes to me that I know, and therefore don't need  This was a matter of 'what the default system behavior should be', not 'how do I load my modules?'

I'll mark this as solved, as in 'the answer to the first line and the final paragraph of the original post is "whatever"'.


----------



## phoenix (Jun 27, 2013)

This is a "duh!" moment, not a bug or a feature.  

When you use nextboot(8), the loader uses that to set the module path that gets sent to the kernel, and which the kernel uses to search for modules.  IOW, it's working as designed... just not how you want.  

And, if you think about it, you really don't want it to work any other way.  You used nextboot for a reason (to load the kernel from a non-standard location, to give you a fall-back to the standard kernel if things go sideways).  And you really don't want this non-standard kernel to be pulling modules from the standard locations as that would mix modules from separate kernels... a definite no-no.  

The kernel has no way of knowing you changed directories out from underneath it.    However, I believe you can tell the kernel that you changed the directories underneath it by editing the module_path kenv variable or via sysctl.  Never tried it, though.

Ideally, you should add "reboot" into your testing framework, as the step right after "move test kernel to /boot/kernel".     After all, that's how I do things, and I'm obviously the greatest FreeBSD user ever.


----------



## DutchDaemon (Jun 27, 2013)

Yeah, I can suffer the extra reboot, as I'm not rebooting between the `make installkernel` and `make installworld` steps anyway


----------

