# HOWTO: Create a FreeBSD 8 i386 Xen PV domU



## Aprogas (Jan 12, 2010)

Xen is virtualization software, which lets you run multiple operating systems on the same physical machine. The goal of this guide is to show you how to create a FreeBSD 8 installation that will run as a guest OS, without the need for a processor that supports hardware-assisted virtualization. It would go beyond the scope of this guide to explain how exactly Xen works or what the differences between PV and HVM are, suffice to say dom0 is the host OS and domU the guest OS. Although FreeBSD fully supports being a Xen domU, it is not yet very well documented and not suited for novice Xen users. Especially the lack of a Xen-aware installer or pre-built Xen-aware images make things more difficult. If you are a Xen novice, I recommend starting first with pre-built Linux images, and then trying a NetBSD domU installation, before proceeding with FreeBSD.

*Preparing the host*
We will start with a normal Debian installation, we will need some free diskspace, beware that Linux names harddisks differently from FreeBSD, so if you have other OSes on this machine, be very careful you don't partition and format the wrong one. I recommend including the Desktop environment, having a graphical browser and the Synaptic Package Manager will be really useful. Now we need to install Xen itself. Debian mostly automates this process if you use apt-get or aptitude or synaptic. Installing xen-linux-system-2.6.26-2-xen should pull in all other needed dependencies (exact version may have changed by the time you read this guide). The required boot entries should be automatically added to GRUB, so in the next reboot we can start the Xen hypervisor.

*Preparing the guest*
There are no pre-built kernels and images we can use (but see below for mine), so we need to make our own. I'll assume you already have a FreeBSD installation somewhere, I used one existing inside a VirtualBox on a Windows 7 host. I have only tested with an i386 kernel and image, I am not certain whether amd64 is supported. The files we are going to create will eventually have to be transported to your dom0, so make sure you have some way to get them there. Go to a directory on your FreeBSD system with enough space.

Create a 256 MB file that will become our image:
`# truncate -s 256M freebsd.img`

Create a virtual disk backed by this file:
`# mdconfig -f freebsd.img`
If this commands yields something other than md0, adjust all the following commands to that.

Create a MBR and single slice on the virtual disk:
`# fdisk -BI md0`

Create a BSD-label inside our slice:
`# bsdlabel -wB md0s1`

Create a UFS2 filesystem with softupdates enabled:
`# newfs -U md0s1a`
It is probably possible to directly create a filesystem inside the image and skip the MBR and label.

Mount the newly created filesystem:
`# mount /dev/md0s1a /mnt`

Make sure you have a fully populated 8.0 /usr/src tree:
`# csup -h nearby.mirror.FreeBSD.org -L 2 /usr/share/examples/cvsup/standard-supfile`

We are about to build our world and kernel. By default this will populate /usr/obj, this will do no harm, but it is possible to specify an alternate location:
`# export MAKEOBJDIRPREFIX=/other/dir`
csh users use setenv

Now it is time to start compiling, if you need multiple attempts to get things working, it is not necessary to do this step again each time (provided you did it correctly the first time):
`# make buildworld && make buildkernel KERNCONF=XEN`

Our file-backed virtual disk should still be mounted, so now we can install to it:
`# export DESTDIR=/mnt && make installworld && make installkernel KERNCONF=XEN && cd etc && make distribution`

Now we need to modify a few files to prepare our installation for living in a Xen world.
Add to /mnt/etc/fstab:

```
/dev/xbd0       /               ufs     rw              1       1
```
Add to /mnt/etc/ttys:

```
xc0     "/usr/libexec/getty Pc"         vt100   on  secure
```

We are almost done, but need to extract our kernel because we will need to put it in your dom0:
`# cp /mnt/boot/kernel/kernel /some/place/freebsd-kernel`

Now we can unmount and destroy the virtual disk:
`# umount /mnt`
`# mdconfig -d -u md0`

The results of all our labour are now in freebsd.img and freebsd-kernel. Put these somewhere safe and reachable.

*Configuring on the dom0*
Go back to your dom0 installation, reboot into Xen if you hadn't already. Put the image and kernel we created somewhere on the dom0. They don't have to be in the same directory.

Check whether your Xen is working:
`# xm list`

Now create /etc/xen/freebsd and use the following configuration:

```
kernel = "/virt/freebsd-8.0p2-i386-xen-domu-kernel"
memory = 512
name = "freebsd"
vif = [ '' ]
disk = [ 'file:/virt/freebsd-8.0p2-i386-xen-domu.img,hda,w' ]
extra = "boot_verbose=1"
extra += ",vfs.root.mountfrom=ufs:/dev/ad0s1a"
extra += ",kern.hz=100"
```
Of course change the kernel and disk paths to wherever you put them. Later you can tweak the configuration, but first we must determine if everything is working.

Also edit /etc/xen/xend-config.sxp to enable bridged networking:

```
#(network-script network-dummy)
(network-script network-bridge)
```

Everything should now be in place to start the domU:
`# xm create -c freebsd`
The -c option will directly attach to the console output, alternative you can attach seperately:
`# xm console freebsd`

At this point you should see the regular output of a booting FreeBSD and eventually the login prompt. If you can login, you have succeeded in creating a FreeBSD 8 Xen domU.

*How to proceed*
Of course all of this was just a small test installation not suited for real use. It's probably possible to increase the size of the image-file and growfs to a larger installation, but file-backed virtual disks aren't very efficient. If you want to make serious use of virtual FreeBSD installations, creating real partitions is the way to go. Let us know if you succesfully set this up, I am especially interesting in knowing whether GPT and ZFS will work in a Xen domU.

*Known problems*
ifconfig_xn0="DHCP" will not work, run dhclient or configure manually

*References*
This guide was heavily based on http://wiki.freebsd.org/AdrianChadd/XenImages and http://wiki.freebsd.org/AdrianChadd/XenHackery

*Closing words*
I started out about a week ago trying to figure out how to run FreeBSD in Xen. At first I was disappointed because I had assumed that since FreeBSD has full domU support, it would be relatively easy to do. Documentation was hard to find and much of it was outdated and being a total novice to Xen did not help either. Luckily I did not give up and soon become more acquainted with Xen terminology, which made it easier to understand Adrian Chadd's notes on the Wiki. I had already decided I did not want a UFS1 image, but a full simulated disk with an MBR, disklabel, UFS2 and softupdates, so I went that route. For simplicity I also decided to go for file-backed disk instead of LVM and to not use pygrub or pvgrub. I haven't yet extensively used and tested my FreeBSD domU, the moment I got it to actually boot (I made many mistakes at first) I started writing this guide.

Attached are the config, kernel and image file I created. This is an old image, you probably don't want to use it. Beware that sshd and dhclient are enabled, an account xenguest with password "password" exists and root is still password-less, so it is not a secure installation, but only meant for testing.
config
kernel (gzipped, 1.8M)
image (gzipped, 83M)
mirror


----------



## Leander (Apr 15, 2010)

Hi Aprogas,

first of all: Thanks for all your effort to create that rar how to and also for uploading significant files.

I tried your how to several times now [ with an i386 version  ] but unfortunatley am I always getting a kernel panic when I'm about booting the virtual FreeBSD. No matter what image I used (I even tryed the actual pfsense which is based on 7.2) plus I tryed your kernel and image as well - but the result was always the same.

```
xm create -c /FreeBSD-i38.cfg
```
started the machine and it looked like it would work, but then all of a sudden it broke up again and switched back to the regular console.

```
xm list
```
showed me that the thing broke while booting.
It must be a Kernel issue?! OR the only thing I could think of is that my Xen might be incompatible with the FreeBSD DomU project cause I'm using NetBSD 5.0.2 with the actual ported Xen 3.3.2 - but as far as I know should that 3.3.2 do the job?!

... I even saw the Login once for a millisecond ... but then it broke again ... ;(

Any Ideas?

P.S. I'll post the last boot information as soon as possible.


Regards,

   Leander


----------



## redweasel (May 2, 2010)

This won't help you alot Leander, but I just tried with a plain Debian install with Xen and the image and kernel provided here, and it's working without any problems. So thats Xen 3.2.1 on Debian 5.0

I'm up to compiling the kernel myself now and creating an image. Thanks for this tutorial!


----------



## black0ut (Jul 4, 2010)

I'd like to post my attempts at creating a FreeBSD 8.1-RC PV domU.

Hardware:
CPU = AMD Athlon II Regor 250

DomU is Debian 5.0.5 (Lenny), linux kernel 2.6.26-2, xen 3.2-1.  I installed FreeBSD 8.1-RC on Virtualbox, created a dev environment, pulled down sources, compiled the kernel with KERNCONF=XEN.  I also installed FreeBSD to an image by creating a FreeBSD HVM (which works fine btw, using the loader option hw.clflush_disable=1).  I then tried to use the XEN kernel as the PV kernel (i.e. kernel='/path/to/xen/kernel'), but I get this kernel panic:


```
panic: HYPERVISOR_update_va_mapping(((unsigned long)(va)), (pa | 0x002 | 
0x001 | pgeflag | pmap_cache_bits(mode, 0)), UVMF_INVLPG| UVMF_ALL) < 0: 
/usr/src/sys/i386/xen/pmap.c:1269
cpuid = 0
KDB: enter: panic
[thread pid 0 tid 0 ]
Stopped at      kdb_enter+0x3a: movl    $0,kdb_why
```

Found very little documentation, other than this "successful" attempt.  I say "successful" because simply using the image and kernel provided here also results in a kernel panic, albeit a different one.

Is the FreeBSD PV domU more favorable on Intel HW?  I can switch if needed, but would prefer not to.


----------



## black0ut (Jul 5, 2010)

And here is someone else having the exact same problem on (recent) AMD hardware:

http://lists.freebsd.org/pipermail/freebsd-xen/2009-October/000323.html

The kernel panic there is the same, even under FreeBSD-CURRENT (9) and Xen 3.4.  Starting to lose hope that FreeBSD PV domU will be supported any time soon, judging by the lack of response to the various kernel panic bug reports


----------



## bsd4me (Jul 27, 2010)

I downloaded your kernel and image files (from a mirror).  They worked without issue.

System:

Lenovo R61 ThinkPad
Intel Core2 Duo
Debian Lenny (5.0.4) Stable (2.6.26-2-xen-amd64)
Xen 3.2.1  (AMD64)

Next, I will follow your howto and try to build my own.

Thank you!


----------



## Nats (Aug 14, 2010)

blackOut - just wanted to let you know I'm experience the same problem on Intel hardware ( http://www.freebsd.org/cgi/query-pr.cgi?pr=148262 ) with 8.1

If I combine the disk image I made with the kernel attached to the OP, it does boot fine. I suppose next thing to try is go back to 8.0


----------



## Nats (Aug 14, 2010)

I've just successfully built a kernel and disk image with RELENG_8_0 . I guess Xen kernel is broken in 8.1


----------



## vtypal (Sep 18, 2010)

Thank you @Aprogas . Excellent guide. 

Just for the record if you try to create your own images and kernel from a *amd64 release* you should simply make the following changes (the kernel configuration file is called XENHVM on amd64) :


```
make buildworld && make buildkernel KERNCONF=[B]XENHVM[/B]
```


```
export DESTDIR=/mnt && make installworld && make installkernel KERNCONF=[B]XENHVM[/B] && cd etc && make distribution
```

Tested with 8.1. The generated amd-64 kernel and the relative image are not suitable for paravitualized guests, but only for full virtualization, so your domu configuration file should be different. I tried to test it with as a full virtualization image - it seems to boot - but i cannot proceed as i need vnc to take output... (i'm lazy to do this now...)


----------



## tege (Sep 21, 2010)

I have spent/wasted a lot of time trying to get FreeBSD run as a DomU under NetBSD Dom0. My NetBSD is the amd64 flavour of release 5.0.2.  I use Xen 3.3.2 (installed via pkgsrc, NetBSD's answer to FreeBSD's ports).

I tried the i386 images presented here, and they yield immediate kernel panics.  I have compiled my own i386 kernels (using KERNCONF=XEN) and they work no better.

I have compiled amd64 images, using KERNCONF=XENHVM and made futile attempts at getting them to boot.  In the apparent absence of documentation (at least such documentation that is not obsolete and/or sloppy) I've had problems figuring out how to boot this image.  I started with /usr/pkg/share/examples/xen/hvm, but it is not clear how to point to the FreeBSD disk image from there; I made it the disk device.  But it is clear that this is *not* the correct approach, since there is no reason that the system should be able to read FreeBSD's file system.

I have tried various version of FreeBSD 8.x, RELEASEs and RELENGs during this struggle.

*My question is: Does anybody here have a working FreeBSD of any sort under a NetBSD dom0, and is willing to explain exactly how to get it to run without triggering bugs?*

(My ultimate goal is setting up a large number of DomU on all my test systems, but I have thus far only succeeded with NetBSD i386 and NetBSD amd64...  My main development platform is FreeBSD, so it that fails, my test array project fails.)


----------



## fwaggle (Sep 25, 2010)

Good news everyone!

Using a combination of these instructions, the NetBSD instructions, some good luck and some help from #bsdcode people, I've managed to get FreeBSD i386 to boot under Linode (which is XEN+PV).

The 8.0 images included above boot on linode reasonably trivially, but upgrading the kernel at all yields immediate panics. Thanks to nox @ #bsdcode, I discovered this morning that the panics many people are seeing under i386 paravirtualized Xen are caused by the new x86bios stuff that wasn't in 8.0-RELEASE... it's only called from the keyboard stuff (nox found that, and vesa which call it, but vesa isn't in the default i386 XEN kernel config). Commenting out atkbdc/atkbd/kbdmux from the default XEN kernel config results in an 8.1-RELEASE-p1 kernel that boots and doesn't seem to blow up at all, though I don't know which device or function is actually the buggy part and I'm not smart enough to track down where the issue is and fix it. 

Since the process of getting *Free*BSD on a linode isn't documented at all, _anywhere_, I'm working on a set of instructions which I'll post on my site, then mirror here and on the linode wiki. The instructions are still a work in progress, because each time I follow through my own notes, something is missing or goes wrong, so obviously I'm doing stuff from muscle memory and not documenting it, then forgetting it in subsequent tries.

I don't know what version of Xen linode are using, nor what they're using for a host, but this is a current FreeBSD that's working-for-me under Xen-PV which is a start. 

Edit: Google-snacks:


```
panic: HYPERVISOR_update_va_mapping(((unsigned long)(va)), (pa |
0x002 | 0x001 | pgeflag | pmap_cache_bits(mode, 0)), UVMF_INVLPG| 
UVMF_ALL) < 0: /usr/src/sys/i386/xen/pmap.c:1269
cpuid = 0
```

This particular panic with i386 kernels >8.0-R can be fixed by removing atkbd (I removed atkbdc + kbdmux too, not sure if that's necessary though) from your kernel, then booting from the updated kernel.


----------



## tege (Sep 28, 2010)

I now have FreeBSD 6.4 amd64 as well as FreeBSD 6.4 i386 running under a NetBSD Dom0.   Unlike 7.x and 8.x installation and operation are smooth.


```
kernel = "/usr/pkg/lib/xen/boot/hvmloader"
builder='hvm'
memory = 256
vcpus = 1
name = "freebsd-64"
vif = [ 'mac=00:16:3e:00:01:64, bridge=bridge0, type=ioemu, model=ne2k_pci' ]
device_model = '/usr/pkg/libexec/qemu-dm'
disk = [
    'file:/home/xen/freebsd/64/6.4/disk,ioemu:hda,w',
#    'file:/home/tege/6.4-RELEASE-amd64-disc1.iso,ioemu:hdc:cdrom,r'
]
boot='c'
#boot='d'
pae = 1
usb=1
usbdevice='tablet'
```

The #-marked lines need to be uncommented for installation.

It seems that using a mac address not starting with 00:16:3e makes NetBSD/Xen confused (sic!).  It seems that the default simulated network card is unreliable, therefore the need for the ne2k_pci moniker.

I have made some progress with 8.1 amd64 too, with a custom kernel.  It boots, but the installation is not yet robust.


----------



## vtypal (Oct 7, 2010)

fwaggle said:
			
		

> Good news everyone!
> 
> 
> ```
> ...



Thank you! Removing atkbd + atkbdc + kbdmux worked also for me. I also removed INVARIANTS, INVARIANT_SUPPORT, WITNESS and WITNESS_SKIPSPIN to improve performance. 

Do you get the following notification about every 3 minutes in your terminal?


```
# [XEN] hypervisor wallclock nudged; nudging TOD.
```

It seems to be generated by the rev.193033. My domu clock it seems to work fine, although I do not know how to stop them (add a comment?).


----------



## fwaggle (Oct 12, 2010)

vtypal said:
			
		

> Thank you! Removing atkbd + atkbdc + kbdmux worked also for me. I also removed INVARIANTS, INVARIANT_SUPPORT, WITNESS and WITNESS_SKIPSPIN to improve performance.



nox via #bsdcode was the one who figured it out. He also learned later that the fix for that problem was in -CURRENT the entire time, but it can be reversed because apparently in -CURRENT that whole section of code is replaced anyway.

He and a developer on the freebsd-xen mailing list asked if I could try boot a -CURRENT kernel with the "fix" reverted, and it exploded on boot in a different place, so I went back to my *kbd*-less kernel and it's been running fine ever since. I was going to post the backtrace from -CURRENT to the freebsd-xen mailing list, but it seems like every time I go to post on a FreeBSD mailing list my posts I stick my foot in my mouth, and laziness eventually won out.

It's spontaneously blown up on me once when I left nload running (nload has garbage in the interface name too, which might be related, but I can't be bothered tracking the issue down), but other than that I have a Mumble server on that box that's been chugging along for a couple weeks now.



> Do you get the following notification about every 3 minutes in your terminal?
> 
> 
> ```
> ...



I actually get that closer to every 30 seconds - so annoying I actually commented out the line that generates it in case it blew away something more pertinent in dmesg (in my notes, the notice is generated at sys/i386/xen/clock.c:344). As you said, the clock seems to be working fine, and I don't feel like tracking down the actual cause (I'm not near familiar enough with Xen/FreeBSD). 

I said I'd mirror my notes on this forum and the Linode wiki, but honestly I keep going back and editing them and putting them in multiple places just seems like way too much effort, so I published them on one of my sites and I'll mirror them when they're a little less turbulent:

http://www.hungryhacker.com/os/freebsd-on-linode/

Hope it helps someone.


----------



## vtypal (Oct 13, 2010)

fwaggle said:
			
		

> ```
> # [XEN] hypervisor wallclock nudged; nudging TOD.
> ```
> 
> I actually get that closer to every 30 seconds - so annoying I actually commented out the line that generates it in case it blew away something more pertinent in dmesg (in my notes, the notice is generated at sys/i386/xen/clock.c:344). As you said, the clock seems to be working fine, and I don't feel like tracking down the actual cause (I'm not near familiar enough with Xen/FreeBSD).



Instead of commenting this line to halt these messages


```
# [XEN] hypervisor wallclock nudged; nudging TOD.
```

I decided to turn on the independent_wallclock in the dom0 side.
For my Centos dom0 I did the following actions (no need to restart):


```
# DOMO(CENTOS)
echo '/bin/echo 1 > /proc/sys/xen/independent_wallclock' >> /etc/rc.local
echo 1 > /proc/sys/xen/independent_wallclock
```

I also turned on the relative mibs on my domU:


```
#add the following lines to /etc/sysctl.conf
machdep.disable_rtc_set = 1
machdep.indipendent_wallclock = 1
```

and voila' the messages stopped appearing in my terminal.

I run ntp to synchronize my domU clock. Perfect 

This is a greatly working FreeBSD paravirtualized guest (I use the tap:aio driver) in XEN and it consumes less memory than all other linux distros!


----------



## nl0pvm (Nov 4, 2010)

I've used this manual to build my own freebsd FreeBSD PV domU but every 1-2 days i I run into this kernel panic message.

Any advice on this?


```
Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0x8b045029
fault code              = supervisor read, page not present
instruction pointer     = 0x21:0xc0176808
stack pointer           = 0x29:0xd593983c
frame pointer           = 0x29:0xd593983c
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 1, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 40786 (sendmail)
[thread pid 40786 tid 100057 ]
Stopped at      strlen+0x8:     cmpb    $0,0(%edx)
```


----------



## nl0pvm (Nov 6, 2010)

A new one;


```
panic: mtx_lock() of spin mutex (null) @ /usr/src/sys/net/netisr.c:830
cpuid = 0
KDB: enter: panic
[thread pid 15756 tid 100049 ]
Stopped at      kdb_enter+0x3a: movl    $0,kdb_wh
```
y


----------



## black0ut (Nov 21, 2010)

fwaggle said:
			
		

> Good news everyone!
> 
> Using a combination of these instructions, the NetBSD instructions, some good luck and some help from #bsdcode people, I've managed to get FreeBSD i386 to boot under Linode (which is XEN+PV).
> 
> ...



Woo that fixed that problem.  Now it's crashing at this point:


```
[XEN] xen_rtc_attach: attaching Hypervisor RTC clock
Timecounters tick every 10.000 msec
kernel trap 9 with interrupts disabled


Fatal trap 9: general protection fault while in kernel mode
cpuid = 0; apic id = 00
instruction pointer     = 0x21:0xc0313a61
stack pointer           = 0x29:0xc21ffca0
frame pointer           = 0x29:0xc21ffca8
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 1, pres 1, def32 1, gran 1
processor eflags        = resume, IOPL = 0
current process         = 11 (idle: cpu0)
[thread pid 11 tid 100003 ]
Stopped at      outb_+0x6d1:    hlt
```


----------



## black0ut (Nov 21, 2010)

Ah and here's the backtrace

```
Tracing pid 11 tid 100003 td 0xc2308a00
outb_(1,c21ffcf8,c00fc7ee,1,0,...) at outb_+0x6d1
cpu_idle(1,0,c0355dd0,9dd,c2308a00,...) at cpu_idle+0x12
sched_idletd(0,c21ffd38,c034f58a,344,c2306aa0,...) at sched_idletd+0x23e
fork_exit(c00fc5b0,0,c21ffd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xc21ffd70, ebp = 0 ---
```


----------



## black0ut (Nov 21, 2010)

and fwaggle, do you mind uploading your kernel somewhere?  Also, what hardware are you running on?

PS: Sorry for the triple post and where's the edit button?


----------



## DutchDaemon (Nov 21, 2010)

black0ut said:
			
		

> PS: Sorry for the triple post and where's the edit button?



Read your sign-up email. We send it for a reason.


----------



## black0ut (Nov 24, 2010)

DutchDaemon said:
			
		

> Read your sign-up email. We send it for a reason.



I see, 10 posts required.  I guess I'm just used to forums automerging posts :r.


----------



## cperciva@ (Nov 29, 2010)

nl0pvm said:
			
		

> ```
> panic: mtx_lock() of spin mutex (null) @ /usr/src/sys/net/netisr.c:830
> ```



I've fixed this in HEAD, and will MFC some time in the next few weeks.


----------



## ajwak95 (Jan 18, 2011)

I do realize this thread is over 1 year old, but FreeBSD 8.2-RC2 works almost flawlessly with Xen. Almost flawlessly being that, if you compiled the kernel on Intel hardware (CPU)...this will not work on AMD hardware (CPU). I don't know if it is the same vice versa, as I don't have an AMD64 FreeBSD box running (yet!).


----------



## brucec@ (Feb 22, 2011)

I've created a mirror of the files at http://www.cran.org.uk/~brucec/freebsd/xen/ .
Could I copy the instructions to other formats such as the Wiki and possibly the Handbook? The current instructions have a broken link so I'd like to replace them.


----------



## Vener (Mar 25, 2011)

There is another document I used to install 2 FreeBSD domU with a Debian dom0 using lvm. There's no difference with the things you wrote and just work really fine but I found the part with lvm interesting:

http://silverwraith.com/blog/?p=83


----------



## skywalker80239 (Aug 24, 2011)

*How do I get started with XEN?*

The most success I've had with XEN is installing XCP but I wasn't able to install any DOMU environments on it. What is a good place to start with XEN virtualization? 

Thanks.


----------



## nervous (Feb 26, 2012)

I'm running your FreeBSD8 xen images on a Debian wheezy dom0 without any problem! Thanks man!


----------



## romihs (Aug 7, 2012)

Hello,

I would like to compile the latest FreeBSD release (amd64) so that I can import my current ver. 28 zfs pool in the FreeBSD DomU.

I am a little confused, and am not finding the answers online. (I have no experience compiling FreeBSD kernels)
I tried building an amd64 kernel with the XENHVM parameter, but it would not work in Xen. Xen complained that the kernel I was using was not a Xen enabled kernel.
I checked the FreeBSD handbook about building custom kernels, and all I see is info on how to setup the kernel config file for i386 architecture.

My questions are:
1) Do I edit the config file in ..../amd64/conf directory, or the ..../i386/conf directory irrespective of whether I am building an 64 bit or 32 bit kernel?
2) Can I add the XENHVM parameter to the config file, in which case how?
3) How do I go about compiling the 64 bit kernel?

Sorry for the noob questions, but I could not find this info online.

Thanks and best regards

Sandi


----------



## ondra_knezour (Aug 7, 2012)

romihs said:
			
		

> I would like to compile the latest FreeBSD release (amd64) so that I can import my current ver. 28 zfs pool in the FreeBSD DomU.




Create new kernel config file /usr/src/sys/amd64/conf/XENHVMZFS

```
#include default kernel config with Xen support
include XENHVM 
#rename your config
ident XENHVMZFS
#reset this option which is set to empty in included config
nomakeoption MODULES_OVERRIDE
#set your requested modules to be build
makeoption MODULES_OVERRIDE="zfs opensolaris"
```
Note: I tried to add zfs and opensolaris modules to default MODULES_OVERRIDE option from included XENHVM config with += or =+ to be sure nothing breaks in future when empty list may be filled with something, but didn't succeed, so I clear this option first and than set my own.
Compile new custom kernel
`# cd /usr/src && make buildkernel KERNCONF=XENHVMZFS`
Install new custom kernel
`# make installkernel KERNCONF=XENHVMZFS`
Reboot
`# reboot`
You are done

Your question is probably little OT in this thread, because is more oriented to basic kernel compilation in general than to Xen support, so new thread may be more appropriate next time.


----------



## romihs (Aug 7, 2012)

ondra_knezour said:
			
		

> Create new kernel config file /usr/src/sys/amd64/conf/XENHVMZFS
> 
> ```
> #include default kernel config with Xen support
> ...




Thanks for this information, I will try this out to see if I can build a kernel that will work in Xen.

Currently, I get this error when I use a kernel I built:

```
xc: error: elf_xen_note_check: ERROR: Not a Xen-ELF image: No ELF notes or '__xen_guest' section found.: Invalid kernel
```

If I use the kernel and image provided here in this thread I get the following error:

```
KDB: enter: panic
[thread pid 0 tid 0 ]
Stopped at      kdb_enter+0x3a: movl    $0,kdb_why
```
And for that reason I have been trying to build my own FreeBSD kernel.

I apologize for posting OT here.


----------

