# Running FreeBSD 8.1 amd64 as a XEN HVM domU locks up



## boersmic (Sep 22, 2010)

I am trying to run FreeBSD 8.1 on XEN 4.0.1 install on Debian Lenny.  I have be able to make it work mostly, but under heavy load with multiple virtual processors, it often locks up and xentop reports the domain is using 100% of all its vcpus. 

I have compiled the XENHVM kernel but I need to remove:


```
options       INVARIANTS              # Enable calls of extra sanity checking
options       INVARIANT_SUPPORT       # Extra sanity checks of internal structures, required by IN
VARIANTS
options       WITNESS                 # Enable checks to detect deadlocks and cycles
options       WITNESS_SKIPSPIN        # Don't run witness on spinlocks for speed
```

or else the machine would crash on startup with:

```
kernel trap 22 with interrupts disabled
fpudna: fpcurthread == curthread 1433 times
fpudna in kernel mode!
kernel trap 22 with interrupts disabled
fpudna: fpcurthread == curthread 1434 times
fpudna in kernel mode!
kernel trap 22 with interrupts disabled
fpudna: fpcurthread == curthread 1435 times
```

If anyone can point me in the right direction, I would be very appreciative.  I have spent about a week banging on this.  Here are some configs in case someone sees something helpful:

more configs/aquaria.cfg

```
name = 'aquaria'
kernel = '/usr/lib/xen/boot/hvmloader'
builder = 'hvm'
memory = '6144'
vcpus = '1'
device_model = '/usr/lib/xen/bin/qemu-dm'

disk = [ 
	'file:/home/xen/disks/aquaria/boot.img,hda,w',
	'phy:/dev/sdc,hdb,w',
	'phy:/dev/sdd,hdc,w'
	 ]

#vif = ['type=ioemu, bridge=eth2']
vif = ['bridge=eth0']

serial='pty'
#localtime='1'
```

FreeBSD Kernel Definition

```
#
# XEN5 -- Custom kernel configuration file for FreeBSD/amd64
#

cpu             HAMMER
ident           XEN5

# To statically compile in device wiring instead of /boot/device.hints
#hints          "GENERIC.hints"         # Default places to look for devices.

# Use the following to compile in values accessible to the kernel
# through getenv() (or kenv(1) in userland). The format of the file
# is 'variable=value', see kenv(1)
#
# env           "GENERIC.env"

#makeoptions    DEBUG=-g                # Build kernel with gdb(1) debug symbols

options         SCHED_ULE               # ULE scheduler
options         PREEMPTION              # Enable kernel thread preemption
options         INET                    # InterNETworking
#options        INET6                   # IPv6 communications protocols
options         SCTP                    # Stream Control Transmission Protocol
options         FFS                     # Berkeley Fast Filesystem
options         SOFTUPDATES             # Enable FFS soft updates support
options         UFS_ACL                 # Support for access control lists
options         UFS_DIRHASH             # Improve performance on big directories
options         UFS_GJOURNAL            # Enable gjournal-based UFS journaling
#options        MD_ROOT                 # MD is a potential root device
#options        NFSCLIENT               # Network Filesystem Client
#options        NFSSERVER               # Network Filesystem Server
#options        NFSLOCKD                # Network Lock Manager
#options        NFS_ROOT                # NFS usable as /, requires NFSCLIENT
#options        MSDOSFS                 # MSDOS Filesystem
options         CD9660                  # ISO 9660 Filesystem
options         PROCFS                  # Process filesystem (requires PSEUDOFS)
options         PSEUDOFS                # Pseudo-filesystem framework
options         GEOM_PART_GPT           # GUID Partition Tables.
options         GEOM_LABEL              # Provides labelization
options         COMPAT_43TTY            # BSD 4.3 TTY compat (sgtty)
options         COMPAT_FREEBSD32        # Compatible with i386 binaries
options         COMPAT_FREEBSD4         # Compatible with FreeBSD4
options         COMPAT_FREEBSD5         # Compatible with FreeBSD5
options         COMPAT_FREEBSD6         # Compatible with FreeBSD6
options         COMPAT_FREEBSD7         # Compatible with FreeBSD7
options         SCSI_DELAY=5000         # Delay (in ms) before probing SCSI

options         KTRACE                  # ktrace(1) support
options         STACK                   # stack(9) support
options         SYSVSHM                 # SYSV-style shared memory
options         SYSVMSG                 # SYSV-style message queues
options         SYSVSEM                 # SYSV-style semaphores
###options      P1003_1B_SEMAPHORES     # POSIX-style semaphores
options         _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions
###options      PRINTF_BUFR_SIZE=128    # Prevent printf output being interspersed.
options         KBD_INSTALL_CDEV        # install a CDEV entry in /dev
options         HWPMC_HOOKS             # Necessary kernel hooks for hwpmc(4)
options         AUDIT                   # Security event auditing
###options      MAC                     # TrustedBSD MAC Framework
###options              FLOWTABLE               # per-cpu routing cache
#options        KDTRACE_FRAME           # Ensure frames are compiled in
#options        KDTRACE_HOOKS           # Kernel DTrace hooks
options         INCLUDE_CONFIG_FILE     # Include this file in kernel

options         NO_ADAPTIVE_MUTEXES
options         NO_ADAPTIVE_RWLOCKS

# Debugging for use in -current
##options       KDB                     # Enable kernel debugger support.
##options       DDB                     # Support DDB.
##options       GDB                     # Support remote GDB.
##options       INVARIANTS              # Enable calls of extra sanity checking
##options       INVARIANT_SUPPORT       # Extra sanity checks of internal structures, required by INVARIANTS
##options       WITNESS                 # Enable checks to detect deadlocks and cycles
##options       WITNESS_SKIPSPIN        # Don't run witness on spinlocks for speed

# Make an SMP-capable kernel by default
options         SMP                     # Symmetric MultiProcessor Kernel

options         XENHVM
device          xenpci

# CPU frequency control
device          cpufreq

# Bus support.
device          acpi
device          pci

# ATA and ATAPI devices
device          ata
device          atadisk         # ATA disk drives
device          atapicd         # ATAPI CDROM drives
options         ATA_STATIC_ID   # Static device numbering

# SCSI peripherals
device          scbus           # SCSI bus (required for SCSI)
device          da              # Direct Access (disks)
device          cd              # CD
device          pass            # Passthrough device (direct SCSI access)
device          ses             # SCSI Environmental Services (and SAF-TE)

# atkbdc0 controls both the keyboard and the PS/2 mouse
device          atkbdc          # AT keyboard controller
device          atkbd           # AT keyboard
device          psm             # PS/2 mouse

device          kbdmux          # keyboard multiplexer

device          vga             # VGA video card driver

device          splash          # Splash screen and screen saver support

# syscons is the default console driver, resembling an SCO console
device          sc

device          agp             # support several AGP chipsets

# Serial (COM) ports
device          uart            # Generic UART driver

# PCI Ethernet NICs that use the common MII bus controller code.
# NOTE: Be sure to keep the 'device miibus' line in order to use these NICs!
device          miibus          # MII bus support
device          re              # RealTek 8139C+/8169/8169S/8110S

# Pseudo devices.
device          loop            # Network loopback
device          random          # Entropy device
device          ether           # Ethernet support
device          vlan            # 802.1Q VLAN support
device          tun             # Packet tunnel.
device          pty             # BSD-style compatibility pseudo ttys
device          md              # Memory "disks"
device          gif             # IPv6 and IPv4 tunneling
device          faith           # IPv6-to-IPv4 relaying (translation)
device          firmware        # firmware assist module

# The `bpf' device enables the Berkeley Packet Filter.
# Be aware of the administrative consequences of enabling this!
# Note that 'bpf' is required for DHCP.
device          bpf             # Berkeley packet filter

device          pf
device          pflog
```


----------



## joag (Sep 28, 2010)

I don't know if the issue is related but at least is close to what is happening to me. 

I have an Intel dp35dp motherboard, Celeron dual core 1.6GHz and 6Gigs of RAM. If I use 4 GIGs or more the system crashes. At the beginning I thought it was a memory issue but after placing 4Gigs of brand new modules I realized the issue is with the OS. I'm running amd64. If I use 2Gigs or less the system works just fine. 

I installed amd64 because I want to use the tinderbox port in order to make some builds but this is issue is bothering me . I had i386 before with 6Gigs or RAM with no problems but you guys know what happen after 3.xxx Gigs . 

I think this is a bug in the 64Bit build as I've installed twice and tested with brand new memories. I hope this can help in someway.


----------

