# bhyve + win2k12r2



## eyegor (Jan 29, 2018)

Hello guys. I have a trouble with bhyve.
My system:

```
root@whive:/etc # uname -a
FreeBSD whive 11.1-STABLE FreeBSD 11.1-STABLE #0 r325763: Mon Nov 13 16:43:10 EET 2017     root@whive:/usr/obj/usr/src/sys/HIVE  amd64
root@whive:/etc #
root@whive:/etc # cat /usr/src/sys/amd64/conf/HIVE
#HIVE
include   GENERIC
ident     HIVE
options   SC_NORM_ATTR=(FG_GREEN|BG_BLACK)
options   SC_NORM_REV_ATTR=(FG_GREEN|BG_BLACK)
options   SC_KERNEL_CONS_ATTR=(FG_GREEN|BG_BLACK)
options   SC_KERNEL_CONS_REV_ATTR=(FG_BLACK|BG_RED)
device    lagg
device    if_bridge
device    tap
device    nmdm
root@whive:/etc #
root@whive:/etc # cat /boot/loader.conf
kern.geom.label.disk_ident.enable="0"
kern.geom.label.gptid.enable="0"
zfs_load="YES"
vmm_load="YES"
uhid_load="YES"  ##apcupsd USB support
root@whive:/etc #
root@whive:/etc # cat /etc/rc.conf
##MISC
hostname="whive"
sshd_enable="YES"
ntpd_enable="YES"
ntpd_sync_on_start="YES"
powerd_enable="YES"
dumpdev="NO" #ON-->AUTO OFF-->NO
zfs_enable="YES"
clear_tmp_enable="YES"
fsck_y_enable="YES"
background_fsck="NO"
sendmail_enable="NONE"
apcupsd_enable="YES"

##Network
ifconfig_igb0="up -rxcsum -txcsum -lro -tso"
ifconfig_igb1="up -rxcsum -txcsum -lro -tso"
cloned_interfaces="lagg0"
ifconfig_lagg0="SYNCDHCP laggproto failover laggport igb0 laggport igb1"

##Server
#ftpd_enable="YES"
#ftpd_flags="-D -d -l -E"
#vsftpd_enable="YES"
vm_enable="YES"
#vm_dir="/etc/vm"
vm_dir="zfs:zroot0/vm"
vm_list="whwin1 whdeb0"
#vm_list="whdeb0"
vm_delay="5"
root@whive:/etc #
root@whive:/etc # zpool status
  pool: zroot0
 state: ONLINE
  scan: none requested
config:

   NAME        STATE     READ WRITE CKSUM
   zroot0      ONLINE       0     0     0
     mirror-0  ONLINE       0     0     0
      ada0p3  ONLINE       0     0     0
      ada1p3  ONLINE       0     0     0

errors: No known data errors
root@whive:/etc #
```
The virtual machine whwin1 (win2k12r2) is switched off periodically, and i have error in log:

```
cat /vm/whwin1/vm-bhyve.log
Jan 29 08:30:19: bhyve exited with status 137
Jan 29 08:30:19: destroying network device tap0
Jan 29 08:30:19: stopped
```
whwin1 conf:

```
root@whive:/etc # cat /vm/whwin1/whwin1.conf
#My
guest="windows"
uefi="yes"
cpu=4
memory=32G
network0_type="virtio-net"
network0_switch="sw0"
disk0_type="ahci-hd"
disk0_opts="sectorsize=512"
disk0_dev="sparse-zvol"
disk0_name="disk0"
graphics="yes"
#graphics_wait="yes"
#xhci_mouse="yes"
uuid="8f783c6c-19cd-11e7-9ef4-0cc47adae502"
network0_mac="58:9c:fc:0c:59:86"
debug="yes"
```

Thanks for helping!


----------



## grehan@ (Jan 30, 2018)

Thanks for the config files - that was very helpful and I think points out the problem.


```
root@whive:/etc # cat /boot/loader.conf
...
zfs_load="YES"
```

When running ZFS with bhyve (and particularly with such a large guest), you'll need to limit the ZFS ARC size or bhyve and ZFS will compete for memory, with bhyve generally losing 

A general rule is to restrict 
	
	



```
vfs.zfs.arc_max
```
 in 
	
	



```
/boot/loader.conf
```
 to the amount of memory, minus that desired for bhyve VMs, and maybe an addition few GB.

For example, if you have a 64GB system, and a single 32GB bhyve guest, a reasonable value would be 64 - 32 - 4 = 28G.

You don't need to reboot to set this - it can be done with the sysctl at run-time, though the setting needs to be put into 
	
	



```
loader.conf
```
 so it can be picked up on next boot.


----------



## eyegor (Jan 30, 2018)

Thanks for your reply. I will try your solution soon.


----------



## eyegor (Mar 26, 2018)

grehan@ said:


> vfs.zfs.arc_max


almost 2 months without problem. Your solution is work for me! Thanks a lot, bro!


----------

