# Bhyve with Ubuntu Linux



## rdunkle (Dec 28, 2014)

I am using FreeBSD 11 amd64  r275582
and have tried on FreeBSD 10.1 amd64 Release and it appears
to work equally well.

Preliminary setup:

Install port grub2-bhyve
`cd /usr/ports/sysutils/grub2-bhyve`
`make install clean`

Allow needed kernel modules at boot time:
Edit: /boot/loader.conf
Add the following lines:

```
vmm_load="YES"
nmdm_load="YES"
if_bridge_load="YES"
if_tap_load="YES"
```

Edit:  /etc/rc.conf

Add the following lines:

```
cloned_interfaces="bridge0 tap0"
ifconfig_bridge0="addm em0 addm tap0"
```

**Note -- change em0 to whatever your real network interface is

Make a sysctl change.
Enter this line to add to /etc/sysctl.conf

```
echo "net.link.tap.up_on_open=1" >> /etc/sysctl.conf
```

Reboot FreeBSD to allow changes to apply.

Create directories for virtual machines:
`mkdir /u1`
`mkdir /u1/dist`
`mkdir /u1/ubuntu`

Download the Ubuntu install ISO file: ubuntu-14.10-server-amd64.iso. It can be found here:  http://releases.ubuntu.com/14.04/.

Save ISO file to /u1/dist.

Create the map and OS image file:
`cd /u1/ubuntu`
`truncate -s 40G ubuntu.img`

Create the device map file:
`cd /u1/ubuntu`
`vi ubuntu.map`

Add these two lines to ubuntu.map

```
(hd0) ./ubuntu.img
(cd0) /u1/dist/ubuntu-14.10-server-amd64.iso
```

Start the Grub installer:
`cd /u1/ubuntu`
`grub-bhyve -m ubuntu.map -r cd0 -M 2048M ubuntu`

The Ubuntu Grub installer menu will appear.


```
Press Enter to select "Install Ubuntu Server"
```
The FreeBSD command prompt will appear.


Now start the installer:

```
bhyve -c 2 -m 2048M -H -P -A -l com1,stdio \
-s 0:0,hostbridge -s 1:0,lpc \
-s 2:0,virtio-net,tap0 \
-s 3,ahci-cd,/u1/dist/ubuntu-14.10-server-amd64.iso \
-s 4,virtio-blk,ubuntu.img ubuntu
```

The Ubuntu text installer will start.

The installer asks some basic questions. Watch out --  select "Guided - use entire disk".

Note: Select Guided - use entire disk
The default -- Guided -use entire disk and set up LVM
Caused Ubuntu to crash later on during boot. YMMV.

I did install the OpenSSH server so I could `ssh` to Ubuntu.

Make sure to select 
	
	



```
YES -- Install the GRUB boot loader on a hard disk
```

When Ubuntu completes the install you will see:

```
The system is going down NOW!
Sent SIGTERM to all processes
Sent SIGKILL to all processes
Requesting system reboot
[  703.362711] reboot: Restarting system
```
Then the FreeBSD shell prompt returns.

Need to stop the VM with this command:
`bhyvectl --destroy --vm=ubuntu`

To make the Ubuntu start up a single command create a small script.

```
cd /u1/ubuntu
vi start_ubuntu.sh
```

Add the following lines to the file:


```
grub-bhyve -m ubuntu.map -r hd0,msdos1 -M 2048M ubuntu
bhyve -w -AI -H -P -s 0:0,hostbridge  \
-s 1:0,lpc -s 2:0,virtio-net,tap0 \
-s 3:0,virtio-blk,./ubuntu.img \
-l com1,stdio -c 2 \
-m 2048M ubuntu
```

Save the file, and make it executable:
`chmod +x start_ubuntu.sh`

You should now be able to start the fresh Ubuntu install with this command: `./start_ubuntu.sh`


Finally the Ubuntu text console prompt will appear:

```
Ubuntu 14.10 ubuntu ttyS0

ubuntu login:
```
You now be able to ssh or work from the text console on Ubuntu.

How to shut down the VM:

Log in to Ubuntu from console or ssh.
`sudo init 0`

When the FreeBSD shell prompt appears:

```
bhyvectl --destroy --vm=ubuntu
```


----------



## BSDBernd (Dec 29, 2014)

Brilliant! Thank you for this.


----------



## grehan@ (Dec 29, 2014)

The LVM setup will work with ahci-hd. For some reason the LVM install seems to trigger a bug in bhyve's virtio-blk emulation that I'll look into. Note that if LVM is used, the -d /grub parameter has to be passed to grub-bhyve since the GRUB config files are placed in a different directory by Ubuntu.

You can shut down the VM externally by sending a TERM signal to the bhyve process - this will trigger an ACPI shutdown in the guest (at least for recent Ubuntu's - older ones may require the acpid package to be installed).


----------



## rdunkle (Jan 2, 2015)

Thanks !
-d grub parameter is just what I needed to get Fedora 21 working

The Ubuntu LVM install quickly crashes with this error:

```
[  110.034570] BUG: unable to handle kernel paging request at 0000000010020000
[  110.035356] IP: [<ffffffff81370281>] __blk_bios_map_sg+0x1b1/0x3c0
[  110.035993] PGD 7a7f1067 PUD 7a653067 PMD 0
[  110.036492] Oops: 0000 [#1] SMP
```

I saw a similar install crash with Fedora 21 with BTRFS selected:

```
Starting package installation process
[  243.197320] BUG: unable to handle kernel paging request at 0000000010020000
[  243.198069] IP: [<ffffffff8135b505>] __blk_bios_map_sg+0x1e5/0x3f0 
[  243.198725] PGD 723d7067 PUD 723d8067 PMD 0
[  243.199207] Oops: 0000 [#1] SMP
```


----------



## diizzy (Feb 23, 2015)

I gave Bhyve a spin (a while ago) on FreeBSD 10.1 with good success, although Debian.
http://randominfo.pyret.net/index.php?controller=post&action=view&id_post=9
Works fine with serial access too.

I do not intend to hijack this thread, just mention that Debian also works.
//Danne


----------



## lib13 (Feb 24, 2015)

I wonder if one can use xpra as server inside the guest Ubuntu, and xpra as client in the FreeBSD host so that to have a GUI for the Linux system.


----------

