# bhyve and system reboot



## fengy (Jul 16, 2019)

Hi
I have a host on freebsd 12, and I'm playing with bhyve.
I have openbsd 6.5 as guest
If I ssh from the host to the guest and if i start firefox, then my host restart.
I don't have such issue with xeyes

is this a known issue ?
Is it possible to catch the oops ? (if there is one) and to save it somewhere in the disk at crash time, or maybe to send it to a remote server at crash time ?
Is it possible for the system to reboot without an oops ? (I suppose my hardware is fine)
thank you


----------



## SirDice (Jul 17, 2019)

fengy said:


> I have openbsd 6.5 as guest
> If I ssh from the host to the guest and if i start firefox, then my host restart.
> I don't have such issue with xeyes


Well, Firefox is an entirely different beast compared to a simple application like xeyes. But a crash inside the VM definitely shouldn't result in the host crashing too, regardless of the application that caused it.

How did you configure the guest? What bhyve(8) settings did you use?


----------



## Phishfry (Jul 17, 2019)

And what about your sshd_config settings on guest?
Did you uncomment these?
X11Forwarding yes
X11DisplayOffset 10
X11UseLocalhost yes

Also start ssh on host with -XY flags.


----------



## SirDice (Jul 17, 2019)

fengy said:


> If I ssh from the host to the guest and if i start firefox, then my host restart.





Phishfry said:


> Also start ssh on host with -XY flags.



Is it the VM that's crashing and taking down the host? Or is it the host that crashes because it has to render the graphics?


----------



## Phishfry (Jul 17, 2019)

I also want to emphasize that -Y strips away security and should only be used in a controlled env.

About debugging. `ssh` offers three layers of verbosity with the -vvv flag.





						ssh(1)
					






					www.freebsd.org


----------



## Phishfry (Jul 17, 2019)

Make sure you have `xauth` on your host. If using a minimalist desktop it might not be installed.
`pkg info xauth`


----------



## SirDice (Jul 17, 2019)

Is there any way you could split things up? Run the secure shell and remote X from a different server or client? Just to find out if the problem is caused by the guest misbehaving or the host?


----------



## fengy (Jul 17, 2019)

SirDice said:


> How did you configure the guest? What bhyve(8) settings did you use?


I'm using vm-bhyve

```
➜  ~ cat /data/vm/obsd/obsd.conf
loader="grub"
cpu=1
memory=256M
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0.img"
grub_install0="kopenbsd -h com0 /6.5/amd64/bsd.rd"
grub_run0="kopenbsd -h com0 -r sd0a /bsd"
bhyve_options="-w"
uuid="a25cb2d4-a7b4-11e9-9c4b-7085c2428a98"
network0_mac="58:9c:fc:02:9f:c4"
```

Ok, 256M is not enought for firefox but the host should not crash because of that

Here is what I did to configure it

```
zfs create data/vm
sysrc vm_enable="YES"
sysrc vm_dir="zfs:data/vm"
vm init
vm switch create public
vm switch add public igb0
vm iso https://cdn.openbsd.org/pub/OpenBSD/6.5/amd64/install65.iso
vm create -t openbsd obsd
vm -f install obsd install65.iso
```

By the way, there is a minor BUG in vm-bhyve because it wrote
"grub_install0="kopenbsd -h com0 /6.2/amd64/bsd.rd"" in obsd.conf (6.2 instead of 6.5. I suppose the OpenBSD version is hardcoded ...)



Phishfry said:


> And what about your sshd_config settings on guest?
> Did you uncomment these?
> X11Forwarding yes
> X11DisplayOffset 10
> ...


Well, I think my ssh config is correct because I could see xeyes.
I have that:

```
X11Forwarding yes
#X11DisplayOffset 10
#X11UseLocalhost yes
```
And I only use -Y flag on the ssh when running ssh



SirDice said:


> Is it the VM that's crashing and taking down the host? Or is it the host that crashes because it has to render the graphics?



Both are possible. But I think the issue is on VM side because my host seems stable and never crashed because of graphics issue. (But I started to use FreeBSD on my system only few days ago...)



Phishfry said:


> Make sure you have `xauth` on your host. If using a minimalist desktop it might not be installed.
> `pkg info xauth`



I do have it

```
➜  ~ which xauth
/usr/local/bin/xauth
```



SirDice said:


> Is there any way you could split things up? Run the secure shell and remote X from a different server or client? Just to find out if the problem is caused by the guest misbehaving or the host?



Good idea. But unfortunately I currently have only one computer with graphics.


----------



## SirDice (Jul 17, 2019)

fengy said:


> By the way, there is a minor BUG in vm-bhyve because it wrote


Check the template. The templates that come with sysutils/vm-bhyve are examples, you need to adjust those for your situation.



fengy said:


> But unfortunately I currently have only one computer with graphics.


That's a shame, it would make diagnosing it a little easier. 


I assume the host restarts due to a panic(9)? Is there anything logged in /var/log/messages of the host? If nothing can be found check the /var/log/messages of the VM.


----------



## fengy (Jul 17, 2019)

SirDice said:


> That's a shame, it would make diagnosing it a little easier.


Well, I should have an extra computer to test that, but only in something like 10 days.



SirDice said:


> I assume the host restarts due to a panic(9)? Is there anything logged in /var/log/messages of the host? If nothing can be found check the /var/log/messages of the VM.



The reboot is too quick. Maybe, and I say maybe, there is something like 3 or 5 lines printed to the screen before the reboot, but it's too fast to be sure, and I definitively do not have the time to read it.

And there is nothing logged in /var/log/messages of the host or in the VM

Will check from an another computer when I will be able to do so, and I will let you know

thanks for your help


----------



## Crivens (Jul 17, 2019)

You can use a camera/smartphone and record a clip. Then pause it and read as you like. Been there done it...


----------



## fengy (Jul 17, 2019)

Crivens said:


> You can use a camera/smartphone and record a clip. Then pause it and read as you like. Been there done it...



it's also too quick for my camera.
I tried anyway and I can see I had a panic but it's unreadable. In the video there is 5 images with the oops, but only 1 is exploitable.
Ok ... not really exploitable. We can see I had a panic because of page fault but I cannot see which module/function triggered it

Here is what I can guess :
`trap number   12
panic: page fault
cpuid = 3
time ?? = ?????
KDB: stack backtrace:
#0 ... at t kdb_backtrace+??
#1 ... at ?? vpanic ??? + ??
#2 ... at panic + ??
#3 ... at trap_fatal + ??
#4 .. at trap_pfault + ??
#5 ... at trap + ??
#6 ..... at calltrap + ???
#7 .... at
#8
#9
#10
#11
#12
#13
#14 ... at
#15 ... at amd64_syscall + ??`

Anb below is the picture (the best I could get)


----------

