# Bhyve host restarts when guest is launched with pptdevs



## aragats (Apr 25, 2020)

A Windows 10 guest works fine without passthru devices on the `bhyve` command line (host is i5-6200U CPU, VT-d is enabled, FreeBSD 12.1-RELEASE).
When the same guest is started with passing through a USB controller (see below), the host *gracefully* restarts. There is nothing in dmesg().

```
% pciconf -lv
....
xhci0@pci0:0:20:0:    class=0x0c0330 card=0x223817aa chip=0x9d2f8086 rev=0x21 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Sunrise Point-LP USB 3.0 xHCI Controller'
    class      = serial bus
    subclass   = USB
....
```
Thanks for ideas!


----------



## SirDice (Apr 28, 2020)

That controller is in use by the host (xhci(4)). You can't pass-through a device that's in use on the host. You need to exclude it with pptdevs:

```
PCI PASSTHROUGH
     When the hardware supports VT-d, and vmm.ko has been loaded at boot time,
     PCI devices can be reserved for use by the hypervisor.  Entries
     consisting of the PCI bus/slot/function are added to the pptdevs
     loader.conf(5) variable.  Additional entries are separated by spaces.
     Host PCI devices that match an entry will be assigned to the hypervisor
     and will not be probed by FreeBSD device drivers.  See the EXAMPLES
     section below for sample usage.

     A large number of PCI device entries may require a string longer than the
     128-character limit of loader.conf(5) variables.  The pptdevs2 and
     pptdevs3 variables can be used for additional entries.

EXAMPLES
     Reserve three PCI devices for use by the hypervisor: bus 10 slot 0
     function 0, bus 6 slot 5 function 0, and bus 6 slot 5 function 1.

           pptdevs="10/0/0 6/5/0 6/5/1"
```


----------



## aragats (Apr 28, 2020)

SirDice said:


> That controller is in use by the host (xhci(4)). You can't pass-through a device that's in use on the host.


No, above is the output of `pciconf` *before* adding the "pptdevs" to /boot/loader.conf ― just to show its original state.
Of course, without the corresponding entry in /boot/loader.conf bhyve() won't even start.


----------



## SirDice (Apr 28, 2020)

I haven't tried to pass-through an USB controller. It definitely works for a LSI SAS/SATA card:


```
pptdevs="2/0/0"
```


```
ppt0@pci0:2:0:0:        class=0x010700 card=0x040015d9 chip=0x00721000 rev=0x02 hdr=0x00
    vendor     = 'Broadcom / LSI'
    device     = 'SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]'
    class      = mass storage
    subclass   = SAS
```


----------



## aragats (Apr 28, 2020)

I have a USB controller perfectly working in MS Windows running in bhyve() in another box:
	
	



```
% pciconf -lv | grep -A4 ppt
ppt0@pci0:5:0:0:    class=0x0c0330 card=0x34831106 chip=0x34831106 rev=0x01 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VL805 USB 3.0 Host Controller'
    class      = serial bus
    subclass   = USB
```
The question here is why the host is restarting. I have had other issues with ppt in the past, but never like this. It reboots as it would after `shutdown -r now` or Ctrl-Alt-Del.


----------



## SirDice (Apr 28, 2020)

Maybe the Sunrise controller consists of more than 1 device and you only excluded the "main" but not the other (sub) devices?


----------



## aragats (Apr 28, 2020)

SirDice said:


> Maybe the Sunrise controller consists of more than 1 device and you only excluded the "main" but not the other (sub) devices?


That's a good point! Thanks!
	
	



```
pciconf -lv | grep -A4 0:20:
xhci0@pci0:0:20:0:    class=0x0c0330 card=0x223817aa chip=0x9d2f8086 rev=0x21 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Sunrise Point-LP USB 3.0 xHCI Controller'
    class      = serial bus
    subclass   = USB
none2@pci0:0:20:2:    class=0x118000 card=0x223817aa chip=0x9d318086 rev=0x21 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Sunrise Point-LP Thermal subsystem'
    class      = dasp
```
I don't really understand the relationship between thermal subsystem and USB, will try to pass through both, but I'm not sure it's a good idea...
Although if it shows "none2" now, it's probably not being handled anyway, right?


----------



## Emrion (Apr 28, 2020)

I don't know if it is the problem, but there is a bug with some Intel CPU concerning bhyve and passthru: PR 229852.

This bug AFAIK is still present in 12.1-RELEASE. It has been corrected on 12-STABLE.
You can apply the patch and recompile the kernel to see if it solves your trouble.


----------



## Phishfry (Apr 28, 2020)

I never had a solid experience with motherboard USB passthrough.
There are so many BIOS options like EHCI/UHCI/XHCI and every board seems to allot them differently.

I am using some PCIe NEC USB3 controllers (Dell branded) and they work good.

```
ppt1@pci0:3:0:0:    class=0x0c0330 card=0x04981028 chip=0x01941033 rev=0x03 hdr=0x00
    vendor     = 'NEC Corporation'
    device     = 'uPD720200 USB 3.0 Host Controller'
    class      = serial bus
```
Full height bracket =Dell P/N YJ94F
Half height bracket =0FWGJ8


----------

