# bhyve Windows VM much slower after upgrading to freebsd 12.1



## joggx (Jul 2, 2020)

Hoping to improve bhyve Windows Server disk IO performance I upgraded 11.1-Stable to 12.1-Release by building from source. But after that CrystalDiskMark benchmarks are less than half of the previous. For example, 765 vs 279 MB/s in Q32T1.

12.1 is supposed to include the latest the bhyve update in kernel. What could go wrong here?


----------



## SirDice (Jul 2, 2020)

Have  a look  in  this thread:   https://forums.freebsd.org/threads/bhyve-windows-server-slow-io.71199/


----------



## joggx (Jul 21, 2020)

I have never done patching a freebsd. Under /usr/src, my file version is different:





compared to the one in that thread:


----------



## SirDice (Jul 21, 2020)

The patch is for -CURRENT aka _head_, that's why that line is different compared to yours.


----------



## zirias@ (Jul 21, 2020)

Here's a patch I use on 12.1-RELEASE-p7, this includes the APIC virtualization patch from the other thread which improves windows performance on _some_ host systems tremendously, and also a newer patch which enables TRIM with virtio-blk devices.

To apply, make sure your source tree actually is -p7, go to /usr/src and issue
`patch -p0 <~/bhyve-improvements-fbsd12.1-p7.diff.txt` (assuming you stored the patch in ~).


----------



## joggx (Jul 21, 2020)

Thanks. I found the current already ahead of those patches. If the rebuild takes the same length of time, then I would go with current.


----------



## Mjölnir (Jul 21, 2020)

joggx said:


> Thanks. I found the current already ahead of those patches. If the rebuild takes the same length of time, then I would go with current.


If I understand that correctly, then you're going to run FreeBSD-CURRENT (aka 13 pre-pre-pre-RELEASE) with all bells & whistles _& bugs_ in other parts of kernel & userland.  That's probably not what you want.  In contrast, Zirias patch is for 12.1-RELEASE-p7.  Usually you do not want the latest & brightest, but a stable system.
EDIT 1. You can speed up your builds by using devel/ccache and setting MAKE_JOBS to `sysctl hw.ncpu` + 1
2. Standard disclaimer : install the docs: `pkg install {de,en}-freebsd-doc`, replace _de_ with your native tongue, and point your favorite browser to /usr/local/share/doc/freebsd.
You can add to the alias section of /usr/local/etc/pkg.conf

```
message: "query '[%C/%n] %M'",
  rmessage: query -i "[%C/%n-%v] %M",
```
 and read through all `pkg message|less`.


----------



## joggx (Jul 21, 2020)

OK. I use the following, how do I know it gives the p7?


```
svnlite checkout https://svn.freebsd.org/base/releng/12.1 /usr/src
```


----------



## SirDice (Jul 21, 2020)

That's the correct URL for 12.1-RELEASE plus security patches.


----------



## zirias@ (Jul 21, 2020)

joggx said:


> OK. I use the following, how do I know it gives the p7?


It gives you the latest patchlevel at any time, which happens to be -p7 right now  The reason I added -p7 to my diff file is that part of the diffs I already had applied locally was included with -p7, see here: https://www.freebsd.org/security/advisories/FreeBSD-EN-20:13.bhyve.asc -- so you probably shouldn't use the diff attached here on any older patchlevel. But it should work just fine in future patchlevels, a "clash" like that is pretty unlikely.


----------



## joggx (Jul 21, 2020)

Checked out 12.1 source and ran the patch command and got the message below,


----------



## Mjölnir (Jul 21, 2020)

The interesting part of the error message, please.  Copy & paste, no image if at all possible.


----------



## zirias@ (Jul 21, 2020)

joggx said:


> Checked out 12.1 source and ran the patch command and got the message below,


Are you in /usr/src? Did you add the `-p0` flag for patch?


----------



## joggx (Jul 21, 2020)

The release was -p6 I _freebsd-update_'d to -p7 then _svnlite up_ again then it checks out the a newer version (newer than 363035). Then the patch went through.

svnlite didn't _checkout _the latest version until release updated to -p7.


----------



## zirias@ (Jul 21, 2020)

joggx said:


> svnlite didn't _checkout _the latest version until release updated to -p7.


That's impossible, at least if you imply causality...


----------



## Mjölnir (Jul 21, 2020)

joggx said:


> The release was -p6 I _freebsd-update_'d to -p7 then _svnlite up_ again then it checks out the a newer version (newer than 363035). Then the patch went through.
> 
> svnlite didn't _checkout _the latest version until release updated to -p7.


Did you really `rm -fr` /usr/src? Seems that old .svn junk was still there with a certain revision locked?


----------



## zirias@ (Jul 21, 2020)

Just for the records, this picture of an error shown above doesn't imply -p6. The patch would probably apply to -p6 as well (I just wouldn't recommend that), but even if not, `patch` would still find all the files to patch. They were all present in earlier patchlevels as well.


----------



## joggx (Jul 22, 2020)

mjollnir said:


> Did you really `rm -fr` /usr/src? Seems that old .svn junk was still there with a certain revision locked?



Right I had to _rm -fr .* _too otherwise co doesn't work.


----------



## joggx (Jul 22, 2020)

The patch has been done and the CrystalDiskMark benchmarks look better (Q32T1 400MB/s ~) but still fall behind 11.1-Stable.


----------



## joggx (Jul 22, 2020)

Zirias said:


> Just for the records, this picture of an error shown above doesn't imply -p6. The patch would probably apply to -p6 as well (I just wouldn't recommend that), but even if not, `patch` would still find all the files to patch. They were all present in earlier patchlevels as well.



It doesn't but I have -p6 confirmed by _uname -r_. It stops there asking for location of file to patch.


----------



## Mjölnir (Jul 22, 2020)

Last resort is to write a bug report.  Significantly slower performance is a valid reason IMHO.


----------



## zirias@ (Jul 22, 2020)

joggx said:


> It doesn't but I have -p6 confirmed by _uname -r_. It stops there asking for location of file to patch.


First of all, you are aware that just updating the source won't update your running system? So, `uname -r` won't tell you _anything_ about the state of your /usr/src.
And then, again, the files to patch are there on any patchlevel of 12.1. You made some other mistake (like e.g. being in the wrong directory when trying to patch).

I now have to ask that: after applying the patch, did you actually compile and install?


----------



## joggx (Jul 24, 2020)

Zirias said:


> First of all, you are aware that just updating the source won't update your running system? So, `uname -r` won't tell you _anything_ about the state of your /usr/src.
> And then, again, the files to patch are there on any patchlevel of 12.1. You made some other mistake (like e.g. being in the wrong directory when trying to patch).
> 
> I now have to ask that: after applying the patch, did you actually compile and install?



It is not updating the source, it is checkout (see I mentioned purging everything in /usr/src). I am not sure how svnlite determines the version being checked out in absence of -r but it seems it is not getting the latest release's source and probably running release is something it relies on. 

As you said before applying the patch make sure to have -p7 that's why I came up with _freebsd-update _to -p7 first to minimize the risk. You later said it _should _work with -p6 (tested?) but it didn't work for me.  The steps of handbook/makeworld.html were followed and after that (less than 2 hrs) _uname -r_ does give a new release -p7.  Otherwise doubled I/O benchmark is out of nowhere.


----------

