# Should FreeBSD port NetBSD's NVMM?



## Deleted member 63822 (Sep 11, 2020)

No problem with Bhyve, just to have more option. Dragonfly doesn't have any hypervisor, so it's more important to them. But I think no thing prevents NVMM on FreeBSD other than the developers' lack of interest.






						DragonFly hypervisor in the works – DragonFly BSD Digest
					






					www.dragonflydigest.com


----------



## Phishfry (Sep 11, 2020)

If you want NVMM on FreeBSD than port it yourself.


gh_legacy said:


> developers' lack of interest.


Yea we got hundreds of developers with nothing to do.

You keep playing this whac-a-mole game with user names and I don't think it is OK.
What is this your third user name?


----------



## ekvz (Sep 11, 2020)

There is no need for a ton of options. A single one that works well is fully practical. Diverting resources to make each of the resulting _options_ weaker is bad and so is bloat.



anonymous9 said:


> FreeBSD's pf is out of sync with OpenBSD's pf, and I thought there may have been talk of removing it or moving it to ports(?).  I'm not sure where ipf sits now, but there was talk of removing it.



That's a good example. I have been wondering what's wrong with ipfw since the day i came back to FreeBSD.


----------



## Phishfry (Sep 11, 2020)

What the OP needs to ask himself is- What does NVMM do that Bhyve does not?
That is what I would ask myself if I was a developer.

We have Bhyve and Xen.
Why do we need more options? We have two perfectly functioning tools for Virtualization.
Why would a developer want to add a third?
Is this a numbers game? The OS with the most choices win?
We have limited manpower and need to focus on refining the choices we have.


----------



## ekvz (Sep 11, 2020)

Phishfry said:


> The OS with the most choices win?



Not for me. _Options_ (at least for such basic functionality) irritate me.



Phishfry said:


> We have limited manpower and need to focus on refining the choices we have.



That doesn't necessarily has anything to do with available manpower. Bloat doesn't make a good system even if you have the resources to build it.


----------



## Phishfry (Sep 11, 2020)

I have never understood why it is a big deal that FreeBSD's pf is out of sync with OpenBSD version.
A firewalls job is to filter packets. What exactly is our older pf version missing? Does it not filter packets?


----------



## Jose (Sep 11, 2020)

Phishfry said:


> I have never understood why it is a big deal that FreeBSD's pf is out of sync with OpenBSD version.
> A firewalls job is to filter packets. What exactly is our older pf version missing? Does it not filer packets?





			https://home.nuug.no/~peter/pf/newest/shaping.queues.prio.html
		


Edit: It would be nice to have arpbalance in carp(4), too


----------



## Mjölnir (Sep 11, 2020)

I stopped clicking _Thanks & Like_ ... it just doesn't make sense on _any_ post...  When I'm bored, I use the forums search facilities and randomly select one of _gh_*_'s threads...  I didn't figure out his account name on reddit or such, because I'm not visiting such


----------



## olli@ (Sep 11, 2020)

VirtualBox also works quite well for me on FreeBSD.


----------



## Deleted member 63822 (Sep 12, 2020)

olli@ said:


> VirtualBox also works quite well for me on FreeBSD.


And VirtualBox 5.x is EOL and VirtualBox 6.1.x is not yet in ports.


----------



## Deleted member 63822 (Sep 12, 2020)

mjollnir said:


> I stopped clicking _Thanks & Like_ ... it just doesn't make sense on _any_ post...  When I'm bored, I use the forums search facilities and randomly select one of _gh_*_'s threads... I didn't figure out his account name on reddit or such, because I'm not visiting such


I named my nick name according to Ublock Origin.

gh_origin => Ublock Origin
gh_legacy => Ublock Origin Legacy

There is only two of _gh_*_'s and I didn't create them at the same time. Remember, gh_origin was lost because lost both of it's password and the email used to register it. So the _sysctl_ account came to existent. The gh_origin and sysctl account was merged into sysctl and this sysctl was deleted by the admins. So this gh_legacy account came to existent. Don't said like I'm a troll with multiple accounts to spam, I'm not. And I don't have a Reddit account, just reading.


----------



## Deleted member 63822 (Sep 12, 2020)

Phishfry said:


> What the OP needs to ask himself is- What does NVMM do that Bhyve does not?


NVMM utilize qemu with qemu-nvmm, it's a complete emulator just like Linux KVM which also utilized qemu, but our Bhyve is just a hypervisor and not a full emulator. I would ask for something like qemu-bhyve first but I don't think the developers will care, they are busy keep their KISS principle. You asked what NVMM can and Bhyve can't. Very simple, a good console. Our nmdm console is shite. And a real graphics hardware emulation. Remember we are still relying on EFI framebuffer to have graphics. And it sucks.


----------



## olli@ (Sep 12, 2020)

gh_legacy said:


> And VirtualBox 5.x is EOL and VirtualBox 6.1.x is not yet in ports.


Actually I don’t care if it’s EOL. As I said, it works fine for me.


----------



## Alexander88207 (Sep 12, 2020)

gh_legacy said:


> And VirtualBox 5.x is EOL and VirtualBox 6.1.x is not yet in ports.



I guess we had this topic already. A PR for this is already open and you can contribute to it. If not then do not complain. As long as there are no dangerous security gaps everything is ok


----------



## a6h (Sep 13, 2020)

gh_legacy said:


> I don't have a Reddit account


Are you sure you are a Linux user?


----------



## jmos (Sep 13, 2020)

gh_legacy said:


> And a real graphics hardware emulation. Remember we are still relying on EFI framebuffer to have graphics. And it sucks.


According to my documents bhyve wasn't designed as an emulator at all, but as a hypervisor. If you need an emulator, don't use bhyve.

VMs are classically used for servers and test environments - not for playing games. In my opinion the output via VNC (next to shell) is absolutely sufficient; If I want to use another operating system with a graphical user interface in relevant depth, a native OS makes more sense than a VM.

And: I don't feel called to change this situation, nor do I think it is appropriate to say to those few who gave me bhyve at the expense of their spare time "do more, that was too little"; In my opinion the word "we" is inappropriate in relation to non-commercial open source projects: There is no "we" as long as you don't contribute.


----------



## Deleted member 63822 (Sep 13, 2020)

vigole said:


> Are you sure you are a Linux user?


I used Linux before Reddit ever popular in my country. And even now, it's not as popular as the native alternatives


----------



## Deleted member 63822 (Sep 13, 2020)

jmos said:


> And: I don't feel called to change this situation, nor do I think it is appropriate to say to those few who gave me bhyve at the expense of their spare time "do more, that was too little";


And you still think the open source projects are done by developers on their spare time?  How naive! All of them are backed by commercial company. Even your Bhyve is developed by Pluribus Networks, a company, not a group of developers on their spare time!


----------



## Hakaba (Sep 13, 2020)

«Should FreeBSD port NetBSD's NVM?»
No.
FreeBSD is not a port maintenance project.
But if you want to port nvm you can, FreeBSD accept new port in the ports list.
Why an another thread like this ? The answer will never vary as this is the organisation of the project.
if you are facing issue or need changes when you port a specific tool, you need to contact the core team, but it is a mailing list and the forum is not the right place to this kind of debate.


----------



## ekvz (Sep 13, 2020)

Alexander88207 said:


> A PR for this is already open and you can contribute to it.



He is probably to busy porting this NVMM thing.



gh_legacy said:


> And a real graphics hardware emulation. Remember we are still relying on EFI framebuffer to have graphics. And it sucks.



Can you enlighten me what's so particularly great about qemus graphics capabilities? It has a bunch of dusty chipsets that are more or less good enough for 2D but beyond that? OK, there seems to some recent development concerning virtio but that's still somewhat bleeding edge even on Linux (aside from the fact that noone cares). Besides the (by a wide margin) biggest market for virtualization is headless and probably wouldn't even raise an eyebrow if there was no graphics support at all.

I am not saying that your line of thinking is invalid but just that (as you might have noticed by the generally dismissive responses) that what you want seems to be pretty much an edge case and "I wan't feature X which might be mostly redundant and non trivial on top of that but..." usually doesn't work there. You either need to convince people how it's going to be useful for them (which seems rather hopeless in this case) or do it yourself.



gh_legacy said:


> And you still think the open source projects are done by developers on their spare time?  How naive! All of them are backed by commercial company. Even your Bhyve is developed by Pluribus Networks, a company, not a group of developers on their spare time!



A lot of the work is in fact done by unpaid volunteers. When companies sponsor a certain part of development or open their own code it's done for a reason. Namely to gain the support of all those unpaid volunteers. A handfull of large projects might be able to somewhat survive purely on the money of some sponsoring company but everything else in Open Source would collapse pretty quickly when you take the volunteers out of the equation and those smaller projects would very likely take most bigger ones down with them as the ecosystem becomes non functional.


----------



## jmos (Sep 13, 2020)

gh_legacy said:


> And you still think the open source projects are done by developers on their spare time?  How naive! All of them are backed by commercial company.


…so you think "we" can demand a company to invest more to get you satisfied? And yes, I'm not only thinking that there are numerous private driven open source projects: I know that


----------



## kpedersen (Sep 13, 2020)

gh_legacy said:


> Even your Bhyve is developed by Pluribus Networks, a company, not a group of developers on their spare time!



You are incorrect. Pluribus only ported it to some niche platforms. The original Bhyve at least now is very much written by FreeBSD developers in their spare time. I don't even believe their work is sponsored (the best technology rarely is unfortunately).



olli@ said:


> Actually I don’t care if it’s EOL. As I said, it works fine for me.



Exactly. The only thing you really miss out on in 6.0 is more pointless integration with Oracle's "Cloud".

If we renamed VirtualBox 5 to FunctionalBox and gave it an arbitrary version number of 10, is it still EOL? I don't think so. XD


----------



## shkhln (Sep 13, 2020)

The original bhyve has something to do with NetApp, although it's not clear if NetApp was actually using bhyve for anything.


----------



## kpedersen (Sep 13, 2020)

gh_legacy said:


> and this sysctl was deleted by the admins. So this gh_legacy account came to existent.



Normally that suggests the admins feel you should get off your computer, go for a walk and chill out for a few weeks before re-attempting to interact with the community.



shkhln said:


> The original bhyve has something to do with NetApp, although it's not clear if NetApp was actually using bhyve for anything.


Some sources say it "was shelved" by NetApp before they used it commercially.
https://klarasystems.com/articles/bhyve-the-freebsd-hypervisor/
At least they opened it before sending it to the grave 
It seems that it was a Friday hobby project for a few developers.


----------



## shkhln (Sep 13, 2020)

kpedersen said:


> It seems that it was a Friday hobby project for a few developers.



Yeah, that's my impression too.


----------



## Deleted member 63822 (Sep 13, 2020)

ekvz said:


> He is probably to busy porting this NVMM thing.


You can't stop mocking me?


----------



## Alexander88207 (Sep 13, 2020)

gh_legacy said:


> You can't stop mocking me?



I don't know if I am the only one who thinks so, but you were a little unfair in some things (not only here iirc). The sentence came across as if you had a right or it should be naturally that the latest version should be in the ports. This is something that can not be done just by the way.


----------



## ekvz (Sep 13, 2020)

gh_legacy said:


> You can't stop mocking me?



Well, actually i try to but when someone attacks other people over "cheap talk" while not contributing to realizing their own "wishlist items" i get a bit grumpy. Alexander88207 put it quite nicely. There is a certain level of entitlement coming across. It might be a language/written communication thing but that's how it comes across for (seemingly not only) me.

There is nothing wrong with desiring certain features (even if it seems pointless to most people - like my wish to customize the amount and locations of running X servers doesn't make sense to the general public). You just can't expect anyone to invest a lot of time/energy (and that's basically what porting a whole different technology comes down to) into your problems for no personal gain at all. While people around here seem generally very generous and helpful there is a certain line where one is just asking to much.

Concerning your gripes with bhyve you might be better off describing your actual problems. People would very likely try to come up with solutions (well, at least as long as your problem isn't something along the lines of "I want to play Super Tux on SpongebobSquarepants Linux 0.1 but there is no 3D acceleration...") or you might even convince some developer to push out improvements but asking for a whole new system to be integrated is not likely to be going anywhere.

I am very well aware that i am talking from somewhat of a high horse since i am actually capable of helping myself most of time which probably isn't an option for an ordinary user. I am also not saying that you should learn how to code and to everything yourself but i guess it would be very useful if you'd at least familiarize yourself with the development process a bit (if only to get a feel for the amount work certain tasks require). Your tendency to try things might make you into a pretty valuable tester if your conclusions just weren't often so radical and out of this world from a developers perspective.


----------



## Phishfry (Sep 13, 2020)

gh_legacy said:


> Our nmdm console is shite


Can you please expand on this.
How can a console be shite? Does it crash or what exactly does not work?


----------



## ekvz (Sep 13, 2020)

Phishfry said:


> Can you please expand on this.
> How can a console be shite? Does it crash or what exactly does not work?



Exactly. Some substance would be a nice change but i guess it's pretty hopeless.


----------



## Deleted member 63822 (Sep 13, 2020)

Phishfry said:


> Can you please expand on this.
> How can a console be shite? Does it crash or what exactly does not work?


It maybe OK for people who really used a console, but for people has never used a console like me it's very difficult, counter-intuitive to use and too limited. I learned to use the cli on modern virtual terminals like xfce4-terminal, mate-terminal... Asking me to deal with this old console, sorry, I will abandon Bhyve, only because that console. Luckily, we could use tmux as the console for Bhyve. It's really revived my interest for Bhyve and I tested many OSes on Bhyve. If you want to drag a younger audience, then this console is unacceptable. Nowadays we have too many choices, in no way we would limit ourselves by forcing us using that terrible console. No way!


----------



## Deleted member 63822 (Sep 13, 2020)

One example: the arrow keys not work as expected. Us, the youngsters, grew up on modern virtual terminals, at least xterm-256, that old console is unacceptable.


----------



## Deleted member 63822 (Sep 13, 2020)

If that console is as good as the virtual console of FreeBSD before I type startx, I will happily use it. The fact is, it's not. It's sucks. It's a pos! Not only minor annoying like the arrow keys not work, it also has many problem displaying the output of the running OS. It should be abandoned as soon as possible and replaced with a better, easier to use solution.


----------



## ekvz (Sep 13, 2020)

To be honest i am having a hard time following what you are even talking about. Sounds like are angry about some kind of serial console but as you seem to be after running GUIs i don't understand why you would even use something like that (which doesn't magically get "modern" with qemu either by the way). Besides if you absolutely want a nice console to manage your guest you could always just SSH in and use whatever you want as terminal. Am i missing something here or are you talking about some management console? Qemus management console isn't very nice either so i likely wouldn't see the point if you did.

Actually, i get the impression that you aren't making the right choice (of which there are so many as you say). This "younger audience" you speak of seems to suspiciously correlate with "non technical audience" for which there is more than enough appropriate choices. FreeBSD just might (luckily in my personal opinion) not be one of them.

I also don't think "demanding" or pretending to represent some group (hint: you don't - you speak for yourself and noone else) is a good approach. If you'd calmly describe your problem in detail and it happens be not overly time consuming to fix there might actually be people willing to hack this in but if they are approached like this their reaction is likely to be just "let this person deal with their problems themselves".


----------



## a6h (Sep 13, 2020)

gh_legacy said:


> If you want to drag a younger audience,





gh_legacy said:


> Us, the youngsters, grew up on modern virtual terminals, at least xterm-256, that old console is unacceptable.


Don't project you incompetency or lack of experience on young people. Many of them are writing system programs, working on device drivers and programming microcontrollers.


----------



## kpedersen (Sep 13, 2020)

gh_legacy said:


> It maybe OK for people who really used a console, but for people has never used a console like me it's very difficult, counter-intuitive to use.



Unfortunately *you* need to be the one to learn and adapt. Otherwise no-one will hire you as a professional and no-one will work with you as a hobbyist.

I also feel you just need to learn more. That terminal you are complaining about is likely the virtual serial connection. It is limited because it has to be because not everyone is on a consumer PC. Just grit your teeth for the few minutes you need until you set up the guest's network and you can SSH into it instead.

Have you tried to run tmux in that terminal? That tool may provide some of the features you expect.

Come on, you at least need spend a couple of weeks trying to solve the darn problem yourself until you are even qualified to complain about it.



gh_legacy said:


> modern virtual terminals like xfce4-terminal, mate-terminal...


You may not know this but FreeBSD's terminal is actually newer and "more modern" than them (https://www.freebsd.org/cgi/man.cgi?query=vt&sektion=4)


----------



## Jose (Sep 13, 2020)

vigole said:


> Don't project you incompetency or lack of experience on young people. Many of them are writing system programs, working on device drivers and programming microcontrollers.


This is one of the wonderful things that has happened recently with the rise of Raspberri Pi, Arduino, etc. More and more people are bit-banging on the hardware like it's 1988. I love it.


----------



## kpedersen (Sep 13, 2020)

Jose said:


> This is one of the wonderful things that has happened recently with the rise of Raspberri Pi, Arduino, etc. More and more people are bit-banging on the hardware like it's 1988. I love it.



Its true! After an (almost criminally) boring period between literally the ZX Spectrum and the Arduino *many* years later, there was almost nothing for technical hobbyists. It gave rise to legions of fairly incompetent "IT" people in my generation haha.

It is like the entire industry (and education system) just wanted cattle who could enter data into into a spreadsheet rather than do anything cool or worthwhile.



anonymous9 said:


> Contributing Bhyve port to illumos - path to native VM environment



It looks like the whole Illumos thing is mixing things up. The following source suggests that Pluribus Networks just did the port to Illumos.









						bhyve Hypervisor
					

The bhyve hypervisor



					omniosce.org
				




I did find this out though. It seems that due to the unportable nature of Docker, for it to run on macOS it needed an entire hypervisor (almost like VirtualBox on Windows before WSL could be used). They actually chose to use a Bhyve derivative. https://github.com/moby/hyperkit


----------



## Jose (Sep 13, 2020)

Bhyve "(s)tarted as a skunkworks project at NetApp in late 2010". The "(p)roject died internally" by 2011.


----------



## Deleted member 63822 (Sep 14, 2020)

Telling people is incompetence doesn't make you look cooler. The fact is your nmdm console sucks, just that. If tmux can't be used as console for Bhyve, I have already ditched your Bhyve for a long time.


----------



## Deleted member 63822 (Sep 14, 2020)

Another bad thing about Bhyve is firmware support. It doesn't support legacy BIOS firmware. Yeah, there is uefi-csm, but it's just not work. The only way to install your guest as BIOS with a MSDOS partition table and boot it is using grub2-bhyve. This is the only way you could continue installing legacy BIOS based OSes without going the EFI route. Yes, only grub2-bhyve works. And that port is unmaintained 

Anyone let me know if we could just grab the SeaBIOS firmware from VirtualBox and use it with Bhyve? And if it just works, why people don't do that from the beginning? Sorry but I don't see the point of using EFI for guest OS. Because most EFI firmwares of virtualization software out there sucks, including VirtualBox's own 

p/s: it seemed KVM also uses SeaBIOS, too.


----------



## kpedersen (Sep 14, 2020)

gh_legacy said:


> Telling people is incompetence doesn't make you look cooler. The fact is your nmdm console sucks



It is more to warn other members of the forum so they don't waste their time.

nmdm is pretty much exactly the same as qemu's -serial stdio  or -nographic option. It has to be fairly conservative because the serial is relatively bandwidth constrained. Emulating that correctly is more important to sysadmins than your happiness.

This stuff is for developers or sysadmins. Not really users like you. *Did you try SSH*?



gh_legacy said:


> I have already ditched your Bhyve for a long time.



I imagine I speak for many of us when I ask, can you *please* consider ditching the rest of our FreeBSD too? XD


----------



## ekvz (Sep 14, 2020)

gh_legacy said:


> Another bad thing about Bhyve is firmware support. It doesn't support legacy BIOS firmware. Yeah, there is uefi-csm, but it's just not work. The only way to install your guest as BIOS with a MSDOS partition table and boot it is using grub2-bhyve. This is the only way you could continue installing legacy BIOS based OSes without going the EFI route. Yes, only grub2-bhyve works. And that port is unmaintained
> 
> Anyone let me know if we could just grab the SeaBIOS firmware from VirtualBox and use it with Bhyve? And if it just works, why people don't do that from the beginning? Sorry but I don't see the point of using EFI for guest OS. Because most EFI firmwares of virtualization software out there sucks, including VirtualBox's own
> 
> p/s: it seemed KVM also uses SeaBIOS, too.



Now that's a least something i would call a specific complaint. It get's a little less serious as there seems to be a viable workaround (being unmaintained doesn't matter as long as it works) but in any case it's something that could be considered even if  "sucking" is somewhat subjective and it would be better if you would list technical limitations instead.


----------



## Phishfry (Sep 14, 2020)

What needs to be expressed here is that nmdm provides a dumb terminal console.
I think this user believes that it should be a virtual terminal console.
Tmux and stdio provides a virtual terminal console.
When I boot up with serial redirection enabled on my motherboard BIOS I cannot hit F2  to enter the BIOS remotely.
That is why all serial redirect capable motherboards use the Tab key to enter BIOS. Tab is supported by the dumb console.
Dumb consoles are limited by their nature.


----------



## olli@ (Sep 15, 2020)

“nmdm” is an abbreviation for null-modem. And you can think of it as exactly that: a software representation of a null-modem cable, no more and no less. People who are a little older probably remember that you used these kind of cables to connect serial terminals (like a DEC VT100 or VT220) to serial ports on your PC. It’s exactly the same with the nmdm interface: You have to connect a terminal (in this case, a virtual terminal, i.e. a piece of software) to it to make it useful.

(PS: By the way, if someone could use an old – but working – DEC VT510 and is willing to pick it up in Munich (DE), send me a message.  )


----------



## ekvz (Sep 15, 2020)

olli@ said:


> “nmdm” is an abbreviation for null-modem. And you can think of it as exactly that: a software representation of a null-modem cable, no more and no less. People who are a little older probably remember that you used these kind of cables to connect serial terminals (like a DEC VT100 or VT220) to serial ports on your PC.



Not only that. Those things also enabled one of the earliest forms of online gaming. In the the most literal sense. You just had to grab your tower and take it to your buddy (very convenient!) and the both of you could then _duke_ it out. All of this while sitting like maybe 2m apart because your PCs were physically _on-a-line _and the cable wouldn't reach any further. Null modem cables were very much ahead of their times!


----------



## Jose (Sep 15, 2020)

ekvz said:


> Not only that. Those things also enabled one of the earliest forms of online gaming. In the the most literal sense. You just had to grab your tower and take it to your buddy (very convenient!) and the both of you could then _duke_ it out. All of this while sitting like maybe 2m apart because your PCs were physically _on-a-line _and the cable wouldn't reach any further. Null modem cables were very much ahead of their times!


I learned a lot of what I know about Novell networking so I could play Doom 1 head-to-head against my buddy. A couple of garbage-picked 3com 3C501s; some thinnet, t-connectors, and terminators, and we're off to the races in what would've been called a LAN party in the naughts. I think LAN parties are old school now too.


----------



## a6h (Sep 15, 2020)

Jose said:


> I learned a lot of what I know about Novell networking so I could play Doom 1 head-to-head against my buddy


Since it's a off-topic thread and it was from the beginning (wrong section, but things happens!) let me tell you my story. I was a Duke Nukem 3D fan, and at the time I thought I was a master in it! Then there was a _Duke Nukem 3D_ competition. I was confident that I'm going to win the cup. PCs was connected to each other with Dialup MODEMs. Race started and in my first match I lost 0 to 50. I can't remember the exact score, but mine definitely was zero! At that very moment I realized that there's a very important concept in playing FPS, namely *Mouse*!


----------



## ziomario (Aug 20, 2021)

Phishfry said:


> What the OP needs to ask himself is- What does NVMM do that Bhyve does not?
> That is what I would ask myself if I was a developer.
> 
> We have Bhyve and Xen.
> ...



on FreeBSD there is only one good hypervisor : bhyve. I don't think that xen is good as bhyve. At least I've read that it lacks the pci passthrough. It is a very needed feature. So,I want to exclude it from the list because it is not mature yet.


----------



## eternal_noob (Aug 20, 2021)

ziomario said:


> it lacks the pci passthrough


According to the wiki, XEN allows PCI passthrough.


----------



## ziomario (Aug 20, 2021)

eternal_noob said:


> According to the wiki, XEN allows PCI passthrough.



that's good if it does. Even if I read somewhere that it is not allowed.


----------



## olli@ (Aug 20, 2021)

Personally, my problem with bhyve is that it doesn’t support passing raw drives through, in particular, I want to use my BD-ROM drive inside a Linux guest. Bhyve doesn’t support this (you can pass a BD file system, but you cannot pass the raw drive so Linux programs can send SCSI commands to it). I was under the impression that it’s supposed to work with `virtio-scsi`, but it doesn’t. In theory I could pass the PCI device that holds the controller to which the BD-ROM drive is connected, but I can’t do that because that drive is not the only device on that PCI.
 
It does work fine with VirtualBox, though, so I’m using that one. The only issue with VirtualBox is that its USB support is rather weak, but I don’t need USB in that case, so it’s not a big deal.


----------



## tarkhil (Aug 20, 2021)

Phishfry said:


> What the OP needs to ask himself is- What does NVMM do that Bhyve does not?
> That is what I would ask myself if I was a developer.
> 
> We have Bhyve and Xen.
> Why do we need more options? We have two perfectly functioning tools for Virtualization.


Well, on 12.2 I've got issues running Windows Server on AMD CPU, had to move to Intel box.
And Bhyve does not support (and looks like it's not going to) USB passthrough and CD-ROM  mount/unmount on the fly. Sometimes it's disappointing.


----------



## Phishfry (Aug 20, 2021)

tarkhil said:


> And Bhyve does not support (and looks like it's not going to) USB passthrough


I disagree. I have tested using four NEC USB3 controller cards passed thru to four different VM's.
So that gives me a mouse and keyboard for each VM. I also used some 4 port VLI USB3 cards and that allowed usb mass storage which hotplugs just like a regular machine. There was a 60 sec. delay in the CAM subsystem bringing up the device though.

Now where you will see problems is using the motherboard USB with pass-thru.


----------



## ziomario (Aug 20, 2021)

From here : https://wiki.freebsd.org/Xen​Known Limitations​

Suspend and Resume may not work
The console mouse may not work on the host
The FreeBSD kernel debugger may not work
MSI-X PV interrupts are not supported (FreeBSD)
---> PCI devices with virtual functions work inconsistently (FreeBSD)
---> PCI Passthrough is not supported (Xen/FreeBSD) ---> it talks about FreeBSD as a guest os ?
ARM64 is not supported
Loading vmm.ko will panic the Dom0, as may VirtualBox

eternal_noob


----------



## Phishfry (Aug 20, 2021)

ziomario said:


> Even if I read somewhere that it is not allowed.


Well Xen can use the native backend or libvirtd. I know libvirt you can use passthru.
You must build it in your xml setup.


----------



## Hakaba (Aug 21, 2021)

OP miss an important point about adding new software vs improve existing one.

If you list the lack of bhyve/Xen, you need to list the lack of nvmm and compare both.

About the ARM64 limitation :

`man nvmm` >


> The following backends are supported:
> *-*   x86-SVM, for x86 AMD CPUs
> *-*   x86-VMX, for x86 Intel CPUs



Actually, there is no ARM64 support in nvmm.
I do not find information about USB passthrough but at least the DragonFlyBSD documentation say :


> DHCP & DNS​A DHCP server can be run on the bridge interface to provide guests with auto IP address configuration. Similarly, a DNS service can be provided to guests.
> 
> TODO: dnsmasq
> 
> ...


It smell like adding nnvm is not a piece of cake and maybe it is better to encourage improvement in Bhyve 
I can have a wrong opinion because I am not a FreeBSD core developer, but as I am not an expert, I have to belive that FreeBSD teams do a great job and support them (and thanks them as a FreeBSD user  ).
If nnvm is a better solution than bhyve, I am sure there is some discussion in 'obscure' mailling list about the opportinity to port nnvm into FreeBSD, no ?


----------



## bendany (Nov 26, 2021)

Currently, bhyve still lack of some features, as others point out.

legacy BIOS support for installing old OSs,
improve compatibility (more guests)
emulate more devices.
improve current emulated devices.
more important, a framework to manager VM without extra packages. (vm-bhyve is nice although)
I suggest you could visit reviews.freebsd.org, and use the keyword bhyve to find any interested patches for bhyve, review them, let them merge to FreeBSD base. also fire your own patches/PR. core team and developers are willing review your code and commit them. one day bhyve is more mature and NVMM also more mature, we can port NVMM to FreeBSD at that time.


----------



## ziomario (Nov 26, 2021)

bendany said:


> Currently, bhyve still lack of some features, as others point out.
> 
> legacy BIOS support for installing old OSs,
> improve compatibility (more guests)
> ...



do you know,if actually qemu-NVMM supports the GPU passthrough of the nvidia graphic cards ? I've planned to install DragonFLyBSD to check it out,even if there I don't see a good support as I've found here. Regarding NetBSD,I can't use it at all,since the two graphic cards I have aren't supported.


----------



## bendany (Nov 29, 2021)

ziomario said:


> do you know,if actually qemu-NVMM supports the GPU passthrough of the nvidia graphic cards ? I've planned to install DragonFLyBSD to check it out,even if there I don't see a good support as I've found here. Regarding NetBSD,I can't use it at all,since the two graphic cards I have aren't supported.


Actually, some developers are working on video PCIe passthrough, patches are exists, intel IGD is working, others are on going. license is biggest issue when working with Linux staff.


----------



## covacat (Nov 29, 2021)

olli@ said:


> Personally, my problem with bhyve is that it doesn’t support passing raw drives through, in particular, I want to use my BD-ROM drive inside a Linux guest. Bhyve doesn’t support this (you can pass a BD file system, but you cannot pass the raw drive so Linux programs can send SCSI commands to it). I was under the impression that it’s supposed to work with `virtio-scsi`, but it doesn’t. In theory I could pass the PCI device that holds the controller to which the BD-ROM drive is connected, but I can’t do that because that drive is not the only device on that PCI.
> 
> It does work fine with VirtualBox, though, so I’m using that one. The only issue with VirtualBox is that its USB support is rather weak, but I don’t need USB in that case, so it’s not a big deal.


isn't it possible to achieve that with istgt ?


----------



## ziomario (Nov 29, 2021)

bendany said:


> Actually, some developers are working on video PCIe passthrough, patches are exists, intel IGD is working, others are on going. license is biggest issue when working with Linux staff.



I've found this :








						r/openbsd - PCI Passthrough on VMM
					

16 votes and 10 comments so far on Reddit




					www.reddit.com
				




I've sent a msg to the developer,but he never replied. I'm really interested to try it. If u are also interested,we are two and we can start trying to understand how it works. I'm totally new of open bsd,but I want to  learn and try.


----------



## bendany (Nov 30, 2021)

ziomario said:


> I've found this :
> 
> 
> 
> ...


PCI passthrough is functional on bhyve, LAN adapter, USB adapter, NVMe SSD etc are working fine. while VGA passthrough is more complex, without internal documentation, it is hard to make it working. especially for AMD/NVIDIA cards. The vga rom should be executed to initialize the card. That part involves EFI bootrom and others too. 

here is some link you can have a look at






						⚙ D26209 GVT-d support for bhyve
					






					reviews.freebsd.org
				








						⚙ D27456 bhyve: GPU Passthrough for dedicated AMD GPUs
					






					reviews.freebsd.org
				




you can have a try. 

Personally, I am not familiar with OpenBSD/NetBSD.


----------



## ziomario (Nov 30, 2021)

man,I know almost well bhyve. I use it everyday. I've even cooperated with two developers and we have been able to enable the pass through for the high quality nvidia graphic cards. I'm becoming a nice expert user of bhyve. Anyway,I'm also looking for alternatives. and after having explored what is offered on the net,I found that only open bsd is enough mature to offer another alternative for the passthrough of the graphic card. At least,from what I'm reading on reddit. Yes,I'm not familiar with OpenBSD too,but In the next future probably I will jump there to learn something new.


----------

