# General question about 13.0 as a working station



## kxyd (Oct 13, 2021)

I have been working with both backend development, as well as embedded software development.

I am fairly new to FreeBSD (almost a year of "casual" usage for small personal projects in my personal computer, not at work).

I would really like to try out FreeBSD(obviously starting from testing) as a workstation.

My main issue is· Yes, vscode is up and running with very good support, but what about docker and kubernetes?

As I understand, the docker package is broken and well, there is no direct support for kubernetes.

I was thinking of either trying to run the binaries with __linux64__ loaded, or try out the linuxJails.

I tried LinuxJails via the wiki offered by FreeBSD (here)[https://wiki.freebsd.org/LinuxJails] but I cant find either of those apps in apt.

Anyone has any ideas and/or have the same issue?
If anyone has ever solved this problem, could you give me some guidance?


----------



## SirDice (Oct 13, 2021)

kxyd said:


> but what about docker and kubernetes?


We have jail(8) and bhyve(8), forget about running docker and/or kubernetes on FreeBSD.


----------



## jbo (Oct 13, 2021)

I've worked several years as sysadmin/devops for some fancy smart & everything-is-cloud product. The "cloud" was composed of almost two dozen physical FreeBSD servers. The backend stuff was architected to make heavy use of microservices mostly done in golang, partly in C++.

This worked out extremely well and compared to the previous solution with Linux, Docker & Kubernetes the administration overhead was much less, performance was better (glorious networking stacks) and the systems were overall running much more stable.

Where sensible we used jail(8) as suggested by SirDice. Initially there were some bhyve(8) VMs running Linux for some 3rd-party stuff we couldn't immediately port to FreeBSD.

If you are willing to invest the time to start using FreeBSD for this kind of stuff it will certainly pay off. Sure, at first glance it seems like "FreeBSD is lacking behind compared to Linux" but personally I found that this is mainly the case because Linux has so many different technology and every year there is some new hyped thing.
If you boil it down to what you actually need (both for development and production), you'll find that FreeBSD easily gets you 95% without doing anything "advanced".

As for administration of jails and bhyve VMs: Personally I found sysutils/cbsd to be a great tool. Compared to other solutions it unifies administration of jails and bhyve VMs (it apparently can also do Xen but I never used that).
The developer is very friendly and super responsive on the corresponding project's GitHub page.
This utility also (optionally) provides the necessary infrastructure to migrate jails and VMs across hosts, manage inter-host inventories and so on.
To this date I still use `cbsd` regularly for both our company internal stuff as well as for personal use (even on desktops).


----------



## kxyd (Oct 13, 2021)

jbodenmann said:


> I've worked several years as sysadmin/devops for some fancy smart & everything-is-cloud product. The "cloud" was composed of almost two dozen physical FreeBSD servers. The backend stuff was architected to make heavy use of microservices mostly done in golang, partly in C++.
> 
> This worked out extremely well and compared to the previous solution with Linux, Docker & Kubernetes the administration overhead was much less, performance was better (glorious networking stacks) and the systems were overall running much more stable.
> 
> ...


I think I am having a hard time understanding the two concepts, mainly because they are probably identical(please correct me if I am wrong), but is bhyne an orchestration tool such as docker? for example, is it some sort of "similar project" that can bind in clusters etc?

On the personal side (working on a personal project) I could definitely make use of both bhyve and jail, on the "normal job" side, can I control a cluster via bhyve?

Excuse my ignorance, I am really new to this side of software engineering


----------



## SirDice (Oct 13, 2021)

kxyd said:


> but is bhyne an orchestration tool such as docker?




```
DESCRIPTION
     bhyve is a	hypervisor that	runs guest operating systems inside a virtual
     machine.
```
bhyve(8). 



			bhyve - FreeBSD Wiki


----------



## jbo (Oct 13, 2021)

kxyd said:


> Excuse my ignorance, I am really new to this side of software engineering


No ignorance detected. You're new to a particular ecosystem and it can take a while to get a good familiar feeling with the tools.

SirDice as usually was very quick on providing the corresponding information. I don't have anything to say that you wouldn't find yourself by reading the docs but I know how it is to be new to FreeBSD when coming from Linux. Therefore, I'm gonna try to put in the efforts of providing a more "digestible" answer.
The following information is supposed to provide a high-level understanding for somebody new to the FreeBSD "ecosystem". I'd urge any experienced reader not to start spamming the thread with debates about details like a level 1 vs. a level 2 hypervisor and so on. We're here to help out a new guy.

*bhyve* is a hypervisor. You can compare it to something like VirtualBox, Hyper-V, Xen and VMware. It basically allows you to run virtual machines on top of FreeBSD. For example, I use it extensively to spawn Linux & windows guests on a FreeBSD machine. On servers for things like Ci/CD build agents, on desktops because due to the nature of my job it's hard to stay 100% clear of Windows.
One of the nice things about bhyve is that it's built right into FreeBSD. Similar to how Hyper-V is built into Windows. It's a well integrated tool that ships with the base system and therefore you don't need to rely on 3rd-party solutions.

*jails* are not VMs. I won't go into the technical details as mentioned above but they are more like a chroot environment. They get a read-only copy of the underlying FreeBSD base operating system. Processes are isolated form the host system and other jails. They are very lightweight tools for isolating services.
In general they are FreeBSD only tho. So if you need some service isolated from the main host/server you can put them in a jail and easily start, stop, restart, backup, snapshot, migrate and so on like you would with a VM.
Whenever possible, I try to use jails instead of VMs as they are a lot more lightweight and they provide some nice features which can be important later on.

*cbsd* is a management tool. It allows you to easily create jails or VMs. It allows you to easily configure them, update them, move them to different hosts and so on.
This would be the "orchestration tool".
Whether or not CBSD provides the features you need/want is something we can't decide for you. Personally I found it to be more than sufficient for even larger production environments where jails and VMs needed to be migrated across networks.

Please don't hesitate to ask if you have any other questions or if you need more detail/clarity on these subjects. We will gladly help wherever we can. It often just takes a couple of exchanges to figure out which level to built upon.


----------



## hardworkingnewbie (Oct 14, 2021)

kxyd said:


> As I understand, the docker package is broken and well, there is no direct support for kubernetes.


In general I would avoid Docker at all and not touch it with a ten foot pole. Not because the technology itself is bad, but the mindset many have connected to it is.

Yes, Docker is quite nifty and convenient to use. But it also means people are giving away control on their machines, and most don't care what's being done internally in the Docker images. Even more harmful, most people who use Docker simply just don't care about that at all, because the reason why they choose to use Docker images in first place is that they don't want to have the hassle of configuring a system by themselves, also probably don't have the skills to do so. Such people though should better never run services connected to the internet in my opinion.

I for myself prefer to know what's going on on my machines. This is why I do install stuff by myself, and don't outsource that task to people I don't know - aka Docker image creators, and even further rely on them to have a proper update policy in case of problems.

And of course then, no surprise - many Docker images have severe critical vulnerabilities.






						Analysis of 4 Million Docker Images Shows Half Have Critical Vulnerabilities | SecurityWeek.Com
					

Analysis of 4 million Docker container images found that over half have critical vulnerabilities and thousands of them include malicious or potentially harmful elements.




					www.securityweek.com


----------

