# Docker is dead



## drhowarddrfine (Mar 10, 2019)

For all those who wish Docker ran on FreeBSD and praise Linux for having such a thing--and the reasons I don't think this is off topic, here is another in a long list of reasons to not even use Linux at all--and we see such things too often as in Windows, too.

Goodbye Docker and Thanks for All the Fish

Docker has adopted Kubernetes. RedHat has dumped Docker altogether.

Yeah, I'm in a Linux bashing mood. I was reading some posts--on reddit (where else?)--where four people spent three paragraphs each praising BSD then stated "But I would never use it".


----------



## Remington (Mar 10, 2019)

I run Linux in Bhyve and no need for Docker.  Beside Docker was never stable in FreeBSD and it has numerous open issues not resolved so it looks like Docker project for FreeBSD is dead.  It's hasn't been updated for 2 years and it should be removed from ports.


----------



## abishai (Mar 10, 2019)

So, we should ask for Kubernetes@FreeBSD now ?


----------



## Remington (Mar 10, 2019)

abishai said:


> So, we should ask for Kubernetes@FreeBSD now ?



There is kubernetes-bootstrap and it's dead too.  It never made it to FreeBSD ports.


----------



## Remington (Mar 10, 2019)

The only solution to get Docker working is to install Linux in Bhyve and install Docker inside Linux.  Docker isn't going to work natively in FreeBSD because Linux is using systemd and Docker depends heavily on Linux Kernel.


----------



## obsigna (Mar 10, 2019)

Why not simply:

`# ln -s /usr/sbin/jail docker`

and/or

`# ln -s /usr/sbin/jail kubernetes`

?

Sorry, but bringing docker/kubernetes to FreeBSD is like bringing coals to Newcastle -- hmm, well nowadays, we may want to count with any such nonsense.


----------



## justinnoor (Mar 10, 2019)

Unfortunately, like it or not, understanding container workflows gives you an advantage in finding employment. So learn Docker but keep it on Linux!


----------



## Crivens (Mar 10, 2019)

What is the schwarzschild radius of  kubernetes? Compared to node.js? Inquiring minds want to know.


----------



## drhowarddrfine (Mar 10, 2019)

justinnoor said:


> knowing Docker gives you an advantage in terms of employment.


But Docker is dead. Or at least it's Kubernetes now.


----------



## forquare (Mar 10, 2019)

drhowarddrfine said:


> But Docker is dead. Or at least it's Kubernetes now.



I never really understood what the difference was?
Someone at work tried to explain it to me, but it was all a bit of a mishmash of “The backend of this is a frontend to that” and didn’t see to make much sense.


----------



## drhowarddrfine (Mar 11, 2019)

forquare said:


> it was all a bit of a mishmash of “The backend of this is a frontend to that” and didn’t see to make much sense.


This is a perfect explanation of the whole Linux ecosystem.

Yeah, I'm just piling on.


----------



## Crivens (Mar 11, 2019)

forquare said:


> Someone at work tried to explain it to me, but it was all a bit of a mishmash of “The backend of this is a frontend to that” and didn’t see to make much sense.


Someone then puts a moebius strip on the table and says "Explain that on paper, please."


----------



## justinnoor (Apr 5, 2019)

forquare said:


> I never really understood what the difference was.



Docker is the container, Kubernetes is the container orchestration tool. Docker has its own orchestrator known as Swarm, but Kubernetes won the popularity contest. Restated, Kubernetes orchestrates and manages containers in massive clusters, including Docker containers. Docker is just one implementation of the OCI specifications.

[EDIT]

Docker uses features of the Linux kernel to isolate resources, resulting in an ultra-lightweight form of virtualization. It is Linux-centric, and should not be part of FreeBSD.

Kubernetes was created by Google. Like Docker, it is written in Go.


----------



## kpedersen (Apr 5, 2019)

I sometimes think that Docker is only really used in the web world as an attempt to package software projects which drag in a completely unmaintainable number of dependencies. Such examples include a library just to split a string in Javascript*.

So rather than package what should i.e be a single folder, they end up packaging the entire OS installation XD

My prediction is that no single docker image made today will still be usable in as little as 5 years. Whereas didn't someone manage to get an ancient FreeBSD 4 Jail working in a FreeBSD 10 install?

* Actually, first they would include a library to help them include a library just to split a string in Javascript!


----------



## justinnoor (Apr 5, 2019)

kpedersen said:


> I sometimes think that Docker is only really used in the web world as an attempt to package software projects which drag in a completely unmaintainable number of dependencies.



Understandable. Like anything, Docker has its pros and cons. By far, one of its greatest strengths lies in development. Docker prevents you, and your colleagues, from installing a bunch of cruft on your local machines when building prototypes. Shoot your container over to Docker Hub, and anyone can test it in a matter of seconds.

Here’s a good demo from _sysadmincasts _of containers in action with a real project:

https://sysadmincasts.com/episodes/53-extracting-image-metadata-to-json-using-imagemagick


----------



## rootbert (Apr 5, 2019)

I think the idea behind docker is great. The implementation sucks as hell. After we went through various upgrades that left us in an non operating state we dumped docker for rkt... its a much better alternative, I would love to have freebsd implement containers.

And before any "we have jails, they are better" comments come: I know and I love jails, but container simply target a different requirement. Jails just do not always fit, sometimes containers would be awesome.


----------



## kpedersen (Apr 5, 2019)

justinnoor said:


> Docker prevents you, and your colleagues, from installing a bunch of cruft on your local machines when building prototypes.



Well thats kind of the problem. No software should require:

Python 2, Python 3, Ruby, Node.js, Java, 100's of small dependencies for each

And yet installing something like GitLab or Emscripten does.

This kind of cruft just smells of sloppy software development to me.
Yes, not everything can be written in C due to time / skill limitations but I wish some projects would simply choose one single language rather than fscking around mashing a load of crap together!


----------



## justinnoor (Apr 5, 2019)

kpedersen said:


> No software should require: Python 2, Python 3, Ruby, Node.js, Java, 100's of small dependencies for each.



Good point, but even a pure NodeJS application might require several NPM packages, which, many people do not want to install directly on their local machines (_especially personal laptops_). Installing those dependencies on every host is not repeatable across teams, breaks machines, and raises security concerns.


----------



## drhowarddrfine (Apr 6, 2019)

justinnoor said:


> even a pure NodeJS application might require several NPM packages


Which is another can of worms. 


kpedersen said:


> Actually, first they would include a library to help them include a library just to split a string in Javascript!


Oooh! There's gotta be a button on this thing for that thing! --Homer Simpson


----------



## rigoletto@ (Apr 6, 2019)

But hey, there is more:






						Raise some horns: Red Hat's MetalKube aims to make Kubernetes on bare machines simple
					

Open-source project borrows bits of OpenStack




					www.theregister.co.uk


----------



## badbrain (Apr 6, 2019)

Many friends of mine working on the outsource business like Docker very much. It's really convenient for them.

On FreeBSD is anyone ever think about a Jailhub? We have ZFS. You prepared a ready to use jail, for example FreeBSD-FAMP then zfs send | xz ... to have FreeBSD-FAMP.xz and upload it to a central server. Very convenient indeed. Something like this for Bhyvehub would be fine.

On FreeBSD now, you only have Vagrant and VirtualBox prebuilt images.


----------



## hukadan (Apr 6, 2019)

badbrain said:


> On FreeBSD is anyone ever think about a Jailhub?


It seems that the iocage(8) plugins provide something similar. I have not tried yet. You can find more information here : https://iocage.readthedocs.io/en/latest/plugins.html


----------



## badbrain (Apr 6, 2019)

hukadan said:


> It seems that the iocage(8) plugins provide something similar. I have not tried yet. You can find more information here : https://iocage.readthedocs.io/en/latest/plugins.html


Great. So we already have a Jailhub. What about Bhyvehub?


----------



## hukadan (Apr 6, 2019)

If you look at iocage(8) plugins, it is just shell scripts (here, the gitlab plugin script). So you can easily adapt them and use them with bhyve(8).


----------



## badbrain (Apr 6, 2019)

hukadan said:


> If you look at iocage(8) plugins, it is just shell scripts (here, the gitlab plugin script). So you can easily adapt them and use them with bhyve(8).


No no I mean a already established central hub like dockerhub that we could download prebuilt images and use immediately. We already have something like that with iocage plugins. I ask if something like this but with Bhyve already exist or not. I don't want to setup everything my own. I'm lazy man


----------



## rigoletto@ (Apr 6, 2019)

badbrain said:


> Great. So we already have a Jailhub. What about Bhyvehub?



I would rather prefer something properly developed like unikernels instead. MirageOS is indeed a good example. And, for the record, MirageOS work on bhyve.


----------



## justinnoor (Apr 6, 2019)

rigoletto@ said:


> MirageOS is indeed a good example.



This is really interesting, and it’s written in OCaml, which I know nothing about hah.


----------



## rigoletto@ (Apr 6, 2019)

They provide some simple skeletons you do not really need to know OCaml to use. Also, if you do some search you will find some others people constructed and made available. I saw some time ago some people willing to construct one for Ocsigen.

[EDIT]

The Muen is a particular interesting case. The site run on a MirageOS unikernel virtualized on top of Muen.


----------



## badbrain (Apr 6, 2019)

rigoletto@ said:


> I would rather prefer something properly developed like unikernels instead. MirageOS is indeed a good example. And, for the record, MirageOS work on bhyve.


That is yours. Not the problem I see. I said about traditional OSes. Prebuilt images of something like CentOS or Ubuntu already made by someone I could fetch and modify to suite my needs.


----------



## justinnoor (Apr 7, 2019)

badbrain said:


> Prebuilt images of something like CentOS or Ubuntu already made by someone I could fetch and modify to suite my needs.



There is no such thing as BhyveHub.


----------



## badbrain (Apr 7, 2019)

justinnoor said:


> There is no such thing as BhyveHub.


Straightforward. I like it. If we go with this answer sooner no useless off-topic discussion needed. Back to the topic about docker.


----------



## justinnoor (Apr 7, 2019)

badbrain said:


> Back to the topic about docker.



Virtualization is related to Docker. The comments about unikernels are relevant to the discussion because they are probably a better way forward.


----------



## rigoletto@ (Apr 7, 2019)

justinnoor said:


> Virtualization is related to Docker. The comments about unikernels are relevant to the discussion because they are probably a better way forward.



Unikernels are a step further indeed.


----------



## kpedersen (Apr 7, 2019)

badbrain said:


> Straightforward. I like it. If we go with this answer sooner no useless off-topic discussion needed. Back to the topic about docker.



Just because a company has not yet added an almost consumer-centric web page to a technology, does in no way mean it is inferior.

Something like DockerHub is great for quick thrills and if someone is unable to install complex software themselves but in real world scenarios, most of the images are not usable; they are not tailored to the job at hand. I would still need to set one up bespoke for any internal developed software.

Do you think people just use Git because of Microsoft GitHub? Why do people still also use CVS, SVN, Hg, etc.. when there are no corresponding *Hub sites for them?

Don't get me wrong, containers are good and Docker is fine. But I find the off-the-shelf container "market place" design a little naive especially when it comes to future maintenance.
No-one is going to maintain a i.e glibc 2.11 container just so I can run some crusty old application. I will either have to update my stuff or maintain the container myself. So I might as well start now and ignore the DockerHub.

Docker is one of those things that is great until it all stops working in 10 years. Then we will need to dig up all the old shite and replace it. Yeah, we will get paid, so I can't complain I suppose.


----------



## badbrain (Apr 7, 2019)

kpedersen said:


> Just because a company has not yet added an almost consumer-centric web page to a technology, does in no way mean it is inferior.
> 
> Something like DockerHub is great for quick thrills and if someone is unable to install complex software themselves but in real world scenarios, most of the images are not usable; they are not tailored to the job at hand. I would still need to set one up bespoke for any internal developed software.
> 
> ...


Such thing like Dockerhub is so helpful for outsourcer. Specifically for devs on our country. They are pretty much just about money, deploy fast and gone, despite the code is spaghetti and shitetty because another random guy of another low cost outsource company will take that shite. Somewhat irresponsible but true. Even their cash is too low compared to the western. And the company keeps most of it.


----------



## rigoletto@ (Apr 7, 2019)

badbrain one thing you didn't got is FreeBSD does not support crap solutions, we prefer to be slow and do it technically right than offer crap and that is, as expected, the way our audience like the things done. Also, dominate the market like Linux is not one of our goals, inclusive because that would involve making enormous amount of compromises to fit "everyone" taste and necessities, which inevitably translate to low quality.

What we are failing is to make that clear marketing-wise, but this is something the we are working on while we talk, in several different ways.


----------



## justinnoor (Apr 8, 2019)

rigoletto@ said:


> Unikernels are a step further indeed.



Did you see IncludeOS?


----------



## rigoletto@ (Apr 8, 2019)

justinnoor said:


> Did you see IncludeOS?



I wasn't aware of that, probably because I "hate" C++. 
I found this unikernel resource.

Thank you for pointing out.


----------



## reddy (Sep 25, 2019)

What I always wonder is why people do not just write Makefile-like scripts to allow developers to setup a clean environment / easily install an application and all its dependencies and configuration in one click. This would anyway be needed to have repeatable builds.

Regarding the argument: to avoid bloating personal computers, Linux has had LXC containers for quite a while now. So I think the real issue is that people are comfortable with crap engineering practices. They're comfortable loading a full OS in RAM while a shell script could have made it. People prefer to learn ever-changing layers of abstraction such as docker "just" so that they do not need to learn system administration which is supposedly too hard.

I even think that in most cases, a configuration management solution is a better solution than Kubernetes. If you need "cloud-grade scalability", all cloud providers sell transparent scaling as a selling point. If the goal is to easily move from on-premises to the cloud as you scale, a Makefile along with a configuration management solution gives you this result. If the idea is to move your 10TB database to the cloud, you'll need to ship your hard drives anyway unless you can wait 6 months for the upload to complete.

I kind of understand why people choose to go the lazy way, but I find it very unreasonable to waste so much resources. But the good thing is that these people keep cloud providers in business so that those of us investing in sound engineering (which is not such a huge investment, only requires goodwill and consciousness) can run multillion-dollar businesses having hundreds of sandboxed applications running on a single box for 200 dollars a month while others make cloud providers rich.

I think if you are running FreeBSD, considering the power of jails and Linux emulation, you really do not need Kubernetes or Docker. Just write setup and deployment scripts. If people put the effort they invested in Docker into learning FreeBSD (which has been providing a great environment for these conveniences for a very long-time, jails, linux compatibility layer, the port system, zfs etc...), the world would probably be a better place. I am not even a FreeBSD wizard but the system is easy to use, stable, well designed and just works.


----------



## drhowarddrfine (Sep 30, 2019)

Not to repeat the title of this thread but ...



> Docker, a one-time highflier in business software that reached a $1 billion valuation in 2015, is struggling mightily these days as it tries to raise some much-needed capital.


----------



## toorski (Oct 1, 2019)

Here we go again. 100 people with used laptops, open source idea, leased office - valuation 1 billion 
Who's doing all the valutions of those "We got the killer app or solution," and  where do all the billions, in IPOs and adverts, are coming from?
Bill Gates invests in his own fundations, Paul Allen blew his money to look for radio signals from outer space, Elon Musk and Jeff Bezos are blowing their funny money on rockets, Google is giving money away to YouTube stars, Warren Buffet doesn't play High Tech ... 
I think the money changers are creating money out of thin air and are giving it away to anyone with a good or bad idea and a laptop - LOL


----------



## joneum@ (Oct 1, 2019)

We're still working on a dead project: https://reviews.freebsd.org/D21570
PS: Helping hands are welcome ;-)


----------



## Crivens (Oct 1, 2019)

toorski said:


> Here we go again. 100 people with used laptops, open source idea, leased office - valuation 1 billion
> Who's doing all the valutions of those "We got the killer app or solution," and  where do all the billions, in IPOs and adverts, are coming from?


For example, it comes from hype companies like facebook. These are highly depending on their share price. So what would happen if Zuckerberg was selling half his shares? The rest would crash down. But if he buys a startup with it that was funded by "a guy whose 3.rd cousin he met in a pub"? Vision! Progress!! In the end he moved a lot of his wealth from fantasy land (shares) into the real world.


----------



## kpedersen (Oct 1, 2019)

drhowarddrfine said:


> Not to repeat the title of this thread but ...



This is great. Just like Adobe Flash, Unity, DotNet and a lot of other highly marketed products, it is useful when things go under; it should be a learning experience for developers to check the dependency and maintenance trail before investing their livelihood on the sole bucket of convenience.

But the reality is that they don't learn; these guys will instead become middle management and then a new batch of naive developers come up and jump on the naxt big flaky bandwagon. Together they continue making terrible technical decisions and software. They then fade away leaving the industry with no real progress.


----------



## somedude (Jan 28, 2020)

wow. this is some negative circle jerk ya'll have going on here.  Are there any mature FreeBSD leaders in this thread?  
I think it's fine if BSD does not want to be docker compatible, there is lots of room in technology for different solutions.  Mainframes even still have their place after all.  However, wishing death upon another technology simply because it doesn't work well on your platform out of the box is rather naive.  
With regards to those saying "bsd does things technically correct and the rest is crapware"....well, that's an interesting perspective given how many "technical facts" about docker in this thread are dead wrong.  For instance, Docker does not load an entire OS into memory.  Also, docker is keenly aware of LXC - it utilizes it!  One could almost say that Docker simply created a friendly user interface for LXC.  And that is just one example, I am actually hard pressed to find a technically accurate statement about docker in this thread.

From a non-technical perspective, there also seems to be big gap in understanding.  I see some folks suggesting Make files - that's literally the anti-thisis to docker!  Make files require an understanding of the platform the software runs one so as it can ensure it has the right dependencies, and with some luck, those dependencies won't conflict with other software on the same system.  By in large, docker solves both of these very real problems and so many more.

This is my first foray into the FreeBSD community and if this thread is indicative of overall culture, yikes!  As a professional sys admin, I am inclined to steer away quickly. Weather Docker succeeds or fails in BSD, the extremely uninformed hyperbolic statements made by it's members - none of it is professional in tone, purpose, or accuracy - are definitely not for me and my peers.  

Good luck to ya'll !


----------



## kpedersen (Jan 28, 2020)

somedude said:


> statements made by it's members - none of it is professional in tone, purpose, or accuracy - are definitely not for me and my peers.



If you judge the merit of a technology based on members of a forum then no... FreeBSD is definitely not for yourself or your peers.

But we are all very thankful that you took the time out of your day to sign up to these forums in order to revive a 2+ month old thread. We are all much more knowledgeable now .

Honestly though, I am surprised Docker is still even a thing in 2020.


----------



## drhowarddrfine (Jan 28, 2020)

somedude said:


> I think it's fine if BSD does not want to be docker compatible


It's not up to FreeBSD to do this. Docker is a Linux thing only. Some try to shoehorn it into FreeBSD but that's not what this thread is about. The "Docker is Dead" title comes from two articles from two non-FreeBSD sources but here you are pointing the finger at FreeBSD.


somedude said:


> This is my first foray into the FreeBSD community and if this thread is indicative of overall culture, yikes!


That's nothing. You should see any one thread in any one technical Linux forum. This is the Off Topic board here.


----------



## mark_j (Jan 28, 2020)

somedude said:


> wow. this is some negative circle jerk ya'll have going on here.  Are there any mature FreeBSD leaders in this thread?
> I think it's fine if BSD does not want to be docker compatible, there is lots of room in technology for different solutions.  Mainframes even still have their place after all.  However, wishing death upon another technology simply because it doesn't work well on your platform out of the box is rather naive.
> With regards to those saying "bsd does things technically correct and the rest is crapware"....well, that's an interesting perspective given how many "technical facts" about docker in this thread are dead wrong.  For instance, Docker does not load an entire OS into memory.  Also, docker is keenly aware of LXC - it utilizes it!  One could almost say that Docker simply created a friendly user interface for LXC.  And that is just one example, I am actually hard pressed to find a technically accurate statement about docker in this thread.
> 
> ...


As a professional sysadmin/programmer, i've never used docker, but then in my professional capacity i dont use linux.

Docker is a linux-ism and I do wonder about the logic of attempting to 'make it run' on FreeBSD. It seems like a huge waste of time; a bit like trying to get xfs to run natively on FreeBSD. Oh well, i guess how a porter or developer spends their time is up to them but boy what a waste of time attempting to port docker or kubernetes.


----------



## rootbert (Jan 28, 2020)

I would love to see a docker version on FreeBSD, specifically a jailed version - ie a docker/OCI container or a group of them in a jail with a nice CLI management tool on the host ... thats what I really dream about, and probably this would make FreeBSD a viable option for a range of things again. For all my largescale projects FreeBSD is of no interest due to lack of Docker. Providing a cloud system with 100 thousands of docker containers is so incredibly easy, all test driven from a nice CI/CD pipeline.

Having just a few lines of config/docker compose lines producing reproducable, testable containers is one thing I don't ever want to miss again, so imho developing this for FreeBSD would save loads of developer resources if you try to implement this with jails etc. - I think in the end it would pay off.


----------



## Crivens (Jan 28, 2020)

Obligatory xkcd


----------



## mark_j (Jan 28, 2020)

rootbert said:


> [...]
> For all my largescale projects FreeBSD is of no interest due to lack of Docker. Providing a cloud system with 100 thousands of docker containers is so incredibly easy, all test driven from a nice CI/CD pipeline.



So stick to Linux. No one's forcing you to use FreeBSD and there's no sense in FreeBSD copying everything Linux does; then it's Linux!

I just don't get this "let's make FreeBSD have everything Linux has". Why not everything Windows 10 has? MacOS? It's nonsensical.


----------



## drhowarddrfine (Jan 29, 2020)

rootbert said:


> I would love to see a docker version on FreeBSD, specifically a jailed version


Talk about piling on. Sounds like web development. Needing a tool that makes things easier for a tool that made things easier that wrapped around another tool that made things easier...ad nauseum.



rootbert said:


> For all my largescale projects FreeBSD is of no interest due to lack of Docker.


That's why I never use Linux. jails don't work on Linux. If they ever get jails to work there, I might have a use for Linux but, until then, I have no interest in it.
Same thing for the new Windows Edge browser. What use if Linux if they can't get that to work there?


----------



## drhowarddrfine (Jan 29, 2020)

mark_j said:


> I just don't get this "let's make FreeBSD have everything Linux has". Why not everything Windows 10 has? MacOS? It's nonsensical.


That's what Linux already does! Linux is a Windows wannabe. Ubuntu's bug list ranks Windows as the #1 bug so they have to copy everything Microsoft does in order to replace it. Just look at all the people who are now thrilled as .NET and other Microsoft tools get integrated into or run on Linux. They can't wait till there's a Microsoft branded Linux!

Many people just want to run things because they read a headline somewhere on reddit and insist it's now gospel. So they run out, use it themselves, with no understanding of why.


----------



## rootbert (Jan 29, 2020)

yeah, obviously I have to stick to Linux on that projects, unfortunately. But one may dream of containers on FreeBSD, the concept is great ... it's the "new statically linked" contained form of applications. IMHO its the same type of great technology for me as virtualization. Probably I desperately want containers on FreeBSD because what Docker Inc did with containers was a classic showcase of how things go wrong in linux land. The API breaks were insane, and they simply wrote another filesystem because their first attempt did not work as expected.

Anyhow, docker is dead, but containers are more alive than ever, and will probably be in the future, and that tech is probably more alive than Unix.


----------



## kpedersen (Jan 29, 2020)

rootbert said:


> Anyhow, docker is dead, but containers are more alive than ever, and will probably be in the future, and that tech is probably more alive than Unix.



Agreed. On FreeBSD, I always distribute server software in jail(8) containers. How else would the cancerous spread of Python applications and their dependencies be prevented?


----------



## drhowarddrfine (Jan 29, 2020)

rootbert said:


> But one may dream of containers on FreeBSD


As we've been saying, use jails. jails are our containers.


----------



## unitrunker (Jan 30, 2020)

rootbert said:
			
		

> Having just a few lines of config/docker compose lines producing reproducable, testable containers is one thing I don't ever want to miss again, so imho developing this for FreeBSD would save loads of developer resources if you try to implement this with jails etc. - I think in the end it would pay off.


Poudriere uses jails to provide *reproducible* builds. Install a particular tool chain into a jail. Build a port in that jail. Get the same binary output.

I see people rave about vagrant and docker community sourced scripts. Sounds like a big productivity boost. The problem is they take these scripts at face value and don't do any analysis to see what side effects - including security implications. When you create your own jail.conf file - you control exactly what your jails can and cannot do. 

Not only can a jail provide a reproducible environment - when combined with ZFS you can take snapshots and easily roll back any change to a jail. Upgrade a library in a jail and don't like what you see? ZFS can take you back to where you were.


----------



## jgod (Feb 4, 2020)

What developers (myself included) desire is really not to run docker on freebsd, but to have something sitting above jails and libs with similar user interface semantics (making jails an implementation detail). "Docker registries" with "images" can be implemented with poudriere or whatever. Out of iocage, cbsd, etc. the closest to this is the new Bastille, afaik.


----------



## drhowarddrfine (Feb 4, 2020)

jgod said:


> What developers (myself included) desire is really not to run docker on freebsd, but to have something sitting above jails



So a wrapper around a wrapper. Or a tool for a tool?

This sounds like web development nowadays. No one just writes HTML, CSS and JavaScript. You have a tool to make that but, then, someone comes along with a tool to make the first tool better but then someone else needs a tool for that tool and, pretty soon, you have what's going on now--layer upon layer of tools that are all supposed to make life easier but they don't.

As I said, if one is going to use FreeBSD, use jails and be done with it.


----------



## 20-100-2fe (Feb 4, 2020)

drhowarddrfine said:


> So a wrapper around a wrapper. Or a tool for a tool?



When a tool is great, that is rich and flexible, its learning curve is necessarily long. And what human beings need when it comes to learning (in any field, not only IT) is a fast and easy way to make ones first steps.

When you work in a given context, you always do more or less the same things. If you have a "tool above a tool" that makes your first steps easier with the underlying one, you can make sense out of what you learn because it pertains to what you're working on. Then, you can easily and gradually learn more because you don't start from scratch and your knowledge increments keep being meaningful to you.

If you later work in a different context, the "tool above the tool" might still be useful, not for learning but as a kick starter if both contexts differ a lot.

Such tools are important in terms of risk management because tools embed corporate knowledge and best practices. If an experienced developer leaves your team for any reason, you'll be happy to have such a tool so the new joiner will not have to go through the full length of his former colleague's learning curve, with all the unpleasant implications this inevitably has for the project.

Tools have a cost, not only an acquisition cost, but also an operating cost, mainly due to the complexity added to the work environment. This cost should be estimated and compared with the costs likely incurred otherwise.


----------



## kpedersen (Feb 4, 2020)

I can see Docker being useful for those who either cannot be "arsed" or do not have enough time to do things properly. And that is fine, sometimes I like to fiddle about or am rushed off my feet with tasks.

However, those who cannot be bothered, or do not have enough time are probably also not going to be able to make a quality solution work on FreeBSD either. There are quite a few subtle differences between operating systems and container solutions that Docker will not be able to abstract. They will stick to Linux.

So Docker, as an abstraction layer between platforms is only really useful as a layer above the mess that is Linux distros (including WSL). With FreeBSD we have *one* OS, and so it isn't really needed.

... Jails.


----------



## drhowarddrfine (Feb 4, 2020)

20-100-2fe People don't do that. Once they start using that abstract, they won't learn the details. And it's the job of any programmer to know the details. If that person leaves the job, and you no longer have the knowledge they had of that tool, then you need someone to fill that void. Otherwise, you left adrift and dependent on outside sources who now, in a way, control what you do. And your professional system now becomes Windows.


----------



## Crivens (Feb 4, 2020)

Some info on topic: https://changelog.com/posts/monoliths-are-the-future


----------



## kpedersen (Feb 4, 2020)

Crivens said:


> Some info on topic: https://changelog.com/posts/monoliths-are-the-future



That article really nailed it with the term "distributed monolith". I believe so many projects have fallen into that trap. Just look at desktop environments, X11 and most web projects.

Yes, it is all split up into many many parts but they are effectively only ever used together. They are fairly useless when isolated. They are monolithic and provide no modular or "microservice" benefits.

It is pretty much what makes CDE still easily buildable after all these years, and Gnome 3 a pain in the butt to build even in this present day!
I really can understand why the OpenBSD guys put in the effort to maintain Xenocara (a recombined fairly vanilla xorg). Perhaps we should point our xorg port towards that rather than upstream. It would greatly simplify the port


----------



## acheron (Feb 4, 2020)

kpedersen said:


> It is pretty much what makes CDE still easily buildable after all these years


ROFL


----------



## 20-100-2fe (Feb 4, 2020)

Crivens said:


> Some info on topic: https://changelog.com/posts/monoliths-are-the-future



I've worked at a company where they used an ESB to slow down their applications with just 100,000+ customers - a really efficient technique to avoid the temptation of gaining market shares.

Now, they're in the process of switching to Docker and REST micro-services, adding infrastructural chaos to poor performance.
A French proverb says 'doing and undoing are both working'...


----------



## Crivens (Feb 4, 2020)

The icing on the cake is _who_ wrote that article.


----------



## sadaszewski (May 19, 2020)

If you would be curious to try out something that takes the best abstractions of Docker (images, volumes, containers) and puts them to good practical use on FreeBSD without the unnecessary fanfare, please take a look at https://github.com/sadaszewski/focker . It can build image recipes similar to Docker and does that using layers so that you can very conveniently work on your build scripts and not suffer from the need to rebuild from scratch every time (the build is picked up from the last successful layer). On top of that there is a docker compose like configuration orchestration that lets you build a couple of images together, create volumes and couple all that in a bunch of jails. All using the FreeBSD primitives only - /etc/jail.conf, ZFS. Fully transparent. Very small footprint, fast operation. Really nice examples I use on my own server - https://github.com/sadaszewski/focker/tree/master/example/gitea and https://github.com/sadaszewski/focker/tree/master/example/gateway . Particularly gateway should be of general interest as it nicely handles setting up Letsencrypt and reverse-proxying. Cheers.


----------



## drhowarddrfine (May 20, 2020)

sadaszewski All that sounds like typical Linux-like work--taking Docker stuff and building into jails which is redundancy and complication. Especially introducing foreign software where one has native and better working software.

Just use the superior jail and be done with it.


----------



## kpedersen (May 20, 2020)

I like it, whilst for my own use-cases I prefer to use the Jails system directly, this can certainly help me to push FreeBSD on a few more servers maintained by Docker die-hards XD.

drhowarddrfine in many ways this is actually a great project. Look at it this way, next time we get some random Linux kid on these forums whining that Docker doesn't exist, we can simply point them towards this project and they can sod off and bother the next OSS community


----------



## drhowarddrfine (May 20, 2020)

kpedersen Not having or needing Docker is a feature, not a bug.


----------



## sadaszewski (May 21, 2020)

drhowarddrfine said:


> kpedersen Not having or needing Docker is a feature, not a bug.


I hated Docker as much as the other guy until I tried. It is useful, FreeBSD begs for it. I would be grateful for some help implementing push/pull of binary images from hub.docker.com in focker. This would make the tool really shine.


----------



## drhowarddrfine (May 22, 2020)

sadaszewski said:


> FreeBSD begs for it.


I know of no reason to use it on FreeBSD. I'm sure someone can come up with some obscure, one-off reason that only works for them, though. Or something like, "I need it to access my Linux work."


----------



## kpedersen (May 22, 2020)

sadaszewski said:


> push/pull of binary images from hub.docker.com in focker. This would make the tool really shine.



It is actually this aspect of Docker that I like the least. Locking a large project down to an "app store" is potentially a very dangerous thing to do. I like to think that my software will outlive hub.docker.com (5 years?) and by relying upon it, it instantly puts a death sentence on my project.

Your work is impressive nontheless, and I am certain that some who crave the Docker workflow can benefit from it.


----------



## drhowarddrfine (May 22, 2020)

pyret said:


> More quackery and buffoonery.


No more than that supplied by Linux people who insist FreeBSD cannot exist without Linux tools in the same way Windows users think Linux and FreeBSD is pointless and useless altogether without .NET


----------



## PMc (May 22, 2020)

drhowarddrfine said:


> sadaszewski All that sounds like typical Linux-like work--taking Docker stuff and building into jails which is redundancy and complication. Especially introducing foreign software where one has native and better working software.
> 
> Just use the superior jail and be done with it.



Please explain.

I never had a glance at that "docker". I now had a short look thru the piece which sadaszewski presents. And, _as far as I seem to understand_, it is a scripted builder for jails, including their applications (e.g ports) and including their config. So to say, a configuration database.

I once had a customer, and they had decided to put all their configurations into a database, and that was their desaster plan. So if they would loose a mirrored compute-center, they would just run the database, and thousands of machines would automatically self-install and go productive. (They were big enough to go thru the effort to write the necessary interfaces.)

So there is certainly a use-case for such an approach. I don't know if we already have tooling that could do such. But if not, this here might be developed into something, and, if done properly, it might rock (but then it should be called rocker  ).


----------



## xtremae (May 22, 2020)

pyret said:


> More quackery and buffoonery.


What gave it away; the fact that the thread is based on a false premise or the OP's admission to bashing linux?


----------



## Jose (May 22, 2020)

Docker is for those who fear and don't understand packaging and wish it would just go away. It's the logical conclusion of the download everything and pipe it into a root shell until things start working approach. Docker automates this shotgun packaging and yields opaque binaries you can download automatically and run on your systems with no knowledge of how they were built or what they contain. What could possibly go wrong?


----------



## kpedersen (May 22, 2020)

pyret said:


> Apple has been very successful with their app store.



Apple has. The ones using it have been less so.


----------



## 20-100-2fe (May 22, 2020)

It is true that some people use it like that (e.g. researchers, as explained by someone on this forum).
However, it does not mean that everyone HAS to use it that way.
And in the companies I've worked for, it was NOT used like that.



Jose said:


> Docker is for those who fear and don't understand packaging and wish it would just go away. It's the logical conclusion of the download everything and pipe it into a root shell until things start working approach. Docker automates this shotgun packaging and yields opaque binaries you can download automatically and run on your systems with no knowledge of how they were built or what they contain. What could possibly go wrong?


----------



## rootbert (May 22, 2020)

Jose said:


> Docker is for those who fear and don't understand packaging and wish it would just go away. It's the logical conclusion of the download everything and pipe it into a root shell until things start working approach. Docker automates this shotgun packaging and yields opaque binaries you can download automatically and run on your systems with no knowledge of how they were built or what they contain. What could possibly go wrong?



Unfortunately we don't have docker in FreeBSD, so from some comments, especially this one, we can conclude that quite a lot of people don't really know what docker is or think that it is equivalent to jails, however, it is not. Let's just not call it docker, let's say container. Docker is just one provider or runtime where you can run open-container-initiative (OCI) based containers. Docker is actually a bad example, rkt (dead) or podman are showing us how it is done right (my opinion). We have jails, so this is an amazing technology where we could base our own container solution. I am afraid I cannot clearly express what advantages containers have, but I will try in a few sentences. 

Consider you have a repository to download your containers. Someone was offending docker.hub, but most commercial providers offer their own hub, so no need to worry when docker hub vanishes. A container is a layered "archive", with eg. a base layer (for instance ubuntu), then the application installed on another layer, then on top your own layer with scripts or other stuff - similar to unionfs, but all in a package. Also, most projects choose a minimal system, so there is no init in the container, no other tools, really just the basic service and the libraries. So running that package guarantees that you are running always the same container. The advantage? - I can run them unprivileged, restrict i/o per second, network, ram, cpu usage. It is testable: we commit our code to gitlab, gitlab-runner builds the container, the containers spins up, tests are being made, if the tests are ok the container is pushed to our docker repository. Clients then do a "docker pull" which is equivalent to "pkg upgrade" and be done. Upgrades are simple because just the delta is synced, the rest of the layers is already on the client machine running the old version. The cool thing about that is we run them on client servers as well as the cloud. Setting it up in this way is much less work than doing equivalent thing with jails. It guarantees that our client - no matter if they run ubuntu, red hat, gentoo, void or any other linux distribution can use the software. For jail? yeah, deliver a 12.1 release jail tarball and get a call from your client because they are still on 10.2. And good luck if you ship a bunch of services in 20 jails and you send your client the jail.conf who then realize that the network we tested it with cannot be used in their environment because it's their prod DMZ - ask the customer what network we can use to send them the right network jail? - no way. Just a short note ... I could write on and on but then I would rather make one of those gazillion blog posts describing the concept of application containers.

It's just sad that we don't have it in FreeBSD because FreeBSD would be perfect for such stuff ... the base technology is there. Why just use FreeBSD for traditional hosting when it would also be a great "cloud" hosting solution. Not everything in Linux land is bad, and not all container runtimes are bad. The market for this is huge, and some more attraction to FreeBSD would not be too bad.


----------



## drhowarddrfine (May 22, 2020)

The articles in the very first post claims that Docker is already dead and everyone is flocking to Kubernetes. But Google can change their mind--or when Microsoft completely absorbs Linux--then one will chase the next Linux or Windows tech that reddit is telling you to use NOW!

More and more I find people here drooling over Linux things instead of what we have ourselves. If Docker never makes it to FreeBSD, what are you going to do? Switch to Linux? I guarantee Kubernetes will never make it here so will that make you switch cause why? 1) It can't be done on FreeBSD at all? or 2) You can do it on FreeBSD but ... but ... it's not Docker!! It's not Kubernetes!!


----------



## Jose (May 22, 2020)

rootbert said:


> Unfortunately we don't have docker in FreeBSD, so from some comments, especially this one, we can conclude that quite a lot of people don't really know what docker is or think that it is equivalent to jails...


Please tell me more about what I know and don't know. This is fascinating!



rootbert said:


> I am afraid I cannot clearly express what advantages containers have, but I will try in a few sentences.


Where have I heard that before?



rootbert said:


> A container is a layered "archive", with eg. a base layer (for instance ubuntu), then the application installed on another layer, then on top your own layer with scripts or other stuff - similar to unionfs, but all in a package.


This is really awesome when the Docker container is running your CI system, and every build creates a new layer. It's ok, though 'cause the genius that set it up took this into account and figured out how to prune layers automatically, right? No! I discovered this monstrosity when the build system failed in the middle of the night.



rootbert said:


> Also, most projects choose a minimal system, so there is no init in the container, no other tools, really just the basic service and the libraries.


This is one of the biggest Docker lies. Fact is, most people discover very quickly that it's very hard to troubleshoot any such lightweight system, and wind up packaging up a whole Linux distro in their "lightweight" container. This was Ubuntu early on, but now lightweight distros like Alpine have gained popularity for this reason.



rootbert said:


> ...we commit our code to gitlab, gitlab-runner builds the container, the containers spins up, tests are being made, if the tests are ok the container is pushed to our docker repository. Clients then do a "docker pull" which is equivalent to "pkg upgrade" and be done.


So basically production deployments are a side effect of commits to your source tree. I ask again, what could possibly go wrong?



rootbert said:


> ...then realize that the network we tested it with cannot be used in their environment because it's their prod DMZ - ask the customer what network we can use to send them the right network jail?


Docker's network  is a feature. Got it.


----------



## rootbert (May 22, 2020)

Jose said:


> This is one of the biggest Docker lies. Fact is, most people discover very quickly that it's very hard to troubleshoot any such lightweight system, and wind up packaging up a whole Linux distro in their "lightweight" container. This was Ubuntu early on, but now lightweight distros like Alpine have gained popularity for this reason.



I have to admit that lots of people use docker the wrong way ... just because you can do lots of (awkward) stuff does not mean you have to do that. alpine is great, and so would a nanobsd based container be.

And yes, unfortunately docker is a valid reason to choose linux for various requirements, and at least with all my clients the environment surrounding container technologies is becoming more and more important. I have no problem with someone porting kubernetes to FreeBSD, if there is a usecase why not. However, implementing a nice slick container technology based on FreeBSD jails is quite simple compared to kubernetes.


----------



## Jose (May 22, 2020)

rootbert said:


> I have to admit that lots of people use docker the wrong way ...


Blame the user...something else I've heard before.


----------



## 20-100-2fe (May 22, 2020)

There's quite a lot of Linux bashing, here and in the mailing lists, way beyond reason!
And this is quite counter-productive: there are certainly bad things in Linux, but also very good ones.
And not just Docker: also package management, for instance.
Throwing the baby out with the bath water is the best way to miss good ideas and improvement opportunities.



pyret said:


> All I read is fear from some (I won't name names) is they think FreeBSD is the be-all, end-all, and egad! that they could take an idea from Linux because it is inferior.


----------



## JAW (May 22, 2020)

rootbert said:


> I have no problem with someone porting kubernetes to FreeBSD, if there is a usecase why not.



We do have sysutils/apache-mesos, if only someone could write a jails isolator for Mesos, that would a piece of the puzzle solved. We may not even need Kubernetes/Docker on FreeBSD if we can come up with a good way to get Mesos to pull an app+dependencies into a jail (could we just use a local pkg repo)?


----------



## drhowarddrfine (May 22, 2020)

Well then I'm convinced. After 16 years of using FreeBSD and running a small company with it, I'm switching to Linux starting Monday cause it's obvious FreeBSD can't innovate or keep up. I've always wondered why FreeBSD users are so anxious to diss FreeBSD and convince everyone that it's a bad choice. If you want to get anywhere, choose Linux (or even Windows for that matter). 

FreeBSD is dead.


----------



## Jose (May 22, 2020)

Wait, is this turning into a Freebsd is dying thread? What is this, Slashdot circa 2003?

And we all know whatever businesses want now is the Future (tm). That's why Rails dominates today.


----------



## Jose (May 22, 2020)

Oh boy, a car analogy. We're really going to fully explore the computer cliche space now.


----------



## xtremae (May 23, 2020)

Jose said:


> Wait, is this turning into a Freebsd is dying thread? What is this, Slashdot circa 2003?



Sadly it is. There are also threads in the forum and mail archives about _that_ factoid where the very first post cites _articles_ claiming that FreeBSD is dead.


----------



## ralphbsz (May 23, 2020)

pyret said:


> If someone is looking at an operating system for business and wants/needs to implement something in that way, FreeBSD loses out because it doesn't have what's needed.


What makes you think that serving business that want/need to implement things that way is FreeBSD target market? That FreeBSD wants to do that? That it would be the right thing for FreeBSB?

By the way, I'm not claiming that I know what the FreeBSD target market is, or that FreeBSD doesn't want to do Docker/Kubernetes/Containers/..., and I definitely don't know what the right thing is. But the mantra "evolve by following what Linux does, or die" does not seem to be the answer.


----------



## shkhln (May 23, 2020)

"How dares anyone disagree with me on the Internet!"


----------



## drhowarddrfine (May 23, 2020)

pyret said:


> Have fun with your dying operating system! Say anything you want because I am not coming back.


I knew it! A troll!


----------



## PMc (May 23, 2020)

pyret said:


> Linux has network namespaces and the company I work for, which is a Fortune 200 company,  uses them for a tier 1 application (and Kubernetes and Docker are also used.)  Even a developer for 9front, a fork of Plan 9, has implemented network namespaces.  People see the benefit, so it is being adopted.  It might be something FreeBSD should consider.  If someone is looking at an operating system for business and wants/needs to implement something in that way, FreeBSD loses out because it doesn't have what's needed.
> 
> Then there are those stubbornly refusing to consider such an idea because they've used FreeBSD for 16 years and can't imagine others have different needs.



Just checked what "network namespaces" are supposed to be. They seem to do exactly what I did with Berkeley 20 years ago - because here it can be done natively. And nowadays it's a lot easier.
For instance: I am sitting here at my machine, I am working on my machine, and I am accessing the webserver on this very machine - via the internet. Or, I am starting my Android gadget, connecting to my WLAN, open a VPN tunnel, again, via the internet, onto my own machine. (The pracical use of these is just testing connectivity, but it shows what can be done.)

Those people who believe they need "network namespaces" just did never bother to understand proper routing. And they will not understand "network namespaces" either; they just know that they need them.
Soon they will notice that nothing works with these either -not because network namespaces are bad, but because of a lack of general understanding- and they will drop the issue and return to Windows.



> Who needs anti-lock brakes, power steering, A/C, collision warning, backup camera, and other safety features in a car?



Those who cannot drive.


----------



## Jose (May 23, 2020)

shkhln said:


> "How dares anyone disagree with me on the Internet!"











						Duty Calls
					






					xkcd.com


----------



## illumulli (May 23, 2020)

Opinion may not be warranted, but I have always disliked Docker. While I see it as ugly and inelegant, it also just seems unable to "flow" properly with any system I have used it on. It is like playing an electric guitar in a symphony, which can sound great, but it feels out of place to me. A truly great system in my opinion should not need to rely on third party containerization. Also, I have not tested or looked, but I imagine the memory usage is grotesque compared to native containerization.


----------



## Hakaba (May 23, 2020)

Before Docker, FreeBSD was not used where I work (and as I am a freelance, I work in some places in the web marketing). The reason ? «Hu... This [specific tool] only work on Linux»
So, OS agnostic is a start condition if a project want mix OSs.

Embrassing Linux tools is not the good solution.
The good solution for me is to help the emergence of a standarized «interface».
A script that make a Docker container on linux, jail or behive on FreeBSD and so on.
Kubernetes can be emulated everywhere. The description of what a user want to do when he use Kubernetes can be standardized.


----------



## kpedersen (May 23, 2020)

pyret said:


> they think FreeBSD is the be-all, end-all, and egad! that they could take an idea from Linux because it is inferior.



They have been saying the same about Windows for years. We didn't listen to them then and we don't need to listen to them now.



pyret said:


> Jails and Solaris zones aren't application containers



Ah, this is why you like Docker. You don't know that Jails and Zones *are* containers. You should try them out. You might realise that you don't need Docker.

Unless you really pine for Linux Containers then... no FreeBSD Containers are not Linux Containers. They are better because we can run the FreeBSD userland in ours .

Besides, I am fairly sure most of Docker use is to avoid dependency issues due to sloppy web development sprawl. Honestly a chroot is enough for that. OS level virtualization just to consolidate a shedload of NPM libraries seems overkill.


----------



## sadaszewski (May 23, 2020)

There seems to be a lot of energy in this topic. I don't know if FreeBSD needs "docker" but with this much polarization something surely is up. I don't think I will have much trouble explaining why containerization is not a bad thing.

CONTAINERIZATION RATIONALE

1. Packaging system was mentioned in one of the previous posts. Supposedly "people using container technologies do not know / like / can't use" (apologies if the quote is not exact) packaging systems. This statement has two problems. For one, it is at least debatable if deployment should only involve creating a package. I mean, deployment is not only about software installation but also about running the necessary setup procedures, pre-creating databases, customizing config files etc. Furthermore, those procedures can depend on metadata going beyond what the dialogs in FreeBSD ports can provide (i.e. not a matter of simple switches). If one gets stubborn about all that belonging simply in a custom package then I guess one would also have to reject systems like Ainsible, Puppet, Chef, etc. After all, if everything should be done by means of packaging why not just keep installing the same (potentially updated) package every day or every hour which would make sure that the system is in the right state? Secondly, as far as I know FreeBSD packages do not have a way to set up jails for themselves? Therefore, at the very least, one would either do it by hand all the time (really bad waste of time if somebody is running services just for his friends and is not a career sysadmin) OR create some kind of a wrapper script to create a jail and install required packages inside. Sounds familiar? This is essentially what container build systems do. Only it consists of running whatever commands you like but if you choose to only install packages then I guess you would have your dream workflow?

2. Containers are not only about setting up a couple of services. They are also about ISOLATION. Both "physical" (chroot, namespaces, separate IP or VNET, etc.) and logical (the way container projects are organized allow to easily look at a cross-section of the entirety of the customizations / diff to a base system image). Layers make a big part of that logical isolation where you can group different customizations into certain themes. FYI focker allows you to use --squeeze parameter when building images or compositions in order not to use layers at all. With that switch you can have a plain image / jail builder with the output not much different from existing solutions. However, the WORKFLOW is much different / better IMHO.

3. The trio of images, containers and volumes is simply a very nice abstraction. Much more natural than "template jails" in BastilleBSD etc. You simply don't need any "templates". You can just base any image on any other image and then create a jail off of any image. Easy. Then if you want persistence you throw volumes into the mix.

4. Volumes are a wonderful approach to isolate the real data that you care about from the rest of the jail. I cannot stress how happy I am that now when I want to move to a new machine I know precisely what I need to copy over. I just grab all my focker volumes + my Git repository of Fockerfile's etc.  Then I don't even need / want to take the old images as I will be at the same time updating to the new version of FreeBSD so it is good to run the build scripts again and let them pull in new versions of software. Perhaps they will also require some gentle tweaks, etc. For me this is a great workflow.

Please give https://github.com/sadaszewski/focker/ a try. Also star it on Github if you please, it will help it gain visibility / recognition.


----------



## rootbert (May 23, 2020)

kpedersen said:


> Ah, this is why you like Docker. You don't know that Jails and Zones *are* containers. You should try them out. You might realise that you don't need Docker.


here is a free tipp for you: since you don't know what application containers are, read up on application and system containers. 



kpedersen said:


> Unless you really pine for Linux Containers then... no FreeBSD Containers are not Linux Containers. They are better because we can run the FreeBSD userland in ours .


... is like saying green is better than blue. ever heard of projects with _different requirements_?



kpedersen said:


> Besides, I am fairly sure most of Docker use is to avoid dependency issues due to sloppy web development sprawl. Honestly a chroot is enough for that. OS level virtualization just to consolidate a shedload of NPM libraries seems overkill.


Yes, thats one nice feature: similar to FreeBSDs compat-* we can run legacy containers and separate dependencies like with jails. But now I am really not sure you understood the difference between chroot and jails, another tipp: read up on jails, https://www.freebsd.org/doc/handbook/jails.html


----------



## kpedersen (May 23, 2020)

rootbert said:


> Yes, thats one nice feature: similar to FreeBSDs compat-* we can run legacy containers and separate dependencies like with jails.



Exactly and we can do it natively with jails. So you agree with me. We don't need additional bloat like Docker.

(There is a reason why Docker's logo is a fat whale 



rootbert said:


> application containers



Jails are containers (hopefully you agree). "Application" containers is a buzzword unique to things like Docker. Docker is not a standard. What Linux has (for a few weeks perhaps) is LXC. *That* is the container technology. Docker is the... set of scripts making it fun and easy to use with a little bit of IPC to control the madness.

What are the cool kids calling it these days? System Container? Infrastructure Container? Whatever. Jails are *the* solution on the FreeBSD platform. Like I said before, give them a try.



rootbert said:


> ever heard of projects with _different requirements_?



Yeah, it rhymes with "bad solution" and "microservice anti-patterns" right? (I am trolling) 

Don't get me wrong, if I was stuck with Windows, macOS or Linux, I probably would look into using Docker because there is no way in hell I would waste my time learning how to do things "natively" on those platforms. It just so happens that I like FreeBSD and it evolves at a stable enough pace that I can keep up with it and do things without much non-standard 3rd party software. From experience this gives so many benefits in terms of future maintenance and security.


----------



## drhowarddrfine (May 23, 2020)

rootbert said:


> ever heard of projects with _different requirements_?


If one has a specific need for a specific program that only runs on one specific operating system then one should choose that operating system. But one can do the same thing on FreeBSD with jail with little to no thought. 

I don't read on Linux about those users desiring jail so much that they want jail there. Why the desperate need for it here. Because Linux told you so?


----------



## kpedersen (May 23, 2020)

> The ZFS on Linux (ZoL) port is healthy and maturing. However, at this point in time it is not recommended to use the zfs Docker storage driver for production use unless you have substantial experience with ZFS on Linux.



https://docs.docker.com/storage/storagedriver/zfs-driver/

I actually didn't realize that Docker would be completely inadequate for ZFS in production.



drhowarddrfine said:


> I don't read on Linux about those users desiring jail so much that they want jail there. Why the desperate need for it here. Because Linux told you so?



Probably because FreeBSD users are actually happy with their OS of choice and don't want to mince it into other operating systems!.
Docker users have moaned about getting it on the Windows platform for some time. Before Microsoft threw them WSL they used to use VirtualBox as a "compatibility layer". Frankly, they can do the same here (if there was enough effort...)


----------



## rootbert (May 23, 2020)

kpedersen said:


> Docker is not a standard. What Linux has (for a few weeks perhaps) is LXC. *That* is the container technology. Docker is the... set of scripts making it fun and easy to use with a little bit of IPC to control the madness.



That is not correct. Linux container technology is actually cgroups and namespaces. LXC (is the Linux equivalent of FreeBSD jails) is the system container solution based on cgroups and namespaces. Docker is the application container solution based on cgroups and namespaces. 
Just as a note: I hate Docker! Docker did so much wrong in so many ways, it is the best example of what is going wrong in Linux land. However, the technology is great, and there are developers out there who realized that Docker is awkward. Noone wants to have a big fat daemon running with root privileges just to do containerization. And luckily, some really cool projects emerged from this: I like podman where you can run those app containers without any daemon or script, on a users perspective it looks like what I call "the new statically linked, sandboxed binary" ... it is just a small, secure environment (compared) to run those OCI-standards based containers.



kpedersen said:


> Don't get me wrong, if I was stuck with Windows, macOS or Linux, I probably would look into using Docker because there is no way in hell I would waste my time learning how to do things "natively" on those platforms. It just so happens that I like FreeBSD and it evolves at a stable enough pace that I can keep up with it and do things without much non-standard 3rd party software. From experience this gives so many benefits in terms of future maintenance and security.


Me neither, I am a big fan of jails, it was the technology which got my attention in the FreeBSD 4.x days and made me switch quite a few systems. Fully agree here, I would not want to tinker with Windows ... no idea about macOS, I have not spent more than maybe 50 hours on macos because it just made me sad how Apple crippled everything, especially in the more recent years. appcontainers are native in Linux, so I have no problem with that. Most of the software we use is available to both Linux and FreeBSD, so for system containers we use jails, for app containers we (have to) stick to Linux. Solaris zones is also a great technology, and I could imagine it would also be a great base for app container technology. We used them quite heavily when Solaris 10 was released, but at some point we realized that Solaris had an uncertain future, and since the best Solaris tech is in FreeBSD we switched ... from a business perspective relying on a handful of opensolaris developers is risky so we fully agreed with the management.

When I tried LXC the first time it was a nice experience, however, we just had to wait for the first major upgrade that broke every damn container, so we went back to jails on many services ;-) Same happened with Docker (at the time where docker was more or less "the standard" in the Linux app container landscape), however, a switch back to system containers was not feasible, it would have been too much work, fixing the docker api breaks first and then looking for an alternative was easier.


----------



## Jose (May 23, 2020)

sadaszewski said:


> 1. Packaging system was mentioned in one of the previous posts. Supposedly "people using container technologies do not know / like / can't use" (apologies if the quote is not exact) packaging systems.


Really? This is beyond lazy. Here's what I actually said: "Docker is for those who fear and don't understand packaging and wish it would just go away."



sadaszewski said:


> ...deployment is not only about software installation but also about running the necessary setup procedures, pre-creating databases, customizing config files etc.


All reasonable packaging systems include the capability of running arbitrary scripts in Turing-complete languages. You're not thinking clearly if you think you need more than this.



sadaszewski said:


> Furthermore, those procedures can depend on metadata going beyond what the dialogs in FreeBSD ports can provide (i.e. not a matter of simple switches).


Example?



sadaszewski said:


> If one gets stubborn about all that belonging simply in a custom package then I guess one would also have to reject systems like Ainsible, Puppet, Chef, etc.


False. I invoke pkg(8) in my Ansible scripts all the time. It's easy and it's fun. Try it! Chef and Puppet are nassty Ruby Hobbitses. Consulting rate goes up if you make me use them.



sadaszewski said:


> Please give https://github.com/sadaszewski/focker/ a try. Also star it on Github if you please, it will help it gain visibility / recognition.


Done! And thanks for all your work on this.


----------



## neel (May 24, 2020)

Here's the truth about Docker: It's a Linux-centric solution for a Linux problem: distro fragmentation. GNU/Linux distros regardless of use case have been incompatible in the most minor ways, making lives hard for application developers. Trying to standardize minor details on Linux has been hard and with little success, while *BSD, Windows, and macOS have avoided these issues for the longest time.

Docker on the server and Snap/Flatpak on the desktop is a way to hide Linux fragmentation, containing it via OS-level virtualization. In this case, a developer only has to worry about a so-called "single platform". It doesn't solve it at all, just hides it.

If everyone using Linux servers instead used FreeBSD (or about anything else), Docker as it exists today wouldn't even exist. Why? FreeBSD/*BSD variants and Windows don't have distro-level fragmentation to the extent GNU/Linux has. However, if Docker is cool developers will use it even if this means apps from GitHub don't work on *BSD.

A particular BSD variant is more or less similar across versions, userland, packaging manager, etc. Derivatives like pfSense or FreeNAS still maintain compatibility with their source for the most part. However, some incompatibility is still tolerated between versions or within "derivatives" like pfSense versus stock FreeBSD, as long as they don't go far enough to become a separate variant.

On Windows, you explicitly have a single source (Microsoft), and on top of that, backwards compatibility is literally a make-it-or-break-it, to the extent that legacy Windows 95/NT 3.1 apps can run unmodified on the latest Windows 10 Insider build.

I don't know much about Solaris/Illumos so I won't comment on that.

If the world ran FreeBSD instead of Linux, Docker would be more of a Ansible rolled in to a Jail manager. If it were a Windows world, I'm not sure, I've only been using Windows more in the past few months thanks to a day job (see disclaimer) and "Windows Containers" are still new when compared to their Unix equivalents.

Disclaimer: I work at Microsoft, not on Windows or Azure (at the time of writing). I still develop at work using the Windows/.NET/Azure stack as opposed to a FOSS stack.


----------



## Jose (May 24, 2020)

neel said:


> On Windows, you explicitly have a single source (Microsoft), and on top of that, backwards compatibility is literally a make-it-or-break-it, to the extent that legacy Windows 95/NT 3.1 apps can run unmodified on the latest Windows 10 Insider build.


It is truly remarkable how binaries compiled 20 years ago will still run unmodified on modern Windows.

On the other hand, the original killer use case for virtualization was that damn legacy app running on the Windows server that had to be rebooted daily. Snapshots or "layers" in Docker parlance are also pretty awesome when that same app tends to crash and corrupt its data every so often.

I still think you should fix the problem rather than working around it, but you don't always have the time, and often don't have permission to do that.


----------



## sadaszewski (May 25, 2020)

Jose said:


> Really? This is beyond lazy. Here's what I actually said: "Docker is for those who fear and don't understand packaging and wish it would just go away."
> 
> 
> All reasonable packaging systems include the capability of running arbitrary scripts in Turing-complete languages. You're not thinking clearly if you think you need more than this.
> ...


Thanks! There is one more clear and I guess overlooked advantage to Focker as compared to Ainsible etc. It has virtually no learning curve. Yes, you just write commands to make your jail look the way you like and that's it. The only two directives you need to master is "run" and "copy". Ainsible's documentation has dozens upon dozens upon dozens of pages. For someone requiring a clean, concise, easy to maintain solution, Focker is clearly the way to go 

As to the example for metadata it could be anything but for me the most important is actually the introspection ability of focker-compose.yml files which lets me for example to automatically set up HTTP(S) reverse-proxying for all the defined jails. Sure you can do that for whatever but then you have to write a sort of focker-compose.yml file anyway so why not just use Focker from the start. That's sort of the point, it is so minimalist that you can only make your life more complicated by using the alternative solutions. Sometimes it might be necessary but not in my and I would guess not in 99% of the people's use cases. And in that 1% of the cases I would use Ainsible to deploy Focker


----------



## Jose (May 26, 2020)

sadaszewski said:


> There is one more clear and I guess overlooked advantage to Focker as compared to Ainsible etc. It has virtually no learning curve. Yes, you just write commands to make your jail look the way you like and that's it.


This to me is a bug and not a feature. I've run into too many "engineers" that had no idea how things they had written and deployed worked. It's all fun and games until you run into a problem you can't fix. That is when all the technical debt you incurred by doing things the fast and easy way comes due.


----------



## sadaszewski (May 27, 2020)

Jose said:


> This to me is a bug and not a feature. I've run into too many "engineers" that had no idea how things they had written and deployed worked. It's all fun and games until you run into a problem you can't fix. That is when all the technical debt you incurred by doing things the fast and easy way comes due.



To me Focker is supposed to be a self-service tool. So essentially you should take care of educating yourself enough to be able to configure a jail with shell commands. I am not a fan of publishing applications in the form of ready-made containers. However if you would like to use them in your deployment then it would still be up to you to verify that the recipes generating them are correct and work as expected. Anyway I have a great idea how to avoid the entire "binary blob" download thing. I will keep you posted


----------



## Toolforger (Jun 14, 2020)

kpedersen said:


> Agreed. On FreeBSD, I always distribute server software in jail(8) containers. How else would the cancerous spread of Python applications and their dependencies be prevented?



And then the best jail management software seems to be iocage, which has a hard dependency on Python...
So you're not preventing the Python spread at all.

Which is fine. Python does have its place in small scripts up to, say, 10,000 lines of code.

Oh, and Python isn't cancer... it is spreading because it is the easiest tool for some kinds of tasks.
If you dislike that (and I can understand why, I am not a big Python fan myself), write or sponsor a better language.


----------



## kpedersen (Jun 14, 2020)

Toolforger said:


> And then the best jail management software seems to be iocage, which has a hard dependency on Python...



I very much avoid iocage for this reason. However it isn't necessarily Python that I find bad; it is the developer mindset of dragging in 100+ dependencies just to i.e split a string.
PIP, NPM, Cargo and CPAN all suffer from the same misuse. In fact, Cargo's Crates is possibly the one reason why I don't think Rust will ever manage to replace C++. It makes it too easy to drag in C bindings rather than commit to writing software in Rust directly.



Toolforger said:


> write or sponsor a better language.



No need. Many better languages exist already and I find it very easy to avoid Python in all of my projects.


----------



## Hakaba (Jun 14, 2020)

No python here. Is use "vanilla" jails on my server and I am testing Bastille on my laptop.


----------



## Jose (Jun 14, 2020)

I use Python, but only because I find it less obnoxious than shell. Yes, it's a low bar.

However, it seems that the Python Steering Committee has learned the wrong lesson from the 2.x -> 3.x upgrade debacle and has started to break backwards compatibility even in dot upgrades. I don't understand how they don't see that this subjects their users to a death of a thousand cuts. Who wants to spend time making dozens of little edits all over the place to accommodate terminology "upgrades"? And if you think that using some automated tool to do this is a good idea, I can tell you've never had to review such changes. Or you have, and decided "screw it, let's commit this" after your eyes glazed over.

The only rationale I've seen for this approach is that Python needs to change in order to stay "relevant." I'm probably just a dinosaur, but the preceding sounds like BS hype to me.

Suppose I went to the mat and convinced skeptical coworkers to try Python. Now every little thing they've written in it breaks every time the system Python is upgraded. How does that make me look? How do my colleagues feel about me now?

I've thought about integrating Lua into a shell, but the resulting abomination wouldn't be POSIX-compliant and nobody would use it anyway.

Maybe Perl or Raku are the answer?


----------



## Toolforger (Jun 14, 2020)

kpedersen said:


> I very much avoid iocage for this reason.



Just to avoid Python?
Aww come on.



> However it isn't necessarily Python that I find bad; it is the developer mindset of dragging in 100+ dependencies just to i.e split a string. PIP, NPM, Cargo and CPAN all suffer from the same misuse.



Same as with pulling in all the dependencies if you install something via pkg.



> In fact, Cargo's Crates is possibly the one reason why I don't think Rust will ever manage to replace C++. It makes it too easy to drag in C bindings rather than commit to writing software in Rust directly.



To be useful, a language needs libraries. You can't tell people to go ahead and write an HTTPS library before making their first network connection. And if the language has an HTTPS library, every project will need some other library - just visit the libXxx packages in any operating system with a package manager to see how many libraries might be found useful in any application, no language creation project can start with all these libraries (and do them well to boot).

So at least initially, all languages need a C binding and the ability to import code.

Over time, any successful language project will create its own libraries.
If it does not, using C libraries isn't painful enough to drive people into writing those libraries; either the language semantics is so close to that of C that it wasn't never worth the burden of yet another variant of C, or the language does not support writing good libraries well enough to be worth the burden of yet another programming language in general. (Like every rule of this kind, it has its exceptions - e.g. I'd very much welcome a C successor that finally eliminated all those nasal demons, but we all need our pipe dreams.)



> No need. Many better languages exist already and I find it very easy to avoid Python in all of my projects.



Now that begs the question: Which one?
I.e. what language would you have preferred to see used for iocage, for example?


----------



## kpedersen (Jun 14, 2020)

Toolforger said:


> Just to avoid Python?
> Aww come on.



First of all, I am not *too* religious. If FreeBSD didn't already offer very usable jail utilities as part of base I certainly wouldn't boycott jails just because of Python.

... but yep, Python dependencies breaks too often. I try to use solutions that remain working for the foreseeable future. Honestly that is why I also use FreeBSD over Linux. I wish the FreeBSD handbook would not make so much mention of ezjail, iocage or the next short-lived wrapper.



Toolforger said:


> Same as with pulling in all the dependencies if you install something via pkg.



Yes, you absolutely want to reduce dependencies. Part of the reason why i.e desktop environments on FOSS platforms are so shakey is because so many of these dependencies are breaking over time. As part of my own solutions, I would only seriously rely on software with 2/3 recursive dependent packages max (for a server... for desktop, it is pretty much a free for all because xorg drags in so much crap ).



Toolforger said:


> Now that begs the question: Which one?
> I.e. what language would you have preferred to see used for iocage, for example?



Preferably C. Harder to develop but much better experience for users and much easier to maintain in the long run.
It would even have a chance at being imported into base then.

It may sound like I am a purist but much of the world chooses C or C++ for this same reason. It becomes less about the "pleasures" of the language and more about the underlying platform and reduction of complexity.



Toolforger said:


> To be useful, a language needs libraries. You can't tell people to go ahead and write an HTTPS library before making their first network connection.


For many libraries, they are a generally good to just merge with the codebase (so they can be updated in a controlled manner). For more complex ones like SSL, sure a single dependency. No problem in C or C++. But check out the dependencies for other languages:




			https://crates.io/crates/openssl
		







						Net::SSL - support for Secure Sockets Layer - metacpan.org
					

support for Secure Sockets Layer




					metacpan.org
				



Each one doesn't drag in *one* dependency. It drags in tonnes of random cruft compared to the same solution in C.
These language based package managers cause developers to make a mess of their solutions.


----------



## a6h (Jun 15, 2020)

Jose said:


> However, it seems that the Python Steering Committee has learned the wrong lesson from the 2.x -> 3.x


Reminds me of perl 5 & 6 but in reverse order (6 -> 5)



Jose said:


> I don't understand how they don't see that this subjects their users to a death of a thousand cuts.


They have no choice. Make a kludge, not a solution; in the end define the problem.



Jose said:


> The only rationale I've seen for this approach is that Python needs to change in order to stay "relevant."


It is not going anywhere soon. Facebook and Google love python.



Jose said:


> I'm probably just a dinosaur, but the preceding sounds like BS hype to me.


No, you aren't. You just don't have copy/paste mindset.



Jose said:


> I've thought about integrating Lua into a shell


Lua and C are happy couples, reminds me of sqlite/C.



Jose said:


> Maybe Perl or Raku are the answer?


Perl (or perl) just sounds more authentic.



kpedersen said:


> Python dependencies breaks too often.


As composer in php. It's the new (php, python, KaliKiddie) scripting common thread.



kpedersen said:


> for a server... for desktop, it is pretty much a free for all because xorg drags in so much crap.


The best part of FreeBSD: Cameron Grant rewrote the sound system for FreeBSD 4.0 (OSS && https://wiki.freebsd.org/Sound).
Looking forward to see Electron as Xorg dependencies



kpedersen said:


> Preferably C. Harder to develop but much better experience for users and much easier to maintain in the long run.


Harder to develop, easier to learn.



kpedersen said:


> It may sound like I am a purist but much of the world chooses C or C++ for this same reason.


C++: There's nothing wrong about OOP, but it sets a bad mindset. kind of: hit and run strategy; ... let's add itnow, fix it later.



kpedersen said:


> These language based package managers cause developers to make a mess of their solutions.


In a smaller scale, "drupal, composer, symfony", comes to mind. That was a big no no. I had to switch somewhere else.


----------



## sadaszewski (Jun 15, 2020)

kpedersen said:


> Preferably C. Harder to develop but much better experience for users and much easier to maintain in the long run.
> It would even have a chance at being imported into base then.
> 
> It may sound like I am a purist but much of the world chooses C or C++ for this same reason. It becomes less about the "pleasures" of the language and more about the underlying platform and reduction of complexity.
> ...



I wonder if it would seriously make a difference if Focker was written in C/C++. With something as mainstream as Python 3 + three extra packages (tabulate, jailconf, pyyaml) I can't help but think rewriting it in C/C++ would be a classical case of form over function. I mean -- seriously aren't there more important challenges to tackle in the whole containerization game than keeping the software written in C/C++?


----------



## kpedersen (Jun 15, 2020)

sadaszewski said:


> I wonder if it would seriously make a difference if Focker was written in C/C++.



Possibly not. The idea and design of Docker is already fairly heavy.


----------



## sadaszewski (Jun 15, 2020)

kpedersen said:


> Possibly not. The idea and design of Docker is already fairly heavy.


You cannot do the same thing Docker does any more lightly. Not speaking about the architecture - the daemon and so on. Those can be optimized, as Podman clearly illustrates. But having a hierarchy of  immutable images (or "code" as I like to think about them) and mutable volumes (the "data") is probably as nice as you can organize a big collection of containerized apps. In case of Focker (not Docker), they all end up as zero-overhead Jails anyway (you can remove Focker and they will keep running flawlessly). I can't see anything heavy about that


----------



## suntzu00 (Jun 15, 2020)

I've been using sysutils/cbsd for years but lately I've been thinking to switch to sysutils/pot and sysutils/nomad to scale things horizontally more easily. It can probably be integrated with Jenkins(running on bare metal) for testing and reproducibility. With a couple more tools like sysutils/consul and net/traefik you can get a kubernetes-like feeling.


----------



## drhowarddrfine (Jun 16, 2020)

sadaszewski said:


> You cannot do the same thing Docker does any more lightly.


I always wonder when someone says that. It was just a few years ago that no one ever mentioned the term and, now, many act like you can't possibly be doing anything unless you use it. No one at my company ever considered it or even brought it up.


----------



## rootbert (Jun 16, 2020)

I am a Python developer with passion, my C and assembler is way too old. "There are only two kinds of languages: the ones people complain about and the ones nobody uses." - Bjarne Stroustrup.

However, back to topic: today heise.de had a nice write-up on security research concerning docker images ... the original: https://www.heise.de/news/Studie-80...haben-schwere-Sicherheitsluecken-4785175.html - and english translated by google: https://translate.googleusercontent...5.html&usg=ALkJrhjfd8AmBUJivOVEnaGulM6TTKxotQ  ... for those preferring the paper this is based on: https://arxiv.org/pdf/2006.02932.pdf


----------



## sadaszewski (Jun 16, 2020)

suntzu00 said:


> I've been using sysutils/cbsd for years


cbsd is basically a wrapper of jail, jls and jexec. It doesn't have any higher level primitives introduced by Docker (checksummed, layered hierarchies of images, volumes, compositions). It also seems to be a primarily interactive tool which is exactly against the point.



> but lately I've been thinking to switch to sysutils/pot


Pot is similar to cbsd with barely some added notions of images and image recipes (flavors). It seems like this could have been a nice solution but I would bet that the author has never used Docker. Somehow in the FreeBSD crowd there seems to be a conviction that it is absolutely impossible that the far bigger Linux/Docker crowd and the industrial crowd might have come up (FOR ONCE) with nice, well thought-out abstractions, better than the ad-hoc, half-hearted, immature and incomplete efforts scattered among so many FreeBSD-compatible projects. Pot has also no notion of compositions.



> and sysutils/nomad to scale things horizontally more easily.


Nomad supports Docker, Podman and Singularity as containerization engines. Doesn't seem to be FreeBSD/containerization friendly in this regard.



> It can probably be integrated with Jenkins(running on bare metal) for testing and reproducibility.


Anything can be integrated with Jenkins 



> With a couple more tools like sysutils/consul


Consul is a service discovery, connectivity, heart-beat, etc. tool. A much higher layer than Docker and certainly not a replacement or alternative 



> and net/traefik you can get a kubernetes-like feeling.


traefik seems to be more or less the same thing as consul. I really appreciate the number of solutions available to some extent for FreeBSD but still the only thing that can give you a Docker-like feeling is Focker


----------



## kpedersen (Jun 16, 2020)

sadaszewski said:


> Docker-like feeling is Focker



As an aside, you might be interested in this: https://www.freebsdfoundation.org/blog/submit-your-freebsd-project-proposal/

In particular the request focus of "Containerization improvements, such as jail orchestration or Docker compatibility"

Would your project be a candidate for that?


----------



## sadaszewski (Jun 16, 2020)

kpedersen said:


> As an aside, you might be interested in this: https://www.freebsdfoundation.org/blog/submit-your-freebsd-project-proposal/
> 
> In particular the request focus of "Containerization improvements, such as jail orchestration or Docker compatibility"
> 
> Would your project be a candidate for that?


Well, yes. I most definitely think that it would!


----------



## sadaszewski (Jun 16, 2020)

sadaszewski said:


> Well, yes. I most definitely think that it would!


However, from what I gather, it is about projects that I would be willing to work on myself. And I am VERY short on time  I would like however to have a Docker-like solution for FreeBSD, either as evolution of Focker or as a Go-rewrite or perhaps C/C++ - rewrite if this is indeed what is the most desirable to the people. With added ability to use remote recipes as base in the build scripts and/or some sort of registry support (Docker-compatible or not).


----------



## suntzu00 (Jun 17, 2020)

Don't you love when a fella comes around and takes things out of context? 


> Consul is a service discovery, connectivity, heart-beat, etc. tool. A much higher layer than Docker and certainly not a replacement or alternative





> With a couple more tools like sysutils/consul and net/traefik you can get a kubernetes-like feeling



you should definitely ready my post again as I don't see where I mentioned consul and docker in the same sentence.



> traefik seems to be more or less the same thing as consul.



wait, what? 



> cbsd is basically a wrapper of jail, jls and jexec



cheers man! I had no idea 



> Doesn't seem to be FreeBSD/containerization friendly in this regard


sure they're not friendly. link link


----------



## a6h (Jun 17, 2020)

466 days (bad chmod) and 6 pages. Apparently, Docker is not dead. He is in Jail.


----------



## sadaszewski (Jun 17, 2020)

suntzu00 said:


> you should definitely ready my post again as I don't see where I mentioned consul and docker in the same sentence.



Well, this is sort of a "Docker" thread. It is not unreasonable to expect that one would be trying to address the topic of Docker and alternatives, not propose unrelated solutions with completely different purposes, duuh.



> wait, what?



Chill down, lest you get a heart attack  According to FreeBSD descriptions, consul is for "Service discovery and configuration" and traefik is a "High availability reverse proxy and load balancer". Close enough, they are definitely the same layer, way above container engines/Docker.



> cheers man! I had no idea



Again, this is sort of a "Docker" and/or containerization engine thread. One would expect adequate suggestions and it feels fine to point out things that are a little bit out of the whack.



> sure they're not friendly. link link



Hehe, ok. It is something one should take a closer look at. Nevertheless it is only a 3rd party plugin to nomad. So we get to the situation that was previously ridiculed in this thread with respect to Linux/Docker, "front-end to this is a back-end to that" lol  So to get out of it a slightest resemblance to Docker (still without hierarchical images and compositions) you would need to put together Nomad, the jail-task-driver and pot. I think this is a strong argument in favor of Focker


----------



## rafnizp (Jun 17, 2020)

Guys
You all are talking about pros and cons in regards to docker, kubernetes etc.
But I do not understand one simple thing. Where exactly is the problem of having those tools on BSD?
If there are people out there who wish to use this tools on BSD why we are limiting them?
Maybe there is some limitations on BSD that prevents from developing those solutions so if there is anyone who can explain more in details where is the problem I will really appreciate that.


----------



## rootbert (Jun 18, 2020)

rafnizp said:


> Where exactly is the problem of having those tools on BSD?
> If there are people out there who wish to use this tools on BSD why we are limiting them?


There is no problem, there is just a lack of interest and/or human resource to develop an application container solution for BSD. Whoever wants or has to use such technology has already switched to a platform that provides that (e.g. Linux + OCI or smartos). I think we do not limit anyone, everybody is free to extend FreeBSD in a way one wants. Having an application container platform with BSD is an absolutely nice imagination for me (probably on top of my list), I could get rid of quite a lot of nasty Linux boxes. However, I am well aware of the tremendous amount of work that involves, so I won't even start without commitment of the FreeBSD Foundation. I hope though I will soon have time to have a look at Focker ... To be honest, the sad truth is, the size of the project is not necessarily an argument pro switching to BSD as an application container platform, even if requirements are met. The size of a project (numer of developers, commitment of other companies) is an important factor in management decisions.


----------



## drhowarddrfine (Jun 18, 2020)

rootbert said:


> Whoever wants or has to use such technology has already switched to a platform that provides that


This is a very good point I try to make to people. I don't use FreeBSD cause I want it to work just like Linux or Windows. I use FreeBSD because I find it superior to the others for what I do.


----------



## Jose (Jun 18, 2020)

rootbert said:


> ...I won't even start without commitment of the FreeBSD Foundation...


I would much rather the Foundation spend its scarce resources getting 802.11ac support working. This is far more likely to drive Freebsd adoption than support for Docker-style containerization.


----------



## scottro (Jun 19, 2020)

It may already be there with some Atheros cards.  I haven't tried myself, but judging from the second to last post at https://forums.freebsd.org/threads/intel-wireless-3160-poor-performance-speed.75733/#post-465837 some people are getting good speeds with it.


----------



## Jose (Jun 19, 2020)

scottro said:


> It may already be there with some Atheros cards.  I haven't tried myself, but judging from the second to last posto at https://forums.freebsd.org/threads/intel-wireless-3160-poor-performance-speed.75733/#post-465837 some people are getting good speeds with it.


It is my understanding that the wireless stack in the kernel does not support 802.11ac. An adapter can claim to support it, but Freebsd won't use it.
The Foundation announced they're going to start sponsoring the work "soon" as of March 11, 2020. I'm no kernel hacker but I suspect it's going to take more than a few weeks for it to be completed and released.


----------



## rafnizp (Jun 19, 2020)

Correct me if I'm wrong, but from what I know BSD uses version control to their entire ecosystem and it is nothing like linux. 
In my opinion if BDS can support most of those container technologies it can become very powerfull BSD CORE CLOUD OS
And this is what we think are missing out there this days, good quality system with well documented changes, fixes etc.
I think if BDS supports that many people will switch ower and build their private cloud based on this BSD CORE CLOUD OS


----------



## Jose (Jun 19, 2020)

Freebsd already has traction in the cloud: https://papers.freebsd.org/2019/fosdem/looney-netflix_and_freebsd/

What's missing is a daily driver for developers. Since programmers tend to be technophiles, many of them are not going to be happy with being stuck with 802.11n in 2020.


----------



## rafnizp (Jun 19, 2020)

Jose said:


> Freebsd already has traction in the cloud: https://papers.freebsd.org/2019/fosdem/looney-netflix_and_freebsd/
> 
> What's missing is a daily driver for developers. Since programmers tend to be technophiles, many of them are not going to be happy with being stuck with 802.11n in 2020.



I do agree with you, that you will need to have fast communication (802.11ac) to build flexible and next gen applications (containerisation)
So both solutions I think will make huge impact on developers


----------



## a6h (Jun 22, 2020)

neel said:


> Linux users have inflicted a pain on themselves by needlessly bloating software and breaking APIs


To be fair, they never said GNU/Linux is Unix (GNU's Not Unix)


neel said:


> people who think it's a good idea to have packages for the most trivial tasks which depends on 100s of packages


Do you have Latex in mind?


neel said:


> All thanks to a short-sighted vision of modern software development which claims to be "agile"


"Agile" in itself is a claim; offer a solution, write a book, search for the problem.
He said, get a job; I told him, read a copy of Structured Programming (Dijkstra, 1972).


neel said:


> These are some interesting articles/videos on DevOps


DevOps is just another marketing term


neel said:


> I had a professor who compared Docker and DevOps to the "Unix philosophy"


Send him/her a copy of FreeBSD Developers' Handbook | Architectural Guidelines


----------



## Hakaba (Jun 22, 2020)

I am pretty sure that poeple wants a "FreeBSD Docker" solution only accept the solution if `Dockerfile` and all `docker` cli option matches.
IMHO a Kubernetes "official port" with jails and/or Bhyve is a better answer (and maybe create a `Jailfile` and/or `Bhyvefile` to push any project inside a container).

Kubernetes for FreeBSD is not impossible


----------



## kpedersen (Jun 22, 2020)

vigole said:


> Do you have Latex in mind?



Honestly all the packages used to be my least favorite thing about latex.

But then I realised that everyone pretty much just installs the *-full version of the distribution and rarely uses an external package other than the *one* hacked together by your intended journal or conference.

So for all intents and purposes, latex is absolutely monolithic and many gigabytes to install haha.

The fact that many packages cause so many side effects and breakages is still an issue. But that is probably more a symptom of latex being evolved rather than designed I suppose 



neel said:


> Disclaimer: I work at Microsoft which owns Azure and npm (the latter via GitHub). I don't work on either team.



We won't hold that against you... too much 

Your mention of technical dept is so very true. We are going to end up with a lot of broken systems if we keep on the way we are. It will also happen fairly abruptly as these large central services start to disappear.

Obviously making developers (and thus the world) rely on them as part of their business model is the important part. I don't think they have much consideration for lifespan or maintainability and other very important areas of engineering.


----------



## sadaszewski (Jun 23, 2020)

Hakaba said:


> I am pretty sure that poeple wants a "FreeBSD Docker" solution only accept the solution if `Dockerfile` and all `docker` cli option matches.
> IMHO a Kubernetes "official port" with jails and/or Bhyve is a better answer (and maybe create a `Jailfile` and/or `Bhyvefile` to push any project inside a container).
> 
> Kubernetes for FreeBSD is not impossible


This is completely unjustified speculation. I am myself a user who does now want this and I know at least a dozen others who don't want that either. Number 1 priority is to have similar abstractions and convenience level to Docker. Focker provides that nicely.


----------



## Purkuapas (Jul 3, 2020)

Jailfile and Bhyvefile already exist (via CBSD): CBSDfile ( bhyve + jail togeter ), or pgadmin4 jail sample


----------



## zirias@ (Jul 3, 2020)

sadaszewski said:


> This is completely unjustified speculation.


No, it's a sane assumption, because for any other usecase, there are already working solutions with FreeBSD. I'm getting increasingly fed up with your aggressive marketing for an open-source(!) tool. Marketing as such is fine, but hijacking a lot of unrelated threads over and over again is not.


----------



## a6h (Jul 3, 2020)

Zirias said:


> I'm getting increasingly fed up with your aggressive marketing for an open-source(!) tool


The account has created on May 19, 2020. The first post was posted, the same day, on May 19, 2020, and it was about Focker. We've been reading about Focker in this forums for last 45 days: _a total of 28 posts, 25 posts about Focker, in 45 days._
Therefore, it is clearly a form of product advertising. But I don't think, the first clause of FreeBSD Forums Rules has been broken by the user. i.e. The FreeBSD Forums are not an advertising billboard. Because this rule is phrased in a rather *ambiguous* way.
I'm not forum moderator, hence I'm only able to share my observations.


----------



## zirias@ (Jul 3, 2020)

It's unlikely there is any financial benefit in advertising an opensource project, so I wouldn't expect this to apply (but it's not my role to interpret the rules...) Still, it's just getting on my (and, obiously, some others') nerves


----------



## a6h (Jul 3, 2020)

Zirias said:


> It's unlikely there is any financial benefit in advertising an opensource projec


I agree with you on this issue. The advertising rule is open to interpretation.


----------



## drhowarddrfine (Jul 3, 2020)

If it hasn't been noted elsewhere, he is the creator of Focker


----------



## sadaszewski (Jul 7, 2020)

I just wanted to make your lives easier before the mentions of Focker get flooded with other sub-par FreeBSD solutions. At this point in time I have tested them all and they are a far cry from Docker-like abstractions provided by Focker. Nvm I will stop mentioning it in old posts but I will keep recommending it in new threads the same way people always reply with the old, sub-par (and honestly not even remotely similar in functionality to Focker) solutions (iocage, ezjail, qjail, etc).


----------



## Hakaba (Jul 7, 2020)

And CBSD and Bastille.


----------



## Mjölnir (Jul 7, 2020)

Time is ripe for a _"objective"_ comparison of these tools? Or two: one for jails, one for bhyve VM's.


----------



## Purkuapas (Jul 9, 2020)

I think the problem is not the lack of an alternative to docker. There is an alternative: Bastillebsd, Focker, CBSD...
The problem is that we cannot convert Linux binaries of real docker images. And the FreeBSD ( BastilleBSD/Focker/CBSD) community is too small to create and maintain its own app images. For this reason, all these utilities do not matter much. All we have for the docker is the bhyve.


----------



## sadaszewski (Jul 10, 2020)

Purkuapas said:


> I think the problem is not the lack of an alternative to docker. There is an alternative: Bastillebsd, Focker, CBSD...
> The problem is that we cannot convert Linux binaries of real docker images. And the FreeBSD ( BastilleBSD/Focker/CBSD) community is too small to create and maintain its own app images. For this reason, all these utilities do not matter much. All we have for the docker is the bhyve.


Last time I checked it is not the Docker developers who maintain images for different distributions and software applications. Now we need to market something to the software developers so that they start preparing and maintaining their own images. I think Focker is much better positioned for that as it resembles Docker a lot, concept-wise and workflow-wise, and the developers presumably all know Docker.


----------



## kagi3624 (Jul 23, 2020)

drhowarddrfine said:


> So a wrapper around a wrapper. Or a tool for a tool?
> 
> This sounds like web development nowadays. No one just writes HTML, CSS and JavaScript. You have a tool to make that but, then, someone comes along with a tool to make the first tool better but then someone else needs a tool for that tool and, pretty soon, you have what's going on now--layer upon layer of tools that are all supposed to make life easier but they don't.
> 
> As I said, if one is going to use FreeBSD, use jails and be done with it.


Your comment reminded me of seitting up java for software developemen.t


----------



## JAW (Jul 23, 2020)

kagi3624 said:


> Your comment reminded me of seitting up java for software developemen.t



`pkg install intellij`


----------



## AM (Jan 1, 2021)

drhowarddrfine said:


> So a wrapper around a wrapper. Or a tool for a tool?
> 
> This sounds like web development nowadays. No one just writes HTML, CSS and JavaScript. You have a tool to make that but, then, someone comes along with a tool to make the first tool better but then someone else needs a tool for that tool and, pretty soon, you have what's going on now--layer upon layer of tools that are all supposed to make life easier but they don't.
> 
> As I said, if one is going to use FreeBSD, use jails and be done with it.


I would like to add my 5¢ to discussion.

The main reason why web devs use docker is to have consistent environment in development, demo and production. For instance, node.js code may behave different on different OS. Most devs have either MacOS/Windows as their dev machine, but production is running on FreeBSD. Because docker uses Linux, I'm as a web dev who loves FreeBSD, forced to use some Linux distro both on the server container and on my dev machine in my local docker client.

Therefore, if FreeBSD developers want to make their OS more popular and as a result to get more donations will have to jump on the trends of web dev and create some FreeBSD alternative of docker. When I say alternative of docker I mean execution of FreeBSD jail layers on MacOS/Windows easier than to run VM, i.e. some sort of containerization.


----------



## drhowarddrfine (Jan 2, 2021)

AM said:


> For instance, node.js code may behave different on different OS.


The purpose of node is to use javascript for consistency on the web. Different behavior based on OS is the opposite of its purpose and containers don't solve that. There are other issues at hand there. Of course, relying on node is a problem in itself.


AM said:


> Most devs have either MacOS/Windows as their dev machine, but production is running on FreeBSD.


Most? Designers maybe but not developers. And especially not FreeBSD.

Note: This month marks 17 years since I started a web dev company using FreeBSD


AM said:


> if FreeBSD developers want to make their OS more popular and as a result to get more donations will have to jump on the trends of web dev and create some FreeBSD alternative of docker.


Make FreeBSD act like Linux? No thank you. FreeBSD is a professional operating system for professionals. Linux is popular among the hobbyists which is why you hear more about it. Remove the hobbyists and usage drops by 80%.

You want great networking? You should be using FreeBSD, not Linux. Look at why NASA, Netflix and Juniper are using BSD for networking as documented everywhere. If you want containers, use jails. Don't be copying Linux's problem solvers that have problems themselves. (If Docker is so great, why do people use Kubernetes?)


----------



## Jose (Jan 2, 2021)

drhowarddrfine said:


> The purpose of node is to use javascript for consistency on the web. Different behavior based on OS is the opposite of its purpose and containers don't solve that. There are other issues at hand there. Of course, relying on node is a problem in itself.


So true. Node's "download the Internet for every build" means that installs on the same machine from a couple of hours ago can be different from an install you do now. The last time I worked on a node project (and I was paid to) we used some tool to keep 2-3 versions of node around because of the significant compatibility problems new versions introduced, and new versions happened early and often.


AM said:


> Therefore, if FreeBSD developers want to make their OS more popular and as a result to get more donations will have to jump on the trends of web dev and create some FreeBSD alternative of docker. When I say alternative of docker I mean execution of FreeBSD jail layers on MacOS/Windows easier than to run VM, i.e. some sort of containerization.


So Freebsd should have embraced Ruby on Rails? Asp.net? Java servlets and JSPs? Apache with mod_perl? SOAP? CORBA? So many missed opportunities!


----------



## mark_j (Jan 2, 2021)

Javascript is evil; it and all its disciples should be burned at the stake (or is that steak?)


----------



## AM (Jan 3, 2021)

drhowarddrfine said:


> The purpose of node is to use javascript for consistency on the web. Different behavior based on OS is the opposite of its purpose and containers don't solve that. There are other issues at hand there. Of course, relying on node is a problem in itself.
> ...



Thanks for your response. I'm not looking to argue who is right and who is wrong.

The thing is that I'm as the user of FreeBSD looking for solution which will make my life easier.
If you personally, don't want to invest time into it, no one is looking to force you to do that.
Just say that you don't want to do that, whatever encouragement I'm offering, whether it is donations or fame, recognition as influential developer etc.

It doesn't matter.

FreeBSD is open source and FreeBSD license is very liberal as far as I understand and anyone can create solution in demand especially if:


drhowarddrfine said:


> ...NASA, Netflix and Juniper are using BSD...



Anyway, thank you for your answer. Happy New Year!


----------



## rootbert (Mar 19, 2021)

wow, it seems someone is working on a cool OCI-compatible container project for FreeBSD ... so we might get all the good stuff that Docker has ... see https://github.com/samuelkarp/runj


----------



## drhowarddrfine (Mar 19, 2021)

> Established in June 2015 by Docker ...


Reads more like a Docker compatible project. 

But Docker is dead. I saw an article about this again a couple of days ago. That everyone is switching to Kubernetes. Docker somehow managed to raise $32 million dollars but that really isn't a lot for a big business.


----------



## rootbert (Mar 19, 2021)

yeah, Docker might be dead, but the technology behind is not (thats what I was refering to) ... whatever, let's call it OCI container, whether it is kubernetes, podman, runc, docker, containerd etc. that runs the application container does not matter. Just wanted to point out that there is an interesting application container project for FreeBSD growing


----------



## Deleted member 30996 (Mar 26, 2021)

AM said:


> The thing is that I'm as the user of FreeBSD looking for solution which will make my life easier.


If by easy you mean simple, it doesn't get any easier than editors/leafpad. I picked up where I left off on NotePad when I switched from Windows. 

I've never used anything but a text editor and valid XHTML or CSS on my sites since I taught myself how to write it at w3schools.com when it came out in 2000.

Docker was a type of khakis Jeans back then. Fireworks was what people who wore them and couldn't write XHTML or CSS used.

Like black Jeans and LeafPad were to those who could.


----------



## neel (Apr 3, 2021)

rootbert said:


> yeah, Docker might be dead, but the technology behind is not (thats what I was refering to) ... whatever, let's call it OCI container, whether it is kubernetes, podman, runc, docker, containerd etc. that runs the application container does not matter. Just wanted to point out that there is an interesting application container project for FreeBSD growing


The truth is that Docker is  commoditized product. Docker's "decline" is a product of their own making, they are just a leader in a piece of software that's easy to replace.

Docker thought they would give their software free and open source so everyone would use Swarm. But then Kubernetes came along and everyone used that, and hence Docker is struggling. Docker is just the container engine, abstracted behind Kubernetes.

Docker might "re-focus" on "Developer Tools", but why would I use something with the "Docker" name instead of the tools my cloud provider gives me and not worry about another bill? Just because Docker was "cool" and "upcoming" in 2015? Look at HTC, they were "cool" 10 years ago and now HTC is history, they lost to Apple, Samsung, and the various number of Chinese OEMs.

Look at Hadoop vendors like Cloudera, they once had the world come around them in 2011 and now Hadoop (and Spark) is just a managed service in AWS or Azure. Why use "Cloudera" when I can just use the Hadoop which comes with my cloud subscription? Same applies to Docker.

In comparison, look at Microsoft. Say what you want about them, but a lot of companies use Microsoft products mainly since they only have to worry about one vendor for OSes, productivity software, and cloud. Many (not all) Microsoft products have "alternatives", but a lot of companies are Microsoft shops since they're the one-stop shop for a lot of an enterprise's software needs. But at the same time, many other companies may go with AWS or Slack or Zoom or Google Workspace and just use Windows and Office.

The one-stop shop analogy can be said about networking with Cisco and Huawei respectively for enterprise and service-provider roles.

Docker has essentially become plumbing. Plumbing that isn't exactly BSD-friendly so we have to rip apart Dockerfiles and build systems in order to put newer software in Ports. Hopefully Docker will eventually get abstracted out of DevOps so we are able to get portable software again. Or maybe what systemd was to the desktop, Docker is for the server: the thing we hate but have to deal with to get software out of it indefinitely. a pain for Ports maintainers but "loved" by developers who only think about Ubuntu on AWS or GNOME on Fedora.

Disclaimer: I work at Microsoft, but not on Azure. I do however deal with the Azure "Big Data" ecosystem heavily.


----------



## patrickjp93 (May 6, 2021)

A good part of me wants to start another thread with a different title just to clear the air, but I think it's useful to at least plant a flag and clarify some of the asks and discussion points.

Kubernetes does not replace Docker. Kubernetes is all about cluster deployment, oversight, and management (most importantly your software-defined network and Head Node equivalent). It is your Infrastructure as Code or Infrastructure as a Service.

Docker is a runtime image in which your deployed code, its applicable middleware (jvm, node, python dist), environment variables/secrets, file system paths, virtual network interfaces, and a DNS white list have been bundled up together. It is your Vagrant equivalent.

Dockerfile and docker-compose.yml are really the fulcrum of this discussion. What THOSE TWO are, are the portable System Definition and Composition files used by Open Container Initiative. Openshift, ContainerD, and Triton (Joyent) all have verbatim copies of these files and use them the same way to construct and deploy a bundle of off-the-shelf tools (middleware, databases, ngnix, ...) and a developer's custom code that uses those tools together in a much more lightweight manner than spinning up Vagrants.

No one (sane) ever runs a production workload where the application server and DB server are the exact same physical machine, but a developer is working off of one workstation/laptop/remote VM for most of their work. The Image Definition and Composition files are portable. They can be handed to any OCI runtime, and a container image with the tools needed is constructed in a manner compatible with the host machine, with far stronger compatibility guarantees than a VM can ever give you (Virtualbox on Mac OS anyone?) WITH the benefit of both the host and guest platforms being separately, continuously upgradable (keeps Security happy) since the OCI runtime only provides a Linux Call Table compatibility layer (OS virtualization).

What OCI environments give us is the beauty of NOT having to defend against unstable crap combos like Python 2+3 and Ansible AND Vagrant (on Mac) getting in the way of developer onboarding and productivity into a software product built to run on a completely different OS from their company-provided machine.

And BECAUSE the Postgres, Cassandra, AdoptOpenJDK, <ad_nauseum>, and OCI dev communities work together so closely, you don't wind up in a scenario where the newbie got the newest model of laptop with a new Mac OS that Apple changed a zillion things on stomping all over a set of Vagrant, Python, and Ansible configs & routines last updated 2 years ago "because they just work" and it takes one of the old wizard senior devs a week to triage it.

BSD doesn't need Docker. It doesn't even need Kubernetes (occasional use of free-tier provided/managed services for sandboxing is probably fine). I do, however, think BSD would benefit immensely from having an Open Container Initiative-compliant/compatible "container" runtime.

Whether you use Jails/Zones and ZFS and network tools du jour to achieve this, that's up to the ports developers.
The benefit of having such a toolchain should be crystal clear even IF you yourselves never intend to deploy your software to a managed container service of any kind. If you want your user base to grow, this will bring in tons who at the very least HATE Docker Desktop For Mac but still need to work with Docker for their jobs. Because it's not about Docker. It's about OCI. Docker is dead. Long live Docker.


----------



## SirDice (May 6, 2021)

patrickjp93 said:


> Kubernetes does not replace Docker. Kubernetes is all about cluster deployment, oversight, and management (most importantly your software-defined network and Head Node equivalent). It is your Infrastructure as Code or Infrastructure as a Service.











						Kubernetes is deprecating Docker: what you need to know
					

What’s Changing with Docker? Kubernetes deprecating Docker is actually not as big of a deal as it sounds, so let’s talk about what is really going on here.




					acloudguru.com


----------



## shkhln (May 6, 2021)

patrickjp93 said:


> No one (sane) ever runs a production workload where the application server and DB server are the exact same physical machine


_looks at forums.freebsd.org with suspicion_


----------



## patrickjp93 (May 6, 2021)

SirDice said:


> Kubernetes is deprecating Docker: what you need to know
> 
> 
> What’s Changing with Docker? Kubernetes deprecating Docker is actually not as big of a deal as it sounds, so let’s talk about what is really going on here.
> ...


I fail to see how that contradicts what I said. The useful thing about Docker is the image and composition spec (which is now the triumvirate bedrock of OCI). Kubernetes doesn't support the runtime that the Docker company started with. You still build a set of "Docker" containers and deploy them TO Kubernetes. What runtime is used for execution (Docker, ContainerD, Triton) is an implementation detail of Kubernetes.

Docker (the runtime) is dead. Long live Docker (OCI) containers.


----------



## kpedersen (May 6, 2021)

patrickjp93 said:


> IF you yourselves never intend to deploy your software to a managed container service of any kind. If you want your user base to grow, this will bring in tons who at the very least HATE Docker Desktop For Mac but still need to work with Docker for their jobs. Because it's not about Docker. It's about OCI.


What this basically translates to is "If you stop using FreeBSD, you will need Docker".

Yes, I possibly agree but if we did stop using FreeBSD, we would be using Linux or macOS like you suggested and they *do* support Docker. However we use FreeBSD to avoid this mess of platforms and instead use tools specialized for FreeBSD rather than basically a heavy abstraction layer.



patrickjp93 said:


> No one (sane) ever runs a production workload where the application server and DB server are the exact same physical machine, but a developer is working off of one workstation/laptop/remote VM for most of their work.



What did people do before Docker got marketed? Basically we still do the same thing (Jails).

Not to mention, Docker is an abstraction layer. It is them that are meant to abstract FreeBSD. Not the other way round. If FreeBSD ever got popular enough, I am pretty sure Docker would have no choice but to do the work to support us themselves.


----------



## patrickjp93 (May 6, 2021)

shkhln said:


> _looks at forums.freebsd.org with suspicion_


I've seen two catastrophic big iron mainframe failures in my short 5-year career. No matter how much the community kicks and screams, the distributed systems philosophy of "servers are cattle, not pets" has proven correct for everything BUT the transactional workloads for which only the biggest iron will suffice. You want to spread your failure risk out and be running app instances on multiple physical machines behind redundant load balancers. Where I currently work, you either do the work to shard your app so Postgres can scale horizontally, or you use multiple production datacenters with data replication constantly running between them for failover. There have been close calls, but no total outages caused by a slight misconfiguration at the OS or network level. Previous employers had multi-million dollar outage fines because of their tiny mistake on a single mission-critical server (mainframe). And even if they hadn't made a mistake, a long-running power outage was inevitable someday, and corporate didn't want to pay for a second electricity provider as emergency backup.

No one (sane) puts everything into one box, and that includes all members of this community. Yes, it's filled with excellent engineers. Engineers are not risk managers constantly asking themselves "should I do this, even though I can and can code it up fast?"


----------



## patrickjp93 (May 6, 2021)

kpedersen said:


> What this basically translates to is "If you stop using FreeBSD, you will need Docker".
> 
> Yes, I possibly agree but if we did stop using FreeBSD, we would be using Linux or macOS like you suggested and they *do* support Docker. However we use FreeBSD to avoid this mess of platforms and instead use tools specialized for FreeBSD rather than basically a heavy abstraction layer.
> 
> ...


No, because Jails are not portable even between between OS versions, let alone other platforms (you want people to migrate to BSD, right?), and not everyone has a private datacenter where they can roll everything themselves as the BSD community seems to have the luxury of in shockingly high concentrations.

Having OCI compatibility (not Docker compatibility) makes BSD much more palatable to developer communities and system admins who would love to be able to use it but for the fact BSD is a walled garden that chains the organization to private datacenters. 

It's not about Docker. Ignore the word Docker. It's about application portability and packaging, as well as developer productivity and symbiosis with the broader software ecosystem in ways that are complementary without being overshadowed by Big Tech powers (like Red Hat and SystemD).


----------



## SirDice (May 6, 2021)

patrickjp93 said:


> No, because Jails are not portable even between between OS versions,


Yes, they are. It's perfectly fine to run a 12.2 jail on a 13.0 host for example. That's a supported way of running it.


----------



## Jose (May 6, 2021)

patrickjp93 said:


> I've seen two catastrophic big iron mainframe failures in my short 5-year career.


He's seen a mainframe. I'm convinced. Docker it is.


----------



## kpedersen (May 6, 2021)

patrickjp93 said:


> No, because Jails are not portable even between between OS versions, let alone other platforms (you want people to migrate to BSD, right?)


As SirDice mentioned, versions aren't a problem.

As for other platforms, do please have a think about this question: Can you run a Windows or macOS binary in a Docker container?

No. So what you are saying is "Target Linux. Compile for Linux. Use Linux". Obviously that isn't going to be a popular suggestion in this community.
For a long time Docker ran on a VM on Windows and macOS so that it could run Linux binaries. Effectively you can do that with FreeBSD. We can run Linux just fine in VirtualBox and Bhyve. So FreeBSD effectively already supports Docker.

Do we really want people to migrate to BSD? The OpenBSD stance is a definate NO! FreeBSD is a little more easy going but we still only want correctly thinking people. Otherwise the project will turn into an eclectic experiment like most Linux distributions.



patrickjp93 said:


> Having OCI compatibility (not Docker compatibility) makes BSD much more palatable to developer communities and system admins who would love to be able to use it but for the fact BSD is a walled garden that chains the organization to private datacenters.


And until OCI compatibility extends to other operating systems rather than just Linux, it is surely a little embarrasing to recommend it to an operating system that exactly isn't Linux. Perhaps Linux should support Jails?

Any organisation that chains itself to OCI is basically yet another  "Linux house". Nothing new or special there.


----------



## patrickjp93 (May 6, 2021)

SirDice said:


> Yes, they are. It's perfectly fine to run a 12.2 jail on a 13.0 host for example. That's a supported way of running it.


The reverse has to also be true for you to have feature parity with OCI. You don't have it.

While the Dockerfile (System Image Spec) and docker-compose.yml (System Image Composition Spec) feature sets may expand over time, when an actual image is made using today's 3.8-compliant spec files, it's portable all the way back to Docker 1.0, on ANY OS Docker 1.0 runs on. Same thing goes for an image built way back when on 1.0. You can run it on Docker 3.8, Kubernetes, OpenShift, or Triton. 

You can't take a BSD 12.x jail spec to a BSD 10 machine without manually rewriting it. And even if you could, that alone is not enough.

What Focker is doing is only half your battle. It provides you a version-agnostic jail composer. Now unify that capability with the OCI container spec to use Jails as the underlying implementation for your execution environment, and you have "Docker" (OCI) on BSD.


----------



## shkhln (May 6, 2021)

This is your daily reminder that BSD _is a license_. (Unless you are talking about the FreeBSD's ancestor OS, in which case we don't have jail compatibility with that.)


----------



## patrickjp93 (May 6, 2021)

kpedersen said:


> As SirDice mentioned, versions aren't a problem.
> 
> *As for other platforms, do please have a think about this question: Can you run a Windows or macOS binary in a Docker container?*


Yes you can. https://docs.microsoft.com/en-us/virtualization/windowscontainers/about/

Windows jumped on the OCI train so Windows binaries could run on Linux and Mac several years ago.



> So what you are saying is "Target Linux. Compile for Linux. Use Linux". Obviously that isn't going to be a popular suggestion in this community.


No I am not. Please tell me English is your second or third language.

Target BSD, Compile for BSD, Use BSD natively. Just give me a toolchain that I can use to transfer what I was being forced to run on Mac and Linux over to BSD without having to rewrite the wheel that is all the configuration hackery to support a full-blown CentOS VM on a Mac host (like building a Vagrant to host everything in the FreeBSD way since lord knows the hacks we set up to get Vagrant to behave on Mac OS Big Sur the way we needed it to were stomach-churning).

BSD has native Postgres, Java, Cassandra, unix shell tools, file systems, etc.. I'm more than happy to use them. I'd LOVE to use them.

Let me hand you a spec file that says "set up some directories, create DNS resolvers for a whitelist of external resources (that only this "container" can see), download a native Java dist., download a native Postgres dist., download my application from my Nexus, set up env vars, set up pre-defined secrets, and expose a port for the world to reach my app. You're free to handle the details of the networking and resource allocation."

You can use Jails and ZFS and whatever else you like to implement the secure sandbox for it to run in.


----------



## mark_j (May 6, 2021)

patrickjp93 said:


> You can't take a BSD 12.x jail spec to a BSD 10 machine without manually rewriting it. And even if you could, that alone is not enough.


Why would you do that? That's a nonsense argument. 10 ended 4 years ago.


----------



## zirias@ (May 6, 2021)

mark_j said:


> Why would you do that? That's a nonsense argument. 10 ended 4 years ago.


I guess that's because you can do such things with Docker … you can even run a Linux container on a Windows host. But of course, under the hood, this will use full virtualization, so IMHO it's just "hiding a problem". One key advantage of using "containers" (jails) is that it's light-weight…


----------



## patrickjp93 (May 6, 2021)

mark_j said:


> Why would you do that? That's a nonsense argument. 10 ended 4 years ago.


So did CentOS 6. Sometimes old servers have to live a long damn time because corporate makes a priority decision that is ITSELF nonsense.

Don't forget that if you work for an organization that already broadly uses FreeBSD (or any of the BSD server OSes), you probably lack a fair amount of nonsense that the rest of the world commonly has to deal with from "management." I can take my Java application stack back to our CentOS 6 servers if some mission-critical security flaw is found in the v8 boxes and they need to be taken down, because CentOS 6 has a JVM compatible with our code, and the rest is networking + directory paths. By the same token, I can take that app to Windows Server or FreeBSD, because they all have JVMs.

But say you have a larger organization that has multiple datacenters, all in different states of being "up-to-date." Say you have an in-house software that is to be rolled out to all datacenters, and it needs to run regardless of OS version. If you wrote it in Python or Java, there's a pretty solid chance it will. If you wrote it against the very latest jail spec that isn't directly transferrable to the N-1 version in another DC, you're going to have to custom-roll another Jail spec and deployment for that DC. An automation of this is Focker.

No one WANTS to go back in time. Disaster Recovery plans sometimes necessitate it, especially in the context where your datacenter provider isn't very timely on spinning up new environments for you and updating OSes on existing machines is another ball of wax unto itself. Decisions being made above engineers' heads sometimes necessitate that in the ecosphere outside this immediate community. Try to not take that for granted too much.


----------



## patrickjp93 (May 6, 2021)

Zirias said:


> I guess that's because you can do such things with Docker … you can even run a Linux container on a Windows host. But of course, under the hood, this will use full virtualization, so IMHO it's just "hiding a problem". One key advantage of using "containers" (jails) is that it's light-weight…


Where is this myth still coming from? The VM hosting requirement is only true on Mac OS anymore. Windows Linux Subsystem is now a lightweight OS virtualization just like BSD's Linux compatibility layer. It's a call table emulator and not much more, has been that way for a little over a year now. The hardware virtualization is gone.


----------



## mark_j (May 6, 2021)

patrickjp93 said:


> Yes you can. https://docs.microsoft.com/en-us/virtualization/windowscontainers/about/
> 
> Windows jumped on the OCI train so Windows binaries could run on Linux and Mac several years ago.
> 
> ...



These are also available on linux. 

I admire your passion but most of your arguments read like over-hyped advertising spiel. The linux foundation is free to fund oci on freebsd; nothing's stopping them.

Freebsd (and this is a broken record retort) has limited developers, funding and overall resources compared to linux. That's the crux of it.


----------



## kpedersen (May 6, 2021)

patrickjp93 said:


> Yes you can. https://docs.microsoft.com/en-us/virtualization/windowscontainers/about/
> 
> Windows jumped on the OCI train so Windows binaries could run on Linux and Mac several years ago.


No they can't run. You are mistaken. Pointing to documentation for a feature that relies on Windows Subsystem for *Linux* shows you don't quite understand the limitations of containers and what a Windows binary is. The fact that Microsoft runs Ubuntu in a Hyper-V based VM doesn't make Docker portable!

Can you point out to me an exact document that shows that Docker can run macOS binaries? Apple would not let it exist for one.

Target Win32, Compile for Win32, Use Win32 natively
Target macOS, Compile for macOS, Use macOS natively
Target BSD, Compile for BSD, Use BSD natively
None of these can you do with Docker. You can only work with Linux binaries. Sure, some platforms can run Linux as VMs or compat layers. But that is nothing more than babysitting unportable Linux code.


----------



## zirias@ (May 6, 2021)

patrickjp93 said:


> Where is this myth still coming from?





patrickjp93 said:


> has been that way for a little over a year now.


Add to that picture we were coming from "oh, a 12.x jail won't run on a 10.x host".

This is a nice strawman


----------



## shkhln (May 6, 2021)

patrickjp93 said:


> Windows Linux Subsystem is now a lightweight OS virtualization just like BSD's Linux compatibility layer. It's a call table emulator and not much more, has been that way for a little over a year now. The hardware virtualization is gone.


[citation needed]

Microsoft getting rid of a syscall-level emulation layer was precisely the point of WSL 2. Are you claiming there is WSL 3 already?


----------



## a6h (May 6, 2021)

If SYSTEM is not a Type-1/Type-2 VMM, and SYSTEM wants to run Windows PE (EXE/DLL),
SYSTEM has to implement at least these bunch of things (TTBOMK) -- either in kernel or user-space, e.g. WineHQ.

* ntoskrnl.exe (syscall)
* KERNEL32.DLL (win32 API)
* GDI32/USER32 (userland libs)

AFAIK, N/A to the Docker.


----------



## zirias@ (May 6, 2021)

vigole said:


> SYSTEM has to implement at least these bunch of things -- either in kernel of user-space (like WineHQ)
> 
> * ntoskrnl.exe (syscall)


Nitpick: Not sure wine is doing this, but it's not strictly necessary. The kernel API on windows is by definition "private" and no userspace program should attempt to access it. They should use a subsystem instead (normally: win32).

Of course, you'll probably find some programs ignoring that, so – as I said, a nitpick


----------



## a6h (May 6, 2021)

Zirias said:


> Of course, you'll probably find some programs ignoring that, so – as I said, a nitpick


Nitpick-ing is great. Thanks.


----------



## mark_j (May 6, 2021)

patrickjp93 said:


> So did CentOS 6. Sometimes old servers have to live a long damn time because corporate makes a priority decision that is ITSELF nonsense.


Servers are not Operating systems. One can have long lived hardware and modern OS. Running an old OS version and using this as an argument for why jails are not as good as linux containers is a spurious argument.



patrickjp93 said:


> Don't forget that if you work for an organization that already broadly uses FreeBSD (or any of the BSD server OSes), you probably lack a fair amount of nonsense that the rest of the world commonly has to deal with from "management." I can take my Java application stack back to our CentOS 6 servers if some mission-critical



I have no idea what centos 6 is, old or new, but I still don't see the correlation or relevance to freebsd. Freebsd is an OS, linux is a kernel. People add various junk to it to make it into an OS. Having stuff like OCI to make a 'standard' seems a reasonable thing for an operating system pulled together differently depending on the distribution. It probably makes sense for 'linux'.

So redeploying some software from one freebsd server to another is child's play. Freebsd is freebsd. Centos is, however, not redhat or ubuntu or gentoo. Your arguments premise on the linux world and presumes their same hotch-potch of OS building applies to freebsd; this simply isn't the case.


patrickjp93 said:


> security flaw is found in the v8 boxes and they need to be taken down, because CentOS 6 has a JVM compatible with our code, and the rest is networking + directory paths. By the same token, I can take that app to Windows Server or FreeBSD, because they all have JVMs.
> 
> But say you have a larger organization that has multiple datacenters, all in different states of being "up-to-date." Say you have an in-house software that is to be rolled out to all datacenters, and it needs to run regardless of OS version. If you wrote it in Python or Java, there's a pretty solid chance it will. If you wrote it against the very



We only ever maintain one version of the freebsd (and netbsd) Os across our datacentres, both regional and state. Problem solved. We're not running ubuntu on one, redhat on another and archlinux on yet another and expecting it all to run like clockwork. Nope, we're on FreeBSD 12.2p5, with a internal package repository and we like to measure uptime in years.

As I've alluded to, I really don't see the need for freebsd to push itself into an area already occupied by linux. It has a potential best case rate of return of zero. It makes no sense in any way.


----------



## kpedersen (May 6, 2021)

mark_j said:


> Having stuff like OCI to make a 'standard' seems a reasonable thing for an operating system pulled together differently depending on the distribution. It probably makes sense for 'linux'.


They were meant to have LSB (Linux Standard Base) but they simply can't seem to keep their sh*t together.

Now the popular thing is to bundle up an entire userland and distribute that ala Flatpak, Snap, Docker, etc. It is getting a little painful to watch.


----------



## drhowarddrfine (May 6, 2021)

I'm going to go find a Linux forum and tell them all that Linux sucks cause it can't run jails.


----------



## SirDice (May 6, 2021)

kpedersen said:


> Now the popular thing is to bundle up an entire userland and distribute that ala Flatpak, Snap, Docker, etc. It is getting a little painful to watch.


I find it quite entertaining. It's fun to watch them all scramble to one thing then abandoning it a few years later for the next "best thing since sliced bread". It just sucks I have to support all that mess at work because managers fall for the same traps every time.


----------



## kpedersen (May 6, 2021)

SirDice said:


> I find it quite entertaining. It's fun to watch them all scramble to one thing then abandoning it a few years later for the next "best thing since sliced bread". It just sucks I have to support all that mess at work because managers fall for the same traps every time.


Very, true. Watching from the sidelines it quite good fun. Though as you mentioned they can't wait to drag us into their self-inflicted mess one way or another.

There was actually a good book here, though originally relating to programming it summarizes the "cool guy" that waltzes into a company with the newest toys. Does their best to get them stuck into the current projects and then disappears after the work becomes too hard for them to handle. Leaving us to maintain the shite when we originally were the ones against using it.

The worst part is that it undermines us. We ironically get seen as the guy who flitters around with different "nerdy" technologies because management conveniently forgets that they originally forced us to because they listened to the "cool guy"! It means that for future tech discussion meetings we get heard even less and you end up being called a luddite and a tech experimentalist in the same damn meeting XD


----------



## Beastie7 (May 6, 2021)

kpedersen said:


> They were meant to have LSB (Linux Standard Base) but they simply can't seem to keep their sh*t together.
> 
> Now the popular thing is to bundle up an entire userland and distribute that ala Flatpak, Snap, Docker, etc. It is getting a little painful to watch.



Band aids on-top of band aids on top of band aids, then they have the nerve to call corporate trends like "OCI" et al., a standard. 

You can't make this s**t up people.


----------



## Menelkir (May 6, 2021)

SirDice said:


> I find it quite entertaining. It's fun to watch them all scramble to one thing then abandoning it a few years later for the next "best thing since sliced bread". It just sucks I have to support all that mess at work because managers fall for the same traps every time.


And sadly, the better things are mostly forgotten or overshadowed by crappy other things. For example, appimage is way better than the counterparts, but guess what...


----------



## drhowarddrfine (May 6, 2021)

"Everything should be made as simple as possible."

You can quote me.


----------



## a6h (May 6, 2021)

> Everything should be made as simple as possible.


-- drhowarddrfine


----------



## a6h (May 6, 2021)

SirDice said:


> I find it quite entertaining. It's fun to watch them all scramble to one thing then abandoning it a few years later for the next "best thing since sliced bread".


What's the solution to Linux package mess: Choose the best one (pacman, apt-get, or whatever it is) across all distros, not to build a new one! The latter leads to more mess.
It's very simple. Isn't it?


----------



## Alain De Vos (May 6, 2021)

But why couldn't they create sustainable business ?


----------



## kpedersen (May 6, 2021)

vigole said:


> What's the solution to Linux package mess: Choose the best one (pacman, apt-get, or whatever it is) across all distros, not to build a new one! The latter leads to more mess.
> It's very simple. Isn't it?


The LSB did specify that rpm had to be present in conforming implementations. It would have been somewhat cool if no matter what package system a distro went for, they always had an rpm plugin to make it integrate consistently with the "standard".

Of course that didn't happen.

I think at this point AIX (which does provide rpm) is probably closer to LSB conformance than 99% of Linux distros.


----------



## mark_j (May 7, 2021)

vigole said:


> What's the solution to Linux package mess: Choose the best one (pacman, apt-get, or whatever it is) across all distros, not to build a new one! The latter leads to more mess.
> It's very simple. Isn't it?


I believe systemd-packageD is in the works. Stand by!


----------



## Jose (May 7, 2021)

kpedersen said:


> The LSB did specify that rpm had to be present in conforming implementations. It would have been somewhat cool if no matter what package system a distro went for, they always had an rpm plugin to make it integrate consistently with the "standard".
> 
> Of course that didn't happen.
> 
> I think at this point AIX (which does provide rpm) is probably closer to LSB conformance than 99% of Linux distros.


Rpm is so two package managers ago. All the cool kids are using dnf nowadays. It's clear whomever named that tool never participated in a race.


----------



## mark_j (May 7, 2021)

Jose said:


> Rpm is so two package managers ago. All the cool kids are using dnf nowadays. It's clear whomever named that tool never participated in a race.


Does this not point to something? I'm generalising here, but, something common to a lot of linux zealots and developers, that change is great just because they can?

While Torvalds might have a mantra of don't break userland, their userland has a mantra of break it at all costs. I don't know why he bothers. 

Their psyche seems to be, if it's more than 3 years old, has zero bugs and is not getting fresh additions then it's time to scrap it and build something else. Then the cycle repeats. SirDice hinted at this in #195.

Don't get me wrong, change is ok, to a point, but adding something like linux-containers to FreeBSD when we already have a very mature alternative, just makes no sense. It makes even less sense if we have developers' scarce resources used in pursuit of such a goal where the only perceived benefit is FreeBSD is more linux-like.


----------



## Jose (May 7, 2021)

No one ever made a name fixing bugs in Rpm or ifconfig. Being the person who wrote the new Linux IP tools looks a lot more impressive on a resume.


----------



## Alain De Vos (May 7, 2021)

You have to mount the linproc filesystem.
Or you have to mount the proc filesystem.
Or you have to use pulse-audio because sndio is not supported or does not work well ..


----------



## drhowarddrfine (May 7, 2021)

mark_j said:


> something common to a lot of linux zealots and developers, that change is great just because they can?


I love it when I see postings where people complain about something that's been around a long time and used by everybody but want it replaced because it's old. I'll usually ask them if that thing does what they want it to do and the usual reply is yes. So then I ask for a technical reason why it should be replaced and, if they reply at all, they usually want a graphical interface cause using the terminal is for neckbeards only.

Any replies that use the term neckbeards always tells me the level of person I'm talking to.


----------



## mark_j (May 7, 2021)

Yes drhowarddrfine , I think Jose said it quite eloquently: "No one ever made a name fixing bugs". I think this is possibly these developers' sole motivation. You correctly say, there is no technical reason for a lot of this stuff to be replaced or even added; à la containers.

No one would have heard of Poettering if he was just fixing bugs and making enhancements to sysVinit.


----------



## Deleted member 30996 (May 7, 2021)

mark_j said:


> No one would have heard of Poettering if he was just fixing bugs and making enhancements to sysVinit.


Everyone who is famous has fame, but not everyone famous has infamy. 

When you're notorious you're famous for your infamy. Which is best of all IMO.


----------



## scottro (May 7, 2021)

drhowarddrfine said


> I love it when I see postings where people complain about something that's been around a long time and used by everybody but want it replaced because it's old.



Sounds as if he's been listening to my wife complain to me.  :-(


----------



## patrickjp93 (May 15, 2021)

Jose said:


> Rpm is so two package managers ago. All the cool kids are using dnf nowadays. It's clear whomever named that tool never participated in a race.


*Laughs in zypper*


----------



## patrickjp93 (May 15, 2021)

mark_j said:


> Does this not point to something? I'm generalising here, but, something common to a lot of linux zealots and developers, that change is great just because they can?
> 
> While Torvalds might have a mantra of don't break userland, their userland has a mantra of break it at all costs. I don't know why he bothers.
> 
> ...


Making it drastically easier to port a lot of software to FreeBSD and improve cloud hosting capability is a pretty sweet benefit.

Or do you think Joyent doing EXACTLY that with Triton and being wildly successful with it is just a fluke?

And once again, they are not "Linux Containers." They are application containers. You can implement them with jails or however you like.

Being able to say "give me an app environment with my provided application files in a specified directory path, some support applications (like nginx or Apache) from your native implementation repo, a set of port mappings I provide, a set of DNS resolutions to external resources I provide, and you secure it/lock it down however you like" all with one yaml file and an environment variables list is enormously beneficial to a lot of developers. Gone are the Ansible playbooks and galaxies, gone are the Virtual Box and VMWare cruft, gone are all of the network admin upkeep tasks like renewing loadbalancer and Apache SSL certs (new deployment = new cert) gone are the differences between local and remote software deployment...

As I have tried to make crystal clear, it's supporting OCI, NOT Docker and NOT Linux. Make it native. And don't use straw man args about Linux to shoot down an idea that isn't even tied to Linux.


----------



## patrickjp93 (May 15, 2021)

neel said:


> Here's the truth about Docker: It's a Linux-centric solution for a Linux problem: distro fragmentation. GNU/Linux distros regardless of use case have been incompatible in the most minor ways, making lives hard for application developers. Trying to standardize minor details on Linux has been hard and with little success, while *BSD, Windows, and macOS have avoided these issues for the longest time.
> 
> Docker on the server and Snap/Flatpak on the desktop is a way to hide Linux fragmentation, containing it via OS-level virtualization. In this case, a developer only has to worry about a so-called "single platform". It doesn't solve it at all, just hides it.
> 
> ...


And believe me, I'd love to be in such a BSD-rich world.

And yes, "Docker/Kubernetes compose" from a local development standpoint would essentially be something that consumed a Jailfile, a Beehive file, and a .env file before proceeding to building an image with the jail spec, the application artifacts, hooks for mounted volumes at runtime, and a short list of DNS resolutions and port mappings.

Runtime orchestration/management of the "cluster of jails" is a whole separate kettle of fish (mostly the virtual network nesting being the biggest Gordian Knot), but certainly much of that can be outright copy/pasted out of the Kubernetes codebase and the "only" major changes would be the interface between "BSD_K8s" and the Jails.


----------



## patrickjp93 (May 15, 2021)

Jose said:


> This to me is a bug and not a feature. I've run into too many "engineers" that had no idea how things they had written and deployed worked. It's all fun and games until you run into a problem you can't fix. That is when all the technical debt you incurred by doing things the fast and easy way comes due.


And this is why Service Oriented Architecture and Microservices came to be. Have clear boundaries around spheres of functionality and responsibility so you can effectively reason about a large system. I may package my primary million-line Java app as a monolith, but there are only 4-5 code files that break 500 lines, and it was at least architected with clear separation of concerns baked in. When we're given an assignment to make a change to the code, it's not all that daunting.

Hell the dependency upgrades are the most dreaded tasks by far. And re-packaging the app into Docker/Kubernetes has been a long, arduous journey too, but the ethos remains.

In a BSD-native implementation of Docker/K8s, as long as you, the developer, know it all boils down to installing your app and supporting components into a Jail and registering that Jail with a "cluster" manager once it's up and running, you can recovery from "Docker" breaking.

I don't think anyone here believes we shouldn't know how to go back to brass tacks, but we get paid to be productive, and we get paid to make things reproducible and discard-able. When we succeed at making ourselves redundant, we have achieved excellence in our craft.

ZFS is one such example. It's so good at what it does and is so well-made it practically never needs a patch. It IS the last word on file systems. Eventually it'll need to support exa-scale storage natively, but that probably isn't "hard."


----------



## Jose (May 15, 2021)

patrickjp93 said:


> And this is why Service Oriented Architecture and Microservices came to be...(snip more buzzword engineering)


Go peddle your silver bullets elsewhere. Your Jedi mind trick won't work on me. I have too much real-world experience.


----------



## patrickjp93 (May 15, 2021)

kpedersen said:


> No they can't run. You are mistaken. Pointing to documentation for a feature that relies on Windows Subsystem for *Linux* shows you don't quite understand the limitations of containers and what a Windows binary is. The fact that Microsoft runs Ubuntu in a Hyper-V based VM doesn't make Docker portable!
> 
> Can you point out to me an exact document that shows that Docker can run macOS binaries? Apple would not let it exist for one.
> 
> ...


Wrong, wrong, and wrong. WSL no longer loads a full VM and has not done so for the best part of a year. All it is now is the Linux System Call Table, the package manager of the distro, and the GNU tools in the flavor of the distro. The Hyper V VM is GONE.

And yes, you can do all of that on Docker/Kubernetes. You can build Win32 binaries and run Win32 binaries in a container on either Windows or Linux.

I make no apologies or defenses for Apple's douchebaggery. You've got me there.


----------



## patrickjp93 (May 15, 2021)

Jose said:


> Go peddle your silver bullet elsewhere. Your Jedi mind trick won't work on me. I have too much real-world experience.


Oh for Pete's sake. No one is saying to blindly use automation tools without understanding what's under the hood. And experience is nothing but bias ingrained. Good engineers are good engineers, and experience has very little to do with it.


----------



## shkhln (May 15, 2021)

patrickjp93 said:


> Wrong, wrong, and wrong. WSL no longer loads a full VM and has not done so for the best part of a year.


Proof?



patrickjp93 said:


> All it is now is the Linux System Call Table, the package manager of the distro, and the GNU tools in the flavor of the distro.


And the whole Linux kernel.



patrickjp93 said:


> The Hyper V VM is GONE.


While it's technically possible to patch the (Linux) kernel coLinux-style, that doesn't really bring any advantages over using a hypervisor.


----------



## patrickjp93 (May 15, 2021)

shkhln said:


> Proof?
> 
> 
> And the whole Linux kernel.
> ...


RTFM

Nope, not the whole kernel. Exact same lightweight emulation of ONLY the system call table as you enjoy in FreeBSD.


----------



## shkhln (May 15, 2021)

patrickjp93 said:


> RTFM


O RLY?


----------



## patrickjp93 (May 15, 2021)

shkhln said:


> O RLY?


Specifically for GPU acceleration which even BSD's Linux emulation needs.

Feature parity. Cry more.


----------



## shkhln (May 15, 2021)

You aren't even trying now. Behave in a civil way or you are going to be garbage collected. (This is probably not your first account, right? This all seems too familiar.)


----------



## patrickjp93 (May 15, 2021)

shkhln said:


> You aren't even trying now. Behave in a civil way or you are going to be garbage collected. (This is probably not your first account, right? This all seems too familiar.)


First account. I'm right. For 99% of WSL use cases, zero reliance on Hyper V or any hardware virtualization. You want to use GPU acceleration (thanks entirely to NVidia), you end up needing it (equal stain on BSD space, and everyone here knows it).

You're the one being uncivil. RTFM. Your seniority does not trump your abject and reproducible lack of knowledge in this moment in time. Cry more.


----------



## scottro (May 16, 2021)

Just for the record, rpm and dnf are two different things. rpm is how a *single* package is handled. Then came yum, which handled dependencies and the like. That is now replaced by dnf, which, in horse racing stands for did not finish, so is rather regrettably named. It does a pretty decent job. 
Dang, I dunno how these things turn into fights so often. Yeah, there are Linux haters here, but as the FreeBSD FAQ says, when one says FreeBSD is better or worse than another operating system, that's user opinion only.


----------



## shkhln (May 16, 2021)

patrickjp93 said:


> You're the one being uncivil. RTFM. Your seniority does not trump your abject and reproducible lack of knowledge in this moment in time. Cry more.


Oh well. Your choice.

(I'll note here to people casually reading this thread: we _do_, in fact, have full 3d acceleration under Linuxulator with Nvidia/Intel/AMD and there is a good chance we are going to have some CUDA support there as well.)


----------



## patrickjp93 (May 16, 2021)

shkhln said:


> Oh well. Your choice.
> 
> (I'll note here to people casually reading this thread: we _do_, in fact, have full 3d acceleration under Linuxulator with Nvidia/Intel/AMD and there is a good chance we are going to have some CUDA support there as well.)


*Requiring HyperV under the hood, which 85% of the people reading this thread will not know the significance of.

That does not change the fact WSL DOES NOT (and I implemented 20% of it and know for a fact) use Hyper V anymore for anything but GPU acceleration.

If you pick it up and use it for "standard" workloads, you will never pay the price of hardware virtualization on WSL anymore assuming you updated anytime past roughly March last year.


----------



## shkhln (May 16, 2021)

patrickjp93 said:


> and I implemented 20% of it and know for a fact


You implemented what?


----------



## patrickjp93 (May 16, 2021)

shkhln said:


> You implemented what?


WSL's bootstrap is partially (almost half, 48.4% by line count) mine.

I remapped half of the linux system calls to NT's and resolved the transactions applicable. That comes out to roughly 19.4% of WSL's codebase now that Hyper V is gone for everything BUT GPU acceleration, which is detected by reflection and essentially something no one pays for.


----------



## shkhln (May 16, 2021)

Let me ask you one last time: where exactly those lightweight skip-the-Linux-kernel syscalls are documented? Surely, it must be good for marketing to make a few blog posts about them? Especially if that makes WSL so much faster? (FYI, I never made this argument, I don't care about Hyper V performance overhead in the slightest.)


----------



## patrickjp93 (May 16, 2021)

shkhln said:


> Let me ask you one last time: where exactly those lightweight skip-the-Linux-kernel syscalls are documented? Surely, it must be good for marketing to make a few blog posts about them? Especially if that makes WSL so much faster? (FYI, I never made this argument, I don't care about Hyper V performance overhead in the slightest.)


Not my fault the Customer Relations team at Microsoft sucks. It's still in the code\. Or is disassembly and decompilation too hard for the almighty BSD dev...?


----------



## shkhln (May 16, 2021)

So, you were repeatedly telling me to RTFM machine code? Makes sense, thank you. I'm assuming you can't confirm you are working for Microsoft as well? NDA and stuff, right?


----------



## patrickjp93 (May 16, 2021)

shkhln said:


> So, you were repeatedly telling me to RTFM machine code? Makes sense, thank you. I'm assuming you can't confirm you are working for Microsoft as well? NDA and stuff, right?


That contract ended last year in June.

And no, I told you to read the decompilable code that isn't remotely obfuscated.


----------



## Deleted member 30996 (May 16, 2021)

patrickjp93 said:


> *Laughs in zypper*


What are you doing with your face by his zypper?


----------



## bakul (May 16, 2021)

patrickjp93, Have you looked at the recently announced support for FreeBSD jails in containerd? Isn't that more or less what you want?


----------



## mark_j (May 16, 2021)

patrickjp93 said:


> Making it drastically easier to port a lot of software to FreeBSD and improve cloud hosting capability is a pretty sweet benefit.


Ok, I'll bite. How does this make software 'drastically easier to port'?


patrickjp93 said:


> Or do you think Joyent doing EXACTLY that with Triton and being wildly successful with it is just a fluke?



So what's the issue?



patrickjp93 said:


> And once again, they are not "Linux Containers." They are application containers. You can implement them with jails or however you like.



And once again, what's the big benefit? Oh wait, isn't it jails? Me, I like clonos.



patrickjp93 said:


> Being able to say "give me an app environment with my provided application files in a specified directory path, some support applications (like nginx or Apache) from your native implementation repo, a set of port mappings I provide, a set of DNS resolutions to external resources I provide, and you secure it/lock it down however you like" all with one yaml file and an environment variables list is enormously beneficial to a lot of developers. Gone are the Ansible playbooks and galaxies, gone are the Virtual Box and VMWare cruft, gone are all of the network admin upkeep tasks like renewing loadbalancer and Apache SSL certs (new deployment = new cert) gone are the differences between local and remote software deployment...


Yes, yes. We know what you're selling, we've said we're not buying, your foot's jammed in the door and we're about to loose the dog on you... 


patrickjp93 said:


> As I have tried to make crystal clear, it's supporting OCI, NOT Docker and NOT Linux. Make it native. And don't use straw man args about Linux to shoot down an idea that isn't even tied to Linux.


OCI is linux foundation sponsored, it is, as I said, free to fund OCI containers in Freebsd; perhaps your efforts would be best served canvassing them rather than USERS of an OS?
Unless things have changed overnight, that foundation is about increasing linux market share, not freebsd's. Perhaps freebsd should adopt the lsb as well?


----------



## kpedersen (May 16, 2021)

patrickjp93 said:


> For 99% of WSL use cases, zero reliance on Hyper V or any hardware virtualization.


Most of the documentation contradicts what you are saying. Even the Docker docs are very clear on the role of Hyper-V in Docker's Linux emulation: https://docs.docker.com/desktop/faqs/#why-is-windows-10-required

As for the GPU passthrough, they are again very clear about this:



> Starting with Docker Desktop 3.1.0, Docker Desktop supports WSL 2 GPU Paravirtualization (GPU-PV) on NVIDIA GPUs. To enable WSL 2 GPU Paravirtualization, you need


In particular, prior to Docker 3.1.0, you still needed WSL 2 which still needed Hyper-V.

But all this aside, some version of Docker supports VirtualBox as a driver: http://docs.docker.oeynet.com/machine/drivers/virtualbox/

So use it and be happy. However I still recommend you will be better off running Linux software on Linux. But better yet, write portable / cross-platform software and run it natively in a FreeBSD Jail. No fuss, no muss and no Docker bullsh*t


----------



## mark_j (May 16, 2021)

bakul said:


> patrickjp93, Have you looked at the recently announced support for FreeBSD jails in containerd? Isn't that more or less what you want?


Is it?

From what I've discerned, apparently anyone who uses FreeBSD is bogged down in a quagmire of mismatched Python versions, or FreeBSD just needs it:  "I do, however, think BSD would benefit immensely from having an Open Container Initiative-compliant/compatible "container" runtime.".

Apparently, this  lack of containers is "getting in the way of developer onboarding (*OMG*!)  and productivity into a software product built to run on a completely different OS from their company-provided machine." (My comments in *bold*).

I freely admit, I couldn't care either way if FreeBSD spends time, money and effort incorporating this container-thingy into the OS. I also freely admit I've never used docker or the like. I have no professional interest in them (they seem to suit a web presence more than anything) and find them just about as interesting as dog poo. I am not a javascript programmer; perhaps this is more their domain? 

I just don't see how this improves FreeBSD any. Maybe as you, bakul, pointed out, it's up to a third-party to build it. If people want it, they will help and rush to improve it and implement it in FreeBSD. A win/win for thems container nerds.


----------



## bakul (May 16, 2021)

mark_j said:


> From what I've discerned, apparently anyone who uses FreeBSD is bogged down in a quagmire of mismatched Python versions, or FreeBSD just needs it: "I do, however, think BSD would benefit immensely from having an Open Container Initiative-compliant/compatible "container" runtime.".


I read the quoted part differently than you: By "BSD would benefit having an OCI compliant runtime" I believe patrickjp93 simply meant such a thing would attract _new_ corporate users to FreeBSD as they would have potentially a much more stable alternative to Linux containers. At least to me that quoted sentence doesn't mean that the FreeBSD project has to provide it. In fact most of us can happily ignore such a thing and continue using FreeBSD as we always have or want.

I do believe FreeBSD already has the necessary pieces for someone to build an OCI compliant runtime as shown by containerd support for jails (not having looked at it I don't know how good it is but at least we have some sort of an existence proof).

I believe the reason for popularity of containers is because people have figured out that especially services a require lot more than running a server program as a daemon. And orchestration layers such as kubernetes have sprung up to centrally manage, monitor  & scale up a set of such services. [Though I do think kubernetes is rather over-engineered]

I'd love it if I can standup mail, dns, dhcp, web, etc. service jails with minimal work. Existence of tools to do this does not mean everyone has to use them or even that they will have any effect on their work (unless they turn out to be genuinely useful).


----------



## Jose (May 16, 2021)

bakul said:


> I believe the reason for popularity of containers is because people have figured out that especially services a require lot more than running a server program as a daemon. And orchestration layers such as kubernetes have sprung up to centrally manage, monitor  & scale up a set of such services. [Though I do think kubernetes is rather over-engineered]


These two sentences contradict each other. Abominations like Kubernetes exist precisely because running a service requires more than what the containerization technologies provide. Why layer "orchestration" hacks on top of the broken architecture instead of stepping back and rethinking the approach? Because it's dogma that containers are the only proper way of deploying your stack. It's religion, not engineering.


----------



## Crivens (May 16, 2021)

Crivens said:


> Obligatory xkcd


^ still valid...


----------



## Beastie7 (May 16, 2021)

Can't you just slap something like puppet on top of jails and have a Docker/Kubernetes-like system? I never really understood the hype about such broken software.


----------



## Alain De Vos (May 16, 2021)

Configiration is a big thing.


----------



## ct85711 (May 16, 2021)

Just thinking it over, even if we slap something on top of a jail; it still wouldn't work.  For the most part, it's like jails in that you have to have a base distro to run everything off from.  Considering, all of the docker packages anymore are all based off like alpine/ubuntu/fedora or so; you'd end up have a linux distro inside a jail (so you still end up having the same issues as you do with linux-compatibility layer).  Even after that, think about all the setup commands the yaml files have...  First it downloads and sets a distro, then it usualy updates and downloads the dependencies (commonly using apt-get), then copies custom configurations and the intended package...  The key issue is going to be the apt-get (which is a linux thing, not freebsd). 

In the end, even if you get docker working, you end up reconfiguring everything all over again anyways.  That that point is it even worth it, since you can't just distribute it between linux and bsd without enclosing everything with a complicated if/else?  You have to remember, the Freebsd kernel is not directly compatible with the linux kernel, same with the compiled libraries.  After everything is done, it is the same of comparing greek to french.  They may share some common words, but beyond that they are 2 completely different languages.


----------



## kpedersen (May 16, 2021)

ct85711 said:


> That that point is it even worth it, since you can't just distribute it between linux and bsd without enclosing everything with a complicated if/else?  You have to remember, the Freebsd kernel is not directly compatible with the linux kernel, same with the compiled libraries.


Yup. Where it is going to get fun is when the 2029 Linux kernel breaks compatibility with 2021 Linux distro userland. It will be just as impossible to support a "neatly packaged" Docker container as on FreeBSD or on some ancient SPARC64 processor.

Docker guys either don't understand the technology behind it (a set of ratty scripts around Linux chroot/cgroups) or they are simply kidding themselves.

I would actually recommend they try to get a 1st generation Docker container running on a current machine. Then they can really see the failure of the entire solution.


----------



## drhowarddrfine (May 16, 2021)

Aaaahhhh! There's gotta be a button on this thing for that thing! --Homer Simpson

Whenever I read conversations about using this tool with that tool on top of some other tool to make another tool work, it makes me think things have gone all wrong.


----------



## patrickjp93 (May 19, 2021)

bakul said:


> I read the quoted part differently than you: By "BSD would benefit having an OCI compliant runtime" I believe patrickjp93 simply meant such a thing would attract _new_ corporate users to FreeBSD as they would have potentially a much more stable alternative to Linux containers. At least to me that quoted sentence doesn't mean that the FreeBSD project has to provide it. In fact most of us can happily ignore such a thing and continue using FreeBSD as we always have or want.
> 
> I do believe FreeBSD already has the necessary pieces for someone to build an OCI compliant runtime as shown by containerd support for jails (not having looked at it I don't know how good it is but at least we have some sort of an existence proof).
> 
> ...


THANK YOU!!!

Finally a pragmatist ignoring the NIH bias.



Jose said:


> These two sentences contradict each other. Abominations like Kubernetes exist precisely because running a service requires more than what the containerization technologies provide. Why layer "orchestration" hacks on top of the broken architecture instead of stepping back and rethinking the approach? Because it's dogma that containers are the only proper way of deploying your stack. It's religion, not engineering.


It's not poor engineering at all. It's separation of concerns.

Kubernetes provides you the tools for rapid provisioning of storage volumes (ephemeral, persistent, w/e), claims-based assignment & security on those volumes, network isolation at the cluster level, a "Factory" pattern to deploy Services (Application Images aka Docker Containers that you built separately), secure patterns for injecting secrets, and an administrative tool suite to manage the cluster scaling and health monitoring. And all of this is automated in an Open Source codebase not reliant on the cleverest esoterica of Bash scripts.

The architecture isn't broken in the least. And no one is saying containers are the only way, but they provide numerous benefits to smaller operations (or those who ended up with a poorly responsive cloud provider that can't keep up provisioning more servers) that don't have in-house datacenter resources to be able to grow rapidly and predictably on very stable infrastructure. And once you have that assurance and capability, there are very few reasons to go back to in-house big iron management, especially when you know it's a never-ending battle of upgrading hardware and middleware in a brittle in-house datacenter that will NEVER have the redundancy and safety of dedicated providers.

You'll always be paying some tiny performance and per-unit scaling cost for that container abstraction rather than running on bare metal, but that's where things like Focker can come in and even erase that cost, albeit only about half as feature-rich as K8s right now (which is not disparaging the author, because he's one of a literal handful of people working on it)

It's the Infrastructure as Code and Environment Setup Automation being highly portable which is useful/valuable. Focker is about halfway there.

And as for kpedersen , I just did exactly that: running a V1 Docker Container on my machine. Trivial. I'm not sure what problems you expected, but the external resource integrations, volume mounting, and virtual networking all snapped together about as simply as one could ask for starting from a clean slate.

The value is not in the underlying implementation. Yes, Docker and Kubernetes are deeply flawed. Shall we put a final nail in their coffin by providing the better and (perhaps) final word on automated deployment and orchestration of services? That would seem the more productive thing to do than get mired in NIH-ism.


----------



## kpedersen (May 19, 2021)

patrickjp93 said:


> NIH


NIH is the process of developing your own solution rather than using an existing or standard approach.

You see, we didn't do that. We already had Jails *long* before Docker (and LXC/cgroups) was being spec'ed up.
Once Docker matures, it might wrap around Jails. However it will probably be gone long before that. So it is more correct to say that Docker itself is the result of NIH.

Just because Docker (Inc.) made some noddy "Open Container Initiative" years after Jails / Zones / LPARs were in wide use as a means to sell their cloud services means very little. So many people new to the game don't see that .

But FreeBSD is flexible. We do support Docker. Install VirtualBox and away you go. That is exactly what they did for first gen containers on non-Linux operating systems.

(I even remember when Docker was called dotCloud. They did consider FreeBSD Jails but Linux was an easier platform to monetize because it was more widespread. The actual technical merits were irrelevant. Linux used OpenVZ at the time and that was much more basic than LXC).



patrickjp93 said:


> And as for kpedersen , I just did exactly that: running a V1 Docker Container on my machine. Trivial. I'm not sure what problems you expected, but the external resource integrations, volume mounting, and virtual networking all snapped together about as simply as one could ask for starting from a clean slate.


It errors saying that the running kernel is too recent for the current libc.

Something similar to this but going the other way. Unlike FreeBSD, the Linux kernel is rarely compiled with backward compatibility in mind:
https://sourceware.org/legacy-ml/libc-help/2017-01/msg00011.html

Also, the very, very first containers will be 32-bit binaries and Docker doesn't really handle multi-lib. In particular many Linux platforms don't.


----------



## a6h (May 19, 2021)

kpedersen said:


> NIH is the process of developing your own solution rather than using an existing or standard approach.


Is NIH an initialism? What does NIH stand for? I've never heard of it.


----------



## mark_j (May 19, 2021)

patrickjp93 said:


> THANK YOU!!!
> 
> Finally a pragmatist ignoring the NIH bias



Well you seem to have missed the point, most likely intentionally. It has nothing to do with not being invented by FreeBSD. That's an idiotic comment. Do you know FreeBSD has been using GNU userland tools up until only recently, or provides Linux emulation? No you conveniently disregard the facts in your head-first evangelical salesmanship of Linux containers.

bakul  may be right that it will attract more corporate users to FreeBSD, but so what? Prove it.
Unlike Linux, and as I wrote earlier, FreeBSD does not have the man-power and funding  to run off emulating everything that comes out ofthe huge "corporation" that is Linux.




patrickjp93 said:


> datacenter resources to be able to grow rapidly and predictably on very stable infrastructure. And once you have that assurance and capability, there are very few reasons to go back to in-house big iron management, especially when you know it's a never-ending battle of upgrading hardware and middleware in a brittle in-house datacenter that will NEVER have the redundancy and safety of dedicated providers


You really do think the world revolves around cloud computing? I've never read so much guff; you've got to be a cloud salesman surely?
This space is already taken by Linux. Why waste developer's time pursuing this to just be another, lesser known provider? Where's the rate of return in this? You might be a good salesman but you stink at accounting... 

Don't get me wrong, I admire your tenacity, but steadfast ignorance of reality isn't doing you any favours. 

I wrote earlier and I will write again, put your efforts into convincing the Linux foundation to fund it.


----------



## kpedersen (May 19, 2021)

vigole said:


> Is NIH an initialism? What does NIH stand for? I've never heard of it.


Not Invented Here [syndrome]

It is often derogative to suggest that someone just wants to hack on their own code rather than integrate with existing software. However in practice it can also be very useful to be in the position to reimplement the solution entirely if it is a mess.

The usage can be:


> I suffered a bit from NIH because that string splitting function we were originally using was dragging in 12 different scripting languages and 100+ libraries. Now it uses only 5 lines of code and zero deps.


----------



## mark_j (May 19, 2021)

I always thought it meant No Idea How....


----------



## rootbert (May 19, 2021)

mark_j said:


> bakul  may be right that it will attract more corporate users to FreeBSD, but so what? Prove it.


prove would be some of my clients then, though that number is negligable. Probably even more companies at least could change their reproducible CI/CD toolchain and also their server fleet to FreeBSD then => more users = more feedback/quality improvement and finally more software vendors stop ignoring FreeBSD. Companies starting over would at least have a choice and not be forced to use Linux. It might not be what the majority of this discussion want, but there are some users like me who do.



mark_j said:


> I wrote earlier and I will write again, put you efforts into convincing the Linux foundation to fund it.


Why should anyone try to convince the Linux Foundation to fund a (probably non-GPL) implementation on FreeBSD? I would not even ask a fancy glittering startup like Docker Inc. for this (the implementation of docker sucks, they should have done it the podman way right from the start), but the FreeBSD Foundation. The concept of application containers is nothing Linux specific but just something jails lack - and before the whole discussion starts all over again about why application containers are different than jails I mean the whole infrastructure to work with those containers namely declarative container configuration, private hub/repo, syncing deltas etc. (and for those who would like to suggest scripting the whole thing and provide jail tarballs on nginx: NO, just NO)


----------



## shkhln (May 19, 2021)

rootbert said:


> concept of application containers


I don't like this terminology drift. The word "container" doesn't come from Docker, it comes Solaris and basically means holistic sandboxing. It's not a container in the cargo sense. The proper word for packaged binaries would be "image".


----------



## Beastie7 (May 19, 2021)

To piggyback. There is no containerization in Docker. It's cgroups and namespaces, that's it. Security is not even a design constraint for docker. 

Docker should be called CFEngine++.


----------



## ct85711 (May 19, 2021)

Either way, I'd be interested in seeing, how do you work around the main part that breaks most docker images on FreeBSD; ie the library/executable incompatibility...  As either way we go, putting a wrapper over Jails are not the problem,  but more of the fundamental issue that linux can't use FreeBSD pre-compiled stuff, and there is only partial support the other way around.  So you are stuck having the entire docker image(all the way from and including the base image of Kali or other base linux distro) being recompiled every time you want to you run; which means you can't just "spin up" another docker image and have it working in minutes.


----------



## drhowarddrfine (May 19, 2021)

mark_j said:


> You really do think the world revolves around cloud computing?


I like reminiscing about back in the day when we would just call them server farms.


----------



## drhowarddrfine (May 19, 2021)

What we should really do is not care what Linux does. What does it matter? We do our thing on FreeBSD and do it better than they can. So...don't care. 

That should be a topic category for things like this. Don't Care.


----------



## kpedersen (May 19, 2021)

ct85711 said:


> Either way, I'd be interested in seeing, how do you work around the main part that breaks most docker images on FreeBSD; ie the library/executable incompatibility...


You are basically asking why you can't run a program compiled for one operating system on another operating system.

People have been asking this question since MS-DOS days (and earlier). The answer is: You can't. Not properly. It is neither operating systems fault. This is life.

Docker has overreached slightly and said it can. But ultimately it relies on emulating one operating system. Linux. One operating system, these forums are specifically not about


----------



## ct85711 (May 19, 2021)

I'm well aware we can't run stuff from one OS on another; but in the end; the stuff running on Docker is compiled to run on Linux.  If anyone wants to get docker to run on FreeBSD, they will have to solve the fundamental issue (which like you said, can't be done properly).  The only way, is giving up all the advantages that Docker gives.  Even if, you do the ugly solution of putting if statements in the docker file, and detect which OS you are running, doesn't work in the end.  As you are still back to be beginning, of not having a common base that everyone has.


----------



## zirias@ (May 19, 2021)

vigole said:


> Is NIH an initialism? What does NIH stand for? I've never heard of it.


*N*ot *I*nvented *H*ere

edit: I just see my browser didn't show me a bunch of newer replies  But, adding to kpedersen's reply, NIH also applies to standards and concepts, not only to existing software  (that's what I was referring to when talking about Powershell in this Linux thread. MS doing their own implementation probably made sense, but they could have followed the POSIX shell standard as a common baseline)


----------



## rootbert (May 19, 2021)

ct85711 said:


> Either way, I'd be interested in seeing, how do you work around the main part that breaks most docker images on FreeBSD; ie the library/executable incompatibility...  As either way we go, putting a wrapper over Jails are not the problem,  but more of the fundamental issue that linux can't use FreeBSD pre-compiled stuff, and there is only partial support the other way around.  So you are stuck having the entire docker image(all the way from and including the base image of Kali or other base linux distro) being recompiled every time you want to you run; which means you can't just "spin up" another docker image and have it working in minutes.


I am primarily talking about running FreeBSD container images. And no I would not recompile everything, just the layer that has changed, like with Linux/OCI containers.
Running Linux containers on this fictional FreeBSD jail-derived tech would probably work for quite some images, and would of course be cool, but is at least not my primary interest.


----------



## kpedersen (May 19, 2021)

ct85711 said:


> Even if, you do the ugly solution of putting if statements in the docker file, and detect which OS you are running, doesn't work in the end.


Docker is really just a distraction and a hinderance to actually solving this problem. Containers could perhaps be provided in source form or as you mentioned a more cross platform rules system to pull in the correct packages per platform.

Linux users and Docker Inc, are basically undermining this more portable development style. Oh well, at least it isn't Microsoft anymore. They were completely constricting.


----------



## mark_j (May 19, 2021)

drhowarddrfine said:


> I like reminiscing about back in the day when we would just call them server farms.


Too true. I remember first reading about 'the cloud' via Sun and their world vision of providing computing resources to business. Then the tech bubble exploded and it died down.
A good article related to the cloud posted, I think, by vermaden: https://martin.kleppmann.com/2021/04/14/goodbye-gpl.html


----------



## mark_j (May 19, 2021)

rootbert said:


> prove would be some of my clients then, though that number is negligable. Probably even more companies at least could change their reproducible CI/CD toolchain and also their server fleet to FreeBSD then => more users = more feedback/quality improvement and finally more software vendors stop ignoring FreeBSD. Companies starting over would at least have a choice and not be forced to use Linux. It might not be what the majority of this discussion want, but there are some users like me who do.


Then fund it.
Seriously, if it's such a benefit, ante up the money and I'm sure someone will do it.

I'm not attacking your stance, you're entitled to your view and perspective, BUT, I find the idea of chasing corporate clients by picking up that latest thing from Linux (and which it already dominates anyway), a fruitless, pointless and costly exercise.

But, hell, if you want to lobby the FreeBSD foundation and the Linux foundation (which sponsors this OCI - of course, because it's on Linux), then I will try it out in FreeBSD 15.


----------



## sadaszewski (Aug 17, 2021)

Should anyone be interested, I have released Focker 2.0 today, addressing many issues raised by the users over the last year and laying a much more solid foundation for future development. https://github.com/sadaszewski/focker. Don't forget to leave a Star if you haven't already


----------



## miguelc (Aug 29, 2021)

Focker looks interesting,  I've played with it sometime ago, I'll surely take a look a 2.0, but for the folks that want some sort of native docker on FreeBSD, I think your asking in the wrong place. 

IMHO and as many said it would take a huge investment from the Fundation into something where Linux already rules and will continue todo so,  in the Clould world Kubernets is in fact very very interesting it become more and more mature and with would be a serious investment because docker and K8s are built from Linux. 

It's been sometime since I use FreeBSD for desktop/laptop daily but I do see the usefulness of being able to run docker images in Freebsd,  but tbh isn't that something to ask from Docker not FreeBSD. 
I work mainly on macOS and docker implementation there use a VM, and in fact these this its actually using a port o xhyve which guess what? is in turn a port of bhyve.
All the Docker would need to do if there's enough interest (which I don't think there is) is to build a UI that works on any Unix, or at most have the cmd line be able to do what it's doing  in macOS and just "pass all the cmds to the VM". 

But for now what we can do is to simply implement that ourselves... If I wanted/needed to do work with docker and FreeBSD was my main desktop I would just setup a bhyve VM.... again this is exactly how it works on macOS and instead of complaining to Apple people are quite happy there's at least a way.


----------



## kpedersen (Aug 29, 2021)

miguelc said:


> again this is exactly how it works on macOS and instead of complaining to Apple people are quite happy there's at least a way.


Yep and until Microsoft's semi-recent EEE strategy to put WSL on Windows, the official Docker Win32 installer would install VirtualBox and run Linux on there too. People were happy with that (FreeBSD also has VirtualBox in ports).

Docker isn't a portable standard or even a program which many people seem to think. It is basically just Linux automation. And people choose not to use Linux for many reasons.


----------



## grahamperrin@ (Aug 29, 2021)

mark_j said:


> … A good article related to the cloud … https://martin.kleppmann.com/2021/04/14/goodbye-gpl.html



A little discussion of the article: <https://news.ycombinator.com/item?id=26834157>


----------



## arcalus (Dec 22, 2021)

drhowarddrfine said:


> For all those who wish Docker ran on FreeBSD and praise Linux for having such a thing--and the reasons I don't think this is off topic, here is another in a long list of reasons to not even use Linux at all--and we see such things too often as in Windows, too.
> 
> Goodbye Docker and Thanks for All the Fish
> 
> ...



I know this is so stale, but FYI your entire premise was faulty. Kubernetes uses Docker, Docker didn't adopt Kubernetes. 

I'm sure by now you have realized Docker is doing just fine.


----------



## drhowarddrfine (Dec 22, 2021)

arcalus said:


> I know this is so stale, but FYI your entire premise was faulty.



Not my article. Nor did I write any of the many articles online about Docker's demise. But I would never stoop so low as to use Linux so it doesn't affect me and I don't bother to follow such things anymore except if I stumble upon them.


----------



## Andriy (Sep 28, 2022)

I know that this is an old topic, but has anyone here tried this https://lists.freebsd.org/archives/freebsd-jail/2022-May/000129.html ?


----------



## drhowarddrfine (Sep 28, 2022)

Andriy It's also not on topic for this thread. You should start a new one.


----------

