# How to compare bastille-bsd, ezjail, cbsd, pot, iocage,ezjail



## Alain De Vos (Oct 4, 2022)

What are the strong and weak points of each system ? Or how do they compare ? When to use one or when another.


			https://www.bsdstore.ru/img/freebsd-jail-chart-2022.png


----------



## wolffnx (Oct 4, 2022)

I can talk from ezjail and bastille , ezjail is solid and works well , but bastille see more the jails as "containers" and had more functions that ezjail, for example, the use of ZFS , ejecute commands on jails without enter the jail
I been a long time user of ezjail, but now I am using bastille


----------



## Ole (Oct 4, 2022)

It's very hard to compare because you have to know all the products well ;-)

additional info:









						Compare Bastille | BastilleBSD
					

Compare Bastille With Other FreeBSD Jail Managers




					bastillebsd.org
				











						Possible comparison with other jail systems? · Issue #224 · cbsd/cbsd
					

I like cbsd very much, and I would like to point it to other people (for example, in similar threads: https://www.reddit.com/r/freebsd/comments/7ex8bw/opinion_jail_utility_cbsd_vs_ezjail_vs_iocage/...




					github.com


----------



## SirDice (Oct 4, 2022)

Used EZJail for a long time too. But the developer stopped a long time ago (there were some issues between the developers of Ezjail and QJail, not a pretty story). So it's getting a bit stale nowadays. Lacks a lot of modern features and still uses the deprecated `jail_*` variables. Switched to Bastille some time ago, and I'm very pleased with it (templates are a really cool feature).


----------



## Alain De Vos (Oct 4, 2022)

If i'm correct bastille is for jailing , i.e. a name-space, ware-as cbsd is also for bhyve and virtualisation ? Not ?
It would be nice to know "Dan Langille" takes on this.
​


----------



## Purkuapas (Oct 4, 2022)

Alain De Vos said:


> If i'm correct bastille is for jailing , i.e. a name-space, ware-as cbsd is also for bhyve and virtualisation ? Not ?
> It would be nice to know "Dan Langille" takes on this.
> ​


pot/bastillebsd/iocage/ezjail for jail only, whereas cbsd can also manage virtual machines (like vm-bhyve).

You can compare the open issues per project (for bastillebsd, for cbsd, for pot , for iocage): it shows what users are missing (feature request)


----------



## gotnull (Oct 4, 2022)

This is my very little opinion which is far far but really far from an expert one. 
I am still discovering the jail world of FreeBSD, so far I tried iocage, bastille and lately pot. 

Iocage is nice, the work is done automatically and quickly. 
Having plugins is a good idea unfortunately there are not ton of them, I was oddly surprised that they were not much popular.   
Sadly its development entered in a slow state, PR are kind of stuck, issues tickets are growing. I hope it won't die.
https://github.com/iocage/iocage/issues/1251 

Bastille looks okay, I didn't play a lot with it for some reason and I didn't like the fact that it was always showing a warning message about pf while ipfw was used instead. It was like it was expected pf no matter what, then I moved on something else at that time and forget about bastille. 
That being said I appreciated the videos the author did about it on YT, or the quick articles on the bastille blog because they help and this type of gesture are always welcome. 

Pot is the last one I play with, I don't have a lot to say about it since I still discovering it, but so far so good. 
Note that jails are deployed slowly because of the xz's archive. Syntax is okay as well as bastille. 
From what I understand it's different kind from the two mentioned before, but since it is above my skills there are certainly good people here to talk about it better than I can.

Overall I just do simple things  when using jails because I still struggle with network and more specifically firewall, add vnet into the equation and my brain freezes.
As an average user this is probably my bigger weak point.
So I try to cure myself I bought "Network For System Administrators" and "FreeBSD Mastery Jails" but damn that's a lot to process I won't lie!
I just wanted to have fun at first 




Alain De Vos said:


> It would be nice to know "Dan Langille" takes on this.


It looks like Dan is more into mkjail I think he prefers when things are simple even if he has to put his hands into it, I may be wrong about it this is only my own deduction.








						GitHub - mkjail/mkjail
					

Contribute to mkjail/mkjail development by creating an account on GitHub.




					github.com


----------



## Alain De Vos (Oct 4, 2022)

For jails, you can also go bare-bones.
Just write a manual /etc/jail.conf i use for poudriere in a jail
cat jail.conf

```
# Common configs for all jails
path = "/jails/$name";
host.hostname = "$name";
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
persist;
ip4 = inherit;
ip6 = inherit;
mount.devfs;
mount.fdescfs;
allow.mount;
allow.mount.devfs;
allow.mount.fdescfs;
allow.mount.nullfs;
allow.mount.tmpfs;
allow.mount.procfs;
allow.mount.zfs;
enforce_statfs=1;
children.max=100;
allow.socket_af;
allow.raw_sockets;
allow.chflags;
allow.sysvipc;
a {
devfs_ruleset="20";
}
```

/etc/devfs.rules

```
[a=20]
add include $devfsrules_hide_all  
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add path 'mixer*' unhide
add path 'dsp*' unhide
add path 'zfs*' unhide
```


----------



## Ole (Oct 4, 2022)

Purkuapas said:


> pot/bastillebsd/iocage/ezjail for jail only, whereas cbsd can also manage virtual machines (like vm-bhyve).


You're wrong. CBSD can also manage XEN and QEMU/NVMM ;-). But these are virtual machines, and therefore irrelevant in this topic. As far as containers, CBSD can create ARMvX/RISCV... jails on x86-64 arch using qemu-user-mode ( usefull for testers/porters ). Personally, it seems to me that due to the lack of support from commercial companies (or FreeBSD Foundation), all such "jail managers" reach the *same* stage and die. Things like "create", "clone", "start/stop", "delete", "import" and "export" containers are the most basic primitives, but we don't see anything beyond that (for example: scaling, service-discovery/healthing, DRS ( dynamic resource scheduling ), public registry ( templates like:


```
pkg install <portname>
service enable <portname>
service start <portname>
```

^ personally, I don't find this useful enough for serious installations (you have to configure the service, right?))
Perhaps the 'pot' project tried to jump over this "great desert" by adopting Hashicorp Nomad. There is also a higher-level abstraction above CBSD in the form of an API that can perform DRS functions (for example: you can create environments on servers that are located in different data centers, with load distribution (on creation) and an interconnected overlay network via  vxlan(4) ). But this, like Nomad, is a different product/service. Otherwise, all managers are the same, only the interface differs (and a set of bugs), to which different users give certain preferences.


----------



## Lamia (Oct 5, 2022)

It would be beneficial to all if one of more members could share their experiences  (or pieces of experience) on the two contending clustering suite for *BSD. And if more than these two, please enlighten me.
1) Nomad/Pot: Here is an additional link: https://honeyguide.eu/posts/virtual-dc1/
2) Corosync/Pacemaker:








						FreeBSD Cluster with Pacemaker and Corosync
					

I always missed ‘proper’ cluster software for FreeBSD systems. Recently I got to run several Pacemaker/Corosync based clusters on Linux systems. I thought how to make similar high availability solutions on FreeBSD and I was really shocked when I figured out that both Pacemaker and Corosync tools...




					forums.freebsd.org
				








						How To Create a High Availability Setup with Corosync, Pacemaker, and Reserved IPs on Ubuntu 14.04  | DigitalOcean
					

This tutorial will demonstrate how you can use Corosync and Pacemaker with a Reserved IP to create a high availability (HA) server infrastructure on DigitalO…




					www.digitalocean.com


----------



## gotnull (Oct 5, 2022)

Alain De Vos said:


> For jails, you can also go bare-bones.
> Just write a manual /etc/jail.conf i use for poudriere in a jail
> cat jail.conf
> 
> ...


Thank you, yep I know it 's what describes the handbook somehow and it reminds me this blog article:


			https://clinta.github.io/freebsd-jails-the-hard-way/
		

Note that I am not against some hand work but for now I try various wrapper just to see how it fits me, I 'll probably go back to the hard way at some point just to see if I can do it in long run, still just for fun haha


----------



## Alain De Vos (Oct 5, 2022)

I think the tools are good when you run alot of jails.

For virtualisation there is vm-vhyve from the churchers,








						GitHub - churchers/vm-bhyve: Shell based, minimal dependency bhyve manager
					

Shell based, minimal dependency bhyve manager. Contribute to churchers/vm-bhyve development by creating an account on GitHub.




					github.com


----------



## Ole (Nov 10, 2022)

Lamia said:


> It would be beneficial to all if one of more members could share their experiences  (or pieces of experience) on the two contending clustering suite for *BSD.


This is a great idea. In two months, I invite you all to a party  - we will celebrate the tenth anniversary of the *CBSD project*..., my ten-year-indie-project .
I plan to write an article about what has been done during this time and where the project is heading. Including an article on building clusters based on FreeBSD+CBSD (yes they exist )

This thread is about jail containers, so a small demo about the possibilities of CBSD in terms of jail clustering: Clustering with CBSD: pool-binded HA/DRS FreeBSD jails ;

As for that link, I'll save it for the future "How to compare vm-bhyve, iohyve, chyves, cbsd" thread in this forum. ¯\_(ツ)_/¯


----------

