# FreeBSD update 9.2->10 ; ezjail config



## _martin (Feb 11, 2014)

Hi, 

Yesterday I upgraded my server from 9.2 to 10.0 using sources. Pretty much business as usual. Everything seemed to be working fine. I did notice a warning when my ezjail jails were starting: 


```
Feb 11 01:43:59 foxi root: /etc/rc.d/jail: WARNING: Per-jail configuration via jail_* variables  is obsolete.  Please consider to migrate to /etc/jail.conf.
```
It does sound like a warm warning. Jails did start, services inside were working as expected. But I did notice something - I was able to control PF within the jail (/dev/pf exists in each jail; here webserver is a jail): 


```
root@webserver:/root # pfctl -d
pf disabled
root@webserver:/root #
```

Uh, oh. What ?
Another example: /dev/ada0 is one disk in rpool: 


```
root@webserver:/root # hd -n 64 /dev/ada0p1
00000000  31 c9 8e c1 8e d9 8e d1  bc 00 7c bb 63 7c 8b 77  |1.........|.c|.w|
00000010  0a 01 de 89 f0 c1 e8 04  83 e6 0f 8e d8 83 c6 f0  |................|
00000020  b8 00 0a bf f0 ff 8e c0  fd 89 f9 41 f3 a4 8e d9  |...........A....|
00000030  8e c1 8b 4f 0a 89 de bf  00 90 01 ce 01 cf 4e 4f  |...O..........NO|
00000040
root@webserver:/root #
```
New 10.0 /etc/defaults/devfs.rules is installed, nothing was changed prior to upgrade (same ezjail configs). 

I do believe that problem is somewhere between keyboard and chair, but this is very unpleasant issue considering the warning I received. I'm rechecking handbook to see what I did wrong, posting here as FYI - maybe somebody else didn't notice this either.


----------



## dpejesh (Feb 11, 2014)

matoatlantis said:
			
		

> yesterday I upgraded my server from 9.2 to 10.0 using sources. Pretty much business as usual. Everything seemed to be working fine. I did notice a warning when my ezjail jails were starting:
> 
> 
> ```
> ...



This is just letting you know that you should migrate to using /etc/jail.conf to configure your jails instead of rc variables.  If I remember right, if you look in /var/run there will be generated files showing you the snippet used for each of the jails.



> But I did notice something - I was able to control PF within the jail (/dev/pf exists in each jail; here webserver is a jail):
> 
> 
> ```
> ...



You probably need to put devfs_load_rulesets="YES" in your rc.conf to get the rulesets to work.

Also, one thing to be aware of if you're using ezjail (or freebsd-update -b for that matter) is that it doesn't properly update your jails (http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/186152).


----------



## _martin (Feb 11, 2014)

Thanks for the tip about new jail configs in /var/run and an update problem. I was actually updating it from sources, so I didn't hit that one.
Right though, warning is harmless and just informs that new config should be used (for which new ezjail script has to be released). 

As everything was working just fine before OS update - jails were not able to see /dev/pf before and /etc/rc.conf is the same as it was on 9.2, I thought problem is with the new jail config location. I didn't have devfs_load_rulesets="YES" on 9.2 and it  was working  fine. 
On 10 it seems this toggle has to be in rc.conf.  

Thanks for your help.


----------



## junovitch@ (Feb 12, 2014)

On FreeBSD 10 devfs rules must be integers rather than names.  You'll have to update your jail configuration files to use numbers instead.

This is what I did, I have some additional rules so I just updated all my files with a sed one-liner.
`sed -i '' -e 's/"devfsrules_jail"/"4"/;s/"devfsrules_jail_with_vpn"/"5"/;s/"devfsrules_jail_with_bpf"/"6"/' /usr/local/etc/ezjail/*`


----------



## zgub (Feb 13, 2014)

Nope, nothing helps. 

With FBSD10 FreeBSD 10 fresh install with sysutils/ezjail, I had to put 
	
	



```
devfs_load_rulesets="YES"
```
 in /etc/rc.conf to hide devices from jails. And then restart devd and jails. It works with numbers and/or names. It would be quite helpful if somebody considered notifying users about a change like this.

Hopefully sysutils/ezjail port maintainers will adapt ezjail to cope with /etc/jail.conf and the devfs surprise


----------



## _martin (Feb 13, 2014)

@junovitch That doesn't seem to be the problem; it's not working if you just switch from names to numbers. It seems devfs_load_rulesets="YES" is a must in /etc/rc.conf. 

I tried to start jail "manually" from command line using `jail` on fresh FreeBSD 10 installation without that toggle in /etc/rc.conf and the same issue occurred. I have yet to find what was changed that it is now needed. 

I'd consider this behavior as a bug. Following command was used: 


```
jail -c path=/local/jails/sandbox devfs_ruleset=4 mount.devfs host.hostname=sandbox ip4.addr=192.168.252.1 command=/bin/sh
```
There was no warning or no information of any kind that ruleset #4 was not loaded.


----------



## junovitch@ (Feb 13, 2014)

You're right. Sorry I jumped the gun there because the ruleset number change was advertised in /usr/src/UPDATING entry 20131010.  I came across the warnings when I upgraded and fixed all my files but I never actually looked at /dev under my jails afterwards.  Now that I have I see that the rules have no effect.  Based off your comments I have found it to be fixable by doing the following (note devfs and not devd was restarted, they are different): 
`sysrc devfs_load_rulesets=YES; service devfs restart; ezjail-admin restart`


----------



## _martin (Feb 13, 2014)

@junovitch Np ..  Correct, that is what I did when @dpejesh mentioned the toggle. After that my jails were OK. 

Question is now whether this can be considered as a bug or not. It seems to me more people hit this problem and were not aware of it. Speaking for myself: as jails started and no apparent issue occurred, I left my jails as is for a day. Some users do have root access there which is OK for jail, but for sure not for host. Root had access to firewall and disks - that's pretty bad. 

I would expect `jail` to report that devfs rules can't be loaded as specified. Hopefully I'll have some spare time this weekend to look into it and check out the history of that toggle (why it was not needed on 9.2 but it is needed now). 
Maybe then I'll open a bug report ..


----------



## zgub (Feb 14, 2014)

Same here

I upgraded 9.2-RELEASE -> 10-RELEASE and realized that root from jails could do for example `pfctl -d` out of sudden.
I would expect a warning  
However I don't know if it's a bug, or I just missed a warning somewhere that devfs_load_rulesets=YES in /etc/rc.conf is now required for jails to load their rulesets.


----------



## dvl@ (Jun 21, 2015)

junovitch said:


> On FreeBSD 10 devfs rules must be integers rather than names.  You'll have to update your jail configuration files to use numbers instead.
> 
> This is what I did, I have some additional rules so I just updated all my files with a sed one-liner.
> `sed -i '' -e 's/"devfsrules_jail"/"4"/;s/"devfsrules_jail_with_vpn"/"5"/;s/"devfsrules_jail_with_bpf"/"6"/' /usr/local/etc/ezjail/*`



This approach just worked for me today.


----------



## _martin (Jun 22, 2015)

Older thread, FYI:   https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=188753


----------

