# buildworld and installworld shell script



## fred974 (Apr 23, 2014)

Hi guys,

I'm trying to patch my FreeBSD 10 system to resolve the openssl heart bleed problem as described at http://www.freebsd.org/security/advisories/FreeBSD-SA-14:06.openssl.asc
I looked at the FreeBSD man Rebuilding “world” chapter but I am really not sure about rebuilding everything by hand as my knowledge is not all that great yet.

I was wondering if anyone had a shell script that automate the `make buildworld` to `make installworld` that you could share

Thank you in advance
Fred


----------



## wblock@ (Apr 23, 2014)

This may help: Building FreeBSD World And Kernel: The Short Form.


----------



## fred974 (Apr 23, 2014)

wblock@ said:
			
		

> This may help: Building FreeBSD World And Kernel: The Short Form.



Wow, that is really a lot shorter 
Could you confirm that it will be ok for FreeBSD 10 zfs?


----------



## wblock@ (Apr 23, 2014)

It's what I use on all my FreeBSD-10 machines, with some refinements (like -j8 for buildworld and kernel targets).


----------



## trh411 (Apr 23, 2014)

wblock@ said:
			
		

> It's what I use on all my FreeBSD-10 machines, with some refinements (like -j8 for buildworld and kernel targets).


So one can use -j4 on buildkernel, too? I thought I remember seeing somewhere the use of -j for kernel builds was not recommended. Same for -DNO_CLEAN?


----------



## fred974 (Apr 23, 2014)

wblock@ said:
			
		

> It's what I use on all my FreeBSD-10 machines, with some refinements (like -j8 for buildworld and kernel targets).




Thank you


----------



## wblock@ (Apr 23, 2014)

trh411 said:
			
		

> wblock@ said:
> 
> 
> 
> ...



-j8 has had no problems for me on 9-STABLE and now 10-STABLE for years (I benchmarked various values a couple of years ago, and -j8 was the fastest on both a Core 2 Duo E8400 and i5-2500K processors).  I have not been using -DNO_CLEAN for as long, but again, no problems.  Well, other than cosmetic: the output of uname(1) does not show the correct version of the kernel unless you delete some files first (as shown by Trond Endrestøl on the mailing lists recently):
`rm /usr/obj/usr/src/sys/[i]CUSTOMKERNELNAME[/i]/vers*`


----------



## fred974 (Apr 23, 2014)

Hi guys,

I get the following when i ran

```
make buildworld -j8
```


```
1 error generated.
*** [bn_lib.So] Error code 1

make[4]: stopped in /usr/src/secure/lib/libcrypto
1 error

make[4]: stopped in /usr/src/secure/lib/libcrypto
A failure has been detected in another branch of the parallel make

make[3]: stopped in /usr/src
*** [libraries] Error code 2

make[2]: stopped in /usr/src
1 error

make[2]: stopped in /usr/src
*** [_libraries] Error code 2

make[1]: stopped in /usr/src
1 error

make[1]: stopped in /usr/src
*** [buildworld] Error code 2

make: stopped in /usr/src
1 error

make: stopped in /usr/src
```

I have no idea what the error refers to...could anyone assist?

Just before that I did 

```
# fetch http://security.FreeBSD.org/patches/SA-14:06/openssl-10.patch
# fetch http://security.FreeBSD.org/patches/SA-14:06/openssl-10.patch.asc
# cd /usr/src
# patch < /path/to/patch
```


----------



## trh411 (Apr 23, 2014)

You need to pull in the entire FreeBSD source repository as the first step. Use `svnlite co [url=https://svn0.us-east.freebsd.org/base/head]https://svn0.us-east.freebsd.org/base/head[/url] /usr/src`. Then you can continue with `make -j8 buildworld` and the rest of the build process as defined in Rebuilding "world".


----------



## wblock@ (Apr 23, 2014)

Remember to delete /usr/src before doing a checkout with `svn`.


----------



## fred974 (Apr 23, 2014)

Thank you guys, I'll do that first thing tomorrow


----------



## wblock@ (Apr 23, 2014)

Oh, and the checkout URL determines which version is checked out.  The one shown by @trh411 is for HEAD, or -CURRENT.  For 10-STABLE, it's base/stable/10.


----------



## trh411 (Apr 24, 2014)

wblock@ said:
			
		

> Oh, and the checkout URL determines which version is checked out.  The one shown by @trh411 is for HEAD, or -CURRENT.  For 10-STABLE, it's base/stable/10.


Thanks for this correction. For some reason I had -CURRENT on my mind when I posted. I missed completely that @fred974 was looking at updating to 10-STABLE.


----------



## bryn1u (Apr 24, 2014)

wblock@ said:
			
		

> This may help: Building FreeBSD World And Kernel: The Short Form.



It's really cool. I have a questions for you.

This procedure is enough to upgrade host and jail ?


> # make buildworld
> # make installworld
> # make installworld DESTDIR=/jails/somejail
> # mergemaster -Ui -D /jails/somejail



Another question, what about `make installkernel` doesn't need it ?


----------



## fred974 (Apr 24, 2014)

Ok guys this is what I am about to run:


```
1.mv /usr/src /usr/src.old
2.svnlite co https://svn0.eu.freebsd.org/base/stable/10 /usr/src
3.fetch http://security.FreeBSD.org/patches/SA-14:06/openssl-10.patch
4.fetch http://security.FreeBSD.org/patches/SA-14:06/openssl-10.patch.asc
5.cd /usr/src
6.patch < /path/to/patch
```

Could you please tell me if I need to do steps 3 to 6? Is the the openssl pach already included in the FreeBSD source repository that I will be downloading?

Fred


----------



## bryn1u (Apr 24, 2014)

fred974 said:
			
		

> Ok guys this is what I am about to run:
> 
> 
> ```
> ...



I think you don't need it. You should have already updated system.


----------



## fred974 (Apr 24, 2014)

bryn1u said:
			
		

> wblock@ said:
> 
> 
> 
> ...


I also have 3 jails on my system, will  
	
	



```
make installworld
```
 destroy my jails?


----------



## SirDice (Apr 24, 2014)

fred974 said:
			
		

> I also have 3 jails on my system, will
> 
> 
> 
> ...


No, it will only update/upgrade the host system, not the jails. You'll have to update/upgrade them separately.


----------



## fred974 (Apr 24, 2014)

@wblock@, 
Would you mind sharing what you have in your etc/mergemaster.rc file?
So far I came up with the following:

```
# Automatically install files that do not exist
AUTO_INSTALL='yes'
# Automatically upgrade files that have not been edited
AUTO_UPGRADE='yes'
```


----------



## fred974 (Apr 24, 2014)

I run 
	
	



```
mergemaster -Ui
```
 and selected i for the prompt. After restarting the server, I have lost my SSH access.
Originally it was set as public/private keyr. When i type the password it is not recognised..

All the web services are gone too, I assume that they hasn't been started upon reboot

Could anyone help please


----------



## kpa (Apr 24, 2014)

You probably overwrote /etc/master.passwd and /etc/group without thinking and all your user accounts are now reset to the ones included in a fresh install. The correct action would have been merge (m). There should be backup copies of those files at /var/backup, restore them from there. Next time pay attention what mergemaster(8) is asking you to do, it does offer options to review the changes that are about to be committed.


----------



## fred974 (Apr 24, 2014)

kpa said:
			
		

> You probably overwrote /etc/master.passwd and /etc/group without thinking and all your user accounts are now reset to the ones included in a fresh install. The correct action would have been merge (m). There should be backup copies of those files at /var/backup, restore them from there. Next time pay attention what mergemaster(8) is asking you to do, it does offer options to review the changes that are about to be committed.



Hi @kpa this is a lesson well learned believe me.
Thank you for the tip. I'll have to check when I have physical access to the box tonight.
Will my root password be the same when I have physical access to the terminal?


----------



## ShelLuser (Apr 24, 2014)

fred974 said:
			
		

> Originally it was set as public/private keyr. When i type the password it is not recognised..
> 
> All the web services are gone too, I assume that they hasn't been started upon reboot
> 
> Could anyone help please


There's nothing anyone can do I'm afraid. The only liable option I see is to gain console access so you can figure out exactly what has gone wrong.

As to what has happened here; that heavily depends on your setup. I mean; if those jails provide specific (web)services then I could imagine scenario's where those may fail to boot after upgrading your base OS. Though I'd also like to quickly mention that jails normally shouldn't be too much affected by the host. On one of my servers I'm still using a 9.1 jail on top of a 9.2-RELEASE-P3 host.

My guess is that you used some wrong options with mergemaster. It will also go over files such as /etc/passwd and /etc/group to update the version "tag" (the first line). If you told it to replace those files, and at the end allowed it to run pwd_mkdb then you effectively reset your environment to the default account settings.

Depending on your settings and used options you might be able to recover from this through local backups (check /var/backups) but you may want to keep your regular backups safe as well "just in case".


----------



## wblock@ (Apr 24, 2014)

fred974 said:
			
		

> @wblock@,
> Would you mind sharing what you have in your etc/mergemaster.rc file?



It varies a bit by system:

```
IGNORE_FILES='/.cshrc /etc/hosts /etc/master.passwd /etc/ntp.conf /etc/printcap /root/.cshrc'
```


----------



## fred974 (Apr 24, 2014)

I restored the group, aliases, master.passwd to /etc but the users are still not recognized..
If I ls /usr/home, I can still see the users home directory.

As for the jails, I have no idea why they won't start, I looked at /var/log but found no logs for ezjail

Has anyone got any suggestion how to solve my problem? 
do I need to restore the files from /var/backup to other location than /etc?


----------



## kpa (Apr 24, 2014)

You need to run (there are other ways but this is probably the easiest) vipw(8) after restoring /etc/master.passwd. Don't change anything in the editor, just exit. This re-creates the binary database file /etc/master.passwd.db and the plain text /etc/passwd file. You don't have to do anything with /etc/group after restoring it from backup, it is used directly in its plain text form.

To make sure your mail aliases are up to date run newaliases(1) after restoring /etc/mail/aliases.


----------



## wblock@ (Apr 24, 2014)

I thought vipw(8) only rebuilt the password file if changes had been made.


----------



## kpa (Apr 24, 2014)

You could be right. You can make a trivial change and save it and revert the change after the database is restored by running vipw(8) again.

Ok, the simplified way to restore the accounts is this:

(I thought the vipw(8) tool would recreate the database unconditionally but that's not the case necessarily)


`cp /var/backups/master.passwd.bak /etc/master.passwd`
`cp /var/backups/group.bak /etc/group`
`pwd_mkdb /etc/master.passwd`

You have to restore the user accounts to have any hope of getting your web service or your database services back. They usually run under special unprivileged users and your system is now missing those users.


----------



## fred974 (Apr 24, 2014)

ShelLuser said:
			
		

> fred974 said:
> 
> 
> 
> ...


This is exactly what I did I did allowed it to run pwd_mkdb.

Now.. have I lost everything or is there hope to save at least the jails 
jail 1 = web server
jail 2 - database

I can get the files back from jail 1 with no bother.. but to how can I get the bata back from the DB is the jail won't start?


----------



## fred974 (Apr 25, 2014)

Hi guys,

Could you please tell me what the mergemaster.mtree.70xnpZ81 file located in /tmp after runing `mergemaster -Ui` is for?
Is that the list of all the files that was changed?


----------



## fred974 (Apr 25, 2014)

Thank you all for your help here...
I managed to get everything back up and running by restoring various files in /etc.
The reason why the jail stopped working in something to do with the `make kernel`.
I have restored my kernel.old file and the jails are now working.

Can I leave it as that or do I need to sort out the kernel issue?


----------



## SirDice (Apr 25, 2014)

You need to keep your world and kernel in sync with each other. So I'd advise to sort it out. Keep the same source tree and do a `make buildkernel` and if that doesn't produce errors a `make installkernel`. Reboot to load the new kernel. Nothing else would be needed.


----------



## fred974 (Apr 25, 2014)

SirDice said:
			
		

> You need to keep your world and kernel in sync with each other. So I'd advise to sort it out. Keep the same source tree and do a `make buildkernel` and if that doesn't produce errors a `make installkernel`. Reboot to load the new kernel. Nothing else would be needed.



Thank you I'll tr that. My only concern is that last time the new kernel stopped my jails from starting up


----------



## wblock@ (Apr 25, 2014)

If you had a custom kernel, specifying that config file is required.  Otherwise, you end up with just the default GENERIC kernel.


----------



## fred974 (Apr 25, 2014)

wblock@ said:
			
		

> If you had a custom kernel, specifying that config file is required.  Otherwise, you end up with just the default GENERIC kernel.



I don't have a customer kernel.
Btw, all my users are also back


----------



## fred974 (Apr 26, 2014)

SirDice said:
			
		

> You need to keep your world and kernel in sync with each other. So I'd advise to sort it out. Keep the same source tree and do a `make buildkernel` and if that doesn't produce errors a `make installkernel`. Reboot to load the new kernel. Nothing else would be needed.



Hi did  `make buildkernel` and no error was displaued, so I did `make installkernel`. Reboot to load kernel.


I am now back to the original problem.. The jails are not sarting.
I have no customer kernel.

Any Idea why this is happening?


----------



## wblock@ (Apr 26, 2014)

Even without a custom kernel, there are differences between various versions of GENERIC.  Compare the one from whatever release you had to the current one in 10-STABLE.

Or it could just be an incomplete upgrade of the jails.  Without seeing the errors shown by the jails, it's really hard to say what problems they are having.


----------



## fred974 (Apr 27, 2014)

wblock@ said:
			
		

> Even without a custom kernel, there are differences between various versions of GENERIC.  Compare the one from whatever release you had to the current one in 10-STABLE.
> 
> Or it could just be an incomplete upgrade of the jails.  Without seeing the errors shown by the jails, it's really hard to say what problems they are having.



Looked at the GENERIC file from my /usr/src/sys/amd64/conf/GENERIC and /usr/src.old/sys/amd64/conf/GENERIC and they are an exact match. no difference between the files.

Is it possible to port the jails to another server? What files are to be copied over.

That's the curent solution that I have. 
Transfer the jails on a new FreeBSD host

will that work?


----------



## bryn1u (May 5, 2014)

What if I want to add all process to the cron()? Is there silent way to upgrade jail?

After these steps:

```
# make buildworld       
# make installworld DESTDIR=/jails/somejail
```

It comes time to:

```
# mergemaster -Ui -D /jails/somejail
```
How to automate this process as asking so many questions during this operation (mean mergemaster) ? I want all my config files to stay untouched.

Greetz,


----------

