# Building a NanoBSD image quickly using pre-compiled binaries



## asteriskRoss (Dec 2, 2013)

Whilst putting together a [link=https://www.freebsd.org/doc/en_US.ISO8859-1/articles/nanobsd/howto.html]NanoBSD[/link] image, it struck me that the standard NanoBSD build method of compiling from source is time-consuming and was for my purposes unnecessary.  There were perfectly adequate pre-compiled binaries and the GENERIC kernel that I could use from the install media, if I could integrate these with the NanoBSD script.  My eventual goal is to use NanoBSD for server installations in the cunning manner outlined by Paul Schenkeveld in [link=http://2010.asiabsdcon.org/papers/abc2010-P4A-paper.pdf]his presentation at AsiaBSDCon 2010[/link], so minimising image size is unimportant for me.

I examined the nanobsd(8) script and saw that it was possible to override the built-in functions with my own versions by including them in my NanoBSD configuration file.  The functions I have written take a steer from the excellent sysutils/ezjail port by Dirk Engling and friends, which provides options for installing jails from binaries rather than compiling from source.

Below, I have included a skeleton NanoBSD configuration file that achieves this.  My new functions mirror the style of the existing functions in nanobsd(8).  It will build a NanoBSD image using installation tarballs that have been downloaded or are located on mounted install media.  It gives the option of including execution of freebsd-update(8) to download and install binary updates, provided the FreeBSD version of your build machine is the same as the target version of NanoBSD.

I have checked I can produce a bootable NanoBSD image with this method on i386 with FreeBSD-9.2-RELEASE.  Before I do some more testing and prior to posting a how-to, I would welcome questions and comments on the script as well as on whether there would be interest in include such an image generation method in the nanobsd(8) script for a future FreeBSD release.

The configuration file (assuming it is called binary_build.nano) can be used like this (although you will likely want to edit it to set other options):
`# /usr/src/tools/tools/nanobsd/nanobsd.sh -c binary_build.nano`


```
# NanoBSD configuration file that modifies functions in the
# /usr/src/tools/tools/nanobsd/nanobsd.sh script to extract
# a FreeBSD system from installation media rather than compiling
# from source.
#
# Copyright (c) 2013 Ross McKelvie
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 1. Redistributions of source code must retain the above copyright
#    notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
#    notice, this list of conditions and the following disclaimer in the
#    documentation and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
# SUCH DAMAGE.
#

# Standard NanoBSD configuration options go here
NANO_NAME=binary_build

## Custom NanoBSD configuration options for binary install
# INSTALLFILESDIR The directory where the installation tarballs
#                 are located. This is where you downloaded the
#                 tarballs or the freebsd-dist directory on your
#                 installation media, without a trailing slash.
INSTALLFILESDIR="/media/cdrom/usr/freebsd-dist"

# WORLD_TXZ the list of tarballs that will be expanded from the
#           installation media; edit this to install additional
#           packages:
#           - base.txz  Base system including man pages (REQUIRED)
#           - doc.txz   Documentation, including FreeBSD handbook
#           - games.txz "Useful and semi-frivolous programs" 	
#           - lib32.txz 32-bit libraries for 64-bit x86 systems
#           - ports.txz Ports tree
#           - src.txz   System source code
WORLD_TXZ="base.txz games.txz doc.txz"

# KERNEL_TXZ the list of kernel tarballs that will be expanded from
#            the installation media. For vanilla FreeBSD installation
#            media this will most likely just be the GENERIC kernel.
KERNEL_TXZ="kernel.txz"

# DO_FETCH_UPDATES Set to true to use freebsd-update to fetch updates
#                  to be applied to NanoBSD. Note this assumes that
#                  you are running a RELEASE version of FreeBSD and the
#                  the version on your build machine matches the version
#                  you are using for NanoBSD.
#                  Control freebsd-update through /etc/freebsd-update.conf
DO_FETCH_UPDATES=true


# DO_INSTALL_UPDATES Set to true to use freebsd-update to install updates
#                    to NanoBSD. Note this assumes that
#                    you are running a RELEASE version of FreeBSD and the
#                    the version on your build machine matches the version
#                    you are using for NanoBSD.
#                    Control freebsd-update through /etc/freebsd-update.conf
DO_INSTALL_UPDATES=true

##########################################################
#
# Custom functions follow. Note these overwrite the
# functions in the nanobsd.sh script, so if you want to
# build your own kernel, you need to comment out the
# build_kernel and install_kernel functions contained in
# this configuration file. 
#
# Functions overwritten (and their new purposes) are:
# build_world - Dummy function that does nothing
# build_kernel - Dummy function that does nothing
# install_world - Extracts world from installation tarball(s)
# install_kernel - Extracts kernel(s) from installation tarball(s)
# install_etc - Creates an empty /etc/make.conf 
#
##########################################################


##########################################################
# Replace build_world with a dummy version

build_world ( ) (
	pprint 2 "Instead of buildworld, a built world will be extracted from installation media"
	pprint 3 "Log: ${MAKEOBJDIRPREFIX}/.bw"
	echo "Custom configuration file used. buildworld not processed; a built world will be extracted from installation media." > ${MAKEOBJDIRPREFIX}/_.bw 2>&1	
)

##########################################################
# Replace build_world with a dummy version

build_kernel ( ) (
	pprint 2 "Instead of building kernel, a built kernel will be extracted from installation media"
	pprint 3  "Log: ${MAKEOBJDIRPREFIX}/.bk"
	echo "Custom configuration file used. buildkernel not processed; a built kernel will be extracted from installation media." > ${MAKEOBJDIRPREFIX}/_.bk 2>&1	
)

##########################################################
# install_world Function to replace standard nanobsd.sh 
#               world installation with expansion of tarballs
#               from installation media.

install_world ( ) (
	pprint 2 "Install world by expanding installation tarball(s)"
	pprint 3 "Log: ${NANO_OBJ}/_.iw"

	# Check the directory holding the tarballs exists
	if [ ! -d "${INSTALLFILESDIR}" ]; then
		echo "Error: Directory for installation tarballs ${INSTALLFILESDIR} does not exist"
		exit 1	
 	fi

	# Check tarballs exist
	for tarball in ${WORLD_TXZ}; do
		if [ ! -f "${INSTALLFILESDIR}/${tarball}" ]; then
			echo "Error: Installation tarball $1 does not exist in $INSTALLATIONFILESDIR."
		exit 1
		fi
	done

	# Install the tarballs
	for tarball in ${WORLD_TXZ}; do
		pprint 3 "Unpacking ${INSTALLFILESDIR}/${tarball}"
		xzdec "${INSTALLFILESDIR}/${tarball}" | tar --unlink -xpJf - -C ${NANO_WORLDDIR} > ${NANO_OBJ}/_.iw 2>&1
	done
)

##########################################################
# install_kernel Function to replace standard nanobsd.sh 
#                kernel installation with expansion of tarball(s)
#                from installation media.

install_kernel ( ) (
	pprint 2 "Install kernel(s) by expanding installation tarballs"
	pprint 3 "Log: ${NANO_OBJ}/_.ik"

	# Check the directory holding the tarballs exists
	if [ ! -d "${INSTALLFILESDIR}" ]; then
		echo "Error: Directory for installation tarballs ${INSTALLFILESDIR} does not exist"
		exit 1	
 	fi

	# Check tarballs exist
	for tarball in ${KERNEL_TXZ}; do
		if [ ! -f "${INSTALLFILESDIR}/${tarball}" ]; then
			echo "Error: Installation tarball $1 does not exist in $INSTALLATIONFILESDIR."
		exit 1
		fi
	done

	# Install the tarballs
	for tarball in ${KERNEL_TXZ}; do
		pprint 3 "Unpacking ${INSTALLFILESDIR}/${tarball}"
		xzdec "${INSTALLFILESDIR}/${tarball}" | tar --unlink -xpJf - -C ${NANO_WORLDDIR} > ${NANO_OBJ}/_.ik 2>&1
	done
)

##########################################################
# install_etc Function to replace standard nanobsd.sh 
#             etc installation with minimal version.

install_etc () (
	pprint 2 "Install /etc"
	pprint 3 "Log: ${NANO_OBJ}/_.etc"

	# make.conf doesn't get created by default, but some
	# ports need it so they can spam it.
	cp /dev/null "${NANO_WORLDDIR}/etc/make.conf"
)

##########################################################
# cust_fetch_updates Use freebsd-update to fetch security
#                    updates for the release.

cust_fetch_updates ( ) (
	if $DO_FETCH_UPDATES; then
		freebsd-update -b ${NANO_WORLDDIR} fetch 
	fi
)
customize_cmd cust_fetch_updates

##########################################################
# cust_install_updates Use freebsd-update to install
#                      security updates for the release.

cust_install_updates ( ) (
	if $DO_INSTALL_UPDATES ; then 
		freebsd-update -b ${NANO_WORLDDIR} install
	fi
)
customize_cmd cust_install_updates
```


----------



## asteriskRoss (Dec 12, 2013)

*Re: Building a NanoBSD image quickly using pre-compiled bina*

There weren't any cries of "that's a terrible idea" so I'll tidy this up and stick it in the how-to section


----------



## VirGin (Jan 4, 2014)

*Re: Building a NanoBSD image quickly using pre-compiled bina*

Hi @asteriskRoss,

Just a quick note of support. I think this is a fabulous idea.  What hardware have you tested your bootable image on?

-V-


----------



## asteriskRoss (Jan 6, 2014)

*Re: Building a NanoBSD image quickly using pre-compiled bina*

Thanks for the vote of support! The i386 image I built (based on 9.2-RELEASE) was perfectly happy in a virtual machine but I haven't yet managed to test it on hardware.  Unfortunately I couldn't get my target machine (an old Dell Optiplex 260) to recognise the Compact Flash cards I had lying around as hard drives.

I was using a very cheap unbranded IDE to Compact Flash adapter (labelled CF-IDE40 V.E0) and a Kingston CF 4 GB card.  Having done some reading on the web I suspect that it is the card that doesn't properly support IDE protocols but I'm yet to make time to investigate further (or as you may have noticed, make time to migrate the configuration file I posted to the how-to section).

Do let me know if you have success with the method I proposed.  It should work fine, since it's effectively just levering a standard FreeBSD installation into the NanoBSD framework, but as any geek will be painfully aware _should work_ and _does work_ are often distant cousins rather than identical twins.


----------



## zennybsd (Mar 3, 2014)

*Re: Building a NanoBSD image quickly using pre-compiled bina*

@*ross: This is a fabulous idea. Thanks for sharing.

But how can one make changes to the kernel of new nanobsd build? In the script, you mentioned to comment 'buildkernel' and 'installkernel' section. There is no mention to point to the desired kernelconfig files (usually NANOBSD and nanobsd.conf).

Any pointer? Thanks!


----------



## asteriskRoss (Mar 3, 2014)

*Re: Building a NanoBSD image quickly using pre-compiled bina*

@@zennybsd You've reminded me that I still need to make time for this.  I also saw that NanoBSD had some changes for 10.0-RELEASE and I haven't tried my configuration file with anything other than 9.2-RELEASE.

The configuration script as posted would extract a compiled GENERIC kernel.  Any kernel build options you added would be ignored as there is no compilation.  The configuration script does this by replacing two functions in the nanobsd.sh script; build_kernel() and install_kernel().  If you remove the custom versions of these functions from the posted configuration script, nanobsd(8) will use its own original functions and build and install the kernel according to whatever options you set in the variables NANO_SRC (path to the source tree) and NANO_KERNEL (kernel configuration file to use, set up as desired).

Your NanoBSD image would then consist of the pre-compiled world extracted from the installation media and the custom kernel you built.  Does that make sense?  You may find it helpful to have a look at the NanoBSD script to see how it works.

Disclaimer: I've not yet properly tested using my script in that way, though would, of course, be pleased to hear how you get on.


----------



## silicium (Mar 7, 2014)

*Re: Building a NanoBSD image quickly using pre-compiled bina*

Thanks for this much needed rewrite of NanoBSD script, using binaries instead of source.
I hacked nanobsd.sh to avoid wasting image space in addition to build time. My goal is to run NanoBSD on thin clients reused as network appliances, but could also be applied to non-i386 embedded platforms. Since they often come with small flash memory, no space should be wasted on unused thousands of files and directories created by  `make installworld` in /etc, /boot and /usr/share.
A single bootable partition almost doubles the available space if remote upgrade is not required.
Kernel build time is shorter after avoiding unused modules with MODULES_OVERRIDE in make.conf().
For faster world build, many options are turned off in src.conf(), but related useless things remain in the target directory under /usr/obj. They are removed with customize_cmd's before creating the filesystem image.
To prevent packages from installing docs and manpages and creating directories with mtree() , they are built from ports, then manually extracted so only binaries and config files are left in /usr/src/tools/tools/nanobsd/Files.


----------



## zennybsd (Jul 13, 2014)

*Re: Building a NanoBSD image quickly using pre-compiled bina*



			
				silicium said:
			
		

> Thanks for this much needed rewrite of NanoBSD script, using binaries instead of source.
> I hacked nanobsd.sh to avoid wasting image space in addition to build time. My goal is to run NanoBSD on thin clients reused as network appliances, but could also be applied to non-i386 embedded platforms. Since they often come with small flash memory, no space should be wasted on unused thousands of files and directories created by  `make installworld` in /etc, /boot and /usr/share.
> A single bootable partition almost doubles the available space if remote upgrade is not required.
> Kernel build time is shorter after avoiding unused modules with MODULES_OVERRIDE in make.conf().
> ...



@@silicium:

Your approach seems very logical and interesting. 

BTW would you mind sharing your version of hacked nanobsd.sh, make.conf, src.conf and other configs? Appreciate it!

/zenny


----------



## silicium (Jul 15, 2014)

*Re: Building a NanoBSD image quickly using pre-compiled bina*

I stopped using nanobsd.sh. I lost the hacked one, and it became too monolithic, with customer-specific heredoc's everywhere.
The target image is partitioned with `gpart`. Some applications with removable USB stick can include a FAT32 data partition first, to allow offline access to logs and configuration files.
The _tmpmfs_ and _varmfs_ variables in rc.conf() allow a read-only root, and my one-off projects do not need the complex management of *mfsroot* or /etc/rc.initdiskless and its /conf directory.
I will attempt to do a bit like TinyBSD, avoiding `make installworld` and `make distribution` that leave mtree dinosaur footprint, and only write files that are actually used on target. I have disabled almost everything with all the WITHOUT= knobs in my src.conf(), except a toolchain (old default *gcc* or new *clang*) is needed for `make buildworld`.
To reduce the kernel build time, do not build unused modules, build only required ones with MODULES_OVERRIDE= in make.conf()
To build kernel with recent gcc port, change CC, CXX, CPP and USE_GCC in the same file and fix the errors coming from /sys/conf/kern.mk and /sys/conf/newvers.sh that only worked with ancient gcc in base.


----------

