# FreeBSD and automated PXE Setup



## fr33bsd (Aug 15, 2021)

Hello people,

I am wondering if there is any *new* tutorial for *automated* installation of FreeBSD 13 (or 12.2.) *using PXE*.

Regular installation tutorials exist, like this older one (in german) and this old one.

This (automated ?) guide goes 404 

Searching for "pxe" in this forum ends up in "No results found." (the search link might not work) 

Is there any config file for the bsdinstaller to automate the setup procedure - or stuff like that? Or do I have to write the full installation procedure in a bash by myself?   I read something about install.cfg in section 6.1 of the old tutorial. But reaching sysinstall(8) ended up in a dead end:


> Sorry, no data found for `_sysinstall_(8)'.



I found this, but this provides information about pxe and diskless setup and informations about *automated* installation are missing.

Also in this context incomplete is this guide.

Maybe useful is this mfsbsd. I don't know. But there is another dead end thread.

This how-to is good, but what I am missing here is the part of the automated installation.

Another incomplete reference...

Did anyone of you experience automated rollouts of FreeBSD using PXE?


----------



## fr33bsd (Aug 16, 2021)

Ah, coming closer: https://wiki.freebsd.org/BSDInstall 
and closer ... https://www.freebsd.org/cgi/man.cgi?query=bsdinstall&sektion=&manpath=freebsd-release-ports ...
and closer ... https://forums.freebsd.org/threads/bsdinstall-scripting-guide-tutorial.79105/

But this is old and has to be verified:


> Installation scripts are located at /usr/src/usr.sbin/bsdinstall/scripts/ . First file to run is ‘auto’, which then calls other files. Almost all of them are shell scripts.


----------



## fundon (Sep 6, 2021)

I cant use PXE.

PXE-E79: *NBP IS TOO BIG TO FIT IN FREE BASE MEMORY.

I found the tftp logs, client fetch pxeboot, but cant next.*


----------



## sko (Sep 6, 2021)

I once (~2017) built an infrastructure and images for unattended installation of TrueOS clients (when it was based on FreeBSD). I only had to add the MAC address to a whitelist (for DHCP and tftp), and everything was set up automagically via pc-sysintstall. Everything after the basic OS setup was handled via Ansible.

This worked fine on _some_ clients - but PXE is (was?) severely broken in most client/desktop PCs. ACER was especially annoying and broken, but they also often only boot from one specific USB port or have other stupidities like automatically changing the boot order...
By far the most reliable and hassle-free ones were genuine Intel NUCs. But even their variants from e.g. ASRock ('beebox') also had/have annoyingly broken UEFI and didn't work more often than they did. Several variants and generations of ASRock NUCs were just freezing on reboot and could only be resurrected by hard shutdown (pressing powerbutton for >5sec) or unplugging the PSU.

So basically the biggest hurdle is the client itself. Make sure they are actually capable of booting via PXE - just because the vendor puts 'PXE' on the box and as an option in the BIOS/UEFI doesn't tell anything!
I found some versions of the loader to be quite picky via PXE and hang on some systems. Back then IIRC I ended up using an old loader from FreeBSD 11.3 (TrueOS was based on 12-CURRENT back then) as it was the most reliable. Thankfully it (mostly) doesn't really matter what version the loader is and what version of FreeBSD you want to install, as it will just pull the tarballs and splat them on the disk.


This is what I have in the README of that project, regarding the PXE-booting and the spaghetti-staging that TrueOS used back then (thanks to openRC that was heavily unreliable and I replaced a lot of that OpenRC-based staging in the installer with one shellscript and ansible. (All of this shouldn't bother you with a proper FreeBSD install)

```
### boot process (includes TrueOS specific quirks):

- BIOS/PXE gets IP, filename, next-server and root-path from DHCP
- BIOS/PXE loads $filename (=pxeboot or loader.efi) via tftp
- pxeboot / loader.efi
        acts as a pre-loader; mounts $root-path from $next-server via NFS and hands over to the standard /boot/loader
- (some basic stuff happens and finally runs rc)
- /etc/rc 
        runs /etc/trueos-rc.sh, then hands over to openrc (runlevels sysinit -> boot -> default)
        - /etc/trueos-rc.sh 
                checks for /dist/TRUEOS.ufs.uzip live-image, fires /etc/SetupLive.sh or /etc/SetupInstall.sh accordingly, then extracts etc/var/root-dist.txz to memfs for the installer environment
                - /etc/SetupInstall.sh
                        checks for /boot/pc-autoinstall.conf and /boot/pc-sysinstall.conf, copies them to /tmp for pc-sysinstall to pick up
        - openrc runlevels (/etc/runlevels/)
                sysinit is empty
                boot: basic system configuration; nothing specific for installation to see here
                default:
                        -> /etc/init.d/local starts everything in /etc/local.d/ (empty)
                        -> /etc/init.d/netmount mounts network shares according to fstab (-> nothing is done here)
- /root/TrueOSStart.sh (extracted from /uzip/root-dist.txz) is fired, starting the actual installation


/root/TrueOSStart.sh has a bug in that it searches for pc-autoinstall.conf and pc-sysinstall.conf in the filesystem root; not in /tmp where /etc/SetupInstall.sh copied it to:
https://github.com/trueos/trueos-core/issues/1497
https://github.com/trueos/trueos-core/pull/1496

UPDATE 2017-11-17: fix has been merged to master

pc-sysinstall makes the hard-coded assumption to find pc-sysinstall.cfg in /tmp, so the renaming from conf to cfg during copy by SetupInstall.sh is essential



#### pc-sysinstall.conf caveats

run setup; right before commiting everything to disk, copy configuration to USB drive and use it as template

packageType=pkg
installMedium=local
localPath=/dist
distFiles=base doc kernel lib32

disk0-part=ZFS 0 [....]   size of 0 causes the zfs slice to get all disk space available (minus efi, swap)


### PXE/NFS setup caveats

script pxe_prep.sh takes a release description and path to local iso file and generates a new pxe bootable dataset with all necessary additions/changes.
All additional files are in git repository @ git.gassner.lan:/var/git/clients/pxe.git releases are managed in seperate branches (ssh keys in repo are for user clients@stor1)

datasets:
/tftpboot/TrueOS-<release> contains the TrueOS-DVD iso (extracted) with the following additions/modifications:
/*      - FreeBSD "pxeboot" or "loader.efi" (trueos version is not mounting NFS?) */  # since TrueOS 17.12 the provided loader by TrueOS correctly boots via PXE
        - /boot/loader.conf (remove vfs.root.mountfrom; NFS mountpoint is received via DHCP)  # mainly cosmetic, after failing to mount a cd-rom loader will proceed with NFS mount; may fail if a cdrom is present!
        - /boot/pc-autoinstall.conf
        - /boot/pc-sysinstall.conf
        - /boot/init_gassner.sh script (called post-installation through pc-sysinstall.conf)
/*      - fixed SetupInstall.sh and TrueOSStart.sh scripts (https://github.com/trueos/trueos-core/pull/1496) in /etc (/uzip/etc-dist.txz) and /root (/uzip/root-dist.txz)
      -> fix has been merged to master on 2017/11/17; should be included in next release */  # included upstream since release 17.12
        - additional packages (ansible) in dist/packages/All + updated package library (see below)
```

PXE-booting and unattended installation is (was?) a bit quirky, but mainly it is relatively transparent, so you can get it working step by step. But by far the biggest and most common show-stopper is broken PXE implementations on client systems...


----------



## msplsh (Sep 7, 2021)

You're going to have to create your own ISO to create an "automated" installation of FreeBSD that requires no user intervention.  There's no manual for that.  I have PXE booted the install discs before, but that just loads the whole thing into RAM over the network.  Slow, dumb, but effective.

You can just PXE boot bootonly.iso or mfsBSD (This is a good project) and modify those ISOs to get your automated scripts to work.  Figuring out how to generate them is left as an exercise to the reader...


----------



## astyle (Sep 7, 2021)

PXE and automated installs are a Cute Idea, but over-relying on automation is a dangerous, thorny and slippery path. If there's an error somewhere along the way, the whole house of cards just collapses, and you spend hours trying to figure out where the mistake was. Those same hours might have been better spent setting the stuff up using the inelegant (but reliable) way. If all that is acceptable as part of the territory to you - as msplsh pointed out, it can be a good project.

Oh, and 


fr33bsd said:


> But reaching sysinstall(8) ended up in a dead end:


Sysinstall has been deprecated for awhile (since FreeBSD 9.0-RELEASE). These days, it's bsdinstall(8).


----------

