# Minimal filesystem hierarchy on a drive



## mefizto (Apr 2, 2021)

Greetings all,

both I and the bdsinstall(8) failed at separating boot and system onto different drives, or better to say making the two work, and my search did not discover any such successful installation.   I was little surprised because I see a motivation for doing so, _e.g._, opportunistic laptop thief would probably just reinstall instead of trying to figure out what is on the drive, people in similar situation to mine could take advantage of the NVMe drives, _etc_.

So, the next question is what would be the minimal filesystem to be installed on the USB drive, so that there is as little wiring into the USB drive, but there is still enough utilities to maintain/attempt to rescue the system in cast the NVMe drive with the reminder of the filesystem fails?

Kindest regards,

M


----------



## im (Apr 2, 2021)

What about moving rootfs (/) on the flash?  Rootfs is FreeBSD-base without /var /usr /home /tmp.
You will have a possibility to boot and use /rescuce/* tools.
I have a server, where rootfs (/) located on separate 2Gb sata flash ssd.
I done it manually, by moving rootfs to the separate disk.

This is real example:
	
	



```
# df -h
Filesystem                             Size    Used   Avail Capacity  Mounted on
/dev/ufs/sataflash                     1,8G    635M    1,0G    38%    /
devfs                                  1,0K    1,0K      0B   100%    /dev
/dev/ada1p1                            2,6T    2,3T    174G    93%    /media/wd3tbvkh
/media/wd3tbvkh/system.bsd/tmp         2,6T    2,3T    174G    93%    /tmp
/media/wd3tbvkh/system.bsd/var         2,6T    2,3T    174G    93%    /var
/media/wd3tbvkh/system.bsd/usr         2,6T    2,3T    174G    93%    /usr
```

If you care about your data, just encrypt /home partition using one of FreeBSD's technologies like geli(), gbde(), maybe even encrypted ZFS.


----------



## Jose (Apr 2, 2021)

Maybe the hier(7) manual page has the information you want.


----------



## mefizto (Apr 2, 2021)

Hi im,

first thank you for the reply.  My comment regarding the potential data theft was only an explanation of motivation of separating the boot and OS on different drive.  I am actually attempted this (unsuccessfully) on a server.

Do I understand it correctly that all the filesystems up to the /usr as described in hier(7) are mounted under root (/)?  If so, it appears that even is the /dev/ada1p1 does fail for some reasons, you still have enough tools to attempt recovery, correct?  If so, could you please describe, how you have accomplish moving the root on the flash drive or point me to a reference.

Kindest regards,

M


----------



## mefizto (Apr 2, 2021)

Hi Jose,

you are undoubtedly correct, and as you can see from my reply to im, I did look, but I am not sure whether I understand it correctly, hence my question.

Kindest regards,

M


----------



## Jose (Apr 2, 2021)

mefizto said:


> Do I understand it correctly that all the filesystems up to the /usr as described in hier(7) are mounted under root (/)?


I don't understand what you mean. The filesystem hierarchy always exists. The path /usr is just a path and should always exist. You can choose to mount a filesystem at that path, but if not, it's just a directory in the root filesystem.

I suggest you familiarize yourself with section 3.7 of the Handbook:








						Chapter 3. FreeBSD Basics
					

Basic commands and functionality of the FreeBSD operating system




					docs.freebsd.org


----------



## ShelLuser (Apr 2, 2021)

im said:


> If you care about your data, just encrypt /home partition using one of FreeBSD's technologies like geli(), gbde(), maybe even encrypted ZFS.


To what end? It's a fantasy that this would somehow provide better protection for your data. The only situation in which encryption would provide extra security is when your computer (or disk(s)) get stolen and the dataset doesn't get automatically mounted (not without having to manually provide a password). Otherwise there is 0 extra protection.


----------



## mefizto (Apr 2, 2021)

Hi Jose,

thank you for pointing out that I did not describe the (perceived) problem correctly.  So let me rephrase.

All SATA ports on my motherboard are taken by drives having zfs pool with data only.  As such, I had been running the OS from an USB flash drive on the motherboard, but it is not the best solution due to wear on the flash drive. Thus, I bough an NVMe drive with the idea of booting from the flash drive and installing and executing the OS from the NVMe drive (since my motherboard dose not boot from the NVMe drive).  However,  I could not make it work.

Hence the idea to physically installing only the minimal filesystem on the flash drive, installing physically the reminder of the filesystem on the NVMe drive and mount these filesytems on the flash drive.  I know how to mount and unmont filesystems; I had done it about million times, trying to make the separate boot/OS work. 

The problem I am contemplating is a potential failure of the NVMe drive, so I would like to have (physically) enough filesystems on the flash drive to attempt rescue/reinstall the missing filesystems.  And I do not know what should the minimal filesystem be.

Does it make more sense?

Kindest regards,

M


----------



## Phishfry (Apr 2, 2021)

mefizto said:


> Hence the idea to physically installing only the minimal filesystem on the flash drive, installing physically the reminder of the filesystem on the NVMe drive and mount these filesytems on the flash drive.


So just run /boot on the flash drive and the rest on NVMe.
Why do you need to bootstrap from flash drive? Will your system not boot from EFI?
If you are looking for redundancy you need to add another NVMe not boot from flash.
Run a gmirror or your zfs preference on the NVMe pair.

Do you only have one PCIe slot free?


----------



## im (Apr 2, 2021)

To ShelLuser


mefizto said:


> _ I see a motivation for doing so, e.g., opportunistic laptop thief would probably just reinstall instead of trying to figure out what is on the drive_


So I advised to encrypt.



mefizto said:


> all the filesystems up to the /usr as described in hier(7) are mounted under root (/)


You have forgotten about /var, it described in hier()below /usr

To mefizto 
rootfs on USB flash be enough for booting and basic recovery process.
If you can place /usr /var /home at your zfs storage - just do it.
Do not forget to backup your boot-flash.



mefizto said:


> could you please describe, how you have accomplish moving the root on the flash drive or point me to a reference


Look at my post below. It is about moving FreeBSD to another disk.
So you have to do something like that, using items from 3 to 7, but copy only rootfs. And use your flash as a target.








						UFS - Experiences with building a softraid 1 using gmirror(8)
					

Hello folks,  I'm using a remote headless server on 11.4-REALEASE-p8 that is used to host services for my personal needs. The motherboard (an Intel DH67BL) handles softraid capabilities and thinking of using a second disk to make a raid 1.  Reading the handbook at chapter 19.3.3, it's possible...




					forums.freebsd.org


----------



## Phishfry (Apr 2, 2021)

You could perhaps gain some redundancy with gmirror of partitions instead of whole NVMe device.
You would basically lose half the capacity of the device if you mirror partitions.
I have never used that method but it does work. If the whole device dies though you are screwed.


----------



## Jose (Apr 2, 2021)

Phishfry said:


> Why do you need to bootstrap from flash drive? Will your system not boot from EFI?
> If you are looking for redundancy you need to add another NVMe not boot from flash.


I have two motherboards that BIOS-boot Freebsd from NVMe drives just fine. I'm typing this on one of them.


Phishfry said:


> Run a gmirror or your zfs preference on the NVMe pair.


Why not boot the thing from ZFS?


----------



## mefizto (Apr 2, 2021)

Hi Phishfry,

no, as we discussed, my motherboard does not boot from EFI.  I actually found another thread where you discussed the same issue with OJ, and you advised that some motherboards boot from EFI when all the options are set to UEFI. So, I went through all the settings for my motherboard's BIOs, set them to UEFI, installed the OS on the NVMe drive, but no luck.  My motherboard has a "Built-in UEFI shell" so I booted into it and examined all the commands, but it is not obvious how I could make it find the NVMe drive.

I do have additional three PCI slots available; I will start thinking about redundancy, once I solve the current problem. Putting the /boot on the flash drive is an attractive idea, except I have no clue how to do it.

Hi im,



> You have forgotten about /var, it described in hier()below /usr


Just to clarify it in my mind.  As I can surmise from a result of `df -h`, the 2GB drive physically holds / and all the filesystems under it up to /usr.  The remaining filesystems, _i.e., _/usr, /var, /tmp, _etc_. , are physically on the /dev/ada1p1 and mounted under /, correct?

Thank you for posting the link.

Kindest regards,

M


----------



## mefizto (Apr 2, 2021)

Hi Jose,



> Why not boot the thing from ZFS?



I find separating the OS and the data to be much cleaner solution.

Kindest regards,

M


----------



## Jose (Apr 2, 2021)

mefizto said:


> I find separating the OS and the data to be much cleaner solution.


Me too. Different ZFS filesystems would be enough of a separation for me.


----------

