# [ZFS] zpool with root on SSD but remainder on spindle HDD



## Beeblebrox (Mar 28, 2013)

I have an SSD and a spindle-type HDD. I take regular backups and I do not use a mirrored device. I have been thinking about placing root on the SSD to get faster boot time, but I decided I do not want my root on UFS because I want the error-check and auto-fsck properties of zpool.

Now I have come up with this "genius idea" 
1. Create the zpool only on the SSD's planned root (gpt partition size 512MB)
2. Give the entire zpool to pool's own zfs file set (the dataset which will be our root)
`# zfs set reservation=512M tank`
3. Now expand pool size by adding the spindle HDD
`# zpool add tank <geom>`
4. Copy root's files to tank, zfs create the other datasets (usr, var, etc) and copy your data to each sub-dataset.

The result "should" (or hopefully might) be the root dataset residing on the SSD, but usr, var, home, etc residing on the spindle HDD.

My question is this: In the beginning, when the pool is first created in this manner, it is possible that what I have described (root fs on the SSD) will be true. However, as time goes by and the system is updated by the likes of installworld/installkernel, there is the possibility that the reserved root space will start migrating off of the SSD and on to the HDD. This would result in part of root being on the SSD, while the other half residing on the HDD, plus the space vacated by root becoming occupied by files from other folders.

What do you guys think? Am I being too smart for my own good?


----------



## jem (Mar 28, 2013)

Seems a bit complicated.  ZFS will want to stripe writes across the two devices in the pool.

Why don't you just create two single-device pools, put your FreeBSD root dataset on one, and all your other datasets on the other?


----------



## Beeblebrox (Mar 28, 2013)

> Why don't you just create two single-device pools, put your FreeBSD root dataset on one


It's very doubtful that a zpool will properly function with a pool size of 512MB (size of root partition on the SSD). Also, I don't want a 3rd pool just for root and would simply go with UFS in that case instead.


> ZFS will want to stripe writes across the two devices in the pool


That was the point of my post: Although I am fairly certain that when first setup, everything will be in its proper place, it's likely that over time the dataset locations will shift. That's what I meant by "migrate".

It's actually very simple and straight-forward, if.and.only.if ZFS adheres to original layout (strict). I think the question comes down to: What does "reservation" mean in terms of the ZFS internal operations?


----------



## Sfynx (Mar 28, 2013)

I rather have a few seconds extra boot time than an administrative nightmare. Just create your pool on the spindle drive and put two GPT partitions on the SSD: one for the gptzfsboot loader and perhaps another for a nice L2ARC to speed things up during the time you're actually working on your computer


----------



## kpa (Mar 28, 2013)

I have exactly such a setup. The disks that make up the ZFS pool don't have partitions or labels. I have a 2GB SSD IDE plug drive that has the boot loader partition and a spare partition in case I need swap for crash dumps.


----------



## Beeblebrox (Mar 28, 2013)

On my SSD I have a fall-back 1G swap, 10G for ccache on UFS, and 2G ZIL (plus a good amount of free space left). The boot sector is on this SSD.

On The spindle I used to have 1 pool and the entire HDD was allocated as RAW to the zpool. I recently changed the setup and now have 2 partitions and 2 pools, where 1st pool is for package building (poudriere jails), $WRKDIRPREFIX, all source files, $DISTDIR, etc. The ZIL is for this pool. The 2nd pool has root, /usr, /home, and likes.

I can tell you that ccache on SSD makes a difference, while ZIL not as much as I would like (because there's only one HDD). I split the HDD into 2 pools because I plan to add a second HDD as stripe (raid0) into the zpool, thereby doubling the HDD throughput and taking greater advantage of the ZIL device. This pool also has a 3GB swap dataset and I can see that zfs takes full advantage of the swap inside the pool.

kpa: by "exactly such set up" do you mean what I described?
sfynx: The box is my PC and not a server that services traffic requests from outside boxes -> an L2ARC would be of no use. I also don't see why it's an administrative nightmare (but I would concur with you if the box were, say a web server).


----------



## kpa (Mar 28, 2013)

I was referring to what Sfynx suggested.


----------



## usdmatt (Mar 28, 2013)

ZFS file systems have no relation to specific areas of the pool, data gets written wherever ZFS wants to put it. A reservation just guarantees that amount of space will always be available - If you reserve 2GB for pool/root/fs and 1GB is in use, ZFS will make sure that there is always another 1GB of pool space available.

As your root filesystem is written to, data will start to spread out onto the HDD as you suggest.

I personally would just create a root pool on the SSD and a separate data pool (I don't know where the 3rd you mention comes from?) or do as suggested above - put bootcode on the SSD, a single disk pool on the HDD and use the remaining SSD space as a cache. Obviously the cache is always empty on boot but I don't think disk performance has much of an effect on boot time anyway, most of it is spent in BIOS, boot loader or device discovery.

A single disk as UFS will probably have raw performance faster than ZFS but you need to decide if you want the ZFS features for your root - checksums, boot environments, snapshots, easy incremental backup to your data pool (or another ZFS pool)


----------

