# Planning a ZFS build



## Again (Dec 25, 2010)

Hello, I'm a 98% *nix noob, but since I saw Vista I knew that where Windows goes is not the place I want to be..A quarter ago I decided to try to leave it with the upcoming PC update. It came earlier than I expected, and though I'm really scared, I'm gonna try it. The decision is to use PC-BSD because it has ZFS and I expect that setting up a workable environment with it is gonna be easier than with regular FreeBSD...
My configuration is:

4 hard drives, each one different, 230 GB - 1.3 TB.
1 HDD is WD EARS.
I want ZFS with copies=2 to have almost-RAID1 with good utilization of what I already have.
I want gzip -9 on OS (as it's mostly static except for /var and /tmp, right?) and LZJB in other places.
Questions:

How does ZFS recover when a HDD dies? Do I have to insert another disk or can I force it to just add another copy to one of the alive ones? If the 230 died soon I'd still have enough space to not have to buy a replacement for a while and I'm wondering if it wouldn't be better to not use it at all.
Should I use just 1 ZFS pool or would it be better to slice the disks? I'd rather avoid slicing because I have no idea how much space do I need for OS and somehow it always ends up with having one slice full before the other fills in half...and I end up moving stuff between them. Blah, I hate making such mess. But I'm unsure how will I handle possible situations like a switch to linux when everything is in 1 pool.
FUSE on linux supports only ZFS pool v13 and I'd rather have it as an option. I didn't read changelogs to see what am I missing yet but I don't think it's a killer. Can I create such pool with the latest version of FreeBSD or do I have to use some older one?
How about ZFS stability in BSD? Reading various related stuff I saw like 2 cases when sb. lost a pool. My backup disk is already 20% too small to keep all my stuff, so I'm gonna loose some music. Nothing really terrible, but ripping it again is quite a lot of work.
WD EARS. I've seen like 5 threads on various forums discussing problems with it. Generally the talk is about it's troubles in RAIDZ, but some say just about ZFS and I'm confused, is it going to cause problems with regular striping?
Do I want to have OS deduplicated? I guess it's going to save little, but with practically no cost.
In order to improve compression I'd like to increase block size. Is it a good idea? If yes, at what point can I expect a notable performance drop? I'm talking about regular desktop use; net, programming, gaming, multimedia.
How to set it up? I reckon that the best way is to use a PC-BSD Live DVD to prepare a pool and then somehow figure out how to install the OS over it. I saw that there are some FreeBSD guides, but the fact that PC-BSD has a different installer makes me wonder whether they apply. Will I be able to do it? Please remember that though rather determined, I'm just a noob...
Anything you think I might be missing?


----------



## graudeejs (Dec 25, 2010)

Again said:
			
		

> * I want gzip -9 on OS (as it's mostly static except for /var and /tmp, right?) and LZJB in other places.


AFAIK root, or at least /boot can't be compressed. If you compress /boot you won't be able to boot (unless you boot from flash)



			
				Again said:
			
		

> 2) Should I use just 1 ZFS pool or would it be better to slice the disks? I'd rather avoid slicing because I have no idea how much space do I need for OS and somehow it always ends up with having one slice full before the other fills in half...and I end up moving stuff between them. Blah, I hate making such mess. But I'm unsure how will I handle possible situations like a switch to linux when everything is in 1 pool.


If you want to boot from HDD, then you will need to make slices.
1x freebsd-boot and 1x for freebsd-zfs gpt schemes.
I'd also suggest adding 1x freebsd-swap scheme



			
				Again said:
			
		

> 3) How about ZFS stability in BSD? Reading various related stuff I saw like 2 cases when sb. lost a pool. My backup disk is already 20% too small to keep all my stuff, so I'm gonna loose some music. Nothing really terrible, but ripping it again is quite a lot of work.


For me zfs is rock-stable on laptop, dekstop pc and server



			
				Again said:
			
		

> 4) WD EARS. I've seen like 5 threads on various forums discussing problems with it. Generally the talk is about it's troubles in RAIDZ, but some say just about ZFS and I'm confused, is it going to cause problems with regular striping?


Relay on zraid if possible, it's FAST



			
				Again said:
			
		

> 5) Do I want to have OS deduplicated? I guess it's going to save little, but with practically no cost.


Dunno, AFAIK, if you use FreeBSD -RELEASE then deduption is not available (yet)



			
				Again said:
			
		

> 6) In order to improve compression I'd like to increase block size. Is it a good idea? If yes, at what point can I expect a notable performance drop? I'm talking about regular desktop use; net, programming, gaming, multimedia.


Music, video archives are already compressed, you won't gain anything.
Compression will introduce little latency.



			
				Again said:
			
		

> 7) How to set it up? I reckon that the best way is to use a PC-BSD Live DVD to prepare a pool and then somehow figure out how to install the OS over it. I saw that there are some FreeBSD guides, but the fact that PC-BSD has a different installer makes me wonder whether they apply. Will I be able to do it? Please remember that though rather determined, I'm just a noob...


I mix knowledge of this thread http://www.daemonforums.org/showthread.php?t=1538
With some howto from http://wiki.freebsd.org/ZFS (depending how you want to setup your PC)
Generally it's easy



			
				Again said:
			
		

> 8) Anything you think I might be missing?


Read: http://forums.freebsd.org/showthread.php?t=6230

NOTE: everything I said is based on my own experience


----------



## dandelion (Dec 25, 2010)

Again said:
			
		

> 3) FUSE on linux supports only ZFS pool v13 and I'd rather have it as an option. I didn't read changelogs to see what am I missing yet but I don't think it's a killer. Can I create such pool with the latest version of FreeBSD or do I have to use some older one?


zpool(8) supports creating pools with lower version numbers (-o version=...), e.g. for grub2/linux/netbsd compatability. However, such pools wouldn't support features like user/group quota (v15) or deduplication (v21).





			
				killasmurf86 said:
			
		

> AFAIK root, or at least /boot can't be compressed. If you compress /boot you won't be able to boot (unless you boot from flash)


Only gzip compression is not supported (PR kern/153173) by (gpt)zfsboot. Booting from lzjb or zle compressed pools should and does work for me.


----------



## graudeejs (Dec 25, 2010)

dandelion said:
			
		

> zpool(8) supports creating pools with lower version numbers (-o version=...), e.g. for grub2/linux/netbsd compatability. However, such pools wouldn't support features like user/group quota (v15) or deduplication (v21).Only gzip compression is not supported (PR kern/153173) by (gpt)zfsboot. Booting from lzjb or zle compressed pools should and does work for me.



Wow, didn't know that, Thanks


----------



## Again (Dec 25, 2010)

Thank you both for the answers.


			
				killasmurf86 said:
			
		

> If you want to boot from HDD, then you will need to make slices.
> 1x freebsd-boot and 1x for freebsd-zfs gpt schemes.
> I'd also suggest adding 1x freebsd-swap scheme


How does it relate to http://wiki.freebsd.org/RootOnZFS/ZFSBootPartition ?

I searched and found that the boot partition is something really small, so the problems are so much smaller anyway.
I wonder whether ZFS will disable caching though...is it a case with BSD too?


			
				killasmurf86 said:
			
		

> Relay on zraid if possible, it's FAST


You mean RAIDZ? Since all my drives are very different and I don't want to buy new ones it's not really an option. If I really wanted I could do this with tiny slices, but it's more hassle than it's worth. It's very possible that I'll get similar drives and do it at some point though.



			
				killasmurf86 said:
			
		

> Dunno, AFAIK, if you use FreeBSD -RELEASE then deduption is not available (yet)


Indeed, I remembered wrong. It's not a problem though, until dedupe can be done offline I have little use for it anyway.



			
				killasmurf86 said:
			
		

> Music, video archives are already compressed, you won't gain anything.


Yep. But I have enough of compressible data to make it worthwhile.


			
				killasmurf86 said:
			
		

> Compression will introduce little latency.


Yes, but is it really something to worry about? I haven't seen any benchmarks, so all I can do is estimate. Pessimistically:
50 MB/s gzip decompression, 8 KB block size = 0.16 ms. There is likely some static overhead, let's double it. 0.32 ms.
HDD seek takes about 9 ms. With SSDs it may be something. Possibly with 128k blocks too. But unless I miss something big, in most cases it shouldn't be even measurable.
Based on this I'm gonna try 8k for most of the filesystem. The data set benchmarked is not really representable for me, but I haven't seen anything else and now I got more important things than benchmarks to do.
For some things like data/usr/src I can't imagine use case that wouldn't involve accessing whole files, so even 128k block size shouldn't hurt. I don't think it's worth bothering though...but I'm a compression junkie and I think I'll do it anyway. 



			
				killasmurf86 said:
			
		

> I mix knowledge of this thread http://www.daemonforums.org/showthread.php?t=1538
> With some howto from http://wiki.freebsd.org/ZFS (depending how you want to setup your PC)
> Generally it's easy


Thanks for the link, hope it works with PC-BSD too...



			
				killasmurf86 said:
			
		

> Read: http://forums.freebsd.org/showthread.php?t=6230
> 
> NOTE: everything I said is based on my own experience


I have it bookmarked already. 



			
				dandelion said:
			
		

> zpool(8) supports creating pools with lower version numbers (-o version=...), e.g. for grub2/linux/netbsd compatability. However, such pools wouldn't support features like user/group quota (v15) or deduplication (v21).


Good, I don't really need either feature so I'll get compatibility painlessly. Thanks.


			
				dandelion said:
			
		

> Only gzip compression is not supported (PR kern/153173) by (gpt)zfsboot. Booting from lzjb or zle compressed pools should and does work for me.


What part of the filesystem does it apply to? /boot?


----------



## dandelion (Dec 26, 2010)

Again said:
			
		

> What part of the filesystem does it apply to? /boot?


(gpt)zfsboot (stage2) and zfsloader (stage3) from boot(8). Note NULLs for gzip in zio_decompress_func_t in zio_compress_table in sys/cddl/boot/zfs/zfssubr.c which likely means it's not implemented.

So, yeah, only dataset containing /boot where zfsloader/grub2 and kernel reside cannot currently be compressed by gzip.


----------

