# ZFS Setup with Geli and 4k alignment



## lockdoc (Feb 12, 2012)

Hi there,

I want to set up a new fileserver having the following hardware:

```
RAM: 8GB RAM
CPU: Intel Core2Duo 3.2Ghz
HDD: 8x 1TB Samsung Spinpoint F3
SSD: 1x Corsair 60GB Force Series 3 SATA 6GB/s
```

*Performance considerations*

So far I have heard that separating the pool from the data is better for performance.
I also _heard_ that using whole disks rather than partition them is also better for performance (correct me if *I* am mistaken), so I have decided to go this way:

+ A pool consisting of 4 x Mirrors (Raid1)
+ SSD for ZIL, L2ARC, Cache/Log and the base system itself

*Q1: *How to best partition the SSD and should i also use zfs for the os or rather ufs?

*Purpose*

It should be mainly for fileserving, but will also have a couple of jails (running inside the pool), such as dns, dhcp, mysql, apache, svn (all separate jails). The pool needs to be fully encrypted with geli.

*1.) Hard disk alignment Methods*

I found a couple of solutions to align the disks for 4k

*a) Windows Way*


```
gpart create -s gpt ada0
gpart add -t freebsd-zfs -b 1M ada0
```

*b) Using gnop*


```
gnop create -S 4096 /dev/ada0
```

I also have seen a combination of them. Now I am not too sure which method to take.

*Q2: *Which alignment method would be best?

*2.) Hard disk alignment methods with geli*

How does geli fit into all this?

*Q3: *Do I have to do another align for geli?


----------



## wblock@ (Feb 12, 2012)

Method A has less overhead--but maybe not signifcantly less--by not going through another GEOM device.  Calling it the "Windows way" seems unfair.


----------



## lockdoc (Feb 12, 2012)

@wblock:
So both work for pure zfs then. No offense, just called it *be*cause that's how windows does it.

So what about if I encrypt the drives with geli afterwards, how does it work with the alignment?


----------



## wblock@ (Feb 12, 2012)

No idea, I've never tried geli.


----------



## bbzz (Feb 13, 2012)

If you use *geli* you don't need to align with say *gnop*.

Assuming you labeled all disks,


```
geli init -b -s4096 -l256 /dev/label/disk0
..
geli init -b -s4096 -l256 /dev/label/disk7
```

That -s4096 means you will be encrypting in 4096 byte blocks. 

Once you geli attach all disks, this will give you /dev/label/disk0.eli .. /dev/label/disk7.eli, which you then add to zpool.

I would use one zpool for data (8 hdd) and another zpool for system. YMMV.
You may not want to use *-b* for geli.


----------



## lockdoc (Feb 13, 2012)

Thanks for the answers.

Is it actually a good idea to use the SSD for both, a) the system and b) ZIL, L2ARC and Cache/Log for the data pool? If yes, how should I best partition it, in terms of how much space for what, to have a stable/performent system?

Or should I rather also separate the system and the other stuff on two separate SSD's?


----------



## bbzz (Feb 13, 2012)

If you could afford it, separation from base system would be better.


----------



## lockdoc (Feb 13, 2012)

Ok so the best would be 1 SSD for the OS, 1 SSD for l2arc and a third SSD for zil.

As I need encryption, zil and l2arc need to run on geli as well. Now here is the point where I don't know if geli will be too much overhead for zfs cache, so that performance will actually drop, even though it is on a spare SSD. Any experience with encrypted l2arc?


----------

