# Perfomance of Geli compared to Luks



## moritz_schlarb (Mar 8, 2012)

My HP Proliant Microserver, which I want to use as a NAS, is using an AMD Turion II Neo N40L processor. The plain disks give me about 100 MB/s using filebench with the fivestreamwrite/multistreamwrite workloads (as software RAID-1 under Linux and zpool mirror under FreeBSD).

In Linux, using LUKS with cipher aes-xts-plain64 on a software RAID-1 I get the same ~100 MB/s for the same benchmarks.

In FreeBSD, using a mirror zfs pool with underlying GELI with AES-XTS I get only ~50 MB/s.

As the encryption algorithms should be the same, I'm wondering why FreeBSD is that slow for nearly exactly the same use case. 
Any ideas how I could tweak my settings in FreeBSD?


----------



## SirDice (Mar 8, 2012)

ZFS doesn't quite fit into GEOM. It works but it's not a tight fit. Try a test with gmirror(8).


----------



## moritz_schlarb (Mar 8, 2012)

Thx, SirDice, but I want to use ZFS (with a four disk RAIDZ1), so I don't want to use gmirror for mirroring...


----------



## PTVi (Apr 2, 2012)

Hi Moritz,

Do you have already a solution? As *I* write in my thread - performance is not really satisf*y*ing .

Please tell me how you have solved your problem. I don*'*t want to switch back to linux.

*B*est regards and many thanks.
Patrick


----------



## moritz_schlarb (Apr 6, 2012)

Hi Patrick,

*M*y solution was indeed to just use Linux. Made m*e* kind of sad, because I really *wanted* to use FreeBSD, but I didn't want to do it by all means.

Did the investigation on "lying" disc sector size bring any new results? I suspected something like that, too, but since I compared performance between plain ZFS and GELI ZFS, I don't think the sector sizes could interfere there in any way, could they?

If someone has some insight, please let us know! 

Greetings
Moritz


----------



## vermaden (Apr 6, 2012)

SirDice said:
			
		

> ZFS doesn't quite fit into GEOM. It works but it's not a tight fit. Try a test with gmirror(8).



Say what?

It*'*s the only possible way to encrypt ZFS volumes on FreeBSD and there are no know issues for it.

You create encrypted GELI devices and then create ZFS pool on them.

I am using that setup in many places and it works like a charm.

Also what does GMirror have to do with ZFS, they are two VERY different things.


----------



## Zare (Apr 6, 2012)

Point is that ZFS does rely on low-level disk I/O and running a GEOM class in between makes things slower due to translations.


----------



## graudeejs (Apr 6, 2012)

It ain't that bad....


----------



## vermaden (Apr 6, 2012)

Its not that GEOM slows ZFS, when ZFS is put on top of GEOM ENCRYPTED provider, ENCRYPTION makes writes/reads slower.


----------



## SageRaven (Apr 6, 2012)

vermaden said:
			
		

> Its not that GEOM slows ZFS, when ZFS is put on top of GEOM ENCRYPTED provider, ENCRYPTION makes writes/reads slower.



Which brings us back to the original question: why does Linux LUKS+RAID1 apparently whomp FreeBSD's GELI+ZFS.  (Not saying it does in all cases -- just that's what was asked.)

As for suggestions on FreeBSD GELI tuning, make certain you use 4096-byte sectors (*geli init -s 4096*), which makes a night-and-day difference in performance from the default of 512.  AES-XTS is slower than CBC.  AES-256 is slower than AES-128.

The crypto will be the bottleneck.  For example, my ada2 device tops out at 140MB/s for reads, and the ada2.eli device at about 75MB/s, almost a 50% performance hit.  With hardware crypto!  AES-XTS/256 using hardware AES-NI on my FX-8150 clocked at 4.1GHz.

I can get around 100MB/s reads using AES-CBC/128 in hardware.


----------

