# Space overhead is much higher on ashift=12 raidz



## belon_cfy (Aug 19, 2012)

Hi,
My current server consume almost double of space on the ashift=12 raidz especially storing a lot of small files in the server even the recordsize has been set to 128K, may I know is there any zfs update to resolve this issue,  or should I need to choice mirror instead of raidz because space saving seems to be more noticable on mirror in this case.


----------



## Savagedlight (Aug 19, 2012)

Your RaidZ with ashift=12 is consuming twice as much space compared to what? Do you store the exact same data in both places?

Do you have drives with a 4k sector size? ashift=12 instructs ZFS to use record sizes which are a multiple of 4kiB (2^12), which prevents several performance problems with such drives.

If you set record size to 128k, minimum space taken by each file will be 128kiB regardless of the ashift setting. (I'm not entirely on this one)
Quoting the zfs() manpage, regarding recordsize; emphasis is mine:

```
For databases that create very large files but access them in small 
random  chunks,  these  algorithms  may be suboptimal. Specifying a
recordsize greater than or equal to the record size of the database
can	result	in significant performance gains. [B]Use of this property
for general purpose file systems is strongly discouraged[/B],  and  may
adversely affect performance.
```


----------



## gkontos (Aug 19, 2012)

The overhead is always an issue when dealing with very small files on 4K drives. You could choose not to align them for 4K but there would be a performance impact. Sometimes it is a matter of choice, speed versus capacity.

http://lists.freebsd.org/pipermail/freebsd-fs/2012-May/014311.html


----------



## t1066 (Aug 19, 2012)

I had some data about changes from 512 to 4k in the thread. The changes on the sizes of the filesystems /usr/src and /usr/ports are particularly interesting since they have lots of small files.


----------



## belon_cfy (Aug 20, 2012)

OK, so I would rather use 512b on a 4k disk for storing backup but sacrifice some of the performance, it will be better in space saving.


----------



## wblock@ (Aug 20, 2012)

Benchmark it first.  Doing misaligned operations can destroy performance.  Or maybe pick a filesystem that handles fragments better.  Setting UFS to optimize for space rather than speed might do it.  I'd think ZFS would do block suballocation also, maybe optionally.


----------



## phoenix (Aug 22, 2012)

Or, enable compression and not worry about it.


----------



## Sebulon (Aug 23, 2012)

phoenix said:
			
		

> Or, enable compression and not worry about it.



Well, if you had compression enabled before- when there was ashift=9, and have compression enabled now with ashift=12, the space consumed will have increased. I too have noticed this, which to me was to be expected. Especially on filesystems like t1066 said, /, /usr, /usr/local, /usr/ports, /var and so on. And no, or a very small increase on other filesystems where most files are larger, like /home e.g.

But I agree with you that between performance and a space increase on small files, performance is preferable.

/Sebulon


----------



## gkontos (Aug 23, 2012)

You can always use SAS drives instead.


----------



## Sebulon (Aug 23, 2012)

gkontos said:
			
		

> You can always use SAS drives instead.



Whaaat?

/Sebulon


----------



## gkontos (Aug 24, 2012)

Sebulon said:
			
		

> Whaaat?
> 
> /Sebulon



They are faster :e


----------



## Sebulon (Aug 24, 2012)

gkontos said:
			
		

> They are faster :e



You have got to be janking my chain, right. YouÂ´re joking or something, cause I donÂ´t follow. Speed has absolutely nothing to do with compression.

/Sebulon


----------



## gkontos (Aug 24, 2012)

Sebulon said:
			
		

> You have got to be janking my chain, right. YouÂ´re joking or something, cause I donÂ´t follow. Speed has absolutely nothing to do with compression.
> 
> /Sebulon



I was not referring to you nor to compression. I simply pointed out that SAS drives are much faster than SATA.


----------



## Sebulon (Aug 24, 2012)

gkontos said:
			
		

> I was not referring to you nor to compression. I simply pointed out that SAS drives are much faster than SATA.



Let me just point out to you that the topic discussed here is:
"*Space overhead is much higher on ashift=12 raidz*"

IÂ´m not trying to be rude or anything. ItÂ´s just that your comment had absolutely nothing to do with the subject. If you have a point, please explain further how that correlates to the subject. Would SAS drives somehow have a smaller space overhead than SATA with ashift=12?

/Sebulon


----------



## gkontos (Aug 24, 2012)

Sebulon said:
			
		

> Let me just point out to you that the topic discussed here is:
> "*Space overhead is much higher on ashift=12 raidz*"
> 
> IÂ´m not trying to be rude or anything. ItÂ´s just that your comment had absolutely nothing to do with the subject. If you have a point, please explain further how that correlates to the subject. Would SAS drives somehow have a smaller space overhead than SATA with ashift=12?
> ...



The answer regarding the overhead is here.

My later comment regarding SAS drives has to do with the fact that SAS drives will have better performance with ASHIFT=9 without the overhead.

Is that clear enough for you now?


----------



## Sebulon (Aug 24, 2012)

gkontos said:
			
		

> The answer regarding the overhead is here.
> 
> My later comment regarding SAS drives has to do with the fact that SAS drives will have better performance with ASHIFT=9 without the overhead.
> 
> Is that clear enough for you now?



Yes, and I'm sorry, I totally missed that post! That was the missing piece

/Sebulon


----------



## gkontos (Aug 24, 2012)

Sebulon said:
			
		

> Yes, and I'm sorry, I totally missed that post! That was the missing piece
> 
> /Sebulon



:beergrin


----------



## belon_cfy (Oct 12, 2012)

How about switching all the HDD to Seagate with smart align technology adopted so that I can maintain the same 512 block size but without sacrifice additional disk space and performance. Running raidz2 on 4k drive natively really waste more than double of space in my scenario. 

Anyway , will try to benchmark once i got it to see what is the performance impact.


----------

