# RAIDZ configuration for multiple SATA controllers



## pva (Jul 17, 2011)

I'm currently running a system with four (Advanced Format) drives attached to an internal SATA controller with no room for further expansion. I plan to add more storage by buying an external eSATA enclosure, stuffing some disks in it and connecting it to the machine using the bundled SilI3132-based SATA controller card.

While the existing four disks are configured as a simple striped ZFS pool (favouring cost and capacity over redundancy), I'm thinking of replacing it with a RAIDZ pool for improved redundancy. However, I'm unsure what kind of configuration I should go with.

The ZFS Best Practices Guide recommends the following:



> Start a single-parity RAIDZ (raidz) configuration at 3 disks (2+1)
> Start a double-parity RAIDZ (raidz2) configuration at 5 disks (3+2)
> Start a triple-parity RAIDZ (raidz3) configuration at 8 disks (5+3)
> (N+P) with P = 1 (raidz), 2 (raidz2), or 3 (raidz3) and N equals 2, 4, or 8
> The recommended number of disks per group is between 3 and 9. If you have more disks, use multiple groups.



Since I'll be using two separate controllers equipped with different makes of controller chips, I assume there might be some difference in performance. I'm especially sceptical about the cheapo SilI3132-based controller, since their performance has been reported to be lacklustre. Although I naturally can't be certain about the matter before I've run some benchmarks, my spider sense tingles that I'll be better off not mixing disks attached to different controllers under a single vdev.

This implies that I'll need to create at least one vdev per controller: one raidz1 vdev for the 4 internal disks and one (or more, see below) vdev for the external disks. The four-disk vdev might, according to some sources, represent a sub-optimal configuration for 4K sector disks, but those are the breaks.

Now, if I've understood correctly, I also won't be able to add any new devices to an existing vdev. Thus, I'll need to either buy 5 disks off the bat (as many as the external enclosure will accommodate) and create one raidz2 vdev, or buy, say, 2 disks to begin with and 3 more disks in the future, and create a mirror and a raidz1 vdev, respectively. The third option would be to buy disks when I need them and when I've filled up the enclosure, destroy the pool (I've got the data backed up elsewhere, remember?) and recreate it with a 4-disk raidz1 vdev and a 5-disk raidz2 vdev as I would've done if I'd bought 5 disks to begin with.

Finally, I could chuck the bundled controller and buy another card with at least four internal SATA ports and one eSATA port which supports port multipliers. This way I could nicely side-step the problem of having to contend with two separate controllers. Any suggestions as to which models would have the required number of ports and work with FreeBSD (preferably without any jiggery-pokery with proprietary drivers)?

Based on this information, what course of action would you recommend?


----------



## Sebulon (Jul 22, 2011)

Hi,

for the most part it's just about doing what you feel like, I think. Every option is valid and would work. I can't vouch personally for Sil3132 in particular, but I can say Sil in general is well supported and has stable quality. I have three Sil3124 PCI-X SATA controllers with the siis driver for the moment and I am satisfied with the performance and hot-swapping too.

One thing to keep in mind though, in your case, is more about not mixing disks than about not mixing controllers. The Advanced Format drives *needs* their own vdev, unless all are going to be like that. It has to do with the ashift value that is written to every new vdev, which is 9 for regular drives and 12 for Advanced Format. I have tried having a AF drive mixed with regular ones and watched with gstat as they worked and I saw that drive was working it's ass off while the others where idling, and the overall performance suffered greatly because of it. But if you set them up correctly, with the AF drives in their own vdev with ashift=12 and regular drives with ashift=9, all is well and they have about them same performance as any other disk.

/Sebulon


----------

