20 1TB drives vs. 10 2TB drives in RAID5/6 server

Question

I will be setting up a server at work and I need some advice on some details. The setup will be one blade-type server (8-core, 16GB RAM) with two subsystems - one for the main storage the other to back it up. I'm shooting for a 20TB array (I know it'll be less after formatting and parity drives).

Is there any advantage one way or the other with either 20 1TB drives or 10 2TB drives?

I'm not sure right now how many controllers I should have either (in the quote I have is a dual-port controller). I would think two controllers for a server of this size would be a better choice than the dual-port controller (but I really don't know).

Would an array of this size have any performance issues in RAID 5 or 6 (I know RAID 5 or 6 are "slower" because of all the parity calculations).

Also, these will be either WD RE3 (1TB) or the RE4 (2TB).

Oh, also, for the backup array would it be ok to use the WD 2TB green drives (also in RAID5 or 6)?

Solution 1

20 drives 1tb - more IO performance for at least reading (depends how you set them up).

10 drives 2tb: less IO perforamnce, problems with RAID 5 quality - go fo RAID 6 to avoid issues in case a disc fails. 2tb drives are the limit to reliably rebuild.

I'm not sure right now how many controllers I should have either (in the quote I have is a dual-port controller).

You should go for a SAS setup. Would be ok to use 1 SAS port, though I would go with a dual port cage an a dual port controller. I ahve a setup similar to that (well, smaller, faster) using WD 300gb Velociraptors and use an Adaptec 5805 and a Supermicro cage with 24 slots in 1 rack unnits (2.5" all) and a SAS backplane for them.

note that SAS can handle SATA drives - they are logically and physically compatible, just plug them in.

The Adaptec handles up to around 190 discs ;) And has plenty of CPU perforamcne for that little IO.

Oh, also, for the backup array would it be ok to use the WD 2TB green drives (also in RAID5 or 6)?

Backup - in general slower is ok. So, yes.

Solution 2

If you're using the controller to do your RAID'ing then having two controllers will obviously mean you'll have two separate arrays (i.e. 2 x 10 disks or 2 x 5 disks), if you're using software RAID then this isn't a problem.

Also something to be aware of; 20 disks in a R6 config = 18*(1000*.95)=17.1TB, Windows (you don't mention what OS you're using) will support 16TB volumes with the largest cluster size but any more than that is 'unsupported' at best, if it's linux then things vary.

Now onto the question, and my answer comes down to how important the data you're storing is. I consider my primary focus to be the maintenance of existing data, everything else is secondary - with that in mind I'd be tempted to go for a 20 x 2TB RAID 10 configuration if you have the budget, if not then the 10 x 2TB RAID 6 config and lastly the 20 x 1TB RAID 5. I'd also make sure I had a few spare drives lying around nearby and ensure I had my drive-failure alerting working fine too.

Regarding backup then for intermittent (i.e. non-24/7) use those green drives are fine.

Comments

[pehrs]

Don't forget that rebuilding an array of that size is likely to be a painful experience.

[John Gardeniers]

Regardless of which way you go you should really factor in a couple of hot spares.

[pauska]

One spare should be sufficient with RAID6

[John Gardeniers]

In principle one spare would be enough for RAID 6 but with such large drives there is an increased risk of another drive going down while the array is rebuilt, which will take a fair bit of time. If it was my system I'd want that second spare.

[Posipiet]

Spares should be avoided on Raid 5/6 this big. The automatic rebuild is a problem. With arrays this big, chances are it will break again during rebuild. Once a drive fails, you want to check your backup, and choices, before a rebuild and start a manual rebuild over the weekend.

[user3759402]

ok, so I asked about a build of a 16TB RAID10 system using RE3 drives and was told upgrading the case and controller to handle 32 1TB drives would be too expensive. New quote is 16TB using RE4 drives (2TB) in RAID10 for the master system and the 2TB green drives for the backup. Erik suggests not going over 1TB drives and it seems like most of you would rather use RAID10 over RAID5/6 -makes sense to me. Faster and more fault tolerant, right? But since this isn't an enterprise environment should I really be concerned about using 2TB RE4 drives? They are designed to be enterprise-class after all.

[Posipiet]

Did you have a successful rebuild on that raid?

[pauska]

Hi. 2TB drives don't have more faults than 1TB drives, it's just that rebuilding an array after a 2TB disk failure takes twice the amount of time that a array with 1 TB disks would. Remember that performance suffers (alot) when doing RAID rebuild.