Why is RAID not recommended for Hadoop HDFS setups?

Question

Various websites (like Hortonworks) recommend to not configure RAID for HDFS setups mainly because of two reasons:

1. Speed limited to slower disk (JBOD performs better).
2. Reliability

It is recommended to use RAID on NameNode.

But what about implementing RAID on each DataNode storage disk?

Solution

RAID is used for two purposes. Depending on the RAID configuration you can get:

1. Better performance: reading a file can be spread over multiple disks or different disks can be transparently used to read multiple files from the same file system.
2. Fault-tolerance: Data is replicated or stored using parity bits on multiple disks. If a disk fails, it can be recovered from another replica or recomputed using the parity bits.

HDFS has similar mechanisms built in software. HDFS splits files into chunks (so-called file blocks) which are replicated across multiple datanodes and stored on their local filesystems. Usually, datanodes have multiple disks which are individually mounted (JBOD). A datanode should distribute its file blocks across all its disks / local filesystems.

This ensures:

1. Fault-tolerance: If a disk or node goes down, other replicas are available on different data nodes and disks.
2. High sequential read/write performance: By splitting a file into multiple chunks and storing them on different nodes (and different disks), a file can be read in parallel by concurrently accessing multiple disks (on different nodes). Each disk can read data with its full bandwidth and its read operations do not interfere with other disks. If the cluster is well utilized all disks will be spinning at full speed delivering the maximum sequential read performance.

Since HDFS is taking care of fault-tolerance and "striped" reading, there is no need to use RAID underneath an HDFS. Using RAID will only be more expensive, offer less storage, and also be slower (depending on the concrete RAID config).

Since the namenode is a single-point-of-failure in HDFS, it requires a more reliable hardware setup. Therefore, the use of RAID is recommended on namenodes.

Comments

[bajran]

great answer @Fabian

[Jonathan]

who said anything about raid 0?

[Maksim Zinal]

The only RAID type which makes technical sense with HDFS is RAID0. However, it only adds more operational burden, so not worth it anyway.