What is the recommended disk configuration for slave nodes in your Hadoop cluster with 6 x 2 TB hard drives?

Question

Options

• ARAID 10
• BJBOD
• CRAID 5
• DRAID 1+0

• A
  11% (4)
• B
  77% (27)
• C
  9% (3)
• D
  3% (1)

Explanation

Note: Let me be clear here...there are absolutely times when using a Enterprise-class storage device makes perfect sense. But for Hadoop it is very much unnecessary, and it is these three areas that I am going to hit as well as some others that I hope will demonstrate that Hadoop works best with inexpensive, internal storage in JBOD mode. Some of you might say "if you lose a disk in a JBOD configuration, you're toast...you lose everything". While this might be true, with Hadoop, it isn't. Not only do you have the benefit that JBOD gives you in speed, you have the benefit that Hadoop Distributed File System (HDFS) negates this risk. HDFS basically creates three copies of the data. This is a very robust way to guard against data loss due to a disk failure or node outage, so you can eliminate the need for performance-reducing RAID.

No community discussion yet for this question.