Using software-RAID vs "firmware"-RAID (a.k.a. FakeRAID)

Question

We recently bought a tower server on which I want to install Debian. I thought the device had hardware RAID, as I could see a BIOS screen. As it turns out, it's still a software RAID. When I configure the RAID drives through that firmware, I can still see both hard-drives in the Debian installer. When I try to repartition from within the installer, Debian warns that the software RAID drives would be lost.

I was slightly disappointed by this, as I always thought hardware-RAID would yield better performance. But anyhow, my question now is whether I should use the Debian installer to configure the RAID drives, and disable this "firmware" RAID from the main BIOS? Or should I create separate drives using this firmware, and not use the installer? Are there any reliability benefits of using the firmware?

This firmware version is:

LSI MegaRaid Software RAID BIOS Version A.10.10211615R
LSI SATA RAID Found at PCI Bus No:00 Dev No:1F

Update I can see from the comments that others were also confused as to whether I was dealing with a Hardware RAID. The machine I have is a Lenovo ThinkServer TS 440. In its datasheet, it says that is has

Integrated SATA SW RAID 0/1

As I understand, there is a hardware component included in a chip on the motherboard (which is why it has a PCI address?), but is not a typical hardware RAID.

Solution 1

How I Learned to Stop Worrying and Love Software RAID:

Back when all of our server storage was on SCSI disks, I used to be pretty particular about using good hardware RAID -- we used HP/Compaq SmartArray controllers and had extremely good luck with them. I operated for a long time under the assumption that software RAID would cause a performance hit and not be as robust to failure as good, battery-backed hardware RAID controllers.

However, as we've moved to cheaper SAS and SATA storage, I've come to appreciate software RAID a whole lot more. Realizing that I could simply build a RAID array in the OS directly created a lot of flexibility and cost savings (true hardware controllers with battery-backed cache are still expensive) without too much of a performance hit (depending of course on the rest of the hardware).

The tradeoff in our case has boiled down to that robustness and performance vs. low-cost and flexibility. Unfortunately, most "fakeraid" solutions are the worst of both worlds: the performance is still less than true hardware RAID, since in most cases the CPU and system memory are doing much of the work rather than the fakeraid controller, robustness to failure is generally less than hardware RAID as the controllers are cheaper and again rely on the rest of the system hardware, and the flexibility is less as you can't necessarily rebuild an array on different hardware (as you could in pure software RAID). Pure hardware or software RAID is almost always a better choice than fakeraid.

With all that said, here are my recommendations for making software RAID work in Debian on a system with a fakeraid controller:

• Disable the "RAID" firmware entirely in the BIOS -- set it to ACHI (if you're using SATA), JBOD or whatever other setting is likely to let you pass the disks through to the OS as directly as possible.

• Use mdraid/mdadm rather than dmraid.

• Use cat /proc/mdstat to check on failure/rebuild status. Check this often, and set up the automated email alerting for when a disk fails.

• For best results, keep RAID at the bottom of your storage stack. If you plan to use encryption and/or LVM, create those volumes on top of the RAID array (see this question for some specifics and note that the issue mentioned seems to have been resolved in more recent debian/ubuntu).

• Keep your kernel as up-to-date as possible, especially if you use SSD - Features like TRIM support are being added and improved continually.

Solution 2

as I always thought hardware-RAID would yield better Performance

How?

At the end the MegaRaid IIRC is a SAS Controller and can address 100+ hard discs, but it will not magically make them faster. Especially not slow SATA discs. You likely hit the IOPS Limit of the drives before (Software) The CPU is even slightly taxed.

Start putting some real stuff behind it and you may see a difference - mostly the Hardware raid will not tax your bandwidth (emmory, pci-e Slot) so much.

If you have the bbu (battery backup unit) you can use write back Caching in the Controller - then things start getting better. But with a smal lset of SATA discs - no, no Speed difference.

I would still use it ;)

Comments

[Zoredache]

The linux MD RAID typically performs better, and is easier to manage the 'fakeraid' controllers you see.

[Chopper3]

Have you no option of just buying a hardware RAID controller? They're not too expensive

[kralyk]

"LSI MegaRaid Software RAID BIOS Version A.10.10211615R LSI SATA RAID Found at PCI Bus No:00 Dev No:1F" - that is hardware. The MegaRaid software BIOS lets you setup the hardware raid before the OS boots and handles the controller that's most likely on the mobo hence it finding it on the PCI bus. Sure sounds like you have a hardware controller to me.

[TomTom]

Oh my. now we get a new Level of pros - MegaRaid (LSI) is one of the BIG brands for Hardware Enterprise raid Controllers, probably followed by Adaptec and then nothing. The OP just really named it badly, but that is basically a question between hardaware and software.

[Roman]

possible duplicate of RAID - software vs. hardware

[Atari911]

There is one thing we need to keep in mind when talking about hardware RAID. If it doesn't have a dedicated CPU for handling the RAID then it is software RAID. Just because you have a hardware add-on card does not mean it is a hardware RAID. Without a dedicated processor and memory for the RAID you are offloading the work onto your systems CPU and memory.

[user1496984]

@Roman I really don't think I am dealing with a hardware RAID in this case, although there doesn't seem to be a consensus among the answers. Like Atari911 wrote, this is a weird middle ground of a RAID system that has a hardware component, but is not an actual hardware RAID.

[Roman]

Woops, right, I only read the question title. Adjusting.

[Philip]

The MegaRaid mentioned in the question is not the MegaRaid you're thinking of; it's SATA + Fakeraid. But otherwise you're correct.

[Chopper3]

Writes to Hardware RAID controllers only cross the bus once, this can help with performance, also hardware driven rebuilds are usually a lot quicker and less impactful on overall performance than software RAID.

[ewwhite]

Software RAID downside... no battery or flash-backed write cache in most implementations... Leaves a lot of performance on the table.

[Atari911]

Most of the performance increase you would see would be because you are offloading your CPU processes to the hardware RAID itself. This is what makes it a hardware RAID. The speed of the disks themselves are not impacted as you said, but I don't think that is what the original poster was referring to.

[user1496984]

Thanks, this sounds like the best option. It's a shame because I could have bought a potentially cheaper machine, and just relied on the software RAID feature in Linux.

[TomTom]

Yes, but as long as those discs a 4,6,8 SATA discs, the bus bandwidth and the CPU load will be neglegible.

[Thomas Guyot-Sionnest]

"But with a smal lset of SATA discs - no, [BBU makes] no Speed difference." -- If you say so then most likely you have write caching enabled on the SATA disks. From experience, many controllers will enable write cache on SATA disks, maybe because makers think ppl buying SATA don't care about reliability??. IF YOU HAVE DISK WRITE CACHE ON, THEN YOUR DATA IS AT RISK! Once you turn disk write caching off, then you will notice an improvement using WriteBack cache with BBU on most write-heavy loads.