Can I detect hardware RAID infromation from inside Linux?

Question

When I'm inside of Linux, I can get the following information from lsblk (irrelevant drives removed from output):

NAME  MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda     8:0    0   298G  0 disk 
sdb     8:16   0   2.7T  0 disk

When I manually pull the drives out of the server, I can tell I'm physically using the following drives:

0  Seagate 320GB
1  Seagate 320GB
2  Hitachi 1TB
3  Hitachi 1TB
4  Hitachi 1TB
5  Hitachi 1TB
6  [empty]
7  [empty]

Because there is more physical storage in the server than available space in Linux, this means I'm obviously using some form of RAID system. With a bit of math, I can often figure out what type of RAID system is being used.

Is there a way for me to detect if I'm using hardware RAID from inside of Linux, and figure out all the information about it (such as type of RAID, available drives) without turning off the server, physically pulling the drives out, and reading their labels?

Can this information be gathered from inside of Linux, or is the point of hardware RAID to make the underlying system "invisible" to the operating system?

Solution 1

How to get the RAID information is going to depend entirely on the RAID controller you are using. Often, manufacturers will have tools that can be downloaded from their website which can be used to query the RAID controller and get this information.

In order to find which RAID controller you are using, try one of the following commands:

lspci

# lspci -knn | grep 'RAID bus controller'
08:00.0 RAID bus controller [0104]: 3ware Inc 9690SA SAS/SATA-II RAID PCIe [13c1:1005] (rev 01)

Here, the information we are looking for is "3ware Inc 9690SA SAS/SATA-II RAID PCIe".

lsscsi

The command is not available on Debian and Ubuntu, but a quick sudo apt-get install lsscsi will fetch it from the repos. Note, if you are not using a RAID controller, the manufacturer and model number of your harddrive will show up here instead.

# lsscsi
[2:0:0:0]    disk    AMCC     9690SA-8I  DISK  4.08  /dev/sda 
[2:0:1:0]    disk    AMCC     9690SA-8I  DISK  4.08  /dev/sdb 

Here we see the manufacturer is "AMCC" and the model number of the RAID card is "9690SA-8I". A quick Google search shows that this card is also known as "AMCC 3Ware 9690SA-8I".

lshw

A third method (which gives quite a bit of output data) is to use the lshw command. Run lshw -class disk as root to only display the details about harddrives (which includes RAID information).

Finding the RAID controller tools

Now that we have the manufacturer and model number, it should be possible to find the tools on their website, or at least be able to Google details on how to find and use the tools for that specific controller.

If the manufacturer shows up in this list, see these answers for more details on how to get the RAID information for your card:

• AMCC - 3ware controllers
• LSI Logic / Symbios Logic
• Adaptec (some devices)

Solution 2

Run something like lspci -knn | grep 'RAID bus controller'.

Using that output, Google (for example) for LSI Logic / Symbios Logic MegaRAID SAS 2208.

Find it uses the storcli utility to interrogate the RAID controller.

Download it and install it.

storcli64 show gives you the model of controller specifically, and the controller index, number of drive groups, and virtual drives.

storcli64 /c0/d0 show shows you first controller, first drive group. Tells you raid levels, including nesting.

storcli64 /c0/eall/sall show all shows you all the information on all the disks.

Further reference data for those commands can be found here:

http://mycusthelp.info/LSI/_cs/AnswerPreview.aspx?sSessionID=&inc=8275

Should be all you have to do. Just use the normal hardware introspection in Linux, then Google, then download and install the utilities that go with it.

@Gene's suggestions and comments are absolutely spot-on.

This answer is entirely specific to the output you get from the lspci command, and the ability for your Google search to identify the manufacturer and to grab the correct command line tool.

But it does show that you can get all of this from a server, on a command line, without halting the machine, opening the case, and pulling drives, which I hope is helpful.

Solution 3

For creating a storage profile, I'd do the following before running lspci or downloading anything.

cat /etc/fstab         # What partitions get mounted at boot?

sudo pvs               # Are you running LVM? List the physical volumes

cat /proc/mdstat       # List software RAID devices & status

for D in /dev/sd?; do parted $D print; done
                       # If hardware RAID is being used, the "Model" field will list
                       # the Mfr. name, otherwise the hard drive Mfr.

Solution 4

This isn't a one-size-fits-all answer and doesn't give you all the information you need, but on one Adaptec hardware RAID controller we've used it. It gave some access to the drives themselves via special devices /dev/sg1, /dev/sg2, etc.

We could run smartctl -a /dev/sg1 to get a lot of info on that physical drive including manufacturer, model number, interface, serial number, size, and other data.

As far as figuring out which controller is being used, I agree with Gene's comment about dmidecode, dmesg, and lspci - those would be my ones to try first as well.

Solution 5

In addition to the lspci and lshw methods described in the other answer, you can parse dmesg and use smartctl to obtain informations about the physical drives. For example, give a look at this server with a PERC H710p controller:

[root@localhost ~]# dmesg | grep -i scsi
[    0.946723] SCSI subsystem initialized
[    1.279256] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)
[    2.853831] scsi host0: Avago SAS based MegaRAID driver
[    2.858058] scsi 0:2:0:0: Direct-Access     DELL     PERC H710P       3.13 PQ: 0 ANSI: 5
[    2.873982] scsi 0:2:1:0: Direct-Access     DELL     PERC H710P       3.13 PQ: 0 ANSI: 5
[    3.011376] sd 0:2:0:0: [sda] Attached SCSI disk
[    3.012031] sd 0:2:1:0: [sdb] Attached SCSI disk
[    7.915918] sd 0:2:0:0: Attached scsi generic sg0 type 0
[    7.916305] sd 0:2:1:0: Attached scsi generic sg1 type 0

You can see two arrays defined: sda and sdb. Running smartctl against such virtual devices retuns something as the following:

[root@localhost ~]# smartctl --all /dev/sdb
smartctl 7.0 2018-12-30 r4883 [x86_64-linux-3.10.0-1062.12.1.el7.x86_64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               DELL
Product:              PERC H710P
Revision:             3.13
Compliance:           SPC-3
User Capacity:        9,930,501,324,800 bytes [9.93 TB]
Logical block size:   512 bytes
Logical Unit id:      REDACTED
Serial number:        REDACTED
Device type:          disk
Local Time is:        Fri May 29 14:46:48 2020 CEST
SMART support is:     Unavailable - device lacks SMART capability.

To check the physical disk health values you can use smartctl --all -d <driver>,<disknum> <sddevice> syntax. For example, to check the first disk of the second array:

[root@localhost ~]# smartctl --all -d megaraid,0 /dev/sdb
smartctl 7.0 2018-12-30 r4883 [x86_64-linux-3.10.0-1062.12.1.el7.x86_64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     REDACTED
Serial Number:    REDACTED
LU WWN Device Id: REDACTED
...                      
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   100   253   021    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       3
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   097   097   000    Old_age   Always       -       2251
 10 Spin_Retry_Count        0x0032   100   253   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   253   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       3
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       1
193 Load_Cycle_Count        0x0032   200   200   000    Old_age   Always       -       3
194 Temperature_Celsius     0x0022   123   117   000    Old_age   Always       -       20
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   253   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   100   253   000    Old_age   Offline      -       0

Finally, to obtain array layout information (ie: RAID type, spare disks, etc) you had to use specific tools. megacli is a generic tool for LSI-based cards, but vendor-customized tools exist also (ie: omreport storage vdisk). For example, on a DELL server:

[root@localhost ~]# omreport storage vdisk
List of Virtual Disks in the System

Controller PERC H710P Mini (Embedded)
ID                                : 0
Status                            : Ok
Name                              : REDACTED
State                             : Ready
Hot Spare Policy violated         : Not Assigned
Encrypted                         : No
Layout                            : RAID-6
Size                              : 64.00 GB (68719411200 bytes)
T10 Protection Information Status : No
Associated Fluid Cache State      : Not Applicable
Device Name                       : /dev/sda
Bus Protocol                      : SATA
Media                             : HDD
Read Policy                       : Adaptive Read Ahead
Write Policy                      : Write Back
Cache Policy                      : Not Applicable
Stripe Element Size               : 64 KB
Disk Cache Policy                 : Enabled

View more solutions

Comments

[Gene]

It depends entirely on the RAID controller you are using and what support it has under linux. If you provide information about your controller you might find help here. You could also do a web search for "model# linux commands" if you don't get the information you need here.

[IQAndreas]

@Gene That's unfortunate. Again, I was hoping there was a solution which does not require looking up the specific model number, but can be used regardless of what hardware you are using.

[Gene]

Different hardware do RAID differently and have different drivers. In a lot of cases you don't have to power your system down, but at the very least you need to know what model controller or chipset you have so you can find commands to query the controller from the operating system.

[Gene]

You typically can find the controller or chipset information via dmesg, dmidecode, lspci, and other similar utilities. If it's built-in RAID on the motherboard knowing the make and model of the motherboard will get you 99% of the way there.

[Zoredache]

adds an answer explaining how to figure out which RAID controller is being used from inside of Linux - @IQAndreas but that isn't in the question. Answers on serverfault should be answer the question being asked. If you Have a different question you should edit your question. That question could almost certainly be answered yourself by a few minutes on Google. Anyway the answer to that is lshw.

[IQAndreas]

Since the answer to my question is going to be hardware specific, what I did was add a community wiki answer on how to find out what RAID controller you are using. This answer then links to answers (including yours) that provide details for specific controllers or manufacturers.

[IQAndreas]

If you want to update the format of your answer to reflect this, it would be nice, but not strictly necessary.