Identify Disks on SuperMicro server running FreeBSD

Question

I am running on a SuperMicro 2U server with 6 disks in a ZPool. I want to identify each drive bay using the Drive Carrier LED from running a command from a tool in FreeBSD. Is there a way to do this?

I know if the drives were running on top of a Raid Card, I could use the raid card utility to identify each disk but in my case, there is no hardware raid.

Is there a tool to do this in FreeBSD or Linux?

Solution 1

For most of my ZFS solutions, you better bet that I have a table and a set of labels identifying drives by their partial SAS WWN. This is a function of the LSI controllers I use, which read like:

    NAME                        STATE     READ WRITE CKSUM
    vol1                        ONLINE       0     0     0
      mirror-0                  ONLINE       0     0     0
        c10t50000393482B340Cd0  ONLINE       0     0     0
        c10t50000393482B4CF0d0  ONLINE       0     0     0
      mirror-1                  ONLINE       0     0     0
        c10t50000393482B4DB4d0  ONLINE       0     0     0
        c10t50000393482BAB48d0  ONLINE       0     0     0
      mirror-2                  ONLINE       0     0     0
        c10t50000393482BDA68d0  ONLINE       0     0     0
        c10t500003935803910Cd0  ONLINE       0     0     0

There are a couple of options to getting this to work. One is the commercial SanTools SMARTmon utility that's available to OEMs and integrators. It leverages the SCSI Enclosure Services (SES) features in external JBOD units, but has some magic for internal disks, too.

You may also have an option depending on the controller you're using. Are you just using a motherboard controller or a purpose-built non-RAID SAS HBA?

Solution 2

The poor man's means to identify disks would be to issue a dd if=/dev/daX of=/dev/null and see which disk's activity LED is blinking most rapidly. This of course needs a period of low activity for the other disks in the system, but is very generic.

If you have a (LSI) SAS controller that would work with sas2ircu then you might use it to issue the "display" command to list available drives and their serials and subsequently run the "LOCATE" command to blink the light on the enclosure.

Solution 3

I know this is an old question, but it gave me a few of the pieces I put together, and i thought i'd shoot back the script i came up with, since this is an almost exact match to my situation: it requires sas2ircu: http://www.avagotech.com/docs-and-downloads/host-bus-adapters/host-bus-adapters-common-files/sas_sata_6g_p20/SAS2IRCU_P20.zip and from the ports, bash and sg3_utils

It does make a few assumptions, i think the main one is that it is attached to controller 0. you can use sas2ircu list to identify your controller number.

It will check the selected pool (via zpool status). If there are no errors it will:

• save a file (at /root/.sas2ircu/drives) with a mapping of device names to enclosure slots
• turn off any leds previously activated by this script (this is stored in /root/.sas2ircu/locs)

If there are errors it will:

• send an email with the full output of zpool status
• activate the leds of any failed drives (and store the locations activates in /root/.sas2ircu locs so they can later be deactivated)

anyway here is the script. I run it as an hourly cron job.

#! /usr/local/bin/bash
if [ ! "$1" ]; then
  echo "Usage: zpscan.sh pool [email]"
  echo "Scan a pool, send email notification and activate leds of failed drives"
  exit
fi
if [ ! -d /root/.sas2ircu ]; then
  mkdir /root/.sas2ircu
  touch /root/.sas2ircu/drives
  touch /root/.sas2ircu/locs
fi
if [ "$2" ]; then
  email="$2"
else
  email="root"
fi
condition=$(/sbin/zpool status $1 | egrep -i '(DEGRADED|FAULTED|OFFLINE|UNAVAIL|REMOVED|FAIL|DESTROYED|corrupt|cannot|unrecover)')
if [ "${condition}" ]; then
  emailSubject="`hostname` - ZFS pool - HEALTH fault"
  mailbody=$(zpool status $1)
  echo "Sending email notification of degraded zpool $1"
  echo "$mailbody" | mail -s "Degraded Zpool $1 on hostname" $email
  drivelist=$(zpool status $1 | grep -E "(DEGRADED|FAULTED|OFFLINE|UNAVAIL|REMOVED|FAIL|DESTROYED)" | grep -vE "^\W+($1|NAME|mirror|logs|spares)" | sed -E $'s/.*was \/dev\/([0-9a-z]+)/\\1/;s/^[\t  ]+([0-9a-z]+)[\t ]+.*$/\\1/')
  echo "Locating failed drives."
  for drive in $drivelist;
  do
  record=$(grep -E "^$drive" /root/.sas2ircu/drives)
  location=$(echo $record | cut -f 3 -d " ")
  echo Locating: $record
  sas2ircu 0 locate $location ON
  if [ ! "$(egrep $location /root/.sas2ircu/locs)" ]; then
  echo $location >> /root/.sas2ircu/locs
  fi
  done
else
  echo "Saving drive list."
  drivelist=$(zpool status $1 | grep -E $'^\t  ' | grep -vE "^\W+($1|NAME|mirror|logs|spares)" | sed -E $'s/^[\t ]+//;s/([a-z0-9]+).*/\\1/')
  saslist=$(sas2ircu 0 display)
  printf "" > /root/.sas2ircu/drives
  for drive in $drivelist;
  do
  sasaddr=$(sg_vpd -i -q $drive 2>/dev/null | sed -E '2!d;s/,.*//;s/  0x//;s/([0-9a-f]{7})([0-9a-f])([0-9a-f]{4})([0-9a-f]{4})/\1-\2-\3-\4/')
  encaddr=$(echo "$saslist" | grep $sasaddr -B 2 | sed -E 'N;s/^.*: ([0-9]+)\n.*: ([0-9]+)/\1:\2/')
  echo $drive $sasaddr $encaddr >> /root/.sas2ircu/drives
  done

  for loc in $(cat /root/.sas2ircu/locs);
  do
  sas2ircu 0 locate $loc OFF
  done
  printf "" > /root/.sas2ircu/locs
fi

  for loc in $(cat /root/.sas2ircu/locs);
  do
  sas2ircu 0 locate $loc OFF
  done
  printf "" > /root/.sas2ircu/locs
fi

Solution 4

I know this is an old question, but for the searchers hitting this, on FreeBSD >= 10.3, you can use sesutil locate da2 on to turn on da2's LED provided it is in an enclosure that provides a /dev/sesN device.

Solution 5

I don't know about FreeBSD, but in Linux, there is a collection of software from Intel called ledmon(8) and I have used the userspace program ledctl(8) to flash the "locate" LED on a particular drive in my SuperMicro SC847E26-RJBOD1 that have drives controlled by an LSI SAS3008 HBA (not MegaRAID):

$ sudo ledctl locate=/dev/sdce

Then once I located it, I turned off the "locate" LED:

$ sudo ledctl locate_off=/dev/sdce

sgpio(1) is required for the communication between the HBA and the backplane.

View more solutions

Comments

[Roman]

Instead of "Low activity", better stop all I/O by unmounting the disk and shutting off patrol jobs.

[the-wabbit]

Note: the cXtYdZ notation is Solaris only. Most Linux distros have the /dev/disk/by-id/ directory containing symlinks from the WWN-containing names to /dev/sdX style names. No idea, if FreeBSD has something similar.

[Therealstubot]

sas2ircu FTW! Thanks, you saved me. Incidentally, I have a IT flashed LSI HBA with a supermicro 12 bay backplane with single mini-sas. Running on supermicro mobo, e3-1230v3, 32gb ecc. running FreeNAS 9.3 latest. LOCATE command flashed red ( error ) LED, leaving blue activity LED with normal function. Display command curiously shows that the LSI controller is #0 but the enclosure shows up as #2 ( Likely the mobo sata 2 and sata 3 controllers being #0 and #1, but that's conjecture ).