How to remove faulted disk from ZFS pool once spare replacement done

zfs zfsonlinux

5,064

To replace a failed disk with a hot spare, you do not need to zpool replace at all (and in fact this might cause you all sorts of grief later; I've never done this). Instead you are supposed to simply zpool detach the failed disk and the hot spare automatically replaces it.

5,064

Will Dennis

Updated on September 18, 2022

Comments

Will Dennis almost 2 years

I have a ZFS pool in the current state:

[root@zfs01 ~]# zpool status
  pool: zdata
 state: DEGRADED
status: One or more devices could not be used because the label is missing or
        invalid.  Sufficient replicas exist for the pool to continue
        functioning in a degraded state.
action: Replace the device using 'zpool replace'.
   see: http://zfsonlinux.org/msg/ZFS-8000-4J
  scan: scrub repaired 0 in 186h53m with 0 errors on Sun Jan 27 20:53:44 2019
config:

        NAME                                                  STATE     READ WRITE CKSUM
        zdata                                                 DEGRADED     0     0     0
          raidz3-0                                            DEGRADED     0     0     0
            ata-HGST_HUH728080ALE604_2EGWK97X                 ONLINE       0     0     0
            spare-1                                           DEGRADED     0     0     2
              ata-HGST_HUH728080ALE604_2EGWHSGX               UNAVAIL      0     0     0
              ata-HGST_HUH728080ALE604_2EGWD3WX               ONLINE       0     0     0
            ata-HGST_HUH728080ALE604_2EGGVUTX                 ONLINE       0     0     0
            ata-HGST_HUH728080ALE604_2EHG14TX                 ONLINE       0     0     0
            ata-HGST_HUH728080ALE604_2EGWW4XX                 ONLINE       0     0     0
            ata-HGST_HUH728080ALE604_2EGW5A5X                 ONLINE       0     0     0
            ata-HGST_HUH728080ALE604_2EGWTPYX                 ONLINE       0     0     0
            ata-HGST_HUH728080ALE604_2EGWALNX                 ONLINE       0     0     0
            ata-HGST_HUH728080ALE604_2EGWNN1X                 ONLINE       0     0     0
            ata-HGST_HUH728080ALE604_2EGWG0BX                 ONLINE       0     0     0
            ata-HGST_HUH728080ALE604_2EGWWTGX                 ONLINE       0     0     0
        logs
          mirror-1                                            ONLINE       0     0     0
            ata-INTEL_SSDSC2BA100G3_BTTV5435005Y100FGN-part1  ONLINE       0     0     0
            ata-INTEL_SSDSC2BA100G3_BTTV54350016100FGN-part1  ONLINE       0     0     0
        cache
          ata-INTEL_SSDSC2BA100G3_BTTV5435005Y100FGN-part2    ONLINE       0     0     0
          ata-INTEL_SSDSC2BA100G3_BTTV54350016100FGN-part2    ONLINE       0     0     0
        spares
          ata-HGST_HUH728080ALE604_2EGWD3WX                   INUSE     currently in use

errors: No known data errors

As you can see, I have added the spare disk "ata-HGST_HUH728080ALE604_2EGWD3WX" via the command zpool replace zdata /dev/disk/by-id/ata-HGST_HUH728080ALE604_2EGWHSGX /dev/disk/by-id/ata-HGST_HUH728080ALE604_2EGWD3WX which has now created a spare-1 device with boths disks in it (was expecting the spare 2EGWD3WX drive to replace the dead 2EGWHSGX one.)

How do I remove the dead drive ata-HGST_HUH728080ALE604_2EGWHSGX now?

Will Dennis over 5 years

Good to know for the next time... (but doesn't really answer my question)
Michael Hampton over 5 years

So what happened when you did it?
Will Dennis over 5 years

As you can see above, it made a "spare-1" device containing both the old failed and spare disks. The spare resilvered, and has "taken the place" of the old failed disk, except that now I need to remove the old failed disk from the pool. I'm just needing the correct command to do that.
Michael Hampton over 5 years

I already gave you the command to do that!
Will Dennis over 5 years

Sorry, that wasn't clear to me... So zpool detach zdata /dev/disk/by-id/ata-HGST_HUH728080ALE604_2EGWHSGX then?
Michael Hampton over 5 years

@WillDennis Just double check that's the UNAVAIL disk. It looks right to me, but never hurts to be sure. BTW, you don't have to use the full path; the basename shown in zpool status is sufficient. You only need the full path to add devices to a zpool.
code_dredd about 2 years

Ran sudo zpool detach <phys-device>. Online spare with AVAIL status did not take over. Physical device detached no longer shows up under /dev (i.e., /dev/sdb is gone now). Running into other issues that I'm now trying to troubleshoot. The worst part is that this answer seemed to make logical sense. It actually appears to be wrong.