Solaris 11 How to replace the faulty disk in SVM using metareplace

Most of the time we are in the situation to replace the faulty disk depends on the Volume manager which we are using. In this post we are going to discussing about the method of procedure to replace the faulty disk in SVM control with using "metareplace" command. 

Please refer VxVM faulty disk replacement

High Level Plan :

• Identify the faulty disk
• Deleting the Meta database
• Replacing the faulty disk
• Copying the Volume Table of Content (VTOC) to New disk
• Creating Meta database
• Executing "Metareplace" command to replace the disk from SVM.
• Checking the status of SVM

Found the below error messages from system logs

root@unixrock# grep md_mirror /var/adm/messages
Mar11 05:55:32 unixrock md_mirror:[kern.warning]:md:d20:/dev/dsk/c1t1d0s0 needs maintenance
Mar11 05:55:32 unixrock md_mirror:[kern.warning]:md:d21:/dev/dsk/c1t1d0s1 needs maintenance
Mar11 05:55:32 unixrock md_mirror:[kern.warning]:md:d25:/dev/dsk/c1t1d0s5 needs maintenance
Mar11 05:55:35 unixrock md_mirror:[kern.warning]:md:d24:/dev/dsk/c1t1d0s4 needs maintenance
root@unixrock#

Checking disk status in OS and SVM level

root@unixrock# echo|format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
       0. c1t0d0 DEFAULT cyl 1304 alt 2 hd 255 sec 63
          /pci@0,0/pci15ad,1976@10/sd@0,0
       1. c1t1d0 drive type unknown
          /pci@0,0/pci15ad,1976@10/sd@1,0
Specify disk (enter its number): Specify disk (enter its number):
root@unixrock#
root@unixrock#
root@unixrock# metastat -c
d1               m  1.0GB d11 d21  (maint)
    d11          s  1.0GB c1t0d0s1
    d21          s  1.0GB c1t1d0s1 (maint)
d5               m  400MB d15 d25  (maint)
    d15          s  400MB c1t0d0s5
    d25          s  400MB c1t1d0s5 (maint)
d4               m  494MB d14 d24  (maint)
    d14          s  494MB c1t0d0s4
    d24          s  494MB c1t1d0s4 (maint)
d0               m  8.0GB d10 d20  (maint)
    d10          s  8.0GB c1t0d0s0
    d20          s  8.0GB c1t1d0s0 (maint)
root@unixrock# 

We have identified our faulty disk (c1t1d0) and also we can see the metadevice status as maintenance.
Well, Let we can check and delete the Meta database with reference to the faulty disk

root@unixrock# metadb
        flags           first blk       block count
     a m  p  luo        16              8192            /dev/dsk/c1t0d0s7
     a    p  luo        8208            8192            /dev/dsk/c1t0d0s7
     a    p  luo        16400           8192            /dev/dsk/c1t0d0s7
      W   p  l          16              8192            /dev/dsk/c1t1d0s7
      W   p  l          8208            8192            /dev/dsk/c1t1d0s7
      W   p  l          16400           8192            /dev/dsk/c1t1d0s7
root@unixrock#
root@unixrock# metadb -d /dev/dsk/c1t1d0s7
root@unixrock#
root@unixrock# metadb
        flags           first blk       block count
     a m  p  luo        16              8192            /dev/dsk/c1t0d0s7
     a    p  luo        8208            8192            /dev/dsk/c1t0d0s7
     a    p  luo        16400           8192            /dev/dsk/c1t0d0s7
root@unixrock#

NOTE : "W" flag is represent that replica has device write errors.
Since faulty disk has completely gone, we can request them to replace the failed disk physically and Once its done we scan the new disk by executing devfsadm command.

root@unixrock# devfsadm -Cc disk
root@unixrock# echo|format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c1t0d0  DEFAULT cyl 1304 alt 2 hd 255 sec 63
          /pci@0,0/pci15ad,1976@10/sd@0,0
       1. c1t1d0  DEFAULT cyl 2086 alt 2 hd 255 sec 63
          /pci@0,0/pci15ad,1976@10/sd@1,0
Specify disk (enter its number): Specify disk (enter its number):
root@unixrock#

Yes, We have new disk in placed. Now we ready to copy the VTOC to the new disk

root@unixrock# prtvtoc /dev/rdsk/c1t0d0s2|fmthard -s - /dev/rdsk/c1t1d0s2
fmthard: Partition 2 specifies the full disk and is not equal
full size of disk.  The full disk capacity is 33511590 sectors.
fmthard:  New volume table of contents now in place.
root@unixrock#

Creating the New meta database on new disk.

root@unixrock#
root@unixrock# metadb -afc3 /dev/rdsk/c1t1d0s7
root@unixrock#
root@unixrock# metadb
        flags           first blk       block count
     a m  p  luo        16              8192            /dev/dsk/c1t0d0s7
     a    p  luo        8208            8192            /dev/dsk/c1t0d0s7
     a    p  luo        16400           8192            /dev/dsk/c1t0d0s7
     a        u         16              8192            /dev/dsk/c1t1d0s7
     a        u         8208            8192            /dev/dsk/c1t1d0s7
     a        u         16400           8192            /dev/dsk/c1t1d0s7
root@unixrock#
root@unixrock#

Now we are good to replace the disk in SVM level

root@unixrock# metastat |grep -i metareplace
    Invoke: metareplace d4 c1t1d0s4 
    Invoke: metareplace d1 c1t1d0s1 
    Invoke: metareplace d5 c1t1d0s5 
    Invoke: metareplace d0 c1t1d0s0 
root@unixrock#
root@unixrock# metareplace -e d4 c1t1d0s4
d4: device c1t1d0s4 is enabled
root@unixrock#
root@unixrock# metareplace -e d1 c1t1d0s1
d1: device c1t1d0s1 is enabled
root@unixrock#
root@unixrock# metareplace -e d5 c1t1d0s5
d5: device c1t1d0s5 is enabled
root@unixrock#
root@unixrock# metareplace -e d0 c1t1d0s0
d0: device c1t1d0s0 is enabled
root@unixrock#

Checking the metadevice status

root@unixrock# metastat -c
d1               m  1.0GB d11 d21  (resync-5%)
    d11          s  1.0GB c1t0d0s1
    d21          s  1.0GB c1t1d0s1 (resyncing)
d5               m  400MB d15 d25  (resync-10%)
    d15          s  400MB c1t0d0s5
    d25          s  400MB c1t1d0s5 (resyncing)
d4               m  494MB d14 d24  (resync-3%)
    d14          s  494MB c1t0d0s4
    d24          s  494MB c1t1d0s4 (resyncing)
d0               m  8.0GB d10 d20  (resync-1%)
    d10          s  8.0GB c1t0d0s0
    d20          s  8.0GB c1t1d0s0 (resyncing)
root@unixrock#

Yes, your are done. Once the resyncing is completed, we good to go from here.

root@unixrock# metastat -c
d1               m  1.0GB d11 d21
    d11          s  1.0GB c1t0d0s1
    d21          s  1.0GB c1t1d0s1
d5               m  400MB d15 d25
    d15          s  400MB c1t0d0s5
    d25          s  400MB c1t1d0s5
d4               m  494MB d14 d24
    d14          s  494MB c1t0d0s4
    d24          s  494MB c1t1d0s4
d0               m  8.0GB d10 d20
    d10          s  8.0GB c1t0d0s0
    d20          s  8.0GB c1t1d0s0
root@unixrock#