Archive

Posts Tagged ‘dynamically scsi fibre channel’

Extend filesystem in vxvm which connects to SAN fibre channel storage

May 28th, 2011 No comments

Firstly, please refer to http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/html/Online_Storage_Reconfiguration_Guide/scanning-storage-interconnects.html for some pre-checking, like memory usage, sync etc.
Then, we should scan for new disk connected to hba one by one(issue_lip, Scenario:fabric on Linux with Emulex hbas), we should check dmpnodes are ALL in ENABLED state before moving on to another hba. This is because when scanning for new disks, we expect to result in disabling paths on that controller – and moving on only when the paths are confirmed enabled again. And during all procedures, tail -f /var/log/messages would help.

1) Checked the messages file for any existing issues. Identified and eliminated concerns of I/O error messages which have occured for some time:
May 29 04:11:05 testserver01 kernel: end_request: I/O error, dev sdft, sector 0^M
May 29 04:11:05 testserver01 kernel: Buffer I/O error on device sdft, logical block 0^M
May 29 04:11:05 testserver01 kernel: end_request: I/O error, dev sdft, sector 0^M
/var/log/messages.1:May 27 22:48:04 testserver01 kernel: end_request: I/O error, dev sdjt, sector 8
/var/log/messages.1:May 27 22:48:04 testserver01 kernel: Buffer I/O error on device sdjt3, logical block 1
2)Saved some output for comparison later:
syminq -pdevfile > /var/tmp/syminq-pdevfile-prior
We expected device as hyper 2C27 so I looked for this device in the output and, as expected, did not find it.
vxdisk -o  alldgs list > /var/tmp/vxdisk-oalldgs.list
3)Checked the current state of the vm devices dmp subpaths:
for disk in `vxdisk -q list | awk '{print $1}'`
do
echo $disk
vxdmpadm getsubpaths dmpnodename=${disk}
done | tee -a /var/tmp/getsubpaths.out
Checked the output to make sure all but the root disk(expected) had two enabled paths: NAME         STATE[A]   PATH-TYPE[M] CTLR-NAME  ENCLR-TYPE   ENCLR-NAME    ATTRS
================================================================================
cciss/c0d0   ENABLED(A)   -          c360       OTHER_DISKS  OTHER_DISKS
sda
NAME         STATE[A]   PATH-TYPE[M] CTLR-NAME  ENCLR-TYPE   ENCLR-NAME    ATTRS
================================================================================
sda          ENABLED(A)   -          c0         EMC          EMC0             -
sddc         ENABLED(A)   -          c1         EMC          EMC0             -
sdae
NAME         STATE[A]   PATH-TYPE[M] CTLR-NAME  ENCLR-TYPE   ENCLR-NAME    ATTRS
================================================================================
sdeh         ENABLED(A)   -          c1         EMC          EMC0             -
sdp          ENABLED(A)   -          c0         EMC          EMC0             -
sdaf
NAME         STATE[A]   PATH-TYPE[M] CTLR-NAME  ENCLR-TYPE   ENCLR-NAME    ATTRS
================================================================================
sdej         ENABLED(A)   -          c1         EMC          EMC0             -
sdq          ENABLED(A)   -          c0         EMC          EMC0             -
sdai
NAME         STATE[A]   PATH-TYPE[M] CTLR-NAME  ENCLR-TYPE   ENCLR-NAME    ATTRS
================================================================================
sdai         ENABLED(A)   -          c0         EMC          EMC0             -
sdfr         ENABLED(A)   -          c1         EMC          EMC0             -
... etc
Ran other commands like
vxdg list (to ensure all diskgroups where enabled)
df -k (no existing filesystem problems)

NOTE:

You can get your  <dmpnodename>  by running:
# vxdisk path | grep emcC5D3
Note, it is listed as DANAME (and not SUBPATH)
4)Tried to scan the first scsi bus
root@testserver01# pwd
/sys/class/scsi_host/host0
root@testserver01# echo '- - -' > scan
Waited to see if the scan would detect any new devices. Monditor the messages log for any messages relating to the scan.
Checked output of syminq and checked the subpaths for each dmpnode as above. All remained ENABLED and no change.
5. Moved on to issuing a force lip to the fibre path
a. issuing the forcelip
root@testserver01# pwd
/sys/class/fc_host/host0
root@testserver01# echo "1" > issue_lip
The command returned and I monitored the messages log. Waited for the disabled path messages to appear (as expected):
May 31 02:44:49 testserver01 kernel: VxVM vxdmp V-5-0-112 disabled path 8/0x80 belonging to the dmpnode 201/0x540
May 31 02:44:49 testserver01 kernel: VxVM vxdmp V-5-0-112 disabled path 67/0xa0 belonging to the dmpnode 201/0x5f0
May 31 02:44:49 testserver01 kernel: VxVM vxdmp V-5-0-112 disabled path 67/0xb0 belonging to the dmpnode 201/0x6c0
May 31 02:44:49 testserver01 kernel: VxVM vxdmp V-5-0-112 disabled path 128/0x60 belonging to the dmpnode 201/0x160
... etc
I checked the output of vxdmpadm getsubpaths for each device to confirm the paths had gone offline:
NAME         STATE[A]   PATH-TYPE[M] CTLR-NAME  ENCLR-TYPE   ENCLR-NAME    ATTRS
================================================================================
sddw         ENABLED(A)   -          c0         EMC          EMC0             -
sdiv         ENABLED(A)   -          c1         EMC          EMC0             -
sddm
NAME         STATE[A]   PATH-TYPE[M] CTLR-NAME  ENCLR-TYPE   ENCLR-NAME    ATTRS
================================================================================
sdbg         DISABLED     -          c0         EMC          EMC0             -
sdgq         ENABLED(A)   -          c1         EMC          EMC0             -
sddn
NAME         STATE[A]   PATH-TYPE[M] CTLR-NAME  ENCLR-TYPE   ENCLR-NAME    ATTRS
================================================================================
sddz         ENABLED(A)   -          c0         EMC          EMC0             -
sdix         ENABLED(A)   -          c1         EMC          EMC0             -
sddo
NAME         STATE[A]   PATH-TYPE[M] CTLR-NAME  ENCLR-TYPE   ENCLR-NAME    ATTRS
================================================================================
sdbh         DISABLED     -          c0         EMC          EMC0             -
sdgr         ENABLED(A)   -          c1         EMC          EMC0             -
sddp

Not all the paths were affected at once - a few minutes wait confirms they all go down as expected. The secondary path remains ENABLED, as expected.
I waited a little longer to get some estimation for how long all the paths took to go down - estimate was around 4 minutes.
b. Rescaning the fibre channel.
Before moving to the other path I get volume manager to rescan the device bus to trigger dmp to wake up the DISABLED paths.
root@testserver01# vxdisk scandisks fabric
I wait until all the primary paths become ENABLED again, checking for the dmp enabled messages in the messages log:
May 31 02:49:43 testserver01 kernel: VxVM vxdmp V-5-0-148 enabled path 129/0x90 belonging to the dmpnode 201/0x10
May 31 02:49:43 testserver01 kernel: VxVM vxdmp V-5-0-148 enabled path 129/0x50 belonging to the dmpnode 201/0x20
May 31 02:49:43 testserver01 kernel: VxVM vxdmp V-5-0-148 enabled path 128/0x40 belonging to the dmpnode 201/0x30
May 31 02:49:43 testserver01 kernel: VxVM vxdmp V-5-0-148 enabled path 129/0x30 belonging to the dmpnode 201/0x40
.... etc
And also check vxdmpadm getsubpaths command until all primary paths return to ENABLED state.
I did the same for the second host controlller at /sys/class/fc_host/host1
6)Checking for new disk device:
Try syminq and/or symcfg disco

After this, you can extend vxvm filesystem to the size you want, and re-import devices after that. Please refer to http://www.doxer.org/extending-filesystems-on-lvm-vxvmvxfs-how-to/ for more details.