Problem
This guide shows how to re-scan and examine FC LUNS on Solaris 10/11. This is required for detecting new LUNs and to assist with basic storage connectivity troubleshooting.
Impact
Re-scanning FC LUNs on Solaris is generally non-disruptive in a well configured environment with normal load.
Solution
Understanding How FC is Configured on a Solaris Host
The following commands are useful for general SAN stack fact-finding on Solaris:
-
List the connected HBA’s:
root@Unixarena-SOL11:~# luxadm -e port |grep CONNECTED
/devices/pci@1d,700000/SUNW,qlc@1,1/fp@0,0:devctl CONNECTED
/devices/pci@1d,700000/SUNW,qlc@3/fp@0,0:devctl CONNECTED
-
Verify FC ports are connected and configured:
Unix@sol# cfgadm -al -o show_FCP_dev |grep fc-fabric
c2 fc-fabric connected configured unknown
c4 fc-fabric connected configured unknown
Unix@sol#
-
To find the HBA’s World Wide Node number (WWN), use the
fcinfo
command:
Unix@sol# fcinfo hba-port |grep Port
HBA Port WWN: 10000000c884bb48
HBA Port WWN: 10000000c884bb49
HBA Port WWN: 10000000c884b85c
HBA Port WWN: 10000000c884b85d
Unix@sol#
-
You can also find the WWN using
luxadm
command, if HBA is already connected to the FC switch.
Unix@sol# luxadm -e dump_map /dev/cfg/c4
Pos Port_ID Hard_Addr Port WWN Node WWN Type
0 29900 0 50080e8008cfb814 50080e8008cfb814 0x0 (Disk device)
1 27400 0 10000000c884b85c 20000000c884b85c 0x1f (Unknown Type,Host Bus Adapter)
Unix@sol#
From the above output, the last line shows the HBA information. In the same way you can find the other controller information as well.
-
Zoning can be verified using the below command:
Unix@sol# cfgadm -al -o show_FCP_dev c2 c4
-
If you see any controller port WWN shown as “unconfigured," then you can initiate FC session using below mentioned command:
Unix@sol# cfgadm -c configure c2::50080e8008cfb814
Unix@sol# cfgadm -c configure c4::50080e8008cfb814
Scanning for New FC Devices and Getting LUN Information:
Scanning for FC/SAN Devices
|
cfgadm -al
|
To scan FC LUNs
|
devfsadm -c disk
|
To make sure all the device files are created
|
tail /var/adm/messages
|
To see the new LUNs information
|
echo | format
|
To get the new LUNs information
|
ls -lrt /dev/rdsk | grep s2 | tail
|
To get the new LUNs information
|
Note
: The command
luxadm
probe
can also be used to scan FC LUNs.
Resetting the HBA
If you are still unable to see the new LUN/DISK, then, if you have multipathing enabled, you can try to reset the HBA. (Do not try this in critical servers unless you are confident that multipathing is correctly configured)
-
List the connected HBA.
root@Unixarena-SOL11:~# luxadm -e port |grep CONNECTED
/devices/pci@1d,700000/SUNW,qlc@1,1/fp@0,0:devctl CONNECTED
/devices/pci@1d,700000/SUNW,qlc@3/fp@0,0:devctl CONNECTED
-
Reset the HBA using
forcelip
option:
root@Unixarena-SOL11:~# luxadm -e forcelip /devices/pci@1d,700000/SUNW,qlc@1,1/fp@0,0:devctl
The
forcelip
command can be issued to the controller names as well:
Unix@sol# cfgadm -al -o show_FCP_dev |grep fc-fabric
c2 fc-fabric connected configured unknown
c4 fc-fabric connected configured unknown
Unix@sol#
Unix@sol# luxadm -e forcelip /dev/cfg/c2
-
Verify the controller status using
cfgadm -al
. Make sure disks didn't loose any SAN paths after the HBA reset. If everything seems to be working, then issue the
forcelip
command to the other controller as well.
If you are still not able to see the new FC/SAN LUNS, then, as a last resort, reboot the server and try again. Once you see the new LUN, make sure that you also see all FC paths. A minimum two FC paths is required for SAN disks.
-
To verify the FC LUN details and multipathing, enter the following command:
root@Unixarena-SOL11:~# luxadm display /dev/rdsk/c1txxxxxxd0s2
Understanding How FC is Configured on a Solaris Host
The following commands are useful for general SAN stack fact-finding on Solaris:
-
List the connected HBA’s:
root@Unixarena-SOL11:~# luxadm -e port |grep CONNECTED
/devices/pci@1d,700000/SUNW,qlc@1,1/fp@0,0:devctl CONNECTED
/devices/pci@1d,700000/SUNW,qlc@3/fp@0,0:devctl CONNECTED
-
Verify FC ports are connected and configured:
Unix@sol# cfgadm -al -o show_FCP_dev |grep fc-fabric
c2 fc-fabric connected configured unknown
c4 fc-fabric connected configured unknown
Unix@sol#
-
To find the HBA’s World Wide Node number (WWN), use the
fcinfo
command:
Unix@sol# fcinfo hba-port |grep Port
HBA Port WWN: 10000000c884bb48
HBA Port WWN: 10000000c884bb49
HBA Port WWN: 10000000c884b85c
HBA Port WWN: 10000000c884b85d
Unix@sol#
-
You can also find the WWN using
luxadm
command, if HBA is already connected to the FC switch.
Unix@sol# luxadm -e dump_map /dev/cfg/c4
Pos Port_ID Hard_Addr Port WWN Node WWN Type
0 29900 0 50080e8008cfb814 50080e8008cfb814 0x0 (Disk device)
1 27400 0 10000000c884b85c 20000000c884b85c 0x1f (Unknown Type,Host Bus Adapter)
Unix@sol#
From the above output, the last line shows the HBA information. In the same way you can find the other controller information as well.
-
Zoning can be verified using the below command:
Unix@sol# cfgadm -al -o show_FCP_dev c2 c4
-
If you see any controller port WWN shown as “unconfigured," then you can initiate FC session using below mentioned command:
Unix@sol# cfgadm -c configure c2::50080e8008cfb814
Unix@sol# cfgadm -c configure c4::50080e8008cfb814
Scanning for New FC Devices and Getting LUN Information:
Scanning for FC/SAN Devices
|
cfgadm -al
|
To scan FC LUNs
|
devfsadm -c disk
|
To make sure all the device files are created
|
tail /var/adm/messages
|
To see the new LUNs information
|
echo | format
|
To get the new LUNs information
|
ls -lrt /dev/rdsk | grep s2 | tail
|
To get the new LUNs information
|
Note
: The command
luxadm
probe
can also be used to scan FC LUNs.
Resetting the HBA
If you are still unable to see the new LUN/DISK, then, if you have multipathing enabled, you can try to reset the HBA. (Do not try this in critical servers unless you are confident that multipathing is correctly configured)
-
List the connected HBA.
root@Unixarena-SOL11:~# luxadm -e port |grep CONNECTED
/devices/pci@1d,700000/SUNW,qlc@1,1/fp@0,0:devctl CONNECTED
/devices/pci@1d,700000/SUNW,qlc@3/fp@0,0:devctl CONNECTED
-
Reset the HBA using
forcelip
option:
root@Unixarena-SOL11:~# luxadm -e forcelip /devices/pci@1d,700000/SUNW,qlc@1,1/fp@0,0:devctl
The
forcelip
command can be issued to the controller names as well:
Unix@sol# cfgadm -al -o show_FCP_dev |grep fc-fabric
c2 fc-fabric connected configured unknown
c4 fc-fabric connected configured unknown
Unix@sol#
Unix@sol# luxadm -e forcelip /dev/cfg/c2
-
Verify the controller status using
cfgadm -al
. Make sure disks didn't loose any SAN paths after the HBA reset. If everything seems to be working, then issue the
forcelip
command to the other controller as well.
If you are still not able to see the new FC/SAN LUNS, then, as a last resort, reboot the server and try again. Once you see the new LUN, make sure that you also see all FC paths. A minimum two FC paths is required for SAN disks.
-
To verify the FC LUN details and multipathing, enter the following command:
root@Unixarena-SOL11:~# luxadm display /dev/rdsk/c1txxxxxxd0s2