Back to the main page
iSCSI on Solaris
Brief introduction
The iSCSI protocol allows SCSI commands to be used over a TCP/IP network.
The main reason people want to use an iSCSI is reducing costs, since they don't need buying FC HBA and infrastructure is already setup.
The default port for iSCSI targets is 3260.
iSCSI versus NFS
Both are used for accessing storage device over network, so what's the difference?
1. NFS is used for accessing remote FILE SYSTEM data.
Many people can assess data so there is functionality of locking data when used by someone, so others have to wait.
2. iSCSI is used for accessing BLOCKS on remote disk.
In this case many users cannot share this access, because there is not lock functionality for block level access.
In this example I have SunFire T2000 directly connected to StorEdge 3510 (with FC cables).
The server is running Solaris 10 update 7.
This server is iSCSI target and is exporting block device (ZFS volume) in order to be accessed by another Solaris box over network (iSCSI initiator).
Configuring iSCSI Target - no Authentication
You need next packages installed:
system SUNWiscsir Sun iSCSI Device Driver (root)
system SUNWiscsitgtr Sun iSCSI Target (Root)
system SUNWiscsitgtu Sun iSCSI Target (Usr)
system SUNWiscsiu Sun iSCSI Management Utilities (usr)
Enable the service svc:/system/iscsitgt:default
1. Create a base directory
The base directory is used to store the iSCSI target configuration data and needs to be defined prior to using the iSCSI target for the first time.
|
# iscsitadm modify admin -d /var/iscsi
|
2. Configure a backing store
The backing store contains the physical storage that is exported as iSCSI target.
With Solaris, next can be backing store: flat files, physical device, SVM, ZFS volume.
Let's create ZFS volumes on StorEdge 3510, one on RAID1 and RAID5.
> format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c1t40d0 <SUN-StorEdge 3510-421F-136.23GB>
/pci@7c0/pci@0/pci@1/pci@0,2/SUNW,qlc@1/fp@0,0/ssd@w216000c0ff89cacc,0
1. c1t40d1 <SUN-StorEdge 3510-421F-68.12GB>
/pci@7c0/pci@0/pci@1/pci@0,2/SUNW,qlc@1/fp@0,0/ssd@w216000c0ff89cacc,1
2. c2t0d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@2/sd@0,0
3. c2t1d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@2/sd@1,0
> zpool create -f drum-raid5 c1t40d0
> zpool create -f drum-raid1 c1t40d1
> zpool status
pool: drum-raid1
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
drum-raid1 ONLINE 0 0 0
c1t40d1 ONLINE 0 0 0
pool: drum-raid5
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
drum-raid5 ONLINE 0 0 0
c1t40d0 ONLINE 0 0 0
> zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
drum-raid1 68G 94K 68.0G 0% ONLINE -
drum-raid5 136G 111K 136G 0% ONLINE -
> zfs create -V 30g drum-raid1/volume-no-CHAP
> zfs list
NAME USED AVAIL REFER MOUNTPOINT
drum-raid1 30.0G 36.9G 18K /drum-raid1
drum-raid1/volume-no-CHAP 30G 66.9G 30K -
drum-raid5 112K 134G 18K /drum-raid5
|
The ZFS Volume (drum-raid1/volume-no-CHAP) is now created.
3. Create a target
Let's now create iSCSI target.
> iscsitadm create target
iscsitadm: at least one option required
iscsitadm create target <OPTIONS> <local-target>
OPTIONS:
-t, --type <disk/tape/osd/raw>
-u, --lun <number>
-z, --size <size k/m/g/t>
-a, --alias <value>
-b, --backing-store <pathname>
For more information, please see iscsitadm(1M)
>iscsitadm create target -b /dev/zvol/dsk/drum-raid1/volume-no-CHAP testors-no-CHAP
|
If you change mind and want to removing iSCSI targets, do:
|
> iscsitadm delete target --lun 0 testors-no-CHAP
|
4. Verify the target configuration
> iscsitadm list target -v
Target: testors-no-chap
iSCSI Name: iqn.1986-03.com.sun:02:92b0edad-52cd-ca06-93bd-d39a31259a2b.testors-no-chap
Connections: 0
ACL list:
TPGT list:
LUN information:
LUN: 0
GUID: 0
VID: SUN
PID: SOLARIS
Type: disk
Size: 30G
Backing store: /dev/zvol/dsk/drum-raid1/volume-no-CHAP
Status: online
|
The iSCSI configuration data file is created in base directory:
# ls /etc/iscsi
total 12
drwxr-xr-x 2 root sys 512 Mar 18 15:23 .
drwxr-xr-x 58 root sys 4096 Mar 18 15:37 ..
-rw------- 1 root root 548 Mar 18 15:23 iscsi_v1.dbc
|
Note about iSCSI name:
It can be in two formats: IQN or EUI.
1. IQN - has date, domain and node identification, like in my example.
2. EUI - has 16 hexadecimal digits, resembles WWN of FC node.
5. Once iSCSI initiator is configured (will be done in next steps), you can list it:
> iscsitadm list initiator
Initiator: cs2
iSCSI Name: iqn.1986-03.com.sun:01:0003ba3559b8.4ba69545
CHAP Name: cs2
|
Configuring iSCSI Initiator - no Authentication
You need next packages installed:
system SUNWiscsir Sun iSCSI Device Driver (root)
system SUNWiscsiu Sun iSCSI Management Utilities (usr)
Enable service svc:/network/iscsi_initiator:default
1. Configure a discovery method
Have only static discovery (use this for small number of targets or restrict targets that initiator can access)
# iscsiadm modify discovery
iscsiadm: at least one option required
iscsiadm modify discovery <OPTIONS>
OPTIONS:
-s, --static <enable|disable>
-t, --sendtargets <enable|disable>
-i, --iSNS <enable|disable>
For more information, please see iscsiadm(1M)
# iscsiadm modify discovery -i disable
# iscsiadm modify discovery -t disable
# iscsiadm list discovery
Discovery:
Static: enabled
Send Targets: disabled
iSNS: disabled
# iscsiadm add static-config iqn.1986-03.com.sun:02:92b0edad-52cd-ca06-93bd-d39a31259a2b.testors-no-chap,192.168.24.35
|
2. Verify the targets
> iscsiadm list target -vS
Target: iqn.1986-03.com.sun:02:92b0edad-52cd-ca06-93bd-d39a31259a2b.testors-no-chap
Alias: testors-no-chap
TPGT: 1
ISID: 4000002a0000
Connections: 1
CID: 0
IP address (Local): 192.168.20.222:32784
IP address (Peer): 192.168.24.35:3260
Discovery Method: Static
Login Parameters (Negotiated):
Data Sequence In Order: yes
Data PDU In Order: yes
Default Time To Retain: 20
Default Time To Wait: 2
Error Recovery Level: 0
First Burst Length: 65536
Immediate Data: yes
Initial Ready To Transfer (R2T): yes
Max Burst Length: 262144
Max Outstanding R2T: 1
Max Receive Data Segment Length: 8192
Max Connections: 1
Header Digest: NONE
Data Digest: NONE
LUN: 0
Vendor: SUN
Product: SOLARIS
OS Device Name: /dev/rdsk/c4t2d0s2
> iscsiadm list target-param -v
Target: iqn.1986-03.com.sun:02:92b0edad-52cd-ca06-93bd-d39a31259a2b.testors-no-chap
Alias: testors-no-chap
Bi-directional Authentication: disabled
Authentication Type: NONE
Login Parameters (Default/Configured):
Data Sequence In Order: yes/-
Data PDU In Order: yes/-
Default Time To Retain: 20/-
Default Time To Wait: 2/-
Error Recovery Level: 0/-
First Burst Length: 65536/-
Immediate Data: yes/-
Initial Ready To Transfer (R2T): yes/-
Max Burst Length: 262144/-
Max Outstanding R2T: 1/-
Max Receive Data Segment Length: 8192/-
Max Connections: 1/-
Header Digest: NONE/-
Data Digest: NONE/-
Configured Sessions: 1
> iscsiadm list initiator-node
Initiator node name: iqn.1986-03.com.sun:01:0003ba3559b8.4ba6c502
Initiator node alias: -
Login Parameters (Default/Configured):
Header Digest: NONE/-
Data Digest: NONE/-
Authentication Type: NONE
RADIUS Server: NONE
RADIUS access: unknown
Configured Sessions: 1
|
3. Initialize and use the new targets
Target(s) is visible, device entries are needed to create with command:
> format
AVAILABLE DISK SELECTIONS:
0. c1t0d0 <SEAGATE-ST373307LC-0006 cyl 49780 alt 2 hd 4 sec 720>
/pci@1f,0/pci@1/scsi@8/sd@0,0
1. c1t1d0 <SEAGATE-ST373307LC-0006 cyl 49780 alt 2 hd 4 sec 720>
/pci@1f,0/pci@1/scsi@8/sd@1,0
2. c4t2d0 <drive type unknown>
/iscsi/disk@0000iqn.1986-03.com.sun%3A02%3A92b0edad-52cd-ca06-93bd-d39a31259a2b.testors-no-chapFFFF,0
4. Create file systems
|
> zpool create iscsi-no-chap c4t2d0
|
> zfs list
NAME USED AVAIL REFER MOUNTPOINT
iscsi-no-chap 89.5K 29.3G 1K /iscsi-no-chap
A. In case you want to remove already configured iSCSI Target
Find your targets.
> iscsiadm list target
Target: iqn.1986-03.com.sun:02:92b0edad-52cd-ca06-93bd-d39a31259a2b.testors-no-chap
Alias: testors-no-chap
TPGT: 1
ISID: 4000002a0000
Connections: 1
|
B. Remove the target
|
> iscsiadm remove static-config iqn.1986-03.com.sun:02:92b0edad-52cd-ca06-93bd-d39a31259a2b.testors-no-chap
|
The iSCSI authentication
The iSCSI authentication may be required since iSCSI target cannot determine if connection request is from valid host.
For example, someone else can connect to already connected (in use) block device, and type 'newfs' and you are screwed.
Target authenticates an initiator by using Challenge-Handshake Authentication Protocol (CHAP).
Authentication can be:
1. Unidirectional: only target identifies initiator.
2. Bidirectional: initiators also identifies target.
Let's create new volume for testing iSCSI setup with Bidirectional authentication :
> zfs create -V 40g drum-raid5/volume-yes-CHAP
> zfs list
NAME USED AVAIL REFER MOUNTPOINT
drum-raid5 40.0G 93.9G 18K /drum-raid5
drum-raid5/volume-yes-CHAP 40G 134G 16K -
An iSCSI initiator configuration for Bidirectional authentication
1. Set a secret key (between 12-16 characters)
> iscsiadm modify initiator-node --CHAP-secret
Enter secret:
Re-enter secret:
|
2. Set iSCSI initiator CHAP name (let's make it same as hostname of iSCSI initiator)
> iscsiadm modify initiator-node --CHAP-name counterstrike2
|
3. Tell iSCSI initiator to use CHAP
> iscsiadm modify initiator-node --authentication CHAP
> iscsiadm list initiator-node
Initiator node name: iqn.1986-03.com.sun:01:0003ba3559b8.4ba6c502
Initiator node alias: -
Login Parameters (Default/Configured):
Header Digest: NONE/-
Data Digest: NONE/-
Authentication Type: CHAP
CHAP Name: counterstrike2
RADIUS Server: NONE
RADIUS access: unknown
Configured Sessions: 1
|
Just as reminder how to add iSCSI target in case it's not already added.
> iscsiadm add static-config iqn.1986-03.com.sun:02:504a7bec-02bb-603a-cb19-ea22f593a799.testors-yes-chap,192.168.24.35
|
4. We want to enable bidirectional authentication on iSCSI target.
> iscsiadm modify target-param --bi-directional-authentication enable iqn.1986-03.com.sun:02:504a7bec-02bb-603a-cb19-ea22f593a799.testors-yes-chap
> iscsiadm list target-param -v
Target: iqn.1986-03.com.sun:02:504a7bec-02bb-603a-cb19-ea22f593a799.testors-yes-chap
Alias: testors-yes-chap
Bi-directional Authentication: enabled
Authentication Type: NONE
Login Parameters (Default/Configured):
Data Sequence In Order: yes/-
Data PDU In Order: yes/-
Default Time To Retain: 20/-
Default Time To Wait: 2/-
Error Recovery Level: 0/-
First Burst Length: 65536/-
Immediate Data: yes/-
Initial Ready To Transfer (R2T): yes/-
Max Burst Length: 262144/-
Max Outstanding R2T: 1/-
Max Receive Data Segment Length: 8192/-
Max Connections: 1/-
Header Digest: NONE/-
Data Digest: NONE/-
Configured Sessions: 1
|
5. And setup authentication method which is CHAP.
> iscsiadm modify target-param --authentication CHAP iqn.1986-03.com.sun:02:504a7bec-02bb-603a-cb19-ea22f593a799.testors-yes-chap
|
6. The iSCSI target must also know secret-key, we already setup on iSCSI initiator in step 1.
> iscsiadm modify target-param --CHAP-secret iqn.1986-03.com.sun:02:504a7bec-02bb-603a-cb19-ea22f593a799.testors-yes-chap
Enter secret:
Re-enter secret:
|
7. Final bidirectional config is below.
> iscsiadm list target-param -v
Target: iqn.1986-03.com.sun:02:504a7bec-02bb-603a-cb19-ea22f593a799.testors-yes-chap
Alias: testors-yes-chap
Bi-directional Authentication: enabled
Authentication Type: CHAP
CHAP Name: iqn.1986-03.com.sun:02:504a7bec-02bb-603a-cb19-ea22f593a799.testors-yes-chap <------ see this
Login Parameters (Default/Configured):
Data Sequence In Order: yes/-
Data PDU In Order: yes/-
Default Time To Retain: 20/-
Default Time To Wait: 2/-
Error Recovery Level: 0/-
First Burst Length: 65536/-
Immediate Data: yes/-
Initial Ready To Transfer (R2T): yes/-
Max Burst Length: 262144/-
Max Outstanding R2T: 1/-
Max Receive Data Segment Length: 8192/-
Max Connections: 1/-
Header Digest: NONE/-
Data Digest: NONE/-
Configured Sessions: 1
|
The iSCSI target configuration for Bidirectional authentication
Quick reminder: how to create iSCSI target:
> iscsitadm create target -b /dev/zvol/dsk/drum-raid5/volume-yes-CHAP testors-yes-CHAP
> iscsitadm list target
Target: testors-yes-chap
iSCSI Name: iqn.1986-03.com.sun:02:504a7bec-02bb-603a-cb19-ea22f593a799.testors-yes-chap
Connections: 0
> iscsitadm list target -v
Target: testors-yes-chap
iSCSI Name: iqn.1986-03.com.sun:02:504a7bec-02bb-603a-cb19-ea22f593a799.testors-yes-chap
Connections: 0
ACL list:
TPGT list:
LUN information:
LUN: 0
GUID: 0
VID: SUN
PID: SOLARIS
Type: disk
Size: 40G
Backing store: /dev/zvol/dsk/drum-raid5/volume-yes-CHAP
Status: online
1. Set iSCSI target CHAP name as its own hostname
> iscsitadm modify admin --chap-name testors
|
2. Set the secret (same secret as for iSCSI initiator)
> iscsitadm modify admin --chap-secret
Enter secret:
Re-enter secret:
|
3. Creat an initiator object (this will be associated to one/more targets)
> iscsitadm create initiator --iqn iqn.1986-03.com.sun:01:0003ba3559b8.4ba69545 cs2
> iscsitadm list initiator -v
Initiator: cs2
iSCSI Name: iqn.1986-03.com.sun:01:0003ba3559b8.4ba69545
CHAP Name: Not set
CHAP Secret: Not set
|
4. Create CHAP name
> iscsitadm modify initiator --chap-name cs2 cs2
> iscsitadm list initiator -v
Initiator: cs2
iSCSI Name: iqn.1986-03.com.sun:01:0003ba3559b8.4ba69545
CHAP Name: cs2 <------- see this
CHAP Secret: Not set
|
5. Set CHAP secret (same secret as for iSCSI initiator)
> iscsitadm modify initiator --chap-secret cs2
Enter secret:
Re-enter secret:
|
On iSCSI initiator:
> format
Searching for disks...done
c4t4d0: configured with capacity of 23.91GB
AVAILABLE DISK SELECTIONS:
0. c1t0d0 <SEAGATE-ST373307LC-0006 cyl 49780 alt 2 hd 4 sec 720>
/pci@1f,0/pci@1/scsi@8/sd@0,0
1. c1t1d0 <SEAGATE-ST373307LC-0006-68.37GB>
/pci@1f,0/pci@1/scsi@8/sd@1,0
2. c4t4d0 <SUN-SOLARIS-1 cyl 3 alt 2 hd 255 sec 65535>
/iscsi/disk@0000iqn.1986-03.com.sun%3A02%3A504a7bec-02bb-603a-cb19-ea22f593a799.testors-yes-chapFFFF,0
Back to the main page