Back to the main page
SAN Add on Solaris with Veritas file system
More disk space is always needed, and SAN add is usual activity for system admin.
And after receiving request from customer, or business owner of file system, this has to be designed properly before implementation.
Here is the example when using Veritas file system.
It's good idea to have standard form to be filled by requestor, like hostname, mount point, ownership,
permissions, size (current, additional), dedicated/shared/replicated LUN.
Once you have this info, you can design the work. Adding new LUN (file system) is simplier then increasing existing one, which has to be planned in more details.
In most cases, you end up adding bigger LUN, mirroring data (logical volume, actually plex within it),
then removing plex from smaller LUN, removing LUN from disk group and finally a Storage team can reclaim LUN.
Discover newly allocated LUNs
Once new LUNs are allocated, run commands so system can recognize them.
# devfsadm
# vxdctl enable
|
Then use script san_sol_info.sh to list SAN LUNs or use one line command (note about ignoring channels 5/6)
# luxadm probe | grep Logical | grep -v /dev/rdsk/c6 | grep -v /dev/rdsk/c5 |awk -F: '{print $2}'
/dev/rdsk/c4t20110002AC0086D2d2s2
....
/dev/rdsk/c4t20110002AC0086D2d29s2
|
Or the example in the sh shell
# for i in `luxadm probe | grep Logical | grep -v /dev/rdsk/c6 | grep -v /dev/rdsk/c5 | awk -F: '{print $2}'`
> do
> echo $i ": \c" ; luxadm -v display $i | grep capacity | awk -F: '{print $2}'
> done
/dev/rdsk/c4t20110002AC0086D2d2s2 : 16384.000 MBytes
....
/dev/rdsk/c4t20110002AC0086D2d28s2 : 524288.000 MBytes
|
Prepare LUNs for Veritas use
# /etc/vx/bin/vxdisksetup -f -i c4t20110002AC0086D2d29
VxVM vxdisksetup ERROR V-5-2-43 c4t20110002AC0086D2d29: Invalid disk device for vxdisksetup
|
If you see above error, format/label disk with Solaris command "format" (this may not be necessary all the time).
If vxdisksetup still doesn't work, try below command:
# cat format.cmd
label
quit
# foreach i (`cat newluns`)
foreach? echo --- formating $i
foreach? format -f format.cmd $i
foreach? end
|
# vxdisk -f init c4t20110002AC0086D2d29
|
So basically, if you have all new LUNs in file "newluns" you can prepare them for Veritas use with:
# foreach i (`cat newluns`)
foreach? echo --- Preparing $i for Veritas use -----
foreach? /etc/vx/bin/vxdisksetup -f -i $i
foreach? vxdisk -f init $i
foreach? end
|
Create Veritas Disk Group
# vxdg init dgtest dgtest_disk001=c4t20110002AC0086D2d29
# vxdg list
NAME STATE ID
dgtest enabled,cds 1389999403.91.sap05.austin.hp.com
|
To add additional disk to DG:
# vxdg -g <dg> adddisk <dg>_disk002=c4t201100xxxxxxxxxxxdx
|
To remove disk from DG (example from sap05):
# vxdg -g dg10cl1 rmdisk dg10cl1_disk001
VxVM vxdg ERROR V-5-1-16414 Disk dg10cl1_disk001 is used by one or more subdisks which are pending to be reclaimed.
Use "vxdisk reclaim 3pardata0_0_6" to reclaim space used by these subdisks, and retry "vxdg rmdisk" command.
Note: reclamation is irreversible.
|
Okay, this is something not seen on HP-UX, but this is Solaris.
# vxdisk reclaim 3pardata0_0_6
Disk 3pardata0_0_6 : Done.
|
Now remove disk command works:
# vxdg -g dg10cl1 rmdisk dg10cl1_disk001
|
Create Veritas logical volume
# vxassist -g dgtest make lvol1 16000m
# vxprint -hrt
Disk group: dgtest
DG NAME NCONFIG NLOG MINORS GROUP-ID
ST NAME STATE DM_CNT SPARE_CNT APPVOL_CNT
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE
RV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRL
RL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNK
CO NAME CACHEVOL KSTATE STATE
VT NAME RVG KSTATE STATE NVOLUME
V NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPE
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
SC NAME PLEX CACHE DISKOFFS LENGTH [COL/]OFF DEVICE MODE
DC NAME PARENTVOL LOGVOL
SP NAME SNAPVOL DCO
EX NAME ASSOC VC PERMS MODE STATE
SR NAME KSTATE
dg dgtest default default 28000 1389999403.91.sap05.austin.hp.com
dm dgtest_disk001 3pardata0_0_27 auto 65536 33483648 -
v lvol1 - ENABLED ACTIVE 16384 SELECT - fsgen
pl lvol1-01 lvol1 ENABLED ACTIVE 16384 CONCAT - RW
sd dgtest_disk001-01 lvol1-01 dgtest_disk001 0 16384 0 3pardata0_0_27 ENA
|
Create Veritas file system
# mkfs -F vxfs -o bsize=8192,largefiles /dev/vx/rdsk/dgtest/lvol1
version 9 layout
32768000 sectors, 2048000 blocks of size 8192, log size 2048 blocks
rcq size 128 blocks
largefiles supported
|
Add line to /etc/vfstab file and mount file system.
# dgtest
/dev/vx/dsk/dgtest/lvol1 /dev/vx/rdsk/dgtest/lvol1 /dgtest-growing vxfs 2 yes largefiles
|
Resizing (mostly growing) existing LVOL
For example, to add 50G to lvol, execute (the file system grows automatically)
# /opt/VRTS/bin/vxresize -g <dg_name> <lvol_name> +50G
|
Or to resize to desired size execute:
# /opt/VRTS/bin/vxresize -g <dg_name> <lvol_name> <desired_size><M|G>
|
The command to show max size of new volume to be created in a DG
# vxassist -g dg# maxsize
|
The command to show max grow of lvol in DG
# vxassist -g dg# maxgrow lvol#
|
Mirroring LVOL to new LUNs and releasing old LUNs
Sometimes it's needed to mirror LVOLs to new LUNs, and release old ones.
The most common situations is because some old stripes are created with too many number of columns, I've seen 18 or 33.
Better practice is to have stripe with ncol=2 or 4 (number of new LUNs is the one that can be divided by 4)
Once new LUNs are added to a DG, start mirroring (in background with -b)
# vxassist -b -g <dg_name> mirror <lvol_name> layout=stripe stripeunit=128k nstripe=4 alloc=<disk1>,<disk2>,<etc>
|
Use vxtask to monitor progress of mirroring:
Also use " vxprint -g <dg> -hrt " to make sure mirroring is done when you see all plexes are active.
Once mirroring is done on all LVOLs, dissociate (dis) and remove (rm) old plexes:
# vxplex -g <dg_name> -o rm dis <plex_name>
|
Then remove disks from DG
# vxdg -g <dg_name> rmdisk <disk_name>
|
Initialize disk again and it's now ready to be returned to Storage Team:
# /opt/VRTS/bin/vxdisksetup -i <disk_name>
|
Adding SWAP
Create a Veritas disk group
# vxdg init dgswap dgswap_disk001=c4t20110002AC0086D2d4
# vxdg list
NAME STATE ID
dgtest enabled,cds 1390002309.94.sap05.austin.hp.com
dgswap enabled,cds 1390003875.96.sap05.austin.hp.com
# swap -l
swapfile dev swaplo blocks free
/dev/zvol/dsk/rpool/swap 256,1 16 67108848 67108848
# swap -s
total: 135976k bytes allocated + 25776k reserved = 161752k used, 88976288k available
|
Create logical volume
# vxassist -g dgswap make lvol1 32000m
# vxprint -g dgswap -hrt
|
Add swap
# swap -a /dev/vx/dsk/dgswap/lvol1
# swap -s
total: 136816k bytes allocated + 25736k reserved = 162552k used, 121742560k available
# swap -l
swapfile dev swaplo blocks free
/dev/zvol/dsk/rpool/swap 256,1 16 67108848 67108848
/dev/vx/dsk/dgswap/lvol1 349,25000 16 65535984 65535984
|
Add to /etc/vfstab
# dgswap
/dev/vx/dsk/dgswap/lvol1 - - swap - no -
|
Back to the main page