juin 29 2011

HPVM disk physical/virtual mapping script

Tag: Scripts,UnixUggla @ 17 h 02 min

This is a short script to create a physical (host)/virtual (guest) map.
This is a basic script written quickly, that was used to simplify data migration operations. So improvements welcomed.

    Requirements :

  • script must be run from host. (not guest)
  • ssh must be allowed using keys (no passwd) to all guests.
  • hpvmstatus command must be available from path.

#!/usr/bin/perl

use strict;
use warnings;
use Data::Dumper;

my @hpvmstatus=`hpvmstatus`;
my @vms;
my $vm;

my %vm_data;

foreach(@hpvmstatus) {
        ($vm)=$_ =~ /^(.+)\s+[0-9]+\sHPUX\s+On/;
        if (defined($vm)){
                $vm =~ s/\s+//g;
                push (@vms,$vm);
        }
}

foreach $vm (@vms){
        my @hpvmstatus_vm=`hpvmstatus -P $vm`;

        foreach(@hpvmstatus_vm){
                #Device  Adaptor    Bus Dev Ftn Tgt Lun Storage   Device
                #======= ========== === === === === === ========= =========================
                #disk    avio_stor    0   2   0   0   0 disk      /dev/rdisk/disk17

                (my $ftn,my $tgt,my $lun,my $disk)=$_ =~ /^disk\s+avio_stor\s+\d+\s+\d+\s+(\d+)\s+(\d+)\s+(\d+)\s+disk\s+(.+)$/;

                if (defined($ftn)){
                $vm_data{$vm}->{$disk}->{"dev"}=$ftn;
                $vm_data{$vm}->{$disk}->{"tgt"}=$tgt;
                $vm_data{$vm}->{$disk}->{"lun"}=$lun;

                #Convert tgt to the good legacy device
                my $tgtconv=sprintf("%02X",$tgt);
                my $tgtconv_part1;
                my $tgtconv_part2;

                ($tgtconv_part1)=$tgtconv=~/(.)./;
                ($tgtconv_part2)=$tgtconv=~/.(.)/;

                $tgtconv_part1=hex($tgtconv_part1);
                $tgtconv_part2=hex($tgtconv_part2);

                #$vm_data{$vm}->{$disk}->{"legacy"}="c".$ftn."t".$tgtconv_part2."d".$tgtconv_part1;
                $vm_data{$vm}->{$disk}->{"legacy"}="c\\dt".$tgtconv_part2."d".$tgtconv_part1; # Don't know how the instance number is defined (cX) so use a more gereric regexp
                }
        }
}

foreach $vm (@vms){
        my @ioscan=`ssh -q -o stricthostkeychecking=no -o batchmode=yes root\@$vm \"ioscan -m dsf\"`;

        foreach(keys(%{$vm_data{$vm}})){
                my $regex=$vm_data{$vm}->{$_}->{"legacy"}."\$";
                my @vdisk=grep(/$regex/,@ioscan);
                my $vdisk_str=join(",",@vdisk);
                ($vdisk_str)=split(",",$vdisk_str);
                $vdisk_str=~s#\s+/dev/rdsk/.+##g;
                $vdisk_str=~s/\s//g;
                $vm_data{$vm}->{$_}->{"vdisk"}=$vdisk_str;
        }
}

# Debuging purpose
#print Dumper(\%vm_data);

foreach $vm (@vms){
        printf("VM\tPhys\t\t\tVirt\n");

        foreach(keys(%{$vm_data{$vm}})){
                printf("%s\t%s\t%s\n",$vm,$_,$vm_data{$vm}->{$_}->{"vdisk"});

        }
}


mai 24 2011

Rescan tape drives

Tag: Scripts,UnixUggla @ 16 h 12 min

Last week we got an issue with VLS. We had to rescan hw to recover tape drives in CLAIMED status.
So I created this short script/command line to rescan HPUX 11.31.

for system in toto titi tata
do
echo "Processing $system"
ssh root@$system 'ioscan > /dev/null && for i in $(dmesg | grep "replace_wwid" | \
perl -ne '"'"'{(my $get) = $_ =~ m/instance = (.+)\) The/; print "$get\n"}'"'"'); \
do scsimgr -f replace_wwid -C tgtpath -I $i;done | sort -u && ioscan -fnkC tape'
done


mar 18 2011

Obtenir une vision synthétique du stockage sous Linux avec une baie XP (HP).

Tag: ScriptsUggla @ 15 h 11 min

Un script perl qui combine les sorties de différents outils (xpinfo, multipath, mount) pour produire une vue complète du stockage disque.

Ce script fonctionne sur une machine Linux (RHEL5.5) avec une baie XP (HP).

Note : Ce script fonctionne mais peut probablement être grandement amélioré


#!/usr/bin/perl
# --------------------------------------------------------------------------------
# Script :   map.pl
# Revision : 1.1
# Author :   Rene Ribaud
# Description :
# --------------------------------------------------------------------------------
use strict;
use warnings;
#use diagnostics;   # Only use it for debugging purpose.

# --------------------------------------------------------------------------------
# Global variables
# --------------------------------------------------------------------------------
# Time
my ($sec,$min,$hour,$day,$month,$year,$wday,$yday,$isdst);
my $timestamp;

# Data
# Define a full structure
#my %mpaths =(
#        name =>{
#               size => undef,
#               uuid => undef,
#               dm => undef,
#               fs => undef,
#               disk =>         {
#                               diskname =>     {
#                                               port => undef,
#                                               lunid => undef,
#                                               sfnum => undef,
#                                               ldev => undef,
#                                               type => undef,
#                                               xp => undef,
#                                               },
#                               },
#               },
#        );

my %mpaths;

# --------------------------------------------------------------------------------
# Functions
# --------------------------------------------------------------------------------

# --------------------------------------------------------------------------------
# Main
# --------------------------------------------------------------------------------

# Get variable from command line

# Get timestamp
($sec,$min,$hour,$day,$month,$year,$wday,$yday,$isdst)=localtime();
$timestamp=sprintf("%4d%02d%02d-%02d:%02d:%02d",($year+1900),($month+1),$day,$hour,$min,$sec);

# Slurp tools data into arrays in memory
my @xpinfo=`xpinfo -i`;
my @multipath=`multipath -ll`;
my @mounts=`mount`;

for (my $line=0; $line<scalar(@multipath); $line++){
if ($multipath[$line] =~ /^mpath/){   # Check & parse line : mpath23 (360060e8015276a000001276a0000d0fc) dm-8 HP,OPEN-V
        (my $mpath, my $uuid, my $dm, my $type) = split ('\s+',$multipath[$line]);
        $uuid=~s/\(//g;
        $uuid=~s/\)//g;
        $mpaths{$mpath}->{"dm"}=$dm;
        $mpaths{$mpath}->{"uuid"}=$uuid;
        $mpaths{$mpath}->{"type"}=$type;
        # Crosscheck into mount to get the fs name
        # /dev/mapper/mpath14 on /oradata/crs/voting1 type ocfs2 (rw,_netdev,datavolume,nointr,heartbeat=local)
        my $fs;
        my @mount = grep(/$mpath\s/,@mounts);
        if (scalar(@mount==0)){
                $fs="N/A";
                }
                else
                {
                (undef,undef,$fs) = split('\s+',$mount[0]);
                }
        $mpaths{$mpath}->{"fs"}=$fs;

        # Next line to parse : [size=36G][features=1 queue_if_no_path][hwhandler=0][rw]
        $line++;
        if ($multipath[$line] =~ /^\[size=/){
                (my $size) = $multipath[$line] =~ /\[size=([0-9]+\w)/;
                $mpaths{$mpath}->{"size"}=$size;

                # Next line to parse : \_ round-robin 0 [prio=0][active]
                $line++;
                if ($multipath[$line] =~ /^\\_/){
                        # Next line to parse :  \_ 0:0:4:13 sdad 65:208 [active][ready]
                        $line++;
                        if ($multipath[$line] =~ /^ \\_/){
                                my $no_more_device=0;
                                do{
                                        (undef,undef,undef,my $disk, my $sf, undef) = split ('\s+',$multipath[$line]);
                                        $mpaths{$mpath}->{"disk"}->{$disk}->{"sfnum"}=$sf;
                                        # Crosscheck into xpinfo
                                        my @device = grep(/$disk\s/,@xpinfo);

                                        # Handle the case where device was not unmapped correctly
                                        # Unmapped from XP and not remoove from multipath.
                                        if (scalar(@device) == 0){
                                                $mpaths{$mpath}->{"disk"}->{$disk}->{"port"} = "Not found";
                                                $mpaths{$mpath}->{"disk"}->{$disk}->{"lunid"}= "N/A";
                                                $mpaths{$mpath}->{"disk"}->{$disk}->{"ldev"}= "N/A";
                                                $mpaths{$mpath}->{"disk"}->{$disk}->{"type"}= "N/A";
                                                $mpaths{$mpath}->{"disk"}->{$disk}->{"xp"}= "N/A";
                                        }
                                        else{
                                                #/dev/sdu                     76  04  02  CL2H  d0:f1  OPEN-V           00075626
                                                (undef,undef,undef, my $lunid, my $port, my $ldev, my $type, my $xp) = split('\s+',$device[0]);
                                                $mpaths{$mpath}->{"disk"}->{$disk}->{"port"}=$port;
                                                $mpaths{$mpath}->{"disk"}->{$disk}->{"lunid"}=$lunid;
                                                $mpaths{$mpath}->{"disk"}->{$disk}->{"ldev"}=$ldev;
                                                $mpaths{$mpath}->{"disk"}->{$disk}->{"type"}=$type;
                                                $mpaths{$mpath}->{"disk"}->{$disk}->{"xp"}=$xp;
                                        }
                                        if ((defined($multipath[$line+1])) && ($multipath[$line+1] =~ /^ \\_/)){
                                                $line++;
                                                }
                                                else {
                                                $no_more_device=1;
                                                }
                                        } while ($no_more_device==0);
                                }
                        }
                }
        }
}

# Print mapping
printf("%-34s %-9s %-7s %-6s %-6s %-8s %-6s %-6s %s\n","UUID","xp","type","size","dm","mapth","lunid","ldev","fs");
foreach my $mpath (sort(keys(%mpaths))){
        my @first_disk=keys(%{$mpaths{$mpath}->{"disk"}});
        # Check to report disk not unmapped correctly
        printf("%-34s %-9s %-7s %-6s %-6s %-8s %-6s %-6s %s\n"  ,$mpaths{$mpath}->{"uuid"}
                                                                ,$mpaths{$mpath}->{"disk"}->{$first_disk[0]}->{"xp"}
                                                                ,$mpaths{$mpath}->{"disk"}->{$first_disk[0]}->{"type"}
                                                                ,$mpaths{$mpath}->{"size"}
                                                                ,$mpaths{$mpath}->{"dm"}
                                                                ,$mpath
                                                                ,$mpaths{$mpath}->{"disk"}->{$first_disk[0]}->{"lunid"}
                                                                ,$mpaths{$mpath}->{"disk"}->{$first_disk[0]}->{"ldev"}
                                                                ,$mpaths{$mpath}->{"fs"}
                                                                );
        my $notfoundcount=0;
        foreach my $disk (keys(%{$mpaths{$mpath}->{"disk"}})){
                printf("%-6s %-8s %-4s\n",$disk,$mpaths{$mpath}->{"disk"}->{$disk}->{"sfnum"},$mpaths{$mpath}->{"disk"}->{$disk}->{"port"});

                if ($mpaths{$mpath}->{"disk"}->{$disk}->{"port"} eq "Not found"){
                        $notfoundcount++;
                        }

                if ($notfoundcount == scalar(keys(%{$mpaths{$mpath}->{"disk"}}))){
                        printf("This device was found in multipath but not reported by xpinfo.\n");
                        printf("Multipath device was probably not flushed after unmapping the device.\n");
                        printf("Try to run multipath -f %s.\n",$mpath);
                        }

                }

print "\n";
}

Output result :

UUID                               xp        type    size   dm     mapth    lunid  ldev   fs
360060e8015276a000001276a0000d0f1  00075626  OPEN-V  201M   dm-13  mpath14  02     d0:f1  /oradata/crs/voting1
sdai   66:32    CL1F
sdc    8:32     CL2F
sday   67:32    CL1H
sds    65:32    CL2H

360060e8015276a000001276a0000d0f2  00075626  OPEN-V  201M   dm-14  mpath15  03     d0:f2  /oradata/crs/voting2
sdd    8:48     CL2F
sdaz   67:48    CL1H
sdaj   66:48    CL1F
sdt    65:48    CL2H

360060e8015276a000001276a0000d0f3  00075626  OPEN-V  201M   dm-15  mpath16  04     d0:f3  /oradata/crs/voting3
sde    8:64     CL2F
sdu    65:64    CL2H
sdba   67:64    CL1H
sdak   66:64    CL1F

360060e8015276a000001276a0000d0f4  00075626  OPEN-V  201M   dm-16  mpath17  05     d0:f4  /oradata/GFSTEUR1/pfile
sdal   66:80    CL1F
sdv    65:80    CL2H
sdbb   67:80    CL1H
sdf    8:80     CL2F

360060e8015276a000001276a0000d0f5  00075626  OPEN-V  36G    dm-17  mpath18  06     d0:f5  /oradata/GFSTEUR1/undo
sdg    8:96     CL2F
sdbc   67:96    CL1H
sdw    65:96    CL2H
sdam   66:96    CL1F

360060e8015276a000001276a0000d0f8  00075626  OPEN-V  99G    dm-18  mpath19  09     d0:f8  /oradata/RMAN/rman01
sdh    8:112    CL2F
sdan   66:112   CL1F
sdbd   67:112   CL1H
sdx    65:112   CL2H
...


mar 01 2011

Gestion des espaces dans les boucles « for » avec IFS

Tag: ScriptsUggla @ 14 h 17 min

Une astuce pour gérer les espaces dans les boucles « for » en shell (bash, ksh).
Rien ne vaut un petit exemple pour montrer le problème :

root@hx04970: /root/home/root/nene # ll
total 0
-rw-r--r--   1 root       sys              0 Feb 28 14:09 1
-rw-r--r--   1 root       sys              0 Feb 28 14:09 2
-rw-r--r--   1 root       sys              0 Feb 28 14:09 3
drwxr-xr-x   2 root       sys             96 Feb 28 12:04 tata tutu
drwxr-xr-x   2 root       sys             96 Feb 28 12:03 titi
drwxr-xr-x   2 root       sys             96 Feb 28 12:03 toto

Imaginons que je fasse une boucle sur l’ensemble des fichiers de l’arborescence ci dessus.

root@hx04970: /root/home/root/nene # for i in *;do echo $i;done
1
2
3
tata tutu
titi
toto

Ici tout se passe bien, pas de problèmes, le répertoire « tata tutu » est bien interprété.

Maintenant je complique un peu en voulant faire une boucle sur les répertoires seulement :

root@hx04970: /root/home/root/nene # for i in $(find . -type d);do echo $i;done
.
./toto
./titi
./tata
tutu

Et la c’est le drame :) .

L’espace sur le répertoire « tata tutu » est vu comme un séparateur et le shell croit que nous avons 2 répertoires tata et tutu.
Le bon administrateur Unix sait qu’il faut éviter les espaces c’est mal. (C’est comme : on ne croise jamais les effluves : c’est mal –> cf Ghostbuster).

Cool ! Mais si le répertoire est un share cifs avec des utilisateurs de windows, il y a de grande chance que l’on retrouve plein d’espaces dans les noms de fichier.

La meilleure méthode de résolution du problème ci dessus est d’utiliser la « variable » IFS.
Un petit man ksh me donne :

IFS Internal field separators, normally space, tab,
and newline that are used to separate command
words resulting from command or parameter
substitution, and for separating words with the
special command read. The first character of the
IFS parameter is used to separate arguments for
the « $* » substitution (see Quoting below).

IFS est la « variable » qui définie le séparateur, par défaut espace, tab et nouvelle ligne.

En redéfinissant le séparateur, sans l’espace et le tab on obtient :

root@hx04970: /root/home/root/nene # IFS=$'\n' && for i in $(find . -type d);do echo $i;done
.
./toto
./titi
./tata tutu

Le problème est résolu.

Pour aller plus loin : http://tldp.org/LDP/abs/html/internalvariables.html#IFSH


fév 21 2011

Script mapping EVA

Tag: ScriptsUggla @ 15 h 07 min

Un petit script perl pour trouver les correspondances entre les luns UUIDs, special files, major numbers, minor numbers.

Dépendances :

evainfo –> outil client pour extraire les informations des EVA.

#!/usr/bin/perl

use strict;
use warnings;
use Data::Dumper;

my @evainfo=`evainfo -a`;
my @ioscan=`ioscan -m dsf`;
my %devmap;
my %devmap2;
my %lunmap;
my %sf;

my $dsf;
my $olddsf;
my $legacy;

# Create map : "legacy device" = "dsf device"
foreach(@ioscan){
        if ( /\/dev/ ){
                ($dsf, $legacy)=split(/\s+/,$_);
                if ( $dsf eq "" ){
                        $dsf=$olddsf;
                }
                $devmap{$legacy}=$dsf;
        $olddsf=$dsf;
        }
}

# Create map : "wwnn id" = "legacy device"
foreach(@evainfo){

        if ( /\/dev/){
                (my $legacy, undef, my $wwnn, my $size, my $ctrl)=split(/\s+/,$_);
                $lunmap{$wwnn}=$legacy;
        }
}

# Create map : "dsf device" = "major and minor"
%devmap2=%devmap; # values() modify the hash, so copy to a new one to avoid %devmap to be modified
foreach(values(%devmap2)){
        open (SHELLCMD,"ll $_ |");
                while (<SHELLCMD>){
                        (undef,undef,undef,undef,my $major, my $minor,undef,undef,undef,my $dev)=split(/\s+/,$_);
                        $sf{$dev}->{"major"}=$major;
                        $sf{$dev}->{"minor"}=$minor;
                }
        close (SHELLCMD)
}

# Debug purpose
#print Dumper(\%devmap);
#print Dumper(\%lunmap);
#print Dumper(\%sf);

# Print output
foreach(sort(keys(%lunmap))){
        my $legacy=$lunmap{$_};
        printf("%s %s %s %s\n",$_ ,$devmap{$legacy}, $sf{$devmap{$legacy}}->{"major"}, $sf{$devmap{$legacy}}->{"minor"});
}

Exemple d'usage :
root@hx000140: /root/home/root # perl eva_mapping.pl
6001-4380-05DE-C738-0000-6000-08B4-0000 /dev/oracle/asm_eva1 13 0x000019
6001-4380-05DE-C738-0000-6000-08B8-0000 /dev/oracle/asm_eva2 13 0x00001a
6001-4380-05DE-C738-0000-6000-08BC-0000 /dev/oracle/asm_eva3 13 0x00001b
6001-4380-05DE-C738-0000-6000-08C0-0000 /dev/oracle/asm_eva4 13 0x00001c
6001-4380-05DE-C738-0000-6000-08C4-0000 /dev/oracle.bak/asm_eva5 13 0x00001d
6001-4380-05DE-C738-0000-6000-08C8-0000 /dev/oracle.bak/asm_eva6 13 0x00001e
6001-4380-05DE-C738-0000-6000-08CC-0000 /dev/oracle.bak/asm_eva7 13 0x00001f
6001-4380-05DE-C738-0000-6000-08D0-0000 /dev/oracle.bak/asm_eva8 13 0x000020