Archive

Archive for the ‘Programming’ Category

resolved – perl get only package/file names and not including package version number

May 9th, 2013 No comments

I got a list of packages installed on my centos linux box via:

rpm -qa|sort|uniq|sort

The result is like:

acl-2.2.39-8.el5
acpid-1.0.4-12.el5
alacarte-0.10.0-1.fc6
alsa-lib-1.0.17-1.el5
alsa-lib-devel-1.0.17-1.el5
amtu-1.0.6-2.el5
anacron-2.3-45.el5.centos
apr-1.2.7-11.el5_6.5
apr-util-1.2.7-11.el5_5.2

But now I only want the package names without version number, that is like:

acl
acpid
alacarte
alsa-lib
alsa-lib-devel
amtu
anacron
apr
apr-util

Here’s the perl script to fulfill this:

#!/usr/bin/perl
open($temp1,”<”,”/doxer/test/package-names.txt”) or die(“error opening”);
while(<$temp1>){
if($_ =~ /([a-zA-Z].*?)\-[0-9]{1,}\.[0-9]{1,}.*/){
print $1.”\n”;
}
}

Categories: Perl, Programming Tags:

perl script for getting sun zfs head project and share usage info

April 26th, 2013 No comments

Sometimes you would like to know on sun zfs head, which project occupies most of the space, and which shares of that occupies most space of that project.

Here’s a perl script to fulfill this(it’s a little cumbersome, but it works anyway)

#!/usr/bin/perl
use strict;
use warnings;
use Net::SSH::Perl;
use List::Util qw/sum/;
my $host = ‘test-zfs-host’;
my $user = ‘root’;
my $password = ‘password’;

my $ssh = Net::SSH::Perl->new($host);
$ssh->login($user,$password);
my ($stdout,$stderr,$exit) = $ssh->cmd(“shares show”);
my @std_arr=split(/:/,$stdout);
my @projects_arr = split(/\n/, $std_arr[2]);

foreach(@projects_arr){
$_ =~ s/^\s+|\s+$//g;
}
shift @projects_arr;
pop @projects_arr;
pop @projects_arr;

my @space_projects;
foreach(@projects_arr){
my ($stdout2,$stderr2,$exit2) = $ssh->cmd(“shares select $_ get”);
my @stdout_arr=split(/\n/,$stdout2);
my $space_temp=join(“\n”,grep(/space_total/,@stdout_arr));
my @space_total_project = “project “.$_.$space_temp;
push(@space_projects,@space_total_project);
}
my @space_projects3;
foreach(@projects_arr){
my ($stdout3,$stderr3,$exit3) = $ssh->cmd(“shares select $_ ls”);
my @stdout_arr3=split(/\n/,$stdout3);
my @space_temp3=grep(/\/export\//,@stdout_arr3);
my @space_temp4=grep(!/mountpoint/,@space_temp3);
push(@space_projects3,@space_temp4);
}
open(my $temp1,’>’,'/var/tmp/temp1′) or die(“cannot open file temp1″);
open(my $temp2,’>’,'/var/tmp/temp2′) or die(“cannot open file temp2″);
open(my $temp3,’>’,'/var/tmp/temp3′) or die(“cannot open file temp3″);
open(my $temp4,’>’,'/var/tmp/temp4′) or die(“cannot open file temp4″);
my @var_T;
my @var_G;
my @var_M;
my @var_K;

foreach(@space_projects){
if($_ =~ /.+space_total\s+(.*)K/){
push(@var_K,$_);
}
elsif($_ =~ /.+space_total\s+(.*)M/){
push(@var_M,$_);
}
elsif($_ =~ /.+space_total\s+(.*)G/){
push(@var_G,$_);
}
elsif($_ =~ /.+space_total\s+(.*)T/){
push(@var_T,$_);
}
}

select $temp1;
foreach(@var_T){
print $_.”\n”;
}
close $temp1;

select $temp2;
foreach(@var_G){
print $_.”\n”;
}
close $temp2;
select $temp3;
foreach(@var_M){
print $_.”\n”;
}
close $temp3;
select $temp4;
foreach(@var_K){
print $_.”\n”;
}
close $temp4;

system(“echo \”======zfs project usage info(Descending)======\”");
system(“sort -r -n -k 5 /var/tmp/temp1;sort -r -n -k 5 /var/tmp/temp2;sort -r -n -k 5 /var/tmp/temp3;sort -r -n -k 5 /var/tmp/temp4″);
open(my $temp5,’>’,'/var/tmp/temp5′) or die(“cannot open file temp5″);
open(my $temp6,’>’,'/var/tmp/temp6′) or die(“cannot open file temp6″);
open(my $temp7,’>’,'/var/tmp/temp7′) or die(“cannot open file temp7″);
open(my $temp8,’>’,'/var/tmp/temp8′) or die(“cannot open file temp8″);
my @var_T_2;
my @var_G_2;
my @var_M_2;
my @var_K_2;

foreach(@space_projects3){
if($_ =~ /\s+.*K\s+.*/){
push(@var_K_2,$_);
}
elsif($_ =~ /\s+.*M\s+.*/){
push(@var_M_2,$_);
}
elsif($_ =~ /\s+.*G\s+.*/){
push(@var_G_2,$_);
}
elsif($_ =~ /\s+.*T\s+.*/){
push(@var_T_2,$_);
}
}

select $temp5;
foreach(@var_T_2){
print $_.”\n”;
}
close $temp5;

select $temp6;
foreach(@var_G_2){
print $_.”\n”;
}
close $temp6;
select $temp7;
foreach(@var_M_2){
print $_.”\n”;
}
close $temp7;
select $temp8;
foreach(@var_K_2){
print $_.”\n”;
}
close $temp8;

system(“echo \”\n\n\n======zfs share usage info(Descending)======\”");
system(“sort -r -n -k 2 /var/tmp/temp5;sort -r -n -k 2 /var/tmp/temp6;sort -r -n -k 2 /var/tmp/temp7;sort -r -n -k 2 /var/tmp/temp8″);

The output would be like:

======zfs project usage info(Descending)======
project DC2_DMZ space_total = 7.68T
project dc2_c9testga space_total = 1.10T
project fa_trialadcf space_total = 277G
project fa_rehydration space_total = 266G
project common space_total = 10.0G
project NODE_8 space_total = 93K
project default space_total = 31K

 

======zfs share usage info(Descending)======
Service_Mid-2 1.44T /export/DC2_DMZ/Service_Mid-2
Service_Web 1.22T /export/DC2_DMZ/Service_Web
dc2_shared_idm 743G /export/DC2_DMZ/dc2_shared_idm
Infra_Web 400G /export/DC2_DMZ/Infra_Web
nuviaq_local02 988M /export/DC2_DMZ/nuviaq_local02
sftp_staging 127M /export/DC2_DMZ/sftp_staging
sftp_local03 14.8M /export/DC2_DMZ/sftp_local03
sftp_manager_local01 85K /export/DC2_DMZ/sftp_manager_local01

Categories: Perl Tags: , ,

GNU Posix Standards Conformance guide – take diffutils for example

January 30th, 2013 No comments

In a few cases, the GNU utilities’ default behavior is incompatible with the POSIX standard. To suppress these incompatibilities, define the POSIXLY_CORRECT environment variable. Unless you are checking for POSIX conformance, you probably do not need to define POSIXLY_CORRECT.

Normally options and operands can appear in any order, and programs act as if all the options appear before any operands. For example, ‘diff lao tzu -C 2’ acts like ‘diff -C 2 lao tzu’, since ‘2’ is an option-argument of -C. However, if the POSIXLY_CORRECT environment variable is set, options must appear before operands, unless otherwise specified for a particular command.

Newer versions of POSIX are occasionally incompatible with older versions. For example, older versions of POSIX allowed the command ‘diff -c -10’ to have the same meaning as ‘diff -C 10’, but POSIX 1003.1-2001 ‘diff’ no longer allows digit-string options like -10.

The GNU utilities normally conform to the version of POSIX that is standard for your system. To cause them to conform to a different version of POSIX, define the _POSIX2_VERSION environment variable to a value of the form yyyymm specifying the year and month the standard was adopted.

Two values are currently supported for _POSIX2_VERSION: ‘199209’ stands for POSIX 1003.2-1992, and ‘200112’ stands for POSIX 1003.1-2001. For example, if you are running older software that assumes an older version of POSIX and uses ‘diff -c -10’, you can work around the compatibility problems by setting ‘_POSIX2_VERSION=199209’ in your environment.

Categories: Programming, SHELL Tags:

a perl script to get all projects space usage in sun zfs heads

January 23rd, 2013 No comments

#!/usr/bin/perl
use strict;
use warnings;
use Net::SSH::Perl;
use List::Util qw/sum/;
my $host = ‘zfs-host’;
my $user = ‘root’;
my $password = ‘welcome1′;

my $ssh = Net::SSH::Perl->new($host);
$ssh->login($user,$password);
my ($stdout,$stderr,$exit) = $ssh->cmd(“shares show”);
my @std_arr=split(/:/,$stdout);
my @projects_arr = split(/\n/, $std_arr[2]);

foreach(@projects_arr){
$_ =~ s/^\s+|\s+$//g;
}
shift @projects_arr;
pop @projects_arr;
pop @projects_arr;

my @space_projects;
foreach(@projects_arr){
my ($stdout2,$stderr2,$exit2) = $ssh->cmd(“shares select $_ get”);
my @stdout_arr=split(/\n/,$stdout2);
my @space_total_project = grep(/space_total/,@stdout_arr);
push(@space_projects,@space_total_project);
}

foreach(@space_projects){
if ($_ =~ /.+=\s+(.+)M/){
$_=0;
}
elsif($_ =~ /.+=\s+(.+)G/){
$_=$1/1024;
}
elsif($_ =~ /.+=\s+(.+)K/){
$_=0;
}
elsif($_ =~ /.+=\s+(.+)T/){
$_=$1;
}
}
my $space_used_sum=sum(@space_projects);

print $space_used_sum.”\n”;

Categories: Perl, Programming Tags:

snmptrapd traphandle configuration example

January 16th, 2013 No comments

This article is going to show the basic configuration of snmptrapd and it’s traphandle command.

Assumptions:
snmptrapd is running on a linux host named “test-centos”;
The host sending snmptrap messages in this example is named “test-zfs-host”

Now first we’re going to set snmptrapd up on the snmptrap server side:

###Server side
[root@test-centos snmp]# cat /etc/snmp/snmptrapd.conf
#traphandle default /bin/mail -s “snmpdtrapd messages” <put your mail address here>
traphandle default /root/lognotify
authCommunity log,execute,net public

[root@test-centos snmp]# service snmptrapd restart

[root@test-centos snmp]# cat /root/lognotify
#!/bin/bash
read host
read ip
vars=

while read oid val
do
if [ "$vars" = "" ]
then
vars=”$oid = $val”
else
vars=”$vars, $oid = $val”
fi
done
echo trap: $host $ip $vars >/var/tmp/snmptrap.out

And to test whether snmptrapd is working as expected:

###On client side
snmptrap -v2c -c public test-centos:162 “” SNMPv2-MIB::sysDescr SNMPv2-MIB::sysDescr.0 s “test-zfs-host test-zfs-host.ip this is a test snmptrap string”

And after running this command, you can have a check of /var/tmp/snmptrap.out on the snmptrapd server side(test-centos):

[root@test-centos ~]# cat /var/tmp/snmptrap.out

PS:
If you’re using sun zfs head, you can set snmptrap destinations in zfs BUI(configuration -> SNMP), here’s the snapshot(click to see the larger image):

perl script for monitoring sun zfs memory usage

January 16th, 2013 No comments

On zfs’s aksh, I can check memory usage with the following:

test-zfs:> status memory show
Memory:
Cache 719M bytes
Unused 15.0G bytes
Mgmt 210M bytes
Other 332M bytes
Kernel 7.79G bytes

So now I want to collect this memory usae information automatically for SNMP’s use. Here’s the steps:

cpan> o conf prerequisites_policy follow
cpan> o conf commit

Since the host is using proxy to get on the internet, so in /etc/wgetrc:

http_proxy = http://www-proxy.us.example.com:80/
ftp_proxy = http://www-proxy.us.example.com:80/
use_proxy = on

Now install the Net::SSH::Perl perl module:

PERL_MM_USE_DEFAULT=1 perl -MCPAN -e ‘install Net::SSH::Perl’

And to confirm that Net::SSH::Perl was installed, run the following command:

perl -e ‘use Net::SSH::Perl’ #no output is good, as it means the package was installed successfully

Now here goes the perl script to get the memory usage of sun zfs head:

[root@test-centos ~]# cat /var/tmp/mrtg/zfs-test-zfs-memory.pl
#!/usr/bin/perl
use strict;
use warnings;
use Net::SSH::Perl;
my $host = ‘test-zfs’;
my $user = ‘root’;
my $password = ‘password’;

my $ssh = Net::SSH::Perl->new($host);
$ssh->login($user,$password);
my ($stdout,$stderr,$exit) = $ssh->cmd(“status memory show”);
$ssh->cmd(“exit”);
if($stderr){
print “ErrorCode:$exit\n”;
print “ErrorMsg:$stderr”;
} else {
my @std_arr = split(/\n/, $stdout);
shift @std_arr;
foreach(@std_arr) {
if ($_ =~ /.+\b\s+(.+)M\sbytes/){
$_=$1/1024;
}
elsif($_ =~ /.+\b\s+(.+)G\sbytes/){
$_=$1;
}
else{}
}
foreach(@std_arr) {
print $_.”\n”;
}
}
exit $exit;

PS:
If you get the following error messages during installation of a perl module:

[root@test-centos ~]# perl -MCPAN -e ‘install SOAP::Lite’
CPAN: Storable loaded ok
CPAN: LWP::UserAgent loaded ok
Fetching with LWP:
ftp://ftp.perl.org/pub/CPAN/authors/01mailrc.txt.gz
LWP failed with code[500] message[LWP::Protocol::MyFTP: connect: Connection timed out]
Fetching with Net::FTP:
ftp://ftp.perl.org/pub/CPAN/authors/01mailrc.txt.gz

Trying with “/usr/bin/links -source” to get
ftp://ftp.perl.org/pub/CPAN/authors/01mailrc.txt.gz
ELinks: Connection timed out

Then you may have a check of whether you’re using proxy to get on the internet(run cpan > o conf init to re-configure cpan; later you should set /etc/wgetrc: http_proxy, ftp_proxy, use_proxy).

 

Categories: NAS, Perl, Storage Tags: ,