This article is going to talk about zfs storage 7320 monitoring using net-snmp and mrtg. Although the monitored system is sun zfs storage 7320, you’ll find the main idea of this article can be applied to many different system monitoring, including but not limited to cpu usage/network/bandwidth/disk/temperature of cisco switches, other linux systems and even windows systems.

As net-snmp extending agent functionality is not supported on sun zfs storage 7320 which is solaris 11 express system, so I’m going to monitor sun zfs storage through using of one linux snmp client by writing monitoring scripts on that linux client rather than on zfs itself.

Now, here goes the steps:

Part 1 – set up snmp client and mrtg on a linux host

yum -y install gcc-* gd-* libpng-* zlib-* httpd
yum -y install net-snmp* net-snmp-libs lm_sensors lm_sensors-devel
yum -y install mrtg
cp /etc/snmp/snmpd.conf{,.bak}
echo “rocommunity public” > /etc/snmp/snmpd.conf
mkdir -p /etc/mrtg
chkconfig –level 2345 snmpd on
service snmpd start

Ensure snmpd is listening:

netstat -tunlp |grep snmp
tcp 0 0 127.0.0.1:199 0.0.0.0:* LISTEN 26427/snmpd
udp 0 0 0.0.0.0:161 0.0.0.0:* 26427/snmpd

And let’s have a test to make sure snmp client is working as expected:

[root@test-centos mrtg]# snmpwalk -v2c -c public localhost interface
IF-MIB::ifNumber.0 = INTEGER: 4
IF-MIB::ifIndex.1 = INTEGER: 1
IF-MIB::ifIndex.2 = INTEGER: 2
IF-MIB::ifIndex.3 = INTEGER: 3
IF-MIB::ifIndex.4 = INTEGER: 4
IF-MIB::ifDescr.1 = STRING: lo
IF-MIB::ifDescr.2 = STRING: eth0
IF-MIB::ifDescr.3 = STRING: eth1
……
……

Now it’s time to configure mrtg on linux:

[root@test-centos mrtg]# cat /etc/httpd/conf.d/mrtg.conf
Alias /mrtg /var/www/mrtg
<Location /mrtg>
Order deny,allow
Allow from all
</Location>

Now, do a httpd restart:

[root@test-centos ~]# apachectl restart

Part 2 – configure snmp on SUN zfs storage 7320 web UI

Log on SUN zfs storage 7320 web UI, navigate through “Configuration” -> “SNMP”, and configure as the following:

After this, you’ll need enable/restart SNMP service on zfs.
Now do a snmpwalk from snmp linux client to SUN zfs storage:

[root@test-centos ~]# snmpwalk -v2c -c public test-zfs-host interface
IF-MIB::ifNumber.0 = INTEGER: 6
IF-MIB::ifIndex.1 = INTEGER: 1
IF-MIB::ifIndex.4 = INTEGER: 4
IF-MIB::ifIndex.5 = INTEGER: 5
IF-MIB::ifIndex.6 = INTEGER: 6
IF-MIB::ifIndex.7 = INTEGER: 7
IF-MIB::ifIndex.8 = INTEGER: 8
IF-MIB::ifDescr.1 = STRING: lo0
……
……

• Part 3 – extending snmp mibs/oids on snmp client(the linux host)

As stated at the beginning of this article, net-snmp extending agent functionality is not supported on sun zfs storage 7320. So I’m going to monitor sun zfs storage through using of one linux snmp client by writing monitoring scripts on that linux client rather than on zfs itself.

To avoid ssh asking for password when connecting from the linux host, you need add the linux host’s pubkey to sun zfs storage:
On snmp Linux client:

[root@test-centos ~]# ssh-keygen -t rsa #if you already have pubkey, skip this step
[root@test-centos ~]# cat /root/.ssh/id_rsa.pub |awk ‘{print $2}’
AAAAB3NzaC1yc2EAAAABIwAAAQEAxYd97A/V5RwdkfzbkmYBqF189pTLOlbYt0dZzO395dfU0Sp/Ykrk+sOJO0bJZEtytuTcCz/bVutWB7vLzeQPxIToRUQnZX7ZoMsjyaFk3LhtAgFhYIycOw2FQL8Qvb5yMBASB2/KthsqaiNqOP/2Vy5e0aCFFIV5DlKQTp/3eceSMq8kTx+e801lZow++yT70rp3p+5WtriN/NKYI0B3cpSQY/36D/TcOF9v5IaqQokp/mLRoc1MLOhN0sy0ipCdT+0bbkZ4Lh8bEeQO48UGKEOnYrYto33tay4mZk8HPWFK4w/TQGxBLthiuPQ4oZzG3gVpQUS4GRwI9zZoGtgELQ==

On Sun zfs storage:
Click “Configuration”, then add the snmp client’s public key(ensure RSA is selected):

Do a ssh connection from snmp linux client to the sun zfs storage host, it should now not asking for password.(ensure to do this, as there’s possibility that sun zfs’s key is not on the linux client as you may never have connected from that linux client to the sun zfs storage system)

Now we’re to the most important part. Assume we want to monitoring space usage on the SUN zfs storage system, to do this, you’ll need do the following on the snmp linux client:

[root@test-centos mrtg]# cat /etc/snmp/snmpd.conf
rocommunity public
extend .1.3.6.1.4.1.2021.31 zfs-test-zfs-host-total /bin/bash /var/tmp/mrtg/zfs-test-zfs-host-total.sh
extend .1.3.6.1.4.1.2021.32 zfs-test-zfs-host-used /bin/bash /var/tmp/mrtg/zfs-test-zfs-host-used.sh

[root@test-centos ~]# cat /var/tmp/mrtg/zfs-test-zfs-host-used.sh
#!/bin/bash
_used=`ssh test-zfs-host “status ls”|grep Used|awk ‘{print gensub(‘/T/’,”",”g”,$2)}’`
echo $_used;

[root@test-centos ~]# cat /var/tmp/mrtg/zfs-test-zfs-host-total.sh
_used=`ssh test-zfs-host “status ls”|grep Used|awk ‘{print gensub(‘/T/’,”",”g”,$2)}’` #I trimed ‘T’, you may need modify this to meet your environment
_avail=`ssh test-zfs-host “status ls”|grep Avail|awk ‘{print gensub(‘/T/’,”",”g”,$2)}’` #I trimed ‘T’, you may need modify this to meet your environment
_all=`echo $_used + $_avail|bc`
echo $_all;

[root@test-centos ~]# chmod +x /var/tmp/mrtg/zfs-test-zfs-host-used.sh
[root@test-centos ~]# chmod +x /var/tmp/mrtg/zfs-test-zfs-host-total.sh

Now, let’s do a snmp restart on snmp linux client and then test the newly added OIDs:

[root@test-centos ~]# service snmpd restart

[root@test-centos ~]# snmpwalk -v2c -c public localhost .1.3.6.1.4.1.2021.32
UCD-SNMP-MIB::ucdavis.32.1.0 = INTEGER: 1
UCD-SNMP-MIB::ucdavis.32.2.1.2.19.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.117.115.101.100 = STRING: “/bin/bash”
UCD-SNMP-MIB::ucdavis.32.2.1.3.19.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.117.115.101.100 = STRING: “/var/tmp/mrtg/zfs-test-zfs-host-used.sh”
UCD-SNMP-MIB::ucdavis.32.2.1.4.19.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.117.115.101.100 = “”
UCD-SNMP-MIB::ucdavis.32.2.1.5.19.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.117.115.101.100 = INTEGER: 5
UCD-SNMP-MIB::ucdavis.32.2.1.6.19.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.117.115.101.100 = INTEGER: 1
UCD-SNMP-MIB::ucdavis.32.2.1.7.19.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.117.115.101.100 = INTEGER: 1
UCD-SNMP-MIB::ucdavis.32.2.1.20.19.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.117.115.101.100 = INTEGER: 4
UCD-SNMP-MIB::ucdavis.32.2.1.21.19.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.117.115.101.100 = INTEGER: 1
UCD-SNMP-MIB::ucdavis.32.3.1.1.19.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.117.115.101.100 = STRING: “9.31″
UCD-SNMP-MIB::ucdavis.32.3.1.2.19.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.117.115.101.100 = STRING: “9.31″
UCD-SNMP-MIB::ucdavis.32.3.1.3.19.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.117.115.101.100 = INTEGER: 1
UCD-SNMP-MIB::ucdavis.32.3.1.4.19.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.117.115.101.100 = INTEGER: 0
UCD-SNMP-MIB::ucdavis.32.4.1.2.19.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.117.115.101.100.1 = STRING: “9.31″
[root@test-centos ~]# snmpwalk -v2c -c public localhost .1.3.6.1.4.1.2021.31
UCD-SNMP-MIB::ucdavis.31.1.0 = INTEGER: 1
UCD-SNMP-MIB::ucdavis.31.2.1.2.20.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.116.111.116.97.108 = STRING: “/bin/bash”
UCD-SNMP-MIB::ucdavis.31.2.1.3.20.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.116.111.116.97.108 = STRING: “/var/tmp/mrtg/zfs-test-zfs-host-total.sh”
UCD-SNMP-MIB::ucdavis.31.2.1.4.20.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.116.111.116.97.108 = “”
UCD-SNMP-MIB::ucdavis.31.2.1.5.20.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.116.111.116.97.108 = INTEGER: 5
UCD-SNMP-MIB::ucdavis.31.2.1.6.20.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.116.111.116.97.108 = INTEGER: 1
UCD-SNMP-MIB::ucdavis.31.2.1.7.20.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.116.111.116.97.108 = INTEGER: 1
UCD-SNMP-MIB::ucdavis.31.2.1.20.20.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.116.111.116.97.108 = INTEGER: 4
UCD-SNMP-MIB::ucdavis.31.2.1.21.20.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.116.111.116.97.108 = INTEGER: 1
UCD-SNMP-MIB::ucdavis.31.3.1.1.20.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.116.111.116.97.108 = STRING: “16.32″
UCD-SNMP-MIB::ucdavis.31.3.1.2.20.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.116.111.116.97.108 = STRING: “16.32″
UCD-SNMP-MIB::ucdavis.31.3.1.3.20.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.116.111.116.97.108 = INTEGER: 1
UCD-SNMP-MIB::ucdavis.31.3.1.4.20.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.116.111.116.97.108 = INTEGER: 0
UCD-SNMP-MIB::ucdavis.31.4.1.2.20.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.116.111.116.97.108.1 = STRING: “16.32″

From the output, we can see that OIDs UCD-SNMP-MIB::ucdavis.32.3.1.1.19.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.117.115.101.100 (used space) and UCD-SNMP-MIB::ucdavis.31.3.1.1.20.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.116.111.116.97.108 (total space) are the two OIDs we want.

• Part 4 – do the mrtg drawing

As we got the OIDs we want, it’s now easier for us to do the mrtg drawing. On the snmp linux host, do the following steps:

[root@test-centos ~]# cat /etc/mrtg/test-zfs-host.cfg
#LoadMIBs: /usr/share/snmp/mibs/UCD-SNMP-MIB.txt,/usr/share/snmp/mibs/TCP-MIB.txt
workdir: /var/www/mrtg/
Title[zfs_space_test-zfs-host]: Percentage used space on zfs
PageTop[zfs_space_test-zfs-host]: <h1>Percentage used space on zfs</h1>
Target[zfs_space_test-zfs-host]: .1.3.6.1.4.1.2021.32.3.1.1.19.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.117.115.101.100&.1.3.6.1.4.1.2021.31.3.1.1.20.122.102.115.45.115.108.99.101.49.48.115.110.48.49.45.116.111.116.97.108:public@localhost
Options[zfs_space_test-zfs-host]: growright,gauge,transparent,nopercent
Unscaled[zfs_space_test-zfs-host]: ymwd
MaxBytes[zfs_space_test-zfs-host]: 100
YLegend[zfs_space_test-zfs-host]: UsedSpace %
ShortLegend[zfs_space_test-zfs-host]: T
LegendI[zfs_space_test-zfs-host]: Used
LegendO[zfs_space_test-zfs-host]: Total
Legend1[zfs_space_test-zfs-host]: Percentage used space on zfs
Legend2[zfs_space_test-zfs-host]: Percentage all space on zfs

PS:
You need replace “UCD-SNMP-MIB::ucdavis” with .1.3.6.1.4.1.2021 or you’ll get error messages like the following:

[root@test-centos ~]# env LANG=C TZ=Asia/Shanghai /usr/bin/mrtg /etc/mrtg/test-zfs-host.cfg
Argument “v4only” isn’t numeric in int at /usr/bin/../lib64/mrtg2/SNMP_Session.pm line 183.
backoff (v4only) must be a number >= 1.0 at /usr/bin/../lib64/mrtg2/SNMP_util.pm line 465

Let’s continue:

[root@test-centos ~]# env LANG=C TZ=Asia/Shanghai /usr/bin/mrtg /etc/mrtg/test-zfs-host.cfg
24-12-2012 01:46:41, Rateup WARNING: /usr/bin/rateup could not read the primary log file for zfs_space_test-zfs-host
24-12-2012 01:46:41, Rateup WARNING: /usr/bin/rateup The backup log file for zfs_space_test-zfs-host was invalid as well
24-12-2012 01:46:41, Rateup WARNING: /usr/bin/rateup Can’t remove zfs_space_test-zfs-host.old updating log file
24-12-2012 01:46:41, Rateup WARNING: /usr/bin/rateup Can’t rename zfs_space_test-zfs-host.log to zfs_space_test-zfs-host.old updating log file

[root@test-centos ~]# env LANG=C TZ=Asia/Shanghai /usr/bin/mrtg /etc/mrtg/test-zfs-host.cfg
24-12-2012 01:46:46, Rateup WARNING: /usr/bin/rateup Can’t remove zfs_space_test-zfs-host.old updating log file

[root@test-centos ~]# env LANG=C TZ=Asia/Shanghai /usr/bin/mrtg /etc/mrtg/test-zfs-host.cfg

[root@test-centos ~]# indexmaker –output=/var/www/mrtg/index.html /etc/mrtg/test-zfs-host.cfg

Now add cronjob:

0-59/5 * * * * env LANG=C TZ=Asia/Shanghai /usr/bin/mrtg /etc/mrtg/test-zfs-host.cfg

Visit http://<your linux box’s ip address>/mrtg/ to get the GUI result(you’ll wait 5 minutes for the initial result, be patient!)

• Part 5 – troubleshooting

• If you met error messages like the following:

[root@test-centos ~]# env LANG=C TZ=Asia/Shanghai /usr/bin/mrtg /etc/mrtg/test-zfs-host.cfg
Argument “v4only” isn’t numeric in int at /usr/bin/../lib64/mrtg2/SNMP_Session.pm line 183.
backoff (v4only) must be a number >= 1.0 at /usr/bin/../lib64/mrtg2/SNMP_util.pm line 465

That’s because you’re using OIDs/MIBs alias rather than number. Change alias to number, i.e. change UCD-SNMP-MIB::ucdavis to .1.3.6.1.4.1.2021, and then re-check.

• If you met error messages like the following:

• If you met error messages like the following:

[root@test-centos mrtg]# env LANG=C TZ=Asia/Shanghai /usr/bin/mrtg /etc/mrtg/test-zfs-host.cfg
SNMP Error:
Received SNMP response with error code
error status: noSuchName
index 2 (OID: 1.3.6.1.4.1.2021.44)
SNMPv1_Session (remote host: “localhost” [127.0.0.1].161)
community: “public”
request ID: 1786815468
PDU bufsize: 8000 bytes
timeout: 2s
retries: 5
backoff: 1)
at /usr/bin/../lib64/mrtg2/SNMP_util.pm line 490
SNMPGET Problem for .1.3.6.1.4.1.2021.44 .1.3.6.1.4.1.2021.44 sysUptime sysName on public@localhost::::::v4only
at /usr/bin/mrtg line 2035
SNMP Error:
Received SNMP response with error code
error status: noSuchName
index 2 (OID: 1.3.6.1.4.1.2021.45)
SNMPv1_Session (remote host: “localhost” [127.0.0.1].161)
community: “public”
request ID: 1786815469
PDU bufsize: 8000 bytes
timeout: 2s
retries: 5
backoff: 1)
at /usr/bin/../lib64/mrtg2/SNMP_util.pm line 490
SNMPGET Problem for .1.3.6.1.4.1.2021.45 .1.3.6.1.4.1.2021.45 sysUptime sysName on public@localhost::::::v4only
at /usr/bin/mrtg line 2035
Monday, 24 December 2012 at 14:41: ERROR: Target[zfs_space_test-zfs-host][_IN_] ‘( $target->[0]{$mode} ) * 10000 / ( $target->[1]{$mode} )’ (warn): Use of uninitialized value in division (/) at (eval 16) line 1.
Monday, 24 December 2012 at 14:41: ERROR: Target[zfs_space_test-zfs-host][_OUT_] ‘( $target->[0]{$mode} ) * 10000 / ( $target->[1]{$mode} )’ (warn): Use of uninitialized value in division (/) at (eval 17) line 1.

Then one possible culprit is that new net-snmp stop supporting “exec”, use “extend” instead and re-try.

Note that you can also use “snmpd -f -Le” to check error messages related to snmpd.

PS:
Here’s more links from where you can read more about net-snmp/mrtg:

1. net-snmp FAQ http://www.net-snmp.org/FAQ.html
2. mrtg configuration opstions http://oss.oetiker.ch/mrtg/doc/mrtg-reference.en.html
3. SUN zfs storage SNMP – http://docs.oracle.com/cd/E22471_01/html/820-4167/configuration__services__snmp.html
4. net-snmp extending agent functionality – http://linux.die.net/man/5/snmpd.conf (search for ‘extending agent functionality’ on this page)
5. Now here’s the image of the baby(click on it to see the larger one:

Today I tried access one website within company’s network which was very important to me. But the site loaded incompletely and so was not usable.

As always, company network is somehow restricted, for example, only output port 80 and 443 are enabled in my company’s network. As many companies do, McAfee software is used to restrict internal employees surfing the internet. So I firstly thought McAfee was the culprit.

As I’m using chrome browser, so I press F12 to open the developer tools. On “Network” tab(you may need ctrl+F5 to forcely refresh the website), I found the following error status:

Then I opened that url source in a new tab, and the following page occured:

So everything sorted! I clicked the button “click here if you have read ….”, and then went back to the site, refresh, and later everything was ok!

After upgrading Oracle VM Server(OVS), I encountered the error message below when trying to startup guests from Oracle VM Manager(OVM):

Start – /OVS/running_pool/74_testhost
PowerOn Failed : Result – failed:<Exception: return=>failed:<Exception: ['xm', 'create', '/var/ovs/mount/4E686F742AB147E591A3EDAEC8F9802D/running_pool/74_testhost/vm.cfg']=> Error: Acquire running lock failed: 256

>
StackTrace:
File “/opt/ovs-agent-2.3/OVSXXenVM.py”, line 59, in xen_start_vm
run_cmd(args=cmd)
File “/opt/ovs-agent-2.3/OVSCommons.py”, line 90, in run_cmd
raise Exception(‘%s => %s’ % (args, err))
>
StackTrace:
File “/opt/ovs-agent-2.3/OVSSiteVM.py”, line 168, in start_vm
raise e

To resolve this issue, you can try the following steps(I’ve successfully resolve the issue after these steps)

shutdown all guests on OVM(do not shutdown OVM itself)
delete servers on OVM(except for OVS master Server)
stop ovs-agent(except for OVS master Server)
delete ovs-agent database #mv /etc/ovs-agent/db /var/tmp/db.bak(except for OVS master Server)
startup ovs-agent(except for OVS master Server)
service ovs-agent configure(except for OVS master Server)
add server back on OVM(except for OVS master Server)
add server back to server pool(except for OVS master Server)
migrate OVM to the non-master server
startup all guests through OVM
migrate OVM back to master

That’s all. Good luck!

This article is going to talk about the concept of hash/md5 vs crc32 hash.

• Concept

Hash is some sort of a fingerprint of some information (data). It is designed to produce a code of specific length (somewhere between 8 and 512 bits) that satisfies the following:
- Every bit in the message contributes to the hash. This means that changing any bit in the message should change the hash.
- Relatively small changes in the message should always result in changes in the hash. We want to be sure that it would take an extremely unlikely combination of errors to produce an identical hash.
- The histogram of output hash values for input messages should tend to be flat. For a given input message, we want the probability of a given hash being produced to be nearly equal across the entire range of possible hash values (for example 0h to FFh for 8 bit hashes).

CRC stands for Cyclic redundancy code. It is a code that provides efficient error detection. It is used by many programs (for example Zip compression programs, communication protocols…). There are many variants of this code.
CRC is also often used for reference to general hash code.

CRC-32 is an acronym for the 32 bit Cyclical Redundancy Check algorithm. CRC-32 generally refers to a specific 32 bit CRC formula sanctioned by the CCITT, an international standards body primarily concerned with telecommunications. CRC-32 is used in communications protocols such as HDLC and ZMODEM to verify the integrity of blocks of data being transferred through various media.

MD5 stands for Message Digest 5 algorithm. Algorithm developed by Professor Ronald L. Rivest of MIT and used in many applications.

• Reliability

CRC-32 code is 32 bit number (code) generated by CRC-32 algorithm based on data input (in our case file content). This code is some sort of a “fingerprint”. However it differs somewhat from the human fingerprint. It is often said that no two people have identical fingerprints. This can’t be the case for our CRC fingerprint. Since there are more than 4,294,967,296 different files in the world, it is a foregone conclusion that some of them must have identical codes. However, the CRC-32 does satisfy all the above mentioned requirements for hash code. They were the goals that the CCITT had in mind when selecting the CRC-32 algorithm. In practice, the chances of inadvertently damaging or modifying a file without modifying the CRC is vanishingly small, so for all practical purposes testing CRC code to detect changes can be considered to be infallible.

However if file would be intentionally damaged (for example by a virus) the CRC could be restored to its previous state. This could be done by using using brute force method to add some bytes to end of the file. So this is one thing it cannot protect you from (although this is almost never done).

MD5 was developed by Professor Ronald L. Rivest of MIT. What it does, to quote the executive summary of rfc1321, is: The MD5 algorithm takes as input a message of arbitrary length and produces as output a 128-bit “fingerprint” or “message digest” of the input. It is conjectured that it is computationally infeasible to produce two messages having the same message digest, or to produce any message having a given prespecified target message digest. The MD5 algorithm is intended for digital signature applications, where a large file must be “compressed” in a secure manner before being encrypted with a private (secret) key under a public-key cryptosystem such as RSA.

In essence, MD5 is a way to verify data integrity, and is much more reliable than checksum and many other commonly used methods.

• Implementation

Hash is implemented using concurrent reading and hash calculation. This is done by writing data from media into intermediate buffer in one thread and reading from buffer and calculating hash values in other thread. Hash values are written to disk as they are calculated to minimize memory usage.

OFED is an abbr for “OpenFabrics Enterprise Distribution”. When installing OFED-1.5.4.1 on a centos/RHEL 5.8 linux system, I met the following problem:

In file included from /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.4.1/drivers/infiniband/core/notice.c:37:
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.4.1/kernel_addons/backport/2.6.18-EL5.7/include/linux/pci.h:164: error: conflicting types for ‘pci_pcie_cap’
include/linux/pci.h:1015: error: previous definition of ‘pci_pcie_cap’ was here
make[4]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.4.1/drivers/infiniband/core/notice.o] Error 1
make[3]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.4.1/drivers/infiniband/core] Error 2
make[2]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.4.1/drivers/infiniband] Error 2
make[1]: *** [_module_/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.4.1] Error 2
make[1]: Leaving directory `/usr/src/kernels/2.6.18-308.4.1.0.1.el5-x86_64′
make: *** [kernel] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.70159 (%build)

After googling and experimenting, I can tell you the definite resolution for this problem -> install another version of OFED, OFED-1.5.3.2. Go to the following official site to download: http://www.openfabrics.org/downloads/OFED/ofed-1.5.3/

PS:

1. Here’s the file of OFED installation howto: OFED installation README.txt

2.Some Infiniband commands that may be useful for you:

service openibd status/start/stop
lspci |grep -i infini
ibnetdiscover
ibhosts
ibswitches
ibv_devinfo #Verify the status of ports by using ibv_devinfo: all connected ports should report a “PORT_ACTIVE” state

Some iLom software(for example oracle iLom) use java jnlp applet(java console) to open remote console of the server. To use this functionality, you need java plugin on firefox installed. Here’s the detailed steps:

• Update firefox: yum update firefox, please note that you better remove firefox 64 bit version and use 32 version ones(reason followed). After this, you’ll have firefox 10 on the system.

@namespace url(http://www.w3.org/1999/xhtml);
#mainpage {
visibility: visible !important;
}

• Install JRE 6: Go to http://www.oracle.com/technetwork/java/javase/downloads/jre6u37-downloads-1859589.html and download jre-6u37-linux-i586-rpm.bin. After that, install the package on the system using rpm -Uvh.(make sure NOT to download 64bits version, as java console only support 32bits java plugin)

• Make java plugin available to firefox: If you are planning to run firefox under root, do the following steps:

mkdir -p /root/.mozilla/plugins/

ln -s /usr/java/jre1.6.0_37/lib/libnpjp2.so /root/.mozilla/plugins/

PS:

if you’ve installed java plugin before, you’ll first need remove it.

• Now restart firefox, and go to about:plugins, you’ll see java plugin enabled. Open java console and enjoy!(select javaWS if prompted)