Archive

Archive for August, 2012

solaris ipmp bonding experiment

August 17th, 2012 No comments

[email protected] ~ # cat /etc/hosts
#
# Internet host table
#
::1 localhost
127.0.0.1 localhost
10.240.3.221 host1-e1000g2
10.240.3.223 host1-e1000g3
10.240.3.222 host1

[email protected] ~ # cat /etc/hostname.e1000g2
host1-e1000g2 group bak deprecated -failover netmask + broadcast + up
addif host1 netmask + broadcast + up
[email protected] ~ #
[email protected] ~ # cat /etc/hostname.e1000g3
host1-e1000g3 group bak deprecated -failover standby netmask + broadcast + up
[email protected] ~ #
[email protected] ~ # cat /etc/default/mpathd
#
#pragma ident "@(#)mpathd.dfl 1.2 00/07/17 SMI"
#
# Time taken by mpathd to detect a NIC failure in ms. The minimum time
# that can be specified is 100 ms.
#
FAILURE_DETECTION_TIME=10000
#
# Failback is enabled by default. To disable failback turn off this option
#
FAILBACK=yes
#
# By default only interfaces configured as part of multipathing groups
# are tracked. Turn off this option to track all network interfaces
# on the system
#
TRACK_INTERFACES_ONLY_WITH_GROUPS=yes

 

After this, reboot host(ensure /usr/lib/inet/in.mpathd is running)

[email protected] ~ # ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 10.240.3.206 netmask ffffff00 broadcast 10.240.3.255
ether 0:c:29:d3:d1:68
e1000g2: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER > mtu 1500 index 3
inet 10.240.3.221 netmask ff000000 broadcast 10.255.255.255
groupname bak
ether 0:c:29:d3:d1:86
e1000g2:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
inet 10.240.3.222 netmask ff000000 broadcast 10.255.255.255
e1000g3: flags=69040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVE R,STANDBY,INACTIVE> mtu 1500 index 4
inet 10.240.3.223 netmask ff000000 broadcast 10.255.255.255
groupname bak
ether 0:c:29:d3:d1:90
[email protected] ~ # if_mpadm -d e1000g2 #(detach or offline an interface. a networking blip will occur here, but soon recover itself)
[email protected] ~ #
[email protected] ~ # ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 10.240.3.206 netmask ffffff00 broadcast 10.240.3.255
ether 0:c:29:d3:d1:68
e1000g2: flags=89040842<BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,OFFLINE> mtu 1500 index 3
inet 10.240.3.221 netmask ff000000 broadcast 10.255.255.255
groupname bak
ether 0:c:29:d3:d1:86
e1000g3: flags=29040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,STANDBY> mtu 1500 index 4
inet 10.240.3.223 netmask ff000000 broadcast 10.255.255.255
groupname bak
ether 0:c:29:d3:d1:90
e1000g3:1: flags=21000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,STANDBY> mtu 1500 index 4
inet 10.240.3.222 netmask ff000000 broadcast 10.255.255.255
[email protected] ~ # if_mpadm -r e1000g2 #(reattach or online an interface that has been offlined with -d)
[email protected] ~ # tail /var/adm/messages
Aug 17 03:31:11 doxer.org at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:318)
Aug 17 03:31:11 doxer.org ... 34 more
Aug 17 03:31:11 doxer.org root: [ID 702911 user.crit] => com.sun.patchpro.cli.PatchServices@910040 <=Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Aug 17 03:31:11 doxer.org at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:174)
Aug 17 03:31:11 doxer.org at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:238)
Aug 17 03:31:11 doxer.org at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:318)
Aug 17 03:31:11 doxer.org ... 34 more
Aug 17 03:44:57 doxer.org snmpXdmid: [ID 290637 daemon.error] Unable to connect to snmpdx
Aug 17 04:17:19 doxer.org in.mpathd[188]: [ID 832587 daemon.error] Successfully failed over from NIC e1000g2 to NIC e1000g3
Aug 17 04:17:48 doxer.org in.mpathd[188]: [ID 620804 daemon.error] Successfully failed back to NIC e1000g2
[email protected] ~ # ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 10.240.3.206 netmask ffffff00 broadcast 10.240.3.255
ether 0:c:29:d3:d1:68
e1000g2: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 3
inet 10.240.3.221 netmask ff000000 broadcast 10.255.255.255
groupname bak
ether 0:c:29:d3:d1:86
e1000g2:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
inet 10.240.3.222 netmask ff000000 broadcast 10.255.255.255
e1000g3: flags=69040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,STANDBY,INACTIVE> mtu 1500 index 4
inet 10.240.3.223 netmask ff000000 broadcast 10.255.255.255
groupname bak
ether 0:c:29:d3:d1:90
[email protected] ~ # ifconfig e1000g2 down
[email protected] ~ # ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 10.240.3.206 netmask ffffff00 broadcast 10.240.3.255
ether 0:c:29:d3:d1:68
e1000g2: flags=9040842<BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 3
inet 10.240.3.221 netmask ff000000 broadcast 10.255.255.255
groupname bak
ether 0:c:29:d3:d1:86
e1000g2:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
inet 10.240.3.222 netmask ff000000 broadcast 10.255.255.255
e1000g3: flags=69040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,STANDBY,INACTIVE> mtu 1500 index 4
inet 10.240.3.223 netmask ff000000 broadcast 10.255.255.255
groupname bak
ether 0:c:29:d3:d1:90
[email protected] ~ # ping 10.240.3.221
^C
[email protected] ~ # ping 10.240.3.223
10.240.3.223 is alive
[email protected] ~ # ifconfig e1000g2 up
[email protected] ~ #
[email protected] ~ #
[email protected] ~ # tail /var/adm/messages
Aug 17 03:31:11 doxer.org ... 34 more
Aug 17 03:31:11 doxer.org root: [ID 702911 user.crit] => com.sun.patchpro.cli.PatchServices@910040 <=Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Aug 17 03:31:11 doxer.org at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:174)
Aug 17 03:31:11 doxer.org at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:238)
Aug 17 03:31:11 doxer.org at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:318)
Aug 17 03:31:11 doxer.org ... 34 more
Aug 17 03:44:57 doxer.org snmpXdmid: [ID 290637 daemon.error] Unable to connect to snmpdx
Aug 17 04:17:19 doxer.org in.mpathd[188]: [ID 832587 daemon.error] Successfully failed over from NIC e1000g2 to NIC e1000g3
Aug 17 04:17:48 doxer.org in.mpathd[188]: [ID 620804 daemon.error] Successfully failed back to NIC e1000g2
Aug 17 04:18:51 doxer.org in.mpathd[188]: [ID 975029 daemon.error] No test address configured on interface e1000g2; disabling probe-based failure detection on it

PS:

1.IPMP(bonding) and Link aggregation(LACP) are different things. Link aggregations(or trunk) provide high availability and higher throughput by aggregating multiple interfaces at the MAC layer. IP Multipathing (IPMP, or bonding) provides features such as higher availability at the IP layer. If you have 4 NICs, you can aggregate 2 nics and bonded them. This way you'll have 2 gig throughput and protect switch and nic level failures. (ipmp or bonding works at IP layer. LACP also needs support on switch side)

2.For more infomation about solaris IPMP, you may refer to the following pdf file solaris IPMP bonding.pdf

Resolved – change ldap expiration policy how to

August 15th, 2012 No comments

If you want to change the default expiration time to 90 days, here's the steps:

  1. Find subject DN cn=posix_account_password_policy,ou=People, dc=doxer,dc=org(search parameters,  objectClass=ldapsubentry under ou=People, dc=doxer,dc=org)
  2. Change passwordMaxAge parameter to '7776000'  (value mentioned in seconds equals to 90 days)
  3. If your ldap server has replication, have a check of whether the replication syncs the modification.
  4. Create a new account, and test it through ldapsearch. For example:

[root@doxer ~]# ldapsearch -x -W -D cn="Directory Manager" -h ldap.doxer.org -b ou=people,dc=doxer,dc=org uid=testme passwordexpirationtime
Enter LDAP Password:
# extended LDIF
#
# LDAPv3
# base <ou=people,dc=doxer,dc=org> with scope subtree
# filter: uid=testme
# requesting: passwordexpirationtime
#

# testme, People, doxer.org
dn: uid=testme,ou=People,dc=doxer,dc=org
passwordexpirationtime: 20121113011623Z #(today is 20120815, meaning that it's the result we want!)

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

Categories: IT Architecture Tags:

vlan configuration on cisco switch Catalyst 6500 Series

August 9th, 2012 No comments

Firstly, have a read of vlan wiki page(benefits compared to physical lan, cisco VTP):

http://en.wikipedia.org/wiki/Virtual_LAN

Then, have a read of the following about configuration of vlan on cisco switch Catalyst 6500 Series(vlan ranges, vlan translation):

http://www.cisco.com/en/US/docs/switches/lan/catalyst6500/ios/12.2SX/configuration/guide/vlans.html

resolved – semget failed with status 28 failed oracle database starting up

August 2nd, 2012 No comments

Today we met a problem with semaphore and unable to start oracle instances. Here's the error message:

ORA-27154: post/wait create failed
ORA-27300: OS system dependent operation:semget failed with status: 28
ORA-27301: OS failure message: No space left on device
ORA-27302: failure occurred at: sskgpcreates

So it turns out, the max number of arrays have been reached:
#check limits of all IPC
root@doxer# ipcs -al

------ Shared Memory Limits --------
max number of segments = 4096
max seg size (kbytes) = 67108864
max total shared memory (kbytes) = 17179869184
min seg size (bytes) = 1

------ Semaphore Limits --------
max number of arrays = 128
max semaphores per array = 250
max semaphores system wide = 1024000
max ops per semop call = 100
semaphore max value = 32767

------ Messages: Limits --------
max queues system wide = 16
max size of message (bytes) = 65536
default max size of queue (bytes) = 65536

#check summary of semaphores
root@doxer# ipcs -su

------ Semaphore Status --------
used arrays = 127
allocated semaphores = 16890

To resolve this, we need increase value of max number of semaphore arrays:

root@doxer# cat /proc/sys/kernel/sem
250 1024000 100 128
                 ^---needs to be increased

PS:

Here's an example with toilets that describes differences between mutex and semaphore LOL http://koti.mbnet.fi/niclasw/MutexSemaphore.html