Archive for the ‘Networking Security’ Category

SELinux security context and its elements – user, role, type identifiers

April 17th, 2014 No comments

All operating system access control is based on some type of access control attribute associated with objects and subjects. In SELinux, the access control attribute is called a security context. All objects (files, interprocess communication channels, sockets, network hosts, and so on) and subjects (processes) have a single security context associated with them. A security context has three elements: user, role, and type identifiers. The usual format for specifying or displaying a security context is as follows:


A valid security context must have one valid user, role, and type identifier, and that the identifiers are defined by the policy writer, and the string identifiers for each element are defined in the SELinux policy language.

Here’s the relationship between unix/linux users -> SELinux identifiers -> roles -> domain:


And here’s one example of SELinux transitions:


And here’s the code you can set to apache httpd server when SELinux runs in enforcing mode:

chcon -t httpd_sys_content_t /var/www/html
chcon -t httpd_sys_content_t /var/www/html -R
ls -Z /var/www/html

Some contents of this article is from book <SELinux by Example: Using Security Enhanced Linux>.

Categories: Linux, Security Tags:

checking MTU or Jumbo Frame settings with ping

February 14th, 2014 No comments

You may set your linux box’s MTU to jumbo frame sized 9000 bytes or larger, but if the switch your box connected to does not have jumbo frame enabled, then your linux box may met problems when sending & receiving packets.

So how can we get an idea of whether Jumbo Frame enabled on switch or linux box?

Of course you can log on switch and check, but we can also verify this from linux box that connects to switch.

On linux box, you can see the MTU settings of each interface using ifconfig:

[root@centos-doxer ~]# ifconfig eth0
eth0 Link encap:Ethernet HWaddr 08:00:27:3F:C5:08
RX packets:50502 errors:0 dropped:0 overruns:0 frame:0
TX packets:4579 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:9835512 (9.3 MiB) TX bytes:1787223 (1.7 MiB)
Base address:0xd010 Memory:f0000000-f0020000

As stated above, 9000 here doesn’t mean that Jumbo Frame enabled on your box to switch. As you can verify with below command:

[root@testbox ~]# ping -c 2 -M do -s 1472 testbox2
PING ( 1472(1500) bytes of data. #so here 1500 bytes go through the network
1480 bytes from ( icmp_seq=1 ttl=252 time=0.319 ms
1480 bytes from ( icmp_seq=2 ttl=252 time=0.372 ms

— ping statistics —
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.319/0.345/0.372/0.032 ms
[root@testbox ~]#
[root@testbox ~]#
[root@testbox ~]# ping -c 2 -M do -s 1473 testbox2
PING ( 1473(1501) bytes of data. #so here 1501 bytes can not go through. From here we can see that MTU for this box is 1500, although ifconfig says it’s 9000
From ( icmp_seq=1 Frag needed and DF set (mtu = 1500)
From ( icmp_seq=1 Frag needed and DF set (mtu = 1500)

— ping statistics —
0 packets transmitted, 0 received, +2 errors

Also, if your the switch is Cisco one, you can verify whether the switch port connecting server has enabled jumbo frame or not by sniffing CDP (Cisco discover protocol) packet. Here’s one example:

-bash-4.1# tcpdump -i eth0 -nn -v -c 1 ether[20:2] == 0×2000 #ether[20:2] == 0×2000 means capture only packets that have a 2 byte value of hex 2000 starting at byte 20
tcpdump: WARNING: eth0: no IPv4 address assigned
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
03:44:14.221022 CDPv2, ttl: 180s, checksum: 692 (unverified), length 287
Device-ID (0×01), length: 46 bytes: ‘’
Address (0×02), length: 13 bytes: IPv4 (1)
Port-ID (0×03), length: 16 bytes: ‘Ethernet111/1/12′
Capability (0×04), length: 4 bytes: (0×00000228): L2 Switch, IGMP snooping
Version String (0×05), length: 66 bytes:
Cisco Nexus Operating System (NX-OS) Software, Version 5.2(1)N1(4)
Platform (0×06), length: 11 bytes: ‘N5K-C5548UP’
Native VLAN ID (0x0a), length: 2 bytes: 123
AVVID trust bitmap (0×12), length: 1 byte: 0×00
AVVID untrusted ports CoS (0×13), length: 1 byte: 0×00
Duplex (0x0b), length: 1 byte: full
MTU (0×11), length: 4 bytes: 1500 bytes #so here MTU size was set to 1500 bytes
System Name (0×14), length: 18 bytes: ‘ucf-c1z3-swi-5k01b’
System Object ID (not decoded) (0×15), length: 14 bytes:
0×0000: 060c 2b06 0104 0109 0c03 0103 883c
Management Addresses (0×16), length: 13 bytes: IPv4 (1)
Physical Location (0×17), length: 13 bytes: 0×00/snmplocation
1 packets captured
1 packets received by filter
0 packets dropped by kernel
110 packets dropped by interface


  1. As for “-M do” parameter for ping, you may refer to man ping for more info. And as for DF(don’t fragment) and Path MTU Discovery mentioned in the manpage, you may read more on and
  2. Here’s more on tcpdump tips and
  3. Maximum packet size is the MTU plus the data-link header length. Packets are not always transmitted at the Maximum packet size. As we can see from output of iptraf -z eth0.
  4. Here’s more about MTU:

The link layer, which is typically Ethernet, sends information into the network as a series of frames. Even though the layers above may have pieces of information much larger than the frame size, the link layer breaks everything up into frames to send them over the network. This maximum size of data in a frame is known as the maximum transfer unit (MTU). You can use network configuration tools such as ip or ifconfig to set the MTU.

The size of the MTU has a direct impact on the efficiency of the network. Each frame in the link layer has a small header, so using a large MTU increases the ratio of user data to overhead (header). When using a large MTU, however, each frame of data has a higher chance of being corrupted or dropped. For clean physical links, a high MTU usually leads to better performance because it requires less overhead; for noisy links, however, a smaller MTU may actually enhance performance because less data has to be re-sent when a single frame is corrupted.


Add static routes in linux which will survive reboot and network bouncing

December 24th, 2013 No comments

We can see that in linux, the file /etc/sysconfig/static-routes is revoked by /etc/init.d/network:

[root@test-linux ~]# grep static-routes /etc/init.d/network
# Add non interface-specific static-routes.
if [ -f /etc/sysconfig/static-routes ]; then
grep “^any” /etc/sysconfig/static-routes | while read ignore args ; do

So we can add rules in /etc/sysconfig/static-routes to let network routes survive reboot and network bouncing. The format of /etc/sysconfig/static-routes is like:

any net netmask gw
any net netmask gw

To make route in effect immediately, you can use route add:

route add -net netmask gw

But remember that to change the default gateway, we need modify /etc/sysconfig/network(modify GATEWAY=).

After the modification, bounce the network using service network restart to make the changes in effect.

Network Performance Analysis

December 5th, 2013 No comments
  • Collisions and network saturation

Ethernet is similar to an old party-line telephone: everybody listens at once, everybody talks at once, and sometimes two talkers start at the same time. In a well-conditioned network, with only two hosts on it, it’s possible to use close to the maximum network’s bandwidth. However, NFS clients and servers live in a burst-filled environment, where many machines try to use the network at the same time. When you remove the well-behaved conditions, usable network bandwidth decreases rapidly.

On the Ethernet, a host first checks for a transmission in progress on the network before attempting one of its own. This process is known as carrier sense. When two or more hosts transmit packets at exactly the same time, neither can sense a carrier, and a collision results. Each host recognizes that a collision has occurred, and backs off for a period of time, t, before attempting to transmit again. For each successive retransmission attempt that results in a collision, t is increased exponentially, with a small random variation. The variation in back-off periods ensures that machines generating collisions do not fall into lock step and seize the network.

As machines are added to the network, the probability of a collision increases. Network utilization is measured as a percentage of the ideal bandwidth consumed by the traffic on the cable at the point of measurement. Various levels of utilization are usually compared on a logarithmic scale. The relative decrease in usable bandwidth going from 5% utilization to 10% utilization, is about the same as going from 10% all the way to 30% utilization.

Measuring network utilization requires a LAN analyzer or similar device. Instead of measuring the traffic load directly, you can use the average collision rate as seen by all hosts on the network as a good indication of whether the network is overloaded or not. The collision rate, as a percentage of output packets, is one of the best measures of network utilization. The Collis field in the output of netstat -in shows the number of collisions:

Code View: Scroll / Show All
% netstat -in 
Name Mtu  Net/Dest     Address      Ipkts   Ierrs  Opkts   Oerrs Collis Queue  
lo0  8232    7188    0      7188     0     0      0     
hme0 1500  139478  11     102155   0     3055   0

The collision rate for a host is the number of collisions seen by that host divided by the number of packets it writes, as shown in Figure 17-1.

Figure 17-1. Collision rate calculation

Figure 17-1. Collision rate calculation

Collisions are counted only when the local host is transmitting; the collision rate experienced by the host is dependent on its network usage. Because network transmissions are random events, it’s possible to see small numbers of collisions even on the most lightly loaded networks. A collision rate upwards of 5% is the first sign of network loading, and it’s an indication that partitioning the network may be advisable.

  • Network partitioning hardware

Network partitioning involves dividing a single backbone into multiple segments, joined by some piece of hardware that forwards packets. There are multiple types of these devices: repeaters, hubs, bridges, switches, routers, and gateways. These terms are sometimes used interchangeably although each device has a specific set of policies regarding packet forwarding, protocol filtering, and transparency on the network:

A repeater joins two segments at the physical layer. It is a purely electrical connection, providing signal amplification and pulse “clean up” functions without regard for the semantics of the signals. Repeaters are primarily used to exceed the single-cable length limitation in networks based on bus topologies, such as 10Base5 and 10Base2. There is a maximum to the number of repeaters that can exist between any two nodes on the same network, keeping the minimum end-to-end transit time for a packet well within the Ethernet specified maximum time-to-live. Because repeaters do not look at the contents of packets (or packet fragments), they pass collisions on one segment through to the other, making them of little use to relieve network congestion.

A hub joins multiple hosts by acting as a wiring concentrator in networks based on star topologies, such as 10BaseT. A hub has the same function as a repeater, although in a different kind of network topology. Each computer is connected, typically over copper, to the hub, which is usually located in a wiring closet. The hub is purely a repeater: it regenerates the signal from one set of wires to the others, but does not process or manage the signal in any way. All traffic is forwarded to all machines connected to the hub.

Bridges function at the data link layer, and perform selective forwarding of packets based on their destination MAC addresses. Some delay is introduced into the network by the bridge, as it must receive entire packets and decipher their MAC-layer headers. Broadcast packets are always passed through, although some bridge hardware can be configured to forward only ARP broadcasts and to suppress IP broadcasts such as those emanating from ypbind.Intelligent or learning bridges glean the MAC addresses of machines through observation of traffic on each interface. “Dumb” bridges must be loaded with the Ethernet addresses of machines on each network and impose an administrative burden each time the network topology is modified. With either type of bridge, each new segment is likely to be less heavily loaded than the original network, provided that the most popular inter-host virtual circuits do not run through the bridge.

You can think of a switch as an intelligent hub having the functionality of a bridge. The switch also functions at the data link layer, and performs selective forwarding of packets based on their destination MAC address. The switch forwards packets only to the intended port of the intended recipient. The switch “learns” the location of the various MAC addresses by observing the traffic on each port. When a switch port receives data packets, it forwards those packets only to the appropriate port for the intended recipient. A hub would instead forward the packet to all other ports on the hub, leaving it to the host connected to the port to determine its interest in the packet. Because the switch only forwards the packet to its destination, it helps reduce competition for bandwidth between the hosts connected to each port.

Repeaters, hubs, bridges, and switches divide the network into multiple distinct physical pieces, but the collection of backbones is still a single logical network. That is, the IP network number of all hosts on all segments will be the same. It is often necessary to divide a network logically into multiple IP networks, either due to physical constraints (i.e., two offices that are separated by several miles) or because a single IP network has run out of host numbers for new machines.Multiple IP networks are joined by routers that forward packets based on their source and destination IP addresses rather than 48-bit Ethernet addresses. One interface of the router is considered “inside” the network, and the router forwards packets to the “outside” interface. A router usually corrals broadcast traffic to the inside network, although some can be configured to forward broadcast packets to the “outside” network. The networks joined by a router need not be of the same type or physical media, and routers are commonly used to join local area networks to point-to-point long-haul internetwork connections. Routers can also help ensure that packets travel the most efficient paths to their destination. If a link between two routers fails, the sending router can determine an alternate route to keep traffic moving. You can install a dedicated router, or install multiple network interfaces in a host and allow it to route packets in addition to its other duties. Appendix A contains a detailed description of how IP packets are forwarded and how routes are defined to Unix systems.

At the top-most level in the network protocol stack, a gateway performs forwarding functions at the application level, and frequently must perform protocol conversion to forward the traffic. A gateway need not be on more than one network; however, gateways are most commonly used to join multiple networks with different sets of native protocols, and to enforce tighter control over access to and from each of the networks.

Replacing an Ethernet hub with a Fast Ethernet hub is like increasing the speed limit of a highway. Replacing a hub with a switch is similar to adding new lanes to the highway. Replacing an Ethernet hub with a Fast Ethernet switch is the equivalent of both improvements, although with a higher cost.



1.This article is from book <Managing NFS and NIS, Second Edition>.

2.Here’s the differences between broadcast domains and collision domains:

Collision Domains
- layer 1 of the OSI model
- a hub is an entire collision domain since it forwards every bit it receives from one interface on every other interfaces
- a bridge is a two interfaces device that creates 2 collision domains, since it forwards the traffic it receives from one interface only to the interface where the destination layer 2 device (based on his mac address) is connected to. A bridge is considered as an “intelligent hub” since it reads the destination mac address in order to forward the traffic only to the interface where it is connected
- a switch is a multi-interface hub, every interface on a switch is a collision domain. A 24 interfaces switch creates 24 collision domains (assuming every interface is connected to something, VLAN don’t have any importance here since VLANs are a layer 2 concept, not layer 1 like collision domains)

Broadcast Domains
- layer 2 of the OSI model
- a switch creates an entire broadcast domain (provided that there’s only one VLAN) since broadcasts are a layer 2 concept (mac address related)
- routers don’t forward layer 2 broadcasts, hence they separate broadcast domains

Categories: Network, Networking Security Tags:

debugging nfs problem with snoop in solaris

December 3rd, 2013 No comments

Network analyzers are ultimately the most useful tools available when it comes to debugging NFS problems. The snoop network analyzer bundled with Solaris was introduced in Section 13.5. This section presents an example of how to use snoop to resolve NFS-related problems.

Consider the case where the NFS client rome attempts to access the contents of the filesystems exported by the server zeus through the /net automounter path:

rome% ls -la /net/zeus/export
total 5
dr-xr-xr-x   3 root     root           3 Jul 31 22:51 .
dr-xr-xr-x   2 root     root           2 Jul 31 22:40 ..
drwxr-xr-x   3 root     other        512 Jul 28 16:48 eng
dr-xr-xr-x   1 root     root           1 Jul 31 22:51 home
rome% ls /net/zeus/export/home
/net/zeus/export/home: Permission denied


The client is not able to open the contents of the directory /net/zeus/export/home, although the directory gives read and execute permissions to all users:

Code View: Scroll / Show All
rome% df -k /net/zeus/export/home
filesystem            kbytes    used   avail capacity  Mounted on
-hosts                     0       0       0     0%    /net/zeus/export/home


The df command shows the -hosts automap mounted on the path of interest. This means that the NFS filesystem rome:/export/home has not yet been mounted. To investigate the problem further, snoopis invoked while the problematic ls command is rerun:

Code View: Scroll / Show All
 rome# snoop -i /tmp/snoop.cap rome zeus
  1   0.00000      rome -> zeus      PORTMAP C GETPORT prog=100003 (NFS) vers=3 
  2   0.00314      zeus -> rome      PORTMAP R GETPORT port=2049
  3   0.00019      rome -> zeus      NFS C NULL3
  4   0.00110      zeus -> rome      NFS R NULL3 
  5   0.00124      rome -> zeus      PORTMAP C GETPORT prog=100005 (MOUNT) vers=1 
  6   0.00283      zeus -> rome      PORTMAP R GETPORT port=33168
  7   0.00094      rome -> zeus      TCP D=33168 S=49659 Syn Seq=1331963017 Len=0 
Win=24820 Options=<nop,nop,sackOK,mss 1460>
  8   0.00142      zeus -> rome      TCP D=49659 S=33168 Syn Ack=1331963018 
Seq=4025012052 Len=0 Win=24820 Options=<nop,nop,sackOK,mss 1460>
  9   0.00003      rome -> zeus      TCP D=33168 S=49659     Ack=4025012053 
Seq=1331963018 Len=0 Win=24820
 10   0.00024      rome -> zeus      MOUNT1 C Get export list
 11   0.00073      zeus -> rome      TCP D=49659 S=33168     Ack=1331963062 
Seq=4025012053 Len=0 Win=24776
 12   0.00602      zeus -> rome      MOUNT1 R Get export list 2 entries
 13   0.00003      rome -> zeus      TCP D=33168 S=49659     Ack=4025012173 
Seq=1331963062 Len=0 Win=24820
 14   0.00026      rome -> zeus      TCP D=33168 S=49659 Fin Ack=4025012173 
Seq=1331963062 Len=0 Win=24820
 15   0.00065      zeus -> rome      TCP D=49659 S=33168     Ack=1331963063 
Seq=4025012173 Len=0 Win=24820
 16   0.00079      zeus -> rome      TCP D=49659 S=33168 Fin Ack=1331963063 
Seq=4025012173 Len=0 Win=24820
 17   0.00004      rome -> zeus      TCP D=33168 S=49659     Ack=4025012174 
Seq=1331963063 Len=0 Win=24820
 18   0.00058      rome -> zeus      PORTMAP C GETPORT prog=100005 (MOUNT) vers=3 
 19   0.00412      zeus -> rome      PORTMAP R GETPORT port=34582
 20   0.00018      rome -> zeus      MOUNT3 C Null
 21   0.00134      zeus -> rome      MOUNT3 R Null 
 22   0.00056      rome -> zeus      MOUNT3 C Mount /export/home
 23   0.23112      zeus -> rome      MOUNT3 R Mount Permission denied


Packet 1 shows the client rome requesting the port number of the NFS service (RPC program number 100003, Version 3, over the UDP protocol) from the server’s rpcbind (portmapper). Packet 2 shows the server’s reply indicating nfsd is running on port 2049. Packet 3 shows the automounter’s call to the server’s nfsd daemon to verify that it is indeed running. The server’s successful reply is shown in packet 4. Packet 5 shows the client’s request for the port number for RPC program number 100005, Version 1, over TCP (the RPC MOUNT program). The server replies with packet 6 with port=33168. Packets 7 through 9 are TCP hand shaking between our NFS client and the server’s mountd. Packet 10 shows the client’s call to the server’s mountd daemon (which implements the MOUNT program) currently running on port 33168. The client is requesting the list of exported entries. The server replies with packet 12 including the names of the two entries exported. Packets 18 and 19 are similar to packets 5 and 6, except that this time the client is asking for the port number of the MOUNT program version 3 running over UDP. Packet 20 and 21 show the client verifying that version 3 of the MOUNT service is up and running on the server. Finally, the client issues the Mount /export/home request to the server in packet 22, requesting the filehandle of the /export/home path. The server’s mountd daemon checks its export list, and determines that the host rome is not present in it and replies to the client with a “Permission Denied” error in packet 23.

The analysis indicates that the “Permission Denied” error returned to the ls command came from the MOUNT request made to the server, not from problems with directory mode bits on the client. Having gathered this information, we study the exported list on the server and quickly notice that the filesystem /export/home is exported only to the host verona:

rome$ showmount -e zeus
export list for zeus:
/export/eng  (everyone)
/export/home verona


We could have obtained the same information by inspecting the contents of packet 12, which contains the export list requested during the transaction:

Code View: Scroll / Show All
rome# snoop -i /tmp/cap -v -p 10,12
      Packet 10 arrived at 3:32:47.73
RPC:  ----- SUN RPC Header -----
RPC:  Record Mark: last fragment, length = 40
RPC:  Transaction id = 965581102
RPC:  Type = 0 (Call)
RPC:  RPC version = 2
RPC:  Program = 100005 (MOUNT), version = 1, procedure = 5
RPC:  Credentials: Flavor = 0 (None), len = 0 bytes
RPC:  Verifier   : Flavor = 0 (None), len = 0 bytes
MOUNT:----- NFS MOUNT -----
MOUNT:Proc = 5 (Return export list)
       Packet 12 arrived at 3:32:47.74
RPC:  ----- SUN RPC Header -----
RPC:  Record Mark: last fragment, length = 92
RPC:  Transaction id = 965581102
RPC:  Type = 1 (Reply)
RPC:  This is a reply to frame 10
RPC:  Status = 0 (Accepted)
RPC:  Verifier   : Flavor = 0 (None), len = 0 bytes
RPC:  Accept status = 0 (Success)
MOUNT:----- NFS MOUNT -----
MOUNT:Proc = 5 (Return export list)
MOUNT:Directory = /export/eng
MOUNT:Directory = /export/home
MOUNT: Group = verona


For simplicity, only the RPC and NFS Mount portions of the packets are shown. Packet 10 is the request for the export list, packet 12 is the reply. Notice that every RPC packet contains the transaction ID (XID), the message type (call or reply), the status of the call, and the credentials. Notice that the RPC header includes the string “This is a reply to frame 10″. This is not part of the network packet. Snoopkeeps track of the XIDs it has processed and attempts to match calls with replies and retransmissions. This feature comes in very handy during debugging. The Mount portion of packet 12 shows the list of directories exported and the group of hosts to which they are exported. In this case, we can see that /export/home was only exported with access rights to the host verona. The problem can be fixed by adding the host rome to the export list on the server.


explain solaris snoop network analyzer with example

December 2nd, 2013 No comments

Here’s the code:

# snoop -i /tmp/capture -v -p 3
ETHER:  ----- Ether Header -----
ETHER:  Packet 3 arrived at 15:08:43.35
ETHER:  Packet size = 82 bytes
ETHER:  Destination = 0:0:c:7:ac:56, Cisco
ETHER:  Source      = 8:0:20:b9:2b:f6, Sun
ETHER:  Ethertype = 0800 (IP)
IP:   ----- IP Header -----
IP:   Version = 4
IP:   Header length = 20 bytes
IP:   Type of service = 0x00
IP:         xxx. .... = 0 (precedence)
IP:         ...0 .... = normal delay
IP:         .... 0... = normal throughput
IP:         .... .0.. = normal reliability
IP:   Total length = 68 bytes
IP:   Identification = 35462
IP:   Flags = 0x4
IP:         .1.. .... = do not fragment
IP:         ..0. .... = last fragment
IP:   Fragment offset = 0 bytes
IP:   Time to live = 255 seconds/hops
IP:   Protocol = 17 (UDP)
IP:   Header checksum = 4503
IP:   Source address =, caramba
IP:   Destination address =, mickey
IP:   No options
UDP:  ----- UDP Header -----
UDP:  Source port = 55559
UDP:  Destination port = 2049 (Sun RPC)
UDP:  Length = 48 
UDP:  Checksum = 3685 
RPC:  ----- SUN RPC Header -----
RPC:  Transaction id = 969440111
RPC:  Type = 0 (Call)
RPC:  RPC version = 2
RPC:  Program = 100003 (NFS), version = 3, procedure = 0
RPC:  Credentials: Flavor = 0 (None), len = 0 bytes
RPC:  Verifier   : Flavor = 0 (None), len = 0 bytes
NFS:  ----- Sun NFS -----
NFS:  Proc = 0 (Null procedure)

And let’s analyze this:

The Ethernet header displays the source and destination addresses as well as the type of information embedded in the packet. The IP layer displays the IP version number, flags, options, and address of the sender and recipient of the packet. The UDP header displays the source and destination ports, along with the length and checksum of the UDP portion of the packet. Embedded in the UDP frame is the RPC data. Every RPC packet has a transaction ID used by the sender to identify replies to its requests, and by the server to identify duplicate calls. The previous example shows a request from the host caramba to the server mickey. The RPC version = 2 refers to the version of the RPC protocol itself, the program number 100003 and Version 3 apply to the NFS service. NFS procedure 0 is always the NULL procedure, and is most commonly invoked with no authentication information. The NFS NULL procedure does not take any arguments, therefore none are listed in the NFS portion of the packet.


  1. Here’s more usage about snoop on solaris:

The amount of traffic on a busy network can be overwhelming, containing many irrelevant packets to the problem at hand. The use of filters reduces the amount of noise captured and displayed, allowing you to focus on relevant data. A filter can be applied at the time the data is captured, or at the time the data is displayed. Applying the filter at capture time reduces the amount of data that needs to be stored and processed during display. Applying the filter at display time allows you to further refine the previously captured information. You will find yourself applying different display filters to the same data set as you narrow the problem down, and isolate the network packets of interest.

Snoop uses the same syntax for capture and display filters. For example, the host filter instructs snoop to only capture packets with source or destination address matching the specified host:

Code View: Scroll / Show All
# snoop host caramba
Using device /dev/hme (promiscuous mode)
     caramba -> schooner     NFS C GETATTR3 FH=B083
    schooner -> caramba      NFS R GETATTR3 OK
     caramba -> schooner     TCP D=2049 S=1023     Ack=3647506101 Seq=2611574902 Len=0 Win=24820


In this example the host filter instructs snoop to capture packets originating at or addressed to the host caramba. You can specify the IP address or the hostname, and snoop will use the name service switch to do the conversion. Snoop assumes that the hostname specified is an IPv4 address. You can specify an IPv6 address by using the inet6 qualifier in front of the host filter:

Code View: Scroll / Show All
# snoop inet6 host caramba
Using device /dev/hme (promiscuous mode)
     caramba -> 2100::56:a00:20ff:fea0:3390    ICMPv6 Neighbor advertisement
2100::56:a00:20ff:fea0:3390 -> caramba         ICMPv6 Echo request (ID: 1294 Sequence number: 0)
     caramba -> 2100::56:a00:20ff:fea0:3390    ICMPv6 Echo reply (ID: 1294 Sequence number: 0)


You can restrict capture of traffic addressed to the specified host by using the to or dst qualifier in front of the host filter:

# snoop to host caramba
Using device /dev/hme (promiscuous mode)
    schooner -> caramba      RPC R XID=1493500696 Success
    schooner -> caramba      RPC R XID=1493500697 Success
    schooner -> caramba      RPC R XID=1493500698 Success


Similarly you can restrict captured traffic to only packets originating from the specified host by using the from or src qualifier:

Code View: Scroll / Show All
# snoop from host caramba
Using device /dev/hme (promiscuous mode)
     caramba -> schooner     NFS C GETATTR3 FH=B083
     caramba -> schooner     TCP D=2049 S=1023     Ack=3647527137 Seq=2611841034 Len=0 Win=24820


Note that the host keyword is not required when the specified hostname does not conflict with the name of another snoop primitive.The previous snoop from host caramba command could have been invoked without the host keyword and it would have generated the same output:

Code View: Scroll / Show All
					# snoop from caramba 
Using device /dev/hme (promiscuous mode)
     caramba -> schooner     NFS C GETATTR3 FH=B083
     caramba -> schooner     TCP D=2049 S=1023     Ack=3647527137 Seq=2611841034 Len=0 Win=24820


For clarity, we use the host keyword throughout this book. Two or more filters can be combined by using the logical operators and and or :

# snoop -o /tmp/capture -c 20 from host caramba and rpc nfs 3
Using device /dev/hme (promiscuous mode)
20 20 packets captured


Snoop captures all NFS Version 3 packets originating at the host caramba. Here, snoop is invoked with the -c and -o options to save 20 filtered packets into the /tmp/capture file. We can later apply other filters during display time to further analyze the captured information. For example, you may want to narrow the previous search even further by only listing TCP traffic by using the proto filter:

# snoop -i /tmp/capture proto tcp
Using device /dev/hme (promiscuous mode)
  1   0.00000     caramba -> schooner    NFS C GETATTR3 FH=B083
  2   2.91969     caramba -> schooner    NFS C GETATTR3 FH=0CAE
  9   0.37944     caramba -> rea         NFS C FSINFO3 FH=0156
 10   0.00430     caramba -> rea         NFS C GETATTR3 FH=0156
 11   0.00365     caramba -> rea         NFS C ACCESS3 FH=0156 (lookup)
 14   0.00256     caramba -> rea         NFS C LOOKUP3 FH=F244
 15   0.00411     caramba -> rea         NFS C ACCESS3 FH=772D (lookup)


Snoop reads the previously filtered data from /tmp/capture, and applies the new filter to only display TCP traffic. The resulting output is NFS traffic originating at the host caramba over the TCP protocol. We can apply a UDP filter to the same NFS traffic in the /tmp/capture file and obtain the NFS Version 3 traffic over UDP from host caramba without affecting the information in the /tmp/capture file:

# snoop -i /tmp/capture proto udp
Using device /dev/hme (promiscuous mode)
  1   0.00000      caramba -> rea          NFS C NULL3


So far, we’ve presented filters that let you specify the information you are interested in. Use the not operator to specify the criteria of packets that you wish to have excluded during capture. For example, you can use the not operator to capture all network traffic, except that generated by the remote shell:

Code View: Scroll / Show All
# snoop not port login
Using device /dev/hme (promiscuous mode)
      rt-086 -> BROADCAST        RIP R (25 destinations)
      rt-086 -> BROADCAST        RIP R (10 destinations)
     caramba -> schooner         NFS C GETATTR3 FH=B083
    schooner -> caramba          NFS R GETATTR3 OK
     caramba -> donald           NFS C GETATTR3 FH=00BD
    jamboree -> donald           NFS R GETATTR3 OK
     caramba -> donald           TCP D=2049 S=657     Ack=3855205229 Seq=2331839250 Len=0 Win=24820
     caramba -> schooner         TCP D=2049 S=1023    Ack=3647569565 Seq=2612134974 Len=0 Win=24820
     narwhal ->    UDP D=9875 S=32825 LEN=368


On multihomed hosts (systems with more than one network interface device), use the -d option to specify the particular network interface to snoop on:

snoop -d hme2


You can snoop on multiple network interfaces concurrently by invoking separate instances of snoop on each device. This is particularly useful when you don’t know what interface the host will use to generate or receive the requests. The -d option can be used in conjunction with any of the other options and filters previously described:

# snoop -o /tmp/capture-hme0 -d hme0 not port login &
# snoop -o /tmp/capture-hme1 -d hme1 not port login &

2.This article is from book <Managing NFS and NIS, Second Edition>

rpc remote procedure call mechanism

December 2nd, 2013 No comments

The rpcbind daemon (also known as the portmapper),[8] exists to register RPC services and to provide their IP port numbers when given an RPC program number. rpcbind itself is an RPC service, but it resides at a well-known IP port (port 111) so that it may be contacted directly by remote hosts. For example, if host fred needs to mount a filesystem from host barney, it must send an RPC request to themountd daemon on barney. The mechanics of making the RPC request are as follows:

[8] The rpcbind daemon and the old portmapper provide the same RPC service. The portmapper implements Version 2 of the portmap protocol (RPC program number 100000), where the rpcbind daemon implements Versions 3 and 4 of the protocol, in addition to Version 2. This means that the rpcbind daemon already implements the functionality provided by the old portmapper. Due to this overlap in functionality and to add to the confusion, many people refer to the rpcbind daemon as the portmapper.

  • fred gets the IP address for barney, using the ipnodes NIS map. fred also looks up the RPC program number for mountd in the rpc NIS map. The RPC program number for mountd is 100005.
  • Knowing that the portmapper lives at port 111, fred sends an RPC request to the portmapper on barney, asking for the IP port (on barney) of RPC program 100005. fred also specifies the particular protocol and version number for the RPC service. barney ‘s portmapper responds to the request with port 704, the IP port at which mountd is listening for incoming mount RPC requests over the specified protocol. Note that it is possible for the portmapper to return an error, if the specified program does not exist or if it hasn’t been registered on the remote host. barney, for example, might not be an NFS server and would therefore have no reason to run the mountd daemon.
  • fred sends a mount RPC request to barney, using the IP port number returned by the portmapper. This RPC request contains an RPC procedure number, which tells the mountd daemon what to do with the request. The RPC request also contains the parameters for the procedure, in this case, the name of the filesystem fred needs to mount.

Note: this is from book <Managing NFS and NIS, Second Edition>

Categories: Kernel, Linux, Network Tags:

resolved – mount clntudp_create: RPC: Program not registered

December 2nd, 2013 No comments

When I did a showmount -e localhost, error occured:

[root@centos-doxer ~]# showmount -e localhost
mount clntudp_create: RPC: Program not registered

So I checked what RPC program number of showmount was using:

[root@centos-doxer ~]# grep showmount /etc/rpc
mountd 100005 mount showmount

As this indicated, we should startup mountd daemon to make showmount -e localhost work. And mountd is part of nfs, so I started up nfs:

[root@centos-doxer ~]# /etc/init.d/nfs start
Starting NFS services: [ OK ]
Starting NFS quotas: [ OK ]
Starting NFS daemon: [ OK ]
Starting NFS mountd: [ OK ]

Now as mountd was running, showmount -e localhost should work.


Categories: Kernel, Linux, Network Tags:

VLAN in windows hyper-v

November 26th, 2013 No comments

Briefly, a virtual LAN (VLAN) can be regarded as a broadcast domain. It operates on the OSI
network layer 2. The exact protocol definition is known as 802.1Q. Each network packet belong-
ing to a VLAN has an identifier. This is just a number between 0 and 4095, with both 0 and 4095
reserved for other uses. Let’s assume a VLAN with an identifier of 10. A NIC configured with
the VLAN ID of 10 will pick up network packets with the same ID and will ignore all other IDs.
The point of VLANs is that switches and routers enabled for 802.1Q can present VLANs to dif-
ferent switch ports in the network. In other words, where a normal IP subnet is limited to a set
of ports on a physical switch, a subnet defined in a VLAN can be present on any switch port—if
so configured, of course.

Getting back to the VLAN functionality in Hyper-V: both virtual switches and virtual NICs
can detect and use VLAN IDs. Both can accept and reject network packets based on VLAN ID,
which means that the VM does not have to do it itself. The use of VLAN enables Hyper-V to
participate in more advanced network designs. One limitation in the current implementation is
that a virtual switch can have just one VLAN ID, although that should not matter too much in
practice. The default setting is to accept all VLAN IDs.

resolved – sshd: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost= user=

November 20th, 2013 No comments

Today when I tried to log on one linux server with a normal account, errors were found in /var/log/secure:

Nov 20 07:43:39 test_linux sshd[11200]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost= user=testuser
Nov 20 07:43:39 test_linux sshd[11200]: pam_ldap: error trying to bind (Invalid credentials)
Nov 20 07:43:42 test_linux sshd[11200]: nss_ldap: failed to bind to LDAP server ldaps:// Invalid credentials
Nov 20 07:43:42 test_linux sshd[11200]: nss_ldap: failed to bind to LDAP server ldap:// Invalid credentials
Nov 20 07:43:42 test_linux sshd[11200]: nss_ldap: could not search LDAP server – Server is unavailable
Nov 20 07:43:42 test_linux sshd[11200]: nss_ldap: failed to bind to LDAP server ldaps:// Invalid credentials
Nov 20 07:43:43 test_linux sshd[11200]: nss_ldap: failed to bind to LDAP server ldap:// Invalid credentials
Nov 20 07:43:43 test_linux sshd[11200]: nss_ldap: could not search LDAP server – Server is unavailable
Nov 20 07:43:55 test_linux sshd[11200]: pam_ldap: error trying to bind (Invalid credentials)
Nov 20 07:43:55 test_linux sshd[11200]: Failed password for testuser from port 34243 ssh2
Nov 20 07:43:55 test_linux sshd[11201]: fatal: Access denied for user testuser by PAM account configuration

After some attempts on linux PAM(sshd, system-auth), I still got nothing. Later, I checked /etc/ldap.conf with one other box, and found the configuration on the problematic host was not right.

I copied the right ldap.conf and tried log on later, and the issue resolved.


You can read more about linux PAM here (I recommend having a reading on the System Administrators’ Guide as that may be the only one linux administrators can reach. You can also get a detailed info on some commonly used PAM modules such as,, pam_cracklib, etc.)

Categories: Linux, Security, Systems Tags:

Enabling NIS on client hosts

November 19th, 2013 No comments

Once you have one or more NIS servers running ypserv, you can set up NIS clients that query them. Make sure you do not enable NIS on any clients until you have at least one NIS server up and running. If no servers are available, the host that attempts to run as an NIS client will hang.

To enable NIS on a client host, first set up the nsswitch.conf file:

newclient# cp /etc/nsswitch.nis /etc/nsswitch.conf


Set up the domain name:

newclient# domainname bedrock
newclient# domainname > /etc/defaultdomain


Run ypinit:

newclient# /usr/sbin/ypinit -c


You will be prompted for a list of NIS servers. Enter the servers in order of proximity to the client.

Kill (if necessary) ypbind, and restart it:

newclient# ps -ef | grep ypbind
newclient# /usr/lib/netsvc/yp/ypstop
newclient# /usr/lib/netsvc/yp/ypstart


Once NIS is running, references to the basic administrative files are handled in two fundamentally different ways, depending on how nsswitch.conf is configured:

  • The NIS database replaces some files. Local copies of replaced files (ethers, hosts, netmasks, netgroups,[3] networks, protocols, rpc, and services) are ignored as soon as the ypbind daemon is started (to enable NIS).

    [3] The netgroups file is a special case. Netgroups are only meaningful when NIS is running, in which case the netgroups map (rather than the file) is consulted. The netgroups file is therefore only used to build the netgroups map; it is never “consulted” in its own right.

  • Some files are augmented, or appended to, by NIS. Files that are appended, or augmented, by NIS are consulted before the NIS maps are queried. The default/etc/nsswitch.conf file for NIS has these appended files: aliases, auto_*, group, passwd, services, and shadow. These files are read first, and if an appropriate entry isn’t found in the local file, the corresponding NIS map is consulted. For example, when a user logs in, an NIS client will first look up the user’s login name in the localpasswd file; if it does not find anything that matches, it will refer to the NIS passwd map.

Although the replaced files aren’t consulted once NIS is running, they shouldn’t be deleted. In particular, the /etc/hosts file is used by an NIS client during the boot process, before it starts NIS, but is ignored as soon as NIS is running. The NIS client needs a “runt” hosts file during the boot process so that it can configure itself and get NIS running. Administrators usually truncate hosts to the absolute minimum: entries for the host itself and the “loopback” address. Diskless nodes need additional entries for the node’s boot server and the server for the diskless node’s /usr filesystem. Trimming the hosts file to these minimal entries is a good idea because, for historical reasons, many systems have extremely long host tables. Other files, like rpc, services, and protocols, could probably be eliminated, but it’s safest to leave the files distributed with your system untouched; these will certainly have enough information to get your system booted safely, particularly if NIS stops running for some reason. However, you should make any local additions to these files on the master server alone. You don’t need to bother keeping the slaves and clients up to date.


This is from book <Managing NFS and NIS, Second Edition>

Categories: IT Architecture, Network, Systems Tags:

make sudo asking for no password on linux

November 1st, 2013 No comments

Assuming that you have a user named ‘test’, and he belongs to ‘admin’ group. So you want user test can sudo to root, and don’t want linux prompting for password. Here’s the way you can do it:

cp /etc/sudoers{,.bak}
sed -i ‘/%admin/ s/^/# /’ /etc/sudoers
echo ‘%admin ALL=(ALL) NOPASSWD: ALL’ >> /etc/sudoers


Categories: Linux, Security Tags:

disable linux strong password policy

November 1st, 2013 No comments

You may enable strong password policy for linux, and can disable it of course. So here’s the way if you want to disable it:

cp /etc/pam.d/system-auth{,.bak}
sed -i ‘/ s/^/# /’ /etc/pam.d/system-auth
sed -i ‘s/use_authtok//’ /etc/pam.d/system-auth
echo “password” | passwd –stdin username


  1. To enable strong password for linux, you can have a try on this
  2. You can read more about linux pam here
Categories: Linux, Security Tags:

F5 big-ip LTM iRULE to redirect http requests to https

October 25th, 2013 No comments

Here’s the irule script:

HTTP::redirect “https://[HTTP::host][HTTP::uri]”


1.You can read more about F5 LTM docs here <select a version of big ip software from the left side first>

2.Here’s one diagram shows a logical configuration example of the F5 solution for Oracle Database, Applications, Middleware, Servers and Storage:


Categories: Network, Networking Security Tags:

tcp flags explanation in details – SYN ACK FIN RST URG PSH and iptables for sync flood

October 11th, 2013 No comments

This is from wikipedia:

To establish a connection, TCP uses a three-way handshake. Before a client attempts to connect with a server, the server must first bind to and listen at a port to open it up for connections: this is called a passive open. Once the passive open is established, a client may initiate an active open. To establish a connection, the three-way (or 3-step) handshake occurs:

  1. SYN: The active open is performed by the client sending a SYN to the server. The client sets the segment’s sequence number to a random value A.
  2. SYN-ACK: In response, the server replies with a SYN-ACK. The acknowledgment number is set to one more than the received sequence number i.e. A+1, and the sequence number that the server chooses for the packet is another random number, B.
  3. ACK: Finally, the client sends an ACK back to the server. The sequence number is set to the received acknowledgement value i.e. A+1, and the acknowledgement number is set to one more than the received sequence number i.e. B+1.

At this point, both the client and server have received an acknowledgment of the connection. The steps 1, 2 establish the connection parameter (sequence number) for one direction and it is acknowledged. The steps 2, 3 establish the connection parameter (sequence number) for the other direction and it is acknowledged. With these, a full-duplex communication is established.

You can read pdf document here

H3C’s implementations of sync flood solution

Using iptables to resolve sync flood issue and

You may also consider using tcpkill to kill half open sessions(using ss -s/netstat -s<SYN_RECV>/tcptrack to see connection summary)

Output from netstat -atun:

The reason for waiting is that packets may arrive out of order or be retransmitted after the connection has been closed. CLOSE_WAIT indicates that the other side of the connection has closed the connection. TIME_WAIT indicates that this side has closed the connection. The connection is being kept around so that any delayed packets can be matched to the connection and handled appropriately.

more on about FIN_wait (one error: 2MSL<Maximum Segment Lifetime>=120s, not 2ms)

All about tcp socket states:

And here’s more about tcp connection(internet socket) states:

iptables tips

October 10th, 2013 No comments
Network packet flow and hook points for NAT

Network packet flow and hook points for NAT

network package flow and hook points for filter

network package flow and hook points for filter

network package flow and hook points for mangling

network package flow and hook points for mangling


hook points details

hook points details


packet flows

packet flows





1.Some of the contents are from book <Linux iptables Pocket Reference>

2.You can read more about iptables in man page of iptables here

Categories: Network, Networking Security, Security, tips Tags:

cisco network tips

August 13th, 2013 No comments

DTE, DCE #Data Terminal Equipment<accept time>,Data Communication Equipment<supply time>;DCE must be specified when adding serial connection.  int se 0;bandwidth 64;clock rate 64000;no shut;end;copy run start
Router>enable #disable
Router#conf t
Router(config)#host R1
R1(config)#ena se c1 #enable secret c1/enable password c1
R1(config)#line vty 0 4 #virtual ports for remote login, this enables 5 sessions
R1(config-line)#pass c2 #config vty password
R1(config-line)#login #enable telnet. login local/username hello password test<login local will need user/pass when telnet>
R1(config)#line console 0 #will need password when connecting to console
R1(config-line)#password boson
R1(config-line)#int eth 0 #int fa0/0
R1(config-if)#ip add #ip address
R1(config-if)#no shut
R1(config-if)#int se 0 #serial port
R1(config-if)#description this is the dte side
R1(config-if)#ip add
R1(config-if)#no shut
R1(config-if)#end #end config, go into enable mode. exit will go to config
R1#copy run start #copy run tftp; copy tftp run; copy tftp start
R1(config)#ip route #static routing, traffic destinated for will go to router with ip of, show ip route
R2(config)#no ip route #remove static routing
R2(config)#ip route #default route
R2(config)#show controllers se 0#check some hardware info, e.g. DTE, promiscuous
R1(config)#int lo0
R1(config-if)#ip add #loop interface
Router#show ip route
ROUTER#show running-config #erase startup-config; reload (from scratch)
show startup-config #stores in NVRAM
show version #NIC info;configuration register
show protocols #layer 3
show flash #image file
show history #10个, ctrl+p/ctrl+n/up down arrow
show clock
show hosts #ip host test
show users #all users connected to router
show interfaces
show interface serial 0 #or show interfaces serial 0. BW means bandwidth
HDLC #High speed Data Link Control, the default encapsulation for serial interfaces on Cisco routers
banner motd z

##PC config
ipconfig /ip
ipconfig /dg
##switch trunk/vtp/vlan
vlan 22 name pcs
int e0/1 #PC1
vlan-membership static 22
int e0/2 #PC2
vlan-membership static 22
show vlan
show vlan-membership
SW3#vlan database
SW3(vlan)#vlan 8 #vlan 8 name xxx
SW3(vlan)#vlan 14
SW3(config)#int range fa0/2 – 5 #space is needed
SW3(config-range)#switchport mode access #force a switch port to always behave as an access port, no trunk automatically
SW3(config-range)#switchport access vlan 8
show vlan #there’s no show vlan-membership command
vtp #VLAN Trunking Protocol, trunk is for delivering vlan info
SW3#vlan database
SW3(vlan)#vtp server
SW3(vlan)#vtp domain Boson
SW3(vlan)#vtp password rules
SW4#vlan database
SW4(vlan)#vtp client
SW4(vlan)#vtp domain Boson
SW4(vlan)#vtp password rules
#create the trunk line that will transport the vlan configurations from SW3 to SW4. To accomplish this, enable trunking on the port that links between the two switches.
SW3(config)#int fa 0/12
SW3(config-if)#switchport mode trunk #or trunk on. switchport mode dynamic desirable(ask for); switchport mode dynamic auto(it’s up to you)
SW4(config)#int fa 0/12
SW4(config-if)#switchport mode trunk #or trunk on
show interface fa0/12 switchport
show vlan brief
show vlan
show vtp
show vtp status
show trunk a #a means fa0/26, b means fa0/27
##router ppp/chap bi-directional authentication
username R2 password samepwd2 #on R1
int se 0
encapsulation ppp
ppp authentication chap
ppp chap hostname R1 #can also set user to peer hostname, and same password. then  no need for ‘ppp chap hostname/password’
ppp chap password samepwd1
no shut
username R1 password samepwd1 #on R2
int se 0
encapsulation ppp
ppp authentication chap #no need for one-way authentication
ppp chap hostname R2
ppp chap password samepwd2
no shut

show ip interface brief #R1
debug ppp negotiation
undebug all #stop all debugging


##Frame Relay(optical fibre, low error; no content control, no error correction<can detect errors and drop them>;better for data transfer(Frame Relay does not provide an ideal path for voice or video transmission, both of which require a steady flow of transmissions)
(each virtual circuit uses DLCI<Data Link Connection Identifier>;pvc,svc<permanent/switching virtual circuit>)
(local management interface LMI, for connectivity confirmation:LMI virtual circuit status messages provide communication and synchronization between Frame Relay DTE and DCE devices. These messages are used to periodically report on the status of PVCs, which prevents data from being sent into black holes)

(Frame Relay switches create virtual circuits to connect remote LANs to a WAN)

R1#int se 0
ip add
encapsulation frame-relay
frame-relay interface-dlci 102

R2#int se 0
ip add
encapsulation frame-relay #establish a connection from R1 through the Frame Relay switch to R2
frame-relay interface-dlci 201 #show frame-relay pvc to show available first

show frame-relay lmi #display LMI traffic that has been exchanged between the router and the Frame Relay switch
show frame-relay traffic #global Frame Relay statistics since the last reload of the router
show frame-relay map #mappings of Layer 2 DLCI to Layer 3 IP addresses
show frame-relay pvc #all of the PVC mappings for the router

#frame-relay lmi-type ansi

##Hub-and-Spoke topology
#int se 0.100 point-to-point #sub-interface
#frame-relay interface-dlci 102
#ip add xxx
no shut
#config other router then
##CDP(cisco discovery protocol, layer 2)
show cdp
show cdp interface #all interfaces that are running cdp
show cdp neighbor detail
show cdp entry *
show cdp entry R1 #case sensitive
cdp timer 45 #send cdp updates package every 45 seconds
cdp holdtime 60 #keep 60s
no cdp run #for specific interface -> int eth 0; no cdp enable
##ARP(address resolution protocol)
show arp
clear arp #ping, then show arp will show again
##RIP(routing information protocol)
network convergence #all routers have an identical view of the internetwork topology
R1(config)#router rip #(classful, A, B, C,ignore subnet mask), does not include the subnet mask in its routing table updates
R1(config-router)#network #ping on R1 first. Later all can ping
R1(config)#router rip #no router rip
R1(config-router)#version 2#classless, include the subnet mask in its routing table updates
R1#clear ip route * #clear and rebuild ip routing table
R1#debug ip route #show RIP updates(every 30 seconds)
R1#undebug all
show ip route #administrative distance: if two routing protocols have the same route, the router will pick the route with the lower administrative distance
show ip protocols
##IGRP(Interior Gateway Routing Protocol)
router IGRP 100 #(classful), does not include the subnet mask in its routing table updates. 100 is autonomous system number
R1#debug ip igrp events #trace IGRP routing updates without showing individual network numbers
R1#debug ip igrp transactions #show routing updates with individual network numbers that are being advertised or received
R1#undebug all

router eigrp 100
show ip eigrp neighbors
show ip eigrp traffic
show ip route

#OSPF(Open Shortest Path First: dynamic/multipath routing/load balancing)
R1(config)#router ospf 100 #100 is process ID number
R1(config-router)#network area 0 #add the network(s) to which R1 is directly connected
R1(config-router)#network area 0
R2(config)#router ospf 100 #100 is process ID number
R2(config-router)#network area 0 #add the network(s) to which R2 is directly connected
R4(config)#router ospf 100 #100 is process ID number
R4(config-router)#network area 0 #add the network(s) to which R4 is directly connected
Now R2 can ping R4
show ip route
show ip protocols
show ip ospf database
show ip ospf interface #all router interfaces that are running OSPF. Designated Router(DR)/Backup Designated Router(BDR)
show ip ospf neighbor
##access lists
access-list [#] [permit|deny] [source_address|any] [source_mask] # -><equals to 255 by adding them>; any means
access-list 1 permit any #standard IP access lists are in the range 1-99 or 1300-1999
int eth 0
ip access-group [access_list_number] [in|out]
show access-lists
access-list 101 permit tcp any eq telnet log # ->; allow only telnet traffic from the subnet off of R1′s s0 to come into R1; log keyword to display output to the router every time this line on the access list is invoked. extended with scope of 100-199 or 2000-2699
#access list 100 permit udp any any eq 520 #RIP uses UDP port 520
access-list 102 permit ip #allow any traffic from R1′s eth0 subnet to travel anywhere
telnet R1 #ctrl+shift+6, then press x; show sessions;resume 1;disconnect 1
ip access-list extended deny_ping
R1(config-ext-acl)#deny icmp host log
R1(config-ext-acl)#permit ip any any log
ip access-group deny_ping in #int se0; no ip access-group 101 in
access-list 100 permit ip log #source, destination
access-list 100 permit ip any #rip broadcast
access-list 102 deny ip host log #deny outbound access to one host
access-list 102 permit ip any any #int se 0; ip access-group 102 out
##ISDN(Integrated Services Digital Network)
R1(config)#isdn switch-type basic-ni #for all BRI interfaces on the router
interface bri 0
isdn spid1 32177820010100 #service profile identifier. number supplied by ISP to identify the line configuration of the BRI service.
show isdn status
int bri 0
dialer string 7782001
R1(config)#dialer-list 1 protocol ip permit
int bri 0
dialer-group 1
R2#show isdn status #layer 3, 1 active layer 3 call
SW1(config)#enable password/secret level 15 test. level 15 is default one, so level 15 can be omitted
SW1(config)#ip add
SW1(config)#ip default-gateway
show ip
show interfaces
show version
delete nvram #factory mode,no need on catalyst 1900 for ‘copy run start’, because it’ll auto save
show spantree #Switched Port Analyzer
show mac #show mac-address-table, all the ports on the switch with the MAC addresses it has learned
SW1(config)#mac-address-table permanent 1111-1111-1111 e0/5 #bind port statically <-> device
SW1(config)#int e0/9
SW1(config-if)#port secure
SW1(config-if)#port secure max-mac-count 1 #allow only one
R1(config)#ip nat inside source static #inside local -> inside global(registered ip)
int eth0 #is where connect to
ip nat inside
int se0 #going outside
ip nat outside
no shut
show ip nat translations

ip nat pool pool1 netmask #dynamic translation
ip nat inside source list 2 pool pool1
access-list 2 permit

R1(config)#ip nat inside source list 2 interface serial0 overload #PAT(port address translation), interface connect to R1 on R2 will have ip of R1′s serial 0

martian package

more about rp_filter/RPF(Reverse Path Forwarding) here (multicast)

Categories: Network, Networking Security, tips Tags:

device bond0/bond1 does not seem to be present delaying

August 1st, 2013 No comments

Today I encountered one problem when trying to start up network:

“device bond0 does not seem to be present delaying”

In my env, bond0 was bridged to v119_BR. And eth0 and eth1 that composed bond0 were not in any vlan in the switch side. Here’s some more info:

-bash-3.2# ifconfig v119_BR

v119_BR Link encap:Ethernet HWaddr 00:21:28:EE:C5:37
inet addr: Bcast: Mask:
inet6 addr: 2606:b400:2010:4051:221:28ff:feee:c537/64 Scope:Global
inet6 addr: fe80::221:28ff:feee:c537/64 Scope:Link
RX packets:3830 errors:0 dropped:0 overruns:0 frame:0
TX packets:334 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:292812 (285.9 KiB) TX bytes:65740 (64.1 KiB)

-bash-3.2# cat ifcfg-v119_BR

And I’ve bond0(eth0 and eth1) bridged to v119_BR:

-bash-3.2# cat ifcfg-bond0
BONDING_OPTS=”primary=eth0 miimon=100 mode=1″


As no multiple VLANs are defined on switch, so no alias interface needed for the bridge v119_BR. If multiple VLANs are defined on eth0 & eth1, then we have to define bond0.117(117 is vlan ID), which should be like the following:
[root@testhost ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond0.117 #and then ifcfg-bond0.118 and more if these vlans are defined on switch

[root@testhost network-scripts]# cat ifcfg-bond0
BONDING_OPTS=”primary=eth0 miimon=100 mode=1″

So what about the problem:

“device bond0/bond0 does not seem to be present delaying”

After some searching, I found that it’s caused by bond module not loaded. To resolve this, I tried:

-bash-3.2# uname -r

-bash-3.2# find . -name “*bond*”

insmod /lib/modules/2.6.18-

Add the following lines in /etc/modprobe.conf

alias bond1 bonding
alias bond0 bonding

Then reboot the host. After these steps, the issue was resolved.

Categories: Network, Networking Security Tags:

get linux server fingerprint through using ssh-keygen

July 31st, 2013 No comments

Run ssh-keygen -lf  /etc/ssh/ to get the server’s fingerprint which respond to:

$ ssh <test-host>
The authenticity of host ‘testhost[ip address]‘ can’t be established.
RSA key fingerprint is <xx:xx:xx….>
Are you sure you want to continue connecting (yes/no)? yes

Categories: Linux, Networking Security, Security Tags:

Best practices for NFS security and performance

July 18th, 2013 No comments

The recommendations here apply to general purpose Network File System (NFS) servers based on Linux or other Unix derivatives.  They do not apply to high-end filers from vendors such as NetApp, EMC, or others, who provide their own proprietary interfaces and settings.


The following example shows how to restrict access on a Linux-based NFS server with settings in /etc/exports. (How to set this configuration varies from manufacturer to manufacture of network appliances.)
Only the network has full access to the export; all other networks have only read access:
/test-export1  (rw,no_root_squash,sync,no_subtree_check) (ro,root_squash,sync,no_subtree_check)

Align partitions with blocks on physical or virtual disks


NFS mount options rsize and wsize not less than 1Mb

The NFS mount options rsize and wsize set the size of data transfers for reading and writing.
For best performance, do not set these sizes to less than 1Mb.


Ethernet jumbo frame size on NICs and switches

For best performance, jumbo frames should be enabled on the NFS server’s network interface cards (NICs), the
network switches or routers they are connected to. A jumbo frame is 9,000 bytes, which greatly reduces packet fragmentation.

You can use the ping -s 8192 command from the cluster to the NFS server to determine if jumbo frames
have been enabled. A successful response from the specified IP address means that jumbo frames are enabled.
For example:

ping -s 8192 IPaddressOrNameOfNfsServer

Identifying NFS retransmissions (this is from book <Managing NFS and NIS, Second Edition>)

Inspection of the load average and disk activity on the servers may indicate that the servers are heavily loaded and imposing the tightest constraint. The NFS client-side statistics provide the most concrete evidence that one or more slow servers are to blame:

% nfsstat -rc 
Client rpc:
calls       badcalls    badxids     timeouts    newcreds    badverfs    
1753584     1412        18          64          0           0           
timers      cantconn    nomem       interrupts  
0           1317        0           18          
calls       badcalls    retrans     badxids     timeouts    newcreds    
12443       41          334         80          166         0           
badverfs    timers      nomem       cantsend    
0           4321        0           206


The -rc option is given to nfsstat to look at the RPC statistics only, for client-side NFS operations. The call type demographics contained in the NFS-specific statistics are not of value in this analysis. The test for a slow server is having badxid and timeout of the same magnitude. In the previous example, badxid is nearly a third the value of timeout for connection-oriented RPC, and nearly half the value oftimeout for connectionless RPC. Connection-oriented transports use a higher timeout than connectionless transports, therefore the number of timeouts will generally be less for connection-oriented transports. The high badxid count implies that requests are reaching the various NFS servers, but the servers are too loaded to send replies before the local host’s RPC calls time out and are retransmitted.badxid is incremented each time a duplicate reply is received for a retransmitted request (an RPC request retains its XID through all retransmission cycles). In this case, the server is replying to all requests, including the retransmitted ones. The client is simply not patient enough to wait for replies from the slow server. If there is more than one NFS server, the client may be outpacing all of them or just one particularly sluggish node.

If the server has a duplicate request cache, retransmitted requests that match a non-idempotent NFS call currently in progress are ignored. Only those requests in progress are recognized and filtered, so it is still possible for a sufficiently loaded server to generate duplicate replies that show up in the badxid counts of its clients. Without a duplicate request cache, badxid and timeout may be nearly equal, while the cache will reduce the number of duplicate replies. With or without a duplicate request cache, if the badxid and timeout statistics reported by nfsstat (on the client) are of the same magnitude, then server performance is an issue deserving further investigation.

A mixture of network and server-related problems can make interpretation of the nfsstat figures difficult. A client served by four hosts may find that two of the hosts are particularly slow while a third is located across a network router that is digesting streams of large write packets. One slow server can be masked by other, faster servers: a retransmission rate of 10% (calculated as timeout/calls) would indicate short periods of server sluggishness or network congestion if the retransmissions were evenly distributed among all servers. However, if all timeouts occurred while talking to just one server, the retransmission rate for that server could be 50% or higher.

A simple method for finding the distribution of retransmitted requests is to perform the same set of disk operations on each server, measuring the incremental number of RPC timeouts that occur when loading each server in turn. This experiment may point to a server that is noticeably slower than its peers, if a large percentage of the RPC timeouts are attributed to that host. Alternatively, you may shift your focus away from server performance if timeouts are fairly evenly distributed or if no timeouts occur during the server loading experiment. Fluctuations in server performance may vary by the time of day, so that more timeouts occur during periods of peak server usage in the morning and after lunch, for example.

Server response time may be clamped at some minimum value due to fixed-cost delays of sending packets through routers, or due to static configurations that cannot be changed for political or historical reasons. If server response cannot be improved, then the clients of that server must adjust their mount parameters to avoid further loading it with retransmitted requests. The relative patience of the client is determined by the timeout, retransmission count, and hard-mount variables.



You can read more about NFS here

Categories: Clouding, Network, Security Tags:

iptables rules after creating NAT to eth0 in virt-manager

July 18th, 2013 No comments

During creating a new virtual network in virt-manager GUI, I specified the following parameters:




Forwarding to physical network(Destination: Physical device eth0, Mode NAT)

After this, the iptables will automatically created:

[root@centos-doxer ~]# iptables-save
:INPUT ACCEPT [19172:2049589]
:OUTPUT ACCEPT [16236:5446648]
-A INPUT -i virbr1 -p udp -m udp –dport 53 -j ACCEPT
-A INPUT -i virbr1 -p tcp -m tcp –dport 53 -j ACCEPT
-A INPUT -i virbr1 -p udp -m udp –dport 67 -j ACCEPT
-A INPUT -i virbr1 -p tcp -m tcp –dport 67 -j ACCEPT
-A FORWARD -d -i eth0 -o virbr1 -m state –state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -s -i virbr1 -o eth0 -j ACCEPT
-A FORWARD -i virbr1 -o virbr1 -j ACCEPT
-A FORWARD -o virbr1 -j REJECT –reject-with icmp-port-unreachable
-A FORWARD -i virbr1 -j REJECT –reject-with icmp-port-unreachable

:PREROUTING ACCEPT [16665:4067972]
:OUTPUT ACCEPT [197:13739]
-A POSTROUTING -s -d ! -o eth0 -p tcp -j MASQUERADE –to-ports 1024-65535
-A POSTROUTING -s -d ! -o eth0 -p udp -j MASQUERADE –to-ports 1024-65535
-A POSTROUTING -s -d ! -o eth0 -j MASQUERADE

And here’s the bridge info:

[root@centos-doxer ~]# brctl show
bridge name bridge id STP enabled interfaces
virbr0 8000.0800273fc508 no eth0
virbr1 8000.000000000000 yes #one vm will have one interface here, format will be like virbr1-nic

[root@centos-doxer ~]# ifconfig virbr1
virbr1 Link encap:Ethernet HWaddr 00:00:00:00:00:00
inet addr: Bcast: Mask:
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:3656 (3.5 KiB)

Categories: Network, Networking Security Tags:

NAT forwarding for ssh and vncviewer

July 11th, 2013 No comments

-A PREROUTING -p tcp –dport 5911 -d -j DNAT –to-destination #now ssh <ip of eth0,> -p 5911 is going to visit
-A PREROUTING -p tcp –dport 222 -d -j DNAT –to-destination #now ssh <ip of eth0,> -p 222 is going to visit
-A POSTROUTING -o eth0 -j MASQUERADE #if eth0 is private ip, you can also do a NAT with one public ip.

:INPUT ACCEPT [247:16364]
:FORWARD ACCEPT [163:13692]
:OUTPUT ACCEPT [228:18664]

Categories: Network, Networking Security Tags: ,

PPTP vpn configuration on linux

July 10th, 2013 No comments
  • Configure pptp on remote machine:

vi /etc/sysctl.conf

sysctl -p
yum install pptpd

vi /etc/pptpd.conf
option /etc/ppp/options.pptpd
ppp /usr/sbin/pppd
localip #configure it first as an alias eth0:1 for example
lcp-echo-interval 180
lcp-echo-failure 10

vi /etc/ppp/options.pptpd
name pptpd
idle 2592000
ms-dns #you can also use OpenDNS’s DNS such as
mtu 1416
mru 1416

vi /etc/ppp/chap-secrets
“username” pptpd “password” *

vi /etc/sysconfig/iptables
-A POSTROUTING -o eth0 -j MASQUERADE #eth0 is where public ip resides

:INPUT ACCEPT [247:16364]
-A INPUT -p tcp -m state –state NEW -m tcp –dport 1723 -j ACCEPT
-A INPUT -p gre -j ACCEPT #PPTP is a form of PPP between two hosts via GRE using encryption (MPPE) and compression (MPPC)
-A OUTPUT -p gre -j ACCEPT
:FORWARD ACCEPT [163:13692]
:OUTPUT ACCEPT [228:18664]

/etc/init.d/pptpd restart

  • Now add a new VPN connection in Windows.


  • Here’s four types of vpn tunneling protocols:
  1. L2TP(Data Link Layer, Layer 2 Tunneling Protocol) over IPSec(IPSec is used to secure L2TP packets by providing confidentiality, authentication and integrity): xl2tpd. IPsec in tunneling mode does not create virtual physical interfaces at the end of the tunnel, since the tunnel is handled directly by the TCP/IP stack. L2TP can be used to provide these interfaces, this technique is called L2TP/IPsec. However, IPSec has a weakness that prevents it from being used all the time—IPSec can’t travel through a Network Address Translation (NAT) server. If you need to go through a NAT server, you can use PPTP or SSTP.
  2. IPSec(Network Layer): openswan/strongswan
  3. PPTP(Session Layer): PoPTop(alias for pptpd)
  4. SSL based vpn(Session Layer): openvpn/openconnect/SSTP(windows)
  5. IKEv2(Internet Key Exchange Version 2): support VPN reconnect
  • After creating the vpn connection(assume you give it a name doxer-vpn), you’ll find the following when you run ‘route print’:

C:\Users\liandy>route print
Interface List

……     21

  • You may want to enlarge netmask to another one, such as Here’s one bat script to automatically connect and change route table:

rasdial doxer-vpn username password

for /f “delims=.” %%i in (‘route print ^| find /i “doxer-vpn”‘) do set if=%%i

route add mask if %if%


  • And you’ll also find that you can not visit the sites blocked by the remote site gateway’s firewall. To change this, right click the vpn connection and navigate through:

Networking -> IPV4 -> Properties -> Advanced, uncheck “use default gateway on remote network”.

After this, you should now be able to connect to sites blocked by remote firewall.

Categories: Network, Networking Security Tags:

NAT binding one priviate ip and one public ip together using linux as router

July 9th, 2013 No comments

If you’re using Linux as router, and you want to bind one public ip and one private ip together, you can do the following:

  • On linux router, define the public ip and iptables rules:

[root@router ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0:1

[root@router~]# cat /etc/sysconfig/iptables

-A POSTROUTING -o eth0 -j MASQUERADE #eth0 is public ip, which acts as a gateway(not router) to internal network. For outbound internal package, eth0 will MASQUERADE their source address
-A PREROUTING -d -j DNAT –to-destination

#or you can do port forwarding like -A PREROUTING -p tcp –dport 5911 -d -j DNAT –to-destination
-A POSTROUTING -s -j SNAT –to-source

#/sbin/iptables -t nat -I PREROUTING -p tcp –dport 2222 -j REDIRECT –to-ports 22

#/sbin/iptables -t nat -I OUTPUT -p tcp -d –dport 2222 -j REDIRECT –to-ports 22

#/sbin/iptables -t nat -I OUTPUT -p tcp -d –dport 2222 -j REDIRECT –to-ports 22

#-A PREROUTING -p tcp -m tcp –dport 389 -j REDIRECT –to-ports 4060
#-A OUTPUT -s -p tcp -m tcp –dport 389 -j REDIRECT –to-ports 4060
#-A OUTPUT -s -p tcp -m tcp –dport 389 -j REDIRECT –to-ports 4060

:INPUT ACCEPT [247:16364]
-A INPUT -p tcp -m state –state NEW -m tcp –dport 1723 -j ACCEPT
-A INPUT -p gre -j ACCEPT
-A OUTPUT -p gre -j ACCEPT
:FORWARD ACCEPT [163:13692]
:OUTPUT ACCEPT [228:18664]

  • So now you have the public ip and private ip bound together.
Categories: Network, Networking Security Tags:

configure linux as a router firewall through iptables NAT

June 25th, 2013 No comments
  • On the linux box that will act as router:

1.Turn on ip_forward:

vi /etc/sysctl.conf

net.ipv4.ip_forward = 1

sysctl -p

2.Edit /etc/sysconfig/iptables:

-A POSTROUTING -o eth1 -j MASQUERADE #eth1 is the NIC connecting to outside network

#-A POSTROUTING -s -o eth0 -j MASQUERADE  #allow to do NAT


3.Reload iptables:

[root@Router ~]# service iptables restart
Flushing firewall rules: [ OK ]
Setting chains to policy ACCEPT: filter nat [ OK ]
Unloading iptables modules: [ OK ]
Applying iptables firewall rules: [ OK ]
Loading additional iptables modules: ip_conntrack_netbios_n[ OK ]

[root@Router ~]# iptables -t nat -nL
target prot opt source destination

target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
[root@Router ~]# iptables -t filter -nL
Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

On the linux box that will act as client:

1.Set default gateway to the ip address of linux router:

vi /etc/sysconfig/network

GATEWAY= #this is ip address of the linux router

2.restart network


On router, the default gateway is:         UG        0 0          0 eth1

And on the linux client, we’ll now be able to connect to outside network too:

[root@client ~]# ping
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=254 time=0.236 ms


  1. You can also make linux as firewall using NAT/iptables, more on this article:
  2. About the numbers in brackets, you can refer to the following:
  3. You should turn on promiscuous mode before applying the configs in this article. If you’re using Vsphere Esxi, this is the step:


mtr – combine traceroute and ping

May 28th, 2013 No comments

mtr combines the functionality of the traceroute and ping programs in a single network diagnostic tool.

As mtr starts, it investigates the network connection between the host mtr runs on and HOSTNAME. by sending packets with purposly low TTLs. It continues to send packets with low TTL, noting the response time of the intervening routers. This allows mtr to print the response percentage and response times of the internet route to HOSTNAME. A sudden increase in packetloss or response time is often an indication of a bad (or simply overloaded) link.


mtr in linux

mtr in linux

Categories: Linux, Network, Networking Security Tags:

snmptrapd traphandle configuration example

January 16th, 2013 1 comment

This article is going to show the basic configuration of snmptrapd and it’s traphandle command.

snmptrapd is running on a linux host named “test-centos”;
The host sending snmptrap messages in this example is named “test-zfs-host”

Now first we’re going to set snmptrapd up on the snmptrap server side:

###Server side
[root@test-centos snmp]# cat /etc/snmp/snmptrapd.conf
#traphandle default /bin/mail -s “snmpdtrapd messages” <put your mail address here>
traphandle default /root/lognotify
authCommunity log,execute,net public

[root@test-centos snmp]# service snmptrapd restart

[root@test-centos snmp]# cat /root/lognotify
read host
read ip

while read oid val
if [ "$vars" = "" ]
vars=”$oid = $val”
vars=”$vars, $oid = $val”
echo trap: $host $ip $vars >/var/tmp/snmptrap.out

And to test whether snmptrapd is working as expected:

###On client side
snmptrap -v2c -c public test-centos:162 “” SNMPv2-MIB::sysDescr SNMPv2-MIB::sysDescr.0 s “test-zfs-host test-zfs-host.ip this is a test snmptrap string”

And after running this command, you can have a check of /var/tmp/snmptrap.out on the snmptrapd server side(test-centos):

[root@test-centos ~]# cat /var/tmp/snmptrap.out

If you’re using sun zfs head, you can set snmptrap destinations in zfs BUI(configuration -> SNMP), here’s the snapshot(click to see the larger image):

SUN zfs storage 7320 monitoring using net-snmp and mrtg

December 25th, 2012 No comments

This article is going to talk about zfs storage 7320 monitoring using net-snmp and mrtg. Although the monitored system is sun zfs storage 7320, you’ll find the main idea of this article can be applied to many different system monitoring, including but not limited to cpu usage/network/bandwidth/disk/temperature of cisco switches, other linux systems and even windows systems.

As net-snmp extending agent functionality is not supported on sun zfs storage 7320 which is solaris 11 express system, so I’m going to monitor sun zfs storage through using of one linux snmp client by writing monitoring scripts on that linux client rather than on zfs itself.

Now, here goes the steps:

Part 1 – set up snmp client and mrtg on a linux host

yum -y install gcc-* gd-* libpng-* zlib-* httpd
yum -y install net-snmp* net-snmp-libs lm_sensors lm_sensors-devel
yum -y install mrtg
cp /etc/snmp/snmpd.conf{,.bak}
echo “rocommunity public” > /etc/snmp/snmpd.conf
mkdir -p /etc/mrtg
chkconfig –level 2345 snmpd on
service snmpd start

Ensure snmpd is listening:

netstat -tunlp |grep snmp
tcp 0 0* LISTEN 26427/snmpd
udp 0 0* 26427/snmpd

And let’s have a test to make sure snmp client is working as expected:

[root@test-centos mrtg]# snmpwalk -v2c -c public localhost interface
IF-MIB::ifNumber.0 = INTEGER: 4
IF-MIB::ifIndex.1 = INTEGER: 1
IF-MIB::ifIndex.2 = INTEGER: 2
IF-MIB::ifIndex.3 = INTEGER: 3
IF-MIB::ifIndex.4 = INTEGER: 4
IF-MIB::ifDescr.1 = STRING: lo
IF-MIB::ifDescr.2 = STRING: eth0
IF-MIB::ifDescr.3 = STRING: eth1

Now it’s time to configure mrtg on linux:

[root@test-centos mrtg]# cat /etc/httpd/conf.d/mrtg.conf
Alias /mrtg /var/www/mrtg
<Location /mrtg>
Order deny,allow
Allow from all

Now, do a httpd restart:

[root@test-centos ~]# apachectl restart

Part 2 – configure snmp on SUN zfs storage 7320 web UI

Log on SUN zfs storage 7320 web UI, navigate through “Configuration” -> “SNMP”, and configure as the following:

After this, you’ll need enable/restart SNMP service on zfs.
Now do a snmpwalk from snmp linux client to SUN zfs storage:

[root@test-centos ~]# snmpwalk -v2c -c public test-zfs-host interface
IF-MIB::ifNumber.0 = INTEGER: 6
IF-MIB::ifIndex.1 = INTEGER: 1
IF-MIB::ifIndex.4 = INTEGER: 4
IF-MIB::ifIndex.5 = INTEGER: 5
IF-MIB::ifIndex.6 = INTEGER: 6
IF-MIB::ifIndex.7 = INTEGER: 7
IF-MIB::ifIndex.8 = INTEGER: 8
IF-MIB::ifDescr.1 = STRING: lo0

  • Part 3 – extending snmp mibs/oids on snmp client(the linux host)

As stated at the beginning of this article, net-snmp extending agent functionality is not supported on sun zfs storage 7320. So I’m going to monitor sun zfs storage through using of one linux snmp client by writing monitoring scripts on that linux client rather than on zfs itself.

To avoid ssh asking for password when connecting from the linux host, you need add the linux host’s pubkey to sun zfs storage:
On snmp Linux client:

[root@test-centos ~]# ssh-keygen -t rsa #if you already have pubkey, skip this step
[root@test-centos ~]# cat /root/.ssh/ |awk ‘{print $2}’

On Sun zfs storage:
Click “Configuration”, then add the snmp client’s public key(ensure RSA is selected):

Do a ssh connection from snmp linux client to the sun zfs storage host, it should now not asking for password.(ensure to do this, as there’s possibility that sun zfs’s key is not on the linux client as you may never have connected from that linux client to the sun zfs storage system)

Now we’re to the most important part. Assume we want to monitoring space usage on the SUN zfs storage system, to do this, you’ll need do the following on the snmp linux client:

[root@test-centos mrtg]# cat /etc/snmp/snmpd.conf
rocommunity public
extend . zfs-test-zfs-host-total /bin/bash /var/tmp/mrtg/
extend . zfs-test-zfs-host-used /bin/bash /var/tmp/mrtg/

[root@test-centos ~]# cat /var/tmp/mrtg/
_used=`ssh test-zfs-host “status ls”|grep Used|awk ‘{print gensub(‘/T/’,””,”g”,$2)}’`
echo $_used;

[root@test-centos ~]# cat /var/tmp/mrtg/
_used=`ssh test-zfs-host “status ls”|grep Used|awk ‘{print gensub(‘/T/’,””,”g”,$2)}’` #I trimed ‘T’, you may need modify this to meet your environment
_avail=`ssh test-zfs-host “status ls”|grep Avail|awk ‘{print gensub(‘/T/’,””,”g”,$2)}’` #I trimed ‘T’, you may need modify this to meet your environment
_all=`echo $_used + $_avail|bc`
echo $_all;

[root@test-centos ~]# chmod +x /var/tmp/mrtg/
[root@test-centos ~]# chmod +x /var/tmp/mrtg/

Now, let’s do a snmp restart on snmp linux client and then test the newly added OIDs:

[root@test-centos ~]# service snmpd restart

[root@test-centos ~]# snmpwalk -v2c -c public localhost .
UCD-SNMP-MIB::ucdavis.32.1.0 = INTEGER: 1
UCD-SNMP-MIB::ucdavis. = STRING: “/bin/bash”
UCD-SNMP-MIB::ucdavis. = STRING: “/var/tmp/mrtg/”
UCD-SNMP-MIB::ucdavis. = “”
UCD-SNMP-MIB::ucdavis. = INTEGER: 5
UCD-SNMP-MIB::ucdavis. = INTEGER: 1
UCD-SNMP-MIB::ucdavis. = INTEGER: 1
UCD-SNMP-MIB::ucdavis. = INTEGER: 4
UCD-SNMP-MIB::ucdavis. = INTEGER: 1
UCD-SNMP-MIB::ucdavis. = STRING: “9.31″
UCD-SNMP-MIB::ucdavis. = STRING: “9.31″
UCD-SNMP-MIB::ucdavis. = INTEGER: 1
UCD-SNMP-MIB::ucdavis. = INTEGER: 0
UCD-SNMP-MIB::ucdavis. = STRING: “9.31″
[root@test-centos ~]# snmpwalk -v2c -c public localhost .
UCD-SNMP-MIB::ucdavis.31.1.0 = INTEGER: 1
UCD-SNMP-MIB::ucdavis. = STRING: “/bin/bash”
UCD-SNMP-MIB::ucdavis. = STRING: “/var/tmp/mrtg/”
UCD-SNMP-MIB::ucdavis. = “”
UCD-SNMP-MIB::ucdavis. = INTEGER: 5
UCD-SNMP-MIB::ucdavis. = INTEGER: 1
UCD-SNMP-MIB::ucdavis. = INTEGER: 1
UCD-SNMP-MIB::ucdavis. = INTEGER: 4
UCD-SNMP-MIB::ucdavis. = INTEGER: 1
UCD-SNMP-MIB::ucdavis. = STRING: “16.32″
UCD-SNMP-MIB::ucdavis. = STRING: “16.32″
UCD-SNMP-MIB::ucdavis. = INTEGER: 1
UCD-SNMP-MIB::ucdavis. = INTEGER: 0
UCD-SNMP-MIB::ucdavis. = STRING: “16.32″

From the output, we can see that OIDs UCD-SNMP-MIB::ucdavis. (used space) and UCD-SNMP-MIB::ucdavis. (total space) are the two OIDs we want.

  • Part 4 – do the mrtg drawing

As we got the OIDs we want, it’s now easier for us to do the mrtg drawing. On the snmp linux host, do the following steps:

[root@test-centos ~]# cat /etc/mrtg/test-zfs-host.cfg
#LoadMIBs: /usr/share/snmp/mibs/UCD-SNMP-MIB.txt,/usr/share/snmp/mibs/TCP-MIB.txt
workdir: /var/www/mrtg/
Title[zfs_space_test-zfs-host]: Percentage used space on zfs
PageTop[zfs_space_test-zfs-host]: <h1>Percentage used space on zfs</h1>
Target[zfs_space_test-zfs-host]: .
Options[zfs_space_test-zfs-host]: growright,gauge,transparent,nopercent
Unscaled[zfs_space_test-zfs-host]: ymwd
MaxBytes[zfs_space_test-zfs-host]: 100
YLegend[zfs_space_test-zfs-host]: UsedSpace %
ShortLegend[zfs_space_test-zfs-host]: T
LegendI[zfs_space_test-zfs-host]: Used
LegendO[zfs_space_test-zfs-host]: Total
Legend1[zfs_space_test-zfs-host]: Percentage used space on zfs
Legend2[zfs_space_test-zfs-host]: Percentage all space on zfs

You need replace “UCD-SNMP-MIB::ucdavis” with . or you’ll get error messages like the following:

[root@test-centos ~]# env LANG=C TZ=Asia/Shanghai /usr/bin/mrtg /etc/mrtg/test-zfs-host.cfg
Argument “v4only” isn’t numeric in int at /usr/bin/../lib64/mrtg2/ line 183.
backoff (v4only) must be a number >= 1.0 at /usr/bin/../lib64/mrtg2/ line 465

Let’s continue:

[root@test-centos ~]# env LANG=C TZ=Asia/Shanghai /usr/bin/mrtg /etc/mrtg/test-zfs-host.cfg
24-12-2012 01:46:41, Rateup WARNING: /usr/bin/rateup could not read the primary log file for zfs_space_test-zfs-host
24-12-2012 01:46:41, Rateup WARNING: /usr/bin/rateup The backup log file for zfs_space_test-zfs-host was invalid as well
24-12-2012 01:46:41, Rateup WARNING: /usr/bin/rateup Can’t remove zfs_space_test-zfs-host.old updating log file
24-12-2012 01:46:41, Rateup WARNING: /usr/bin/rateup Can’t rename zfs_space_test-zfs-host.log to zfs_space_test-zfs-host.old updating log file

[root@test-centos ~]# env LANG=C TZ=Asia/Shanghai /usr/bin/mrtg /etc/mrtg/test-zfs-host.cfg
24-12-2012 01:46:46, Rateup WARNING: /usr/bin/rateup Can’t remove zfs_space_test-zfs-host.old updating log file

[root@test-centos ~]# env LANG=C TZ=Asia/Shanghai /usr/bin/mrtg /etc/mrtg/test-zfs-host.cfg

[root@test-centos ~]# indexmaker –output=/var/www/mrtg/index.html /etc/mrtg/test-zfs-host.cfg

Now add cronjob:

0-59/5 * * * * env LANG=C TZ=Asia/Shanghai /usr/bin/mrtg /etc/mrtg/test-zfs-host.cfg

Visit http://<your linux box’s ip address>/mrtg/ to get the GUI result(you’ll wait 5 minutes for the initial result, be patient!)

  • Part 5 – troubleshooting

  • If you met error messages like the following:

[root@test-centos ~]# env LANG=C TZ=Asia/Shanghai /usr/bin/mrtg /etc/mrtg/test-zfs-host.cfg
Argument “v4only” isn’t numeric in int at /usr/bin/../lib64/mrtg2/ line 183.
backoff (v4only) must be a number >= 1.0 at /usr/bin/../lib64/mrtg2/ line 465

That’s because you’re using OIDs/MIBs alias rather than number. Change alias to number, i.e. change UCD-SNMP-MIB::ucdavis to ., and then re-check.

  • If you met error messages like the following:

[root@test-centos ~]# env LANG=C TZ=Asia/Shanghai /usr/bin/mrtg /etc/mrtg/test-zfs-host.cfg
cannot encode Object ID first subid too big in Object ID at /usr/bin/mrtg line 2035
Tuesday, 25 December 2012 at 0:34: ERROR: Target[zfs-test-zfs-host-io-average-latency-igb2][_IN_] ‘ $target->[3]{$mode} ‘ did not eval into defined data
Tuesday, 25 December 2012 at 0:34: ERROR: Target[zfs-test-zfs-host-io-average-latency-igb2][_OUT_] ‘ $target->[3]{$mode} ‘ did not eval into defined data

Then you should carefully check the scripts you write for the OIDs, and run several times of snmpwalk to ensure the values are correct(is your script’s output variable, this may cause problems)

  • If you met error messages like the following:

[root@test-centos mrtg]# env LANG=C TZ=Asia/Shanghai /usr/bin/mrtg /etc/mrtg/test-zfs-host.cfg
SNMP Error:
Received SNMP response with error code
error status: noSuchName
index 2 (OID:
SNMPv1_Session (remote host: “localhost” [].161)
community: “public”
request ID: 1786815468
PDU bufsize: 8000 bytes
timeout: 2s
retries: 5
backoff: 1)
at /usr/bin/../lib64/mrtg2/ line 490
SNMPGET Problem for . . sysUptime sysName on public@localhost::::::v4only
at /usr/bin/mrtg line 2035
SNMP Error:
Received SNMP response with error code
error status: noSuchName
index 2 (OID:
SNMPv1_Session (remote host: “localhost” [].161)
community: “public”
request ID: 1786815469
PDU bufsize: 8000 bytes
timeout: 2s
retries: 5
backoff: 1)
at /usr/bin/../lib64/mrtg2/ line 490
SNMPGET Problem for . . sysUptime sysName on public@localhost::::::v4only
at /usr/bin/mrtg line 2035
Monday, 24 December 2012 at 14:41: ERROR: Target[zfs_space_test-zfs-host][_IN_] ‘( $target->[0]{$mode} ) * 10000 / ( $target->[1]{$mode} )’ (warn): Use of uninitialized value in division (/) at (eval 16) line 1.
Monday, 24 December 2012 at 14:41: ERROR: Target[zfs_space_test-zfs-host][_OUT_] ‘( $target->[0]{$mode} ) * 10000 / ( $target->[1]{$mode} )’ (warn): Use of uninitialized value in division (/) at (eval 17) line 1.

Then one possible culprit is that new net-snmp stop supporting “exec”, use “extend” instead and re-try.

Note that you can also use “snmpd -f -Le” to check error messages related to snmpd.

Here’s more links from where you can read more about net-snmp/mrtg:

  1. net-snmp FAQ
  2. mrtg configuration opstions
  3. SUN zfs storage SNMP –
  4. net-snmp extending agent functionality – (search for ‘extending agent functionality’ on this page)
  5. To get only numbers of snmpwalk, add -On. More info on
  6. Now here’s the image of the baby(click on it to see the larger one:

hash md5 crc32 – terminology explaination

December 17th, 2012 No comments

This article is going to talk about the concept of hash/md5 vs crc32 hash.

  • Concept

Hash is some sort of a fingerprint of some information (data). It is designed to produce a code of specific length (somewhere between 8 and 512 bits) that satisfies the following:
- Every bit in the message contributes to the hash. This means that changing any bit in the message should change the hash.
- Relatively small changes in the message should always result in changes in the hash. We want to be sure that it would take an extremely unlikely combination of errors to produce an identical hash.
- The histogram of output hash values for input messages should tend to be flat. For a given input message, we want the probability of a given hash being produced to be nearly equal across the entire range of possible hash values (for example 0h to FFh for 8 bit hashes).

CRC stands for Cyclic redundancy code. It is a code that provides efficient error detection. It is used by many programs (for example Zip compression programs, communication protocols…). There are many variants of this code.
CRC is also often used for reference to general hash code.

CRC-32 is an acronym for the 32 bit Cyclical Redundancy Check algorithm. CRC-32 generally refers to a specific 32 bit CRC formula sanctioned by the CCITT, an international standards body primarily concerned with telecommunications. CRC-32 is used in communications protocols such as HDLC(High speed Data Link Control, the default encapsulation for serial interfaces on Cisco routers) and ZMODEM to verify the integrity of blocks of data being transferred through various media.

MD5 stands for Message Digest 5 algorithm. Algorithm developed by Professor Ronald L. Rivest of MIT and used in many applications.

  • Reliability

CRC-32 code is 32 bit number (code) generated by CRC-32 algorithm based on data input (in our case file content). This code is some sort of a “fingerprint”. However it differs somewhat from the human fingerprint. It is often said that no two people have identical fingerprints. This can’t be the case for our CRC fingerprint. Since there are more than 4,294,967,296 different files in the world, it is a foregone conclusion that some of them must have identical codes. However, the CRC-32 does satisfy all the above mentioned requirements for hash code. They were the goals that the CCITT had in mind when selecting the CRC-32 algorithm. In practice, the chances of inadvertently damaging or modifying a file without modifying the CRC is vanishingly small, so for all practical purposes testing CRC code to detect changes can be considered to be infallible.

However if file would be intentionally damaged (for example by a virus) the CRC could be restored to its previous state. This could be done by using using brute force method to add some bytes to end of the file. So this is one thing it cannot protect you from (although this is almost never done).

MD5 was developed by Professor Ronald L. Rivest of MIT. What it does, to quote the executive summary of rfc1321, is: The MD5 algorithm takes as input a message of arbitrary length and produces as output a 128-bit “fingerprint” or “message digest” of the input. It is conjectured that it is computationally infeasible to produce two messages having the same message digest, or to produce any message having a given prespecified target message digest. The MD5 algorithm is intended for digital signature applications, where a large file must be “compressed” in a secure manner before being encrypted with a private (secret) key under a public-key cryptosystem such as RSA.

In essence, MD5 is a way to verify data integrity, and is much more reliable than checksum and many other commonly used methods.

  • Implementation

Hash is implemented using concurrent reading and hash calculation. This is done by writing data from media into intermediate buffer in one thread and reading from buffer and calculating hash values in other thread. Hash values are written to disk as they are calculated to minimize memory usage.


Categories: Security Tags:

resolved – error: conflicting types for ‘pci_pcie_cap’ – Infiniband driver OFED installation

December 10th, 2012 No comments

OFED is an abbr for “OpenFabrics Enterprise Distribution”. When installing OFED- on a centos/RHEL 5.8 linux system, I met the following problem:

In file included from /var/tmp/OFED_topdir/BUILD/ofa_kernel-
/var/tmp/OFED_topdir/BUILD/ofa_kernel- error: conflicting types for ‘pci_pcie_cap’
include/linux/pci.h:1015: error: previous definition of ‘pci_pcie_cap’ was here
make[4]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-] Error 1
make[3]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-] Error 2
make[2]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-] Error 2
make[1]: *** [_module_/var/tmp/OFED_topdir/BUILD/ofa_kernel-] Error 2
make[1]: Leaving directory `/usr/src/kernels/2.6.18-308.′
make: *** [kernel] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.70159 (%build)

After googling and experimenting, I can tell you the definite resolution for this problem -> install another version of OFED, OFED- Go to the following official site to download:


1. Here’s the file of OFED installation howto: OFED installation README.txt

2.Some Infiniband commands that may be useful for you:

service openibd status/start/stop
lspci |grep -i infini
ibv_devinfo #Verify the status of ports by using ibv_devinfo: all connected ports should report a “PORT_ACTIVE” state