Collisions and network saturation
Ethernet is similar to an old party-line telephone: everybody listens at once, everybody talks at once, and sometimes two talkers start at the same time. In a well-conditioned network, with only two hosts on it, it’s possible to use close to the maximum network’s bandwidth. However, NFS clients and servers live in a burst-filled environment, where many machines try to use the network at the same time. When you remove the well-behaved conditions, usable network bandwidth decreases rapidly.
On the Ethernet, a host first checks for a transmission in progress on the network before attempting one of its own. This process is known as carrier sense. When two or more hosts transmit packets at exactly the same time, neither can sense a carrier, and a collision results. Each host recognizes that a collision has occurred, and backs off for a period of time, t, before attempting to transmit again. For each successive retransmission attempt that results in a collision, t is increased exponentially, with a small random variation. The variation in back-off periods ensures that machines generating collisions do not fall into lock step and seize the network.
As machines are added to the network, the probability of a collision increases. Network utilization is measured as a percentage of the ideal bandwidth consumed by the traffic on the cable at the point of measurement. Various levels of utilization are usually compared on a logarithmic scale. The relative decrease in usable bandwidth going from 5% utilization to 10% utilization, is about the same as going from 10% all the way to 30% utilization.
Measuring network utilization requires a LAN analyzer or similar device. Instead of measuring the traffic load directly, you can use the average collision rate as seen by all hosts on the network as a good indication of whether the network is overloaded or not. The collision rate, as a percentage of output packets, is one of the best measures of network utilization. The Collis field in the output of netstat -in shows the number of collisions:Code View: Scroll / Show All% netstat -in Name Mtu Net/Dest Address Ipkts Ierrs Opkts Oerrs Collis Queue lo0 8232 127.0.0.0 127.0.0.1 7188 0 7188 0 0 0 hme0 1500 22.214.171.124 126.96.36.199 139478 11 102155 0 3055 0
The collision rate for a host is the number of collisions seen by that host divided by the number of packets it writes, as shown in Figure 17-1.
Collisions are counted only when the local host is transmitting; the collision rate experienced by the host is dependent on its network usage. Because network transmissions are random events, it’s possible to see small numbers of collisions even on the most lightly loaded networks. A collision rate upwards of 5% is the first sign of network loading, and it’s an indication that partitioning the network may be advisable.
Network partitioning hardware
Network partitioning involves dividing a single backbone into multiple segments, joined by some piece of hardware that forwards packets. There are multiple types of these devices: repeaters, hubs, bridges, switches, routers, and gateways. These terms are sometimes used interchangeably although each device has a specific set of policies regarding packet forwarding, protocol filtering, and transparency on the network:
- A repeater joins two segments at the physical layer. It is a purely electrical connection, providing signal amplification and pulse “clean up” functions without regard for the semantics of the signals. Repeaters are primarily used to exceed the single-cable length limitation in networks based on bus topologies, such as 10Base5 and 10Base2. There is a maximum to the number of repeaters that can exist between any two nodes on the same network, keeping the minimum end-to-end transit time for a packet well within the Ethernet specified maximum time-to-live. Because repeaters do not look at the contents of packets (or packet fragments), they pass collisions on one segment through to the other, making them of little use to relieve network congestion.
- A hub joins multiple hosts by acting as a wiring concentrator in networks based on star topologies, such as 10BaseT. A hub has the same function as a repeater, although in a different kind of network topology. Each computer is connected, typically over copper, to the hub, which is usually located in a wiring closet. The hub is purely a repeater: it regenerates the signal from one set of wires to the others, but does not process or manage the signal in any way. All traffic is forwarded to all machines connected to the hub.
- Bridges function at the data link layer, and perform selective forwarding of packets based on their destination MAC addresses. Some delay is introduced into the network by the bridge, as it must receive entire packets and decipher their MAC-layer headers. Broadcast packets are always passed through, although some bridge hardware can be configured to forward only ARP broadcasts and to suppress IP broadcasts such as those emanating from ypbind.Intelligent or learning bridges glean the MAC addresses of machines through observation of traffic on each interface. “Dumb” bridges must be loaded with the Ethernet addresses of machines on each network and impose an administrative burden each time the network topology is modified. With either type of bridge, each new segment is likely to be less heavily loaded than the original network, provided that the most popular inter-host virtual circuits do not run through the bridge.
- You can think of a switch as an intelligent hub having the functionality of a bridge. The switch also functions at the data link layer, and performs selective forwarding of packets based on their destination MAC address. The switch forwards packets only to the intended port of the intended recipient. The switch “learns” the location of the various MAC addresses by observing the traffic on each port. When a switch port receives data packets, it forwards those packets only to the appropriate port for the intended recipient. A hub would instead forward the packet to all other ports on the hub, leaving it to the host connected to the port to determine its interest in the packet. Because the switch only forwards the packet to its destination, it helps reduce competition for bandwidth between the hosts connected to each port.
- Repeaters, hubs, bridges, and switches divide the network into multiple distinct physical pieces, but the collection of backbones is still a single logical network. That is, the IP network number of all hosts on all segments will be the same. It is often necessary to divide a network logically into multiple IP networks, either due to physical constraints (i.e., two offices that are separated by several miles) or because a single IP network has run out of host numbers for new machines.Multiple IP networks are joined by routers that forward packets based on their source and destination IP addresses rather than 48-bit Ethernet addresses. One interface of the router is considered “inside” the network, and the router forwards packets to the “outside” interface. A router usually corrals broadcast traffic to the inside network, although some can be configured to forward broadcast packets to the “outside” network. The networks joined by a router need not be of the same type or physical media, and routers are commonly used to join local area networks to point-to-point long-haul internetwork connections. Routers can also help ensure that packets travel the most efficient paths to their destination. If a link between two routers fails, the sending router can determine an alternate route to keep traffic moving. You can install a dedicated router, or install multiple network interfaces in a host and allow it to route packets in addition to its other duties. Appendix A contains a detailed description of how IP packets are forwarded and how routes are defined to Unix systems.
- At the top-most level in the network protocol stack, a gateway performs forwarding functions at the application level, and frequently must perform protocol conversion to forward the traffic. A gateway need not be on more than one network; however, gateways are most commonly used to join multiple networks with different sets of native protocols, and to enforce tighter control over access to and from each of the networks.
Replacing an Ethernet hub with a Fast Ethernet hub is like increasing the speed limit of a highway. Replacing a hub with a switch is similar to adding new lanes to the highway. Replacing an Ethernet hub with a Fast Ethernet switch is the equivalent of both improvements, although with a higher cost.
1.This article is from book <Managing NFS and NIS, Second Edition>.
2.Here’s the differences between broadcast domains and collision domains:
- layer 1 of the OSI model
- a hub is an entire collision domain since it forwards every bit it receives from one interface on every other interfaces
- a bridge is a two interfaces device that creates 2 collision domains, since it forwards the traffic it receives from one interface only to the interface where the destination layer 2 device (based on his mac address) is connected to. A bridge is considered as an “intelligent hub” since it reads the destination mac address in order to forward the traffic only to the interface where it is connected
- a switch is a multi-interface hub, every interface on a switch is a collision domain. A 24 interfaces switch creates 24 collision domains (assuming every interface is connected to something, VLAN don’t have any importance here since VLANs are a layer 2 concept, not layer 1 like collision domains)
- layer 2 of the OSI model
- a switch creates an entire broadcast domain (provided that there’s only one VLAN) since broadcasts are a layer 2 concept (mac address related)
- routers don’t forward layer 2 broadcasts, hence they separate broadcast domains