Standards

Data center convergence and virtualization offer the promise of improved efficiency and flexibility. Developing a network visibility strategy when planning the network upgrades needed to support convergence is essential since only network equipment with embedded traffic monitoring will provide the data center wide visibility essential for effective control.

This article examines the proprietary and standards-based protocols for embedded traffic monitoring as well as their current status and level of support.

Before looking at the standards, it's important to understand how current traffic monitoring protocols fit in the protocol stack. The article, sFlow and NetFlow, describes how sFlow operates at layer 2 (switches) and Cisco NetFlow (and variants from other vendors such as j-Flow, NetStream, LFAP etc.) operates at layer 3 (routers).

There are two organizations responsible for most networking standards:

1. IETF (Internet Engineering Task Force) is responsible for layer 3-7 protocols (IP, routing, DNS, telnet, smtp etc.)
2. IEEE (Institute of Electrical and Electronics Engineers) is responsible for layer 2 protocols (Ethernet, switching, 802.11 etc.)

The IETF has recently developed a standard alternative to proprietary IP flow monitoring protocols (Cisco NetFlow, j-Flow, LFAP, NetStream etc.). The IPFIX standard was created in order "to transfer IP flow data from IPFIX exporters to collectors." This focus on IP flow export is consistent with the IETF's responsibility for the TCP/IP suite of protocols and explicitly avoids addressing layer 2 monitoring (switches). The IPFIX standard was published in 2008. Unfortunately, IPFIX has found little support among router vendors, who continue to implement proprietary solutions.

The IEEE publishes the standards for Ethernet and bridging/switching, and is currently developing the set of standards for Data Center Bridging (DCB) that are driving data center convergence. The IEEE would seem to be the natural place to standardize a protocol for monitoring switched Ethernet traffic, however, the IEEE's focus is on the mechanics of layer 2 connectivity and network management protocols have not been a priority.

The sFlow.org industry consortium was formed in order to develop a multi-vendor standard to address the need for network visibility in layer 2 devices. Many of the members of sFlow.org actively participate in the IEEE standards process, ensuring that sFlow is well matched to the challenge of monitoring current and emerging IEEE standard networks (Ethernet, 40/100G, DCB etc.). The sFlow standard was published in 2001 and it is now implemented by most switch vendors (see sFlow.org).

In selecting a standard for data center visibility, it is important to understand how data center networks are changing. The traditional three layer architecture, in which traffic is moved up to the core and back, does not scale well. Instead, the trend is to integrate access and aggregation layer switches into a flat layer 2 network using shortest-path bridging (IEEE 802.1aq) so that traffic bypasses the core to deliver the increased scalability, bandwidth and reduced latency needed to support converged data center workloads.

The shift in data center network architecture requires a corresponding shift in network management. Instead of monitoring and controlling at layer 3 in the core routers, visibility and control functions move to layer 2 switches and the network edge.

sFlow is the only standard specifically designed for embedded monitoring of layer 2 devices. Selecting switches with embedded sFlow provides a low cost, scalable means of obtaining layer 2-7 visibility into all traffic flowing over the switched network (including storage traffic). Building a network visibility strategy around the sFlow standard maximizes the choice of vendors and ensures interoperable monitoring in mixed vendor environments, eliminating vendor lock-in and facilitating "best in class" product selection.

802.1aq and TRILL

There are a number of drivers increasing demand for bandwidth in the data center:

• Multi-core processors and blade servers greatly increase computational density, creating a corresponding demand for bandwidth.
• Networked storage increases demand for bandwidth.
• Virtualization and server consolidation ensures that servers are fully utilized, further increasing demand for bandwidth.

Virtual machine mobility (e.g. VMWare vMotion, Citrix XenMotion or Xen Live Migration) require a large, flat layer 2 network so that machines can be moved without reconfiguring their network settings. The increasing size of the layer 2 network, combined with the increasing demand for bandwidth challenges the scalability of current Ethernet switching technologies.

The diagram illustrates the problem. Currently, Ethernet uses the spanning tree protocol to determine forwarding paths. The tree structure forces traffic to the network core, creating a bottleneck. In addition, the tree structure doesn't allow traffic to flow on backup links, further limiting usable bandwidth. An alternative forwarding technique (used by routers), is to select shortest paths through the network. Shortest path forwarding allows traffic flows to bypass the core, reducing the bottleneck. The added benefit of shortest path forwarding is its ability to make use of all the links, including backup links, further increasing capacity.

There are two emerging standards for shortest path forwarding in switches:

• TRILL (Transparent Interconnect of Lots of Links)
• IEEE 802.1aq (Shortest Path Bridging)

Both TRILL and 802.1aq use IS-IS (Intermediate System to Intermediate System) routing to select shortest paths through the network. The protocols are very similar, but are being proposed by two different standards bodies: TRILL is being developed by IETF and 802.1aq by the IEEE.

It is surprising to see the IETF working on a LAN bridging protocol. The IETF is responsible for Internet protocols (TCP/IP, routing) and the IEEE is responsible for LAN protocols (802.11, Ethernet, bridging/switching). Adopting the IEEE 802.1aq standard makes the most sense, since it will ensure interoperability with the IEEE data center bridging standards being developed to support FCoE and facilitate data center convergence.

Finally, while more efficient network topologies will help increase network capacity, the days of relying on network over-provisioning are over. Much tighter control of bandwidth is going to be required in order to cope with converged data center workloads. Selecting switches that support the sFlow standard provides the visibility and control needed to manage the increasing demand for bandwidth.