Ethernet growth

There are a number of drivers increasing demand for bandwidth in the data center:

• Multi-core processors and blade servers greatly increase computational density, creating a corresponding demand for bandwidth.
• Networked storage increases demand for bandwidth.
• Virtualization and server consolidation ensures that servers are fully utilized, further increasing demand for bandwidth.

The chart shows a projection of the adoption of higher speed Ethernet server interconnects as the increasing demand for bandwidth drives current deployment of 10G Ethernet and accelerates adoption of emerging 40G and 100G Ethernet products.

The scalability, performance and relatively low cost makes Ethernet the clear choice of networking technology for providing converged SAN and LAN connectivity. The sFlow standard, already supported by most Ethernet switch vendors, provides the network visibility that is essential for managing data centers in this rapidly evolving environment.

Multi-vendor support

Multi-vendor support of the sFlow standard has been increasing rapidly over the last 5 years. Initially published by InMon Corp. as RFC 3176, sFlow has grown into the leading, multi-vendor, standard for monitoring high-speed switched networks. The growth in vendor support of sFlow has been driven by the move to 1G and more recently 10G Ethernet switches. The sFlow.org industry consortium, responsible for developing and promoting the sFlow standard, lists the large number of switches that implement sFlow. The switch vendors supporting sFlow now include: HP, IBM, Dell, Brocade, Juniper, BLADE, 3Com, H3C, Force10, Hitachi, AlaxalA, NEC, Alcatel-Lucent, D-Link, Extreme Networks, Allied Telesis and Comtec.

Broad vendor support delivers the network-wide visibility that is essential for managing the convergence of voice, data and storage on the campus and in the data center. It is likely that you have products from one or more of these vendors - if you would like more information on options to extend visibility in your network, ask them about sFlow.

September 11, 2012 Update: An updated version of this article describes the continued growth (nearly doubling) of sFlow support among vendors over the last couple of years, including Cisco's recent support for sFlow.

LAN and WAN

There is widespread confusion about the differences between sFlow and NetFlow and they are often simply referred to collectively as xFlow, implying that the two technologies are interchangeable. The sFlow and NetFlow posting described some of the technical differences between the two technologies, in particular, describing how sFlow operates at the Switch/Ethernet/LAN level and NetFlow operates at the Router/IP/WAN level. This division helps understand where the two technologies fit in the market.

The chart breaks down networking into switching and routing on the x-axis and network speed on the y-axis and then plots the application areas for sFlow and NetFlow. Since sFlow is built into switch ASICs, it offers monitoring solutions that span the full range of layer 2-3 switching products, from inexpensive switches aimed at office and small business environments, to the most demanding applications in supercomputer data centers, Internet exchange points and digital effects render farms. NetFlow is typically found in enterprise class routers. Since performance critical components of NetFlow are often implemented in software, NetFlow isn't widely used for monitoring at the high end of the router market (tier-1 ISPs). The cost of NetFlow enabled equipment limits its use at the low end of the router market.

Dividing the market into routing and switching products and the related applications helps explain why some vendors support sFlow while others support NetFlow. The chart also explains why a vendor might offer sFlow on their switch products and NetFlow on their router products. In practice, most networks blend switching and routing in order to meet the varied requirements of the different services running on the network. In many cases, a network monitoring strategy that embraces both sFlow and NetFlow delivers the most complete visibility into network activity.

Networked storage

A previous posting discussed how sFlow is used to provide visibility in the data center. This post looks more closely at the challenge posed by networked storage.

There are many good reasons to use networked storage: the storage resources can be shared, replicated and backed up independently of the systems that use them. In a virtual server environment, using networked storage for the virtual machine images simplifies the replication of virtual machines and the migration of virtual machines between servers (e.g. VMWare vMotion, Citrix XenMotion or Xen Live Migration).

In addition, the migrating of storage from a dedicated storage area network (SAN) to a single Converged Enhanced Ethernet (CEE) network promises to reduce cost and create a more flexible data center infrastructure. However, this migration also places additional demands on the LAN infrastructure.

Regardless of the type of networked storage (iSCSI, NFS, AoE or FCoE), the management of network bandwidth is critical to successful deployment and operation. For example, the chart above shows site-wide traffic from a large campus network broken out by protocol. The storage traffic (iSCSI) is clearly the largest load on the network, dwarfing the amount of web (HTTP) traffic.

The visibility into network traffic provided by sFlow is critical to effectively managing network resources. If the network is poorly provisioned, congestion associated with storage traffic will degrade quality of service (QoS) for other applications on the network and impair system performance since network congestion will also manifest itself as slow disk performance.

Network visibility in the data center

Current trends toward Virtualization, Converged Enhanced Ethernet (CEE), Fibre Channel over Ethernet (FCoE), Service Oriented Architectures (SOA) and Cloud Computing are part of a broader re-architecture of the data center in which enterprise applications are decomposed into simpler elements that can be deployed, moved, replicated and connected using high-speed switched Ethernet.

The following example, shown in the diagram, illustrates the management challenges faced in this new environment. A system manager decides to move a virtual machine from one server to another. The system management tools show that there is plenty of capacity on the destination server and this looks like a safe move. Unfortunately, the move causes the storage traffic, which had previously been confined to a single switch, to congest links across the data center causing system wide performance problems.

In this new environment the traditional siloed approach in which different teams manage the network, storage and servers does not work. An integrated approach to management is needed if the full benefits of a converged data center are to be achieved. Ensuring network-wide visibility into the storage, network and services running in the data center, their traffic volumes and dependencies is a critical component of an integrated management strategy.

In order to achieve data center wide visibility, every layer of the data center network, including the core, distribution, top of rack and blade server switches, needs to be instrumented. This might seem like a daunting (and expensive!) challenge. However, most vendors have integrated sFlow into their switch products. The sFlow standard provides a proven solution that is available in network products from the leading computer and network vendors, including HP, IBM, Dell, Brocade, BLADE, Juniper, Force10 and 3Com (for a complete list, see sFlow.org). Making sFlow a requirement when building out a new data center is a sound investment, adding very little to the cost of the network, but ensuring that the visibility needed safely deploy, optimize and scale up new services is available to the operations team.