SC11 OpenFlow testbed

ESnet/ECSEL Demo at SC11

The SCinet Research Sandbox, a part of the 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC11), is being used to demonstrate OpenFlow applications using switches from IBM (BNT), HP, NEC and Pronto. Of the four switch models that form the testbed, three support standard sFlow monitoring (IBM, HP and NEC), providing detailed visibility into traffic within the testbed.

In the End-to-End Circuit Service at Layer 2 (ECSEL) demonstration, OpenFlow is used to provision RDMA over Converged Ethernet (RoCE) connectivity through an NEC switch.

Chart 1: Top Connections in OpenFlow Testbed

Chart 1 shows RoCE as the dominant traffic in the testbed, saturating the 10G links. This chart demonstrates how convergence is changing the nature of data center traffic as new storage and clustering workloads place heavy demands on the shared network. The sFlow standard embeds monitoring within the network switches to provide the network-wide visibility needed to manage increased demand for bandwidth.

Note: The RoCE protocol, along with FCoE, AoE and many other data center protocols operate at layer 2 so traditional approaches to monitoring that rely on layer 3 monitoring using IP flow records from routers and firewalls are of limited value.

The OpenFlow protocol provides a way for external software to control the forwarding decisions within switches. In this case, OpenFlow is being used to provision the network to carry the RoCE connections. Maintaining connectivity between the switches and the controllers is essential in order to maintain control of the network.

Chart 2: Traffic between OpenFlow Switches and Open Flow Controllers (NOX)

Chart 2 shows that OpenFlow control traffic is being carried in-band and that it is consuming very little network bandwidth. However, in-band control traffic is potentially vulnerable to interference from large bandwidth consumers (like RoCE).

Chart 3: OpenFlow Traffic and Network Priority

Chart 3 shows that the Quality of Service (QoS) policy for all the OpenFlow connections is Best Effort (BE). Best Effort is the default priority class for traffic in the network, leaving the OpenFlow control channels vulnerable to network congestion. Assigning a high priority to the OpenFlow protocol would ensure that OpenFlow messages could traverse the network during periods of congestion, allowing the controller to take corrective action to mitigate the congestion.

OpenFlow and sFlow

The paper, sFlow and OpenFlow, describes how the OpenFlow and sFlow standards complement one another. Monitoring using sFlow provides visibility into network traffic that allows the OpenFlow controller to dynamically allocate network resources based on changing network demands.

More generally, the Data center convergence, visibility and control presentation describes the critical role that measurement plays in managing costs and optimizing performance in converged, virtualized and cloud environments.