ONOS traffic analytics

Open Network Operating System (ONOS) is "a software defined networking (SDN) OS for service providers that has scalability, high availability, high performance, and abstractions to make it easy to create applications and services." The open source project is hosted by the Linux Foundation.

Mininet and onos.py workflow describes how to run ONOS using the Mininet network emulator. Mininet allows virtual networks to be quickly constructed and is a simple way to experiment with ONOS. In addition, Mininet flow analytics describes how to enable industry standard sFlow streaming telemetry in Mininet, proving a simple way monitor traffic in the ONOS controlled network.

For example, the following command creates a Mininet network, controlled by ONOS, and monitored using sFlow:

sudo mn --custom ~/onos/tools/dev/mininet/onos.py,sflow-rt/extras/sflow.py \
--link tc,bw=10 --controller onos,1 --topo tree,2,2

The screen capture above shows the network topology in the ONOS web user interface.

Install Mininet dashboard to visualize the network traffic. The screen capture above shows a large flow over the same topology being displayed by ONOS, see Mininet weathermap for more examples.

In this case, the traffic was created by the following Mininet command:

mininet-onos> iperf h1 h3

The screen capture above shows top flows, busiest switch ports, and the diameter of the network topology.

The Mininet dashboard is a simple application running on the sFlow-RT analytics platform. For a more realistic example, watch the demonstration of SDN leaf and spine traffic engineering recorded at the Open Networking Summit. In the demonstration, a redundant pair of ONOS controllers implement segment routing, using OpenFlow 1.3 to control an eight switch leaf and spine network of commodity switches. Real-time flow analytics drives the dashboards in the demonstration and trigger load balancing of flows across the fabric. Leaf and spine traffic engineering using segment routing and SDN provides a more detailed explanation.

Mininet with ONOS and sFlow-RT is a great way to rapidly develop and test SDN applications, avoiding the time and expense involved in setting up a physical network.

OVS Orbit podcast with Ben Pfaff

OVS Orbit Episode 6 is a wide ranging discussion between Ben Pfaff and Peter Phaal of the industry standard sFlow measurement protocol, implementation of sFlow in Open vSwitch, network analytics use cases and application areas supported by sFlow, including: OpenStack, Open Network Virtualization (OVN), DDoS mitigation, ECMP load balancing, Elephant and Mice flows, Docker containers, Network Function Virtualization (NFV), and microservices.

Follow the link to see listen to the podcast, read the extensive show notes, follow related links, and to subscribe to the podcast.

OpenFlow integration

Northbound APIs for traffic engineering describes how sFlow and OpenFlow provide complementary monitoring and control capabilities that can be combined to create software defined networking (SDN) solutions that automatically adapt the network to changing traffic and address high value use cases such as: DDoS mitigation, enforcing black lists, ECMP load balancing, and packet brokers.

The article describes the challenge of mapping between the different methods used by sFlow and OpenFlow to identify switch ports:

• Agent IP address ⟷ OpenFlow switch ID
• SNMP ifIndex ⟷ OpenFlow port ID

The recently published sFlow OpenFlow Structures extension addresses the challenge by providing a way for switches to export the mapping as an sFlow structure.

The Open vSwitch recently implemented the extension, unifying visibility and control of the virtual network edge. In addition, most physical that support OpenFlow also support sFlow. Ask vendors about their plans to implement the sFlow OpenFlow Structures extension since it is a key enabler for SDN control applications.

Hybrid OpenFlow ECMP testbed

SDN fabric controller for commodity data center switches describes how the real-time visibility and hybrid control capabilities of commodity data center switches can be used to automatically adapt the network to changing traffic patterns and optimize performance. The article identifies hybrid OpenFlow as a critical component of the solution, allowing SDN to be combined with proven distributed routing protocols (e.g. BGP, ISIS, OSPF, etc) to deliver scaleable, production ready solutions that fully leverage the capabilities of commodity hardware.

This article will take the example of large flow marking that has been demonstrated using physical switches and show how Mininet can be used to emulate hybrid control of data center networks and deliver realistic results.

The article Elephant Detection in Virtual Switches & Mitigation in Hardware describes a demonstration by VMware and Cumulus Networks that shows how real-time detection and marking of large "Elephant" flows can dramatically improve application response time for small latency sensitive "Mouse" flows without impacting the throughput of the Elephants - see Marking large flows for additional background.

Performance optimizing hybrid OpenFlow controller demonstrated how hybrid OpenFlow can be used to mark Elephant flows on a top of rack switch. However, building test networks with physical switches to test the controller with realistic topologies is expensive and time consuming.

Mininet offers an attractive alternative, providing a lightweight network emulator that can be run in a virtual machine on a laptop and realistically simulate network topologies. In this example, Mininet will be used to emulate the four switch leaf and spine network shown in the diagram at the top of this article.

The sFlow-RT SDN controller includes a leafandspine.py script that configures Mininet to emulate ECMP leaf and spine fabrics with hybrid OpenFlow capable switches. To run the emulation, copy the leafandspine.py script from the sFlow-RT extras directory to your Mininet system and run the following command to create the leaf and spine network:

sudo ./leafandspine.py --collector=10.0.0.162 --controller=10.0.0.162 --topofile
=/var/www/html/topology.json

There are a few points to note about the emulation:

1. While physical networks might have link speeds ranging from 1Gbit/s to 100Gbit/s, the emulation scales link speeds down to 10Mbit/s so that they can be emulated in software.
2. The sFlow sampling rate is scaled proportionally - see Large flow detection
3. A pair of OpenFlow 1.3 tables is used to emulate normal ECMP forwarding and hybrid OpenFlow overrides
4. Linux Traffic Control (tc) commands are used to emulate hardware priority queueing based on Differentiated Services Code Point (DSCP) class, mapping DSCP class 8 to a lower priority or "less than best effort" queue.
5. The script posts the topology as a JSON file under the default Apache document root so that it can be retrieved remotely by an SDN controller
6. In this example the sFlow-RT controller is running on host 10.0.0.162 - change the address to match your setup.

The following script runs the ping command to test response time and plots the results as a simple text-based bar chart:

#!/bin/bash
SCALE=1
SCALE=${2-$SCALE}
ping $1 | awk -v SCALE=$SCALE 'BEGIN {FS="[= ]"; } NF==11 { n = $10 * SCALE; bar
 = ""; while(n >= 1) { bar = bar "*"; n--} print bar " " $10 " ms" }

Open and xterm on host h1 and run the command:

./pingtest 10.0.1.1 10

Next type the following command at the Mininet prompt to generate a large flow:

iperf h2 h3

The following screen capture shows the result of the iperf test:

The reported throughput of around the 10Mbit/s shows that the traffic is saturating the emulated 10Mbit/s links.

The following screen capture shows the ping results during the iperf test.

The ping test clearly shows the impact that the Elephant flow is having on response time. In addition, the increased response times of around 3ms are consistent with values shown in VMware / Cumulus Networks charts shown earlier.

The following sFlow-RT mark.js script implements an SDN controller that marks Elephant flows:

include('extras/leafandspine-hybrid.js');

// get topology from Mininet
setTopology(JSON.parse(http('http://10.0.0.30/topology.json')));

// Define large flow as greater than 1Mbits/sec for 1 second or longer
var bytes_per_second = 1000000/8, duration_seconds = 1;

// define TCP flow cache
setFlow('tcp',
 {keys:'ipsource,ipdestination,tcpsourceport,tcpdestinationport', filter:'direct
ion=ingress',
  value:'bytes', t:duration_seconds}
);

// set threshold identifying Elephant flows
setThreshold('elephant', {metric:'tcp', value:bytes_per_second, byFlow:true, tim
eout:4});

// set OpenFlow marking rule when Elephant is detected
var idx = 0;
setEventHandler(function(evt) {
 if(topologyInterfaceToLink(evt.agent,evt.dataSource)) return;
 var port = ofInterfaceToPort(evt.agent,evt.dataSource);
 if(port) {
  var dpid = port.dpid;
  var id = "mark" + idx++;
  var k = evt.flowKey.split(',');
  var rule= {
    match:{in_port: port.port, dl_type:2048, ip_proto:6, nw_src:k[0], nw_dst:k[1], tcp_src:k[2], tcp_dst:k[3]},
    actions:["set_ip_dscp=8","output=normal"], priority:1000, idleTimeout:5
  };
  logInfo(JSON.stringify(rule,null,1));
  setOfRule(dpid,id,rule);
 }
},['elephant']);

About the script:

1. The included leafandspine-hybrid.js script emulates hybrid OpenFlow by rewriting the NORMAL OpenFlow action to jump to the table that contains the ECMP forwarding rules. 
2. The script assumes that Mininet emulation is running on host 10.0.0.30. Modify the address in the setTopology() function for your setup.
3. The setFlow() function instructs the controller build a flow cache to track TCP connections
4. The setThreshold() function defines Elephant flows as TCP connections that exceed 10% of the link's bandwidth (in this case 1Mbit/second) for 1 second or more.
5. The setEventHandler() function processes each Elephant flow notification and applies an OpenFlow marking rules to the ingress port on the edge switch where the traffic enters the fabric.
6. The OpenFlow rules have an idleTimeout of 5 seconds, ensuring that they are automatically deleted by the switch when the flow ends.

Modify the sFlow-RT start.sh script to include the following settings:

-Dopenflow.start=yes 
-Dopenflow.flushRules=no
-Dscript.file=mark.js

Start sFlow-RT:

./start.sh

Repeat the iperf test.

The iperf results show that throughput of large flows is unaffected by the controller.

The screen capture shows the controller actions. The controller installs an OpenFlow rule as soon as the large flow is detected, settings the ip_dscp value to 8 and outputting the packets to the normal ECMP forwarding pipeline. The marked packets are treated as lower priority than the ping packets. Since the ping packets aren't stuck behind the deep queues caused by the iperf tests, the reported response times should be unaffected by the large flow.

The ping test confirms that with the controller running, response times are unaffected by Elephant flows, an approximately 10 times improvement in response time that is consistent with the results shown for a physical switch in the VMware / Cumulus charts shown earlier.

More broadly, hybrid OpenFlow provides an effective way to deliver SDN solutions in production, using OpenFlow to enhance the performance of existing networks. In addition to large flow marking, other cases described on this blog include: DDoS mitigation,  enforcing black lists, ECMP load balancing, and packet brokers.

Increasingly, vendors recognize the critical importance of hybrid OpenFlow in delivering practical SDN solutions - HP proposes hybrid OpenFlow discussion at Open Daylight design forum. The article Super NORMAL offers some suggestions for enhancing hybrid OpenFlow to address additional use cases, reduce operational complexity and increase reliability in production settings.

Finally, the sFlow measurement standard is critical to unlocking the full potential of hybrid OpenFlow. Support for sFlow is build into commodity switch hardware, providing cost effective visibility into traffic on production networks. The comprehensive real-time traffic analytics delivered by sFlow allows an SDN controller to effectively target actions, managing the limited hardware resources on the switches, to enhance network performance and security.

Open vSwitch 2014 Fall Conference

Open vSwitch is an open source software virtual switch that is popular in cloud environments such as OpenStack. Open vSwitch is a standard Linux component that forms the basis of a number of commercial and open source solutions for network virtualization, tenant isolation, and network function virtualization (NFV) - implementing distributed virtual firewalls and routers.

The recent Open vSwitch 2014 Fall Conference agenda included a wide variety speakers addressing a range of topics, including: large scale operation experiences at Rackspace, implementing stateful firewalls, Docker networking,  and acceleration technologies (Intel DPDK and Netmap/VALE).

The video above is a recording of the following sFlow related talk from the conference:

Traffic visibility and control with sFlow (Peter Phaal, InMon)
sFlow instrumentation has been included in Open vSwitch since version 0.99.1 (released 25 Jan 2010). This talk will introduce the sFlow architecture and discuss how it differs from NetFlow/IPFIX, particularly in regards to delivering real-time flow analytics to an SDN controller. The talk will demonstrate that sFlow measurements from Open vSwitch are identical to sFlow measurements made in hardware on bare metal switches, providing unified, end-to-end, measurement across physical and virtual networks. Finally, Open vSwitch / Mininet will be used to demonstrate Elephant flow detection and marking using a combination of sFlow and OpenFlow.

Slides and videos for all the conference talks will soon be available on the Open vSwitch web site.

Super NORMAL

KennyK/Shutterstock

HP proposes hybrid OpenFlow discussion at Open Daylight design forum describes some of the benefits of integrated hybrid OpenFlow and the reasons why the OpenDaylight community would be a good venue for addressing operational and multi-vendor interoperability issues relating to hybrid OpenFlow.

HP's slide presentation from the design forum, OpenFlow-hybrid Mode, gives an overview of hybrid mode OpenFlow and its benefits. The advantage of hybrid mode in leveraging the proven scaleability and operational robustness of existing distributed control mechanisms and complementing them with centralized SDN control is compelling and a number of vendors have released support, including: Alcatel Lucent Enterprise, Brocade, Extreme, Hewlett-Packard, Mellanox, and Pica8. HP's presentation goes on to propose enhancements to the OpenDaylight controller to support hybrid OpenFlow agents.

InMon recently built a hybrid OpenFlow controller and, based on our experiences, this article will discuss how integrated hybrid mode is currently implemented on the switches, examine operational issues, and propose an agent profile for hybrid OpenFlow designed to reduce operational complexity, particularly when addressing traffic engineering use cases such as DDoS mitigation, large flow marking and large flow steering on ECMP/LAG networks.

Mechanisms for Optimizing LAG/ECMP Component Link Utilization in Networks is an IETF Internet Draft, authored by Brocade, Dell, Huawei, Tata, and ZTE that discussed the benefits and operational challenges of the flow steering use case. In particular:

6.2. Handling Route Changes
Large flow rebalancing must be aware of any changes to the FIB.  In cases where the nexthop of a route no longer to points to the LAG, or to an ECMP group, any PBR entries added as described in Section 4.4.1 and 4.4.2 must be withdrawn in order to avoid the creation of forwarding loops. 

The essential feature of hybrid OpenFlow is that it leverages the capabilities of existing routing, switching and link state mechanisms to handle traffic without controller intervention. The controller only needs to install rules when it wants to override the default behavior. However, hybrid OpenFlow, as currently implemented, does not fully integrate with the on-switch control plane, resulting in complex and unpredictable behavior that is hard to align with forwarding policy established through the on-switch control plane (BGP, ISIS, LACP, etc), particularly when steering flows.

In order to best understand the challenges, it is worth taking a look at the architecture of an OpenFlow agent.

Figure 1: OpenFlow 1.3 switch

Figure 1 shows the functional elements of an OpenFlow 1.3 agent. Multiple tables in the Data Plane are exposed through OpenFlow to the OpenFlow controller. Packets entering the switch pass from table to table, matching different packet headers. If there is no match, the packet is discarded, if there is a match, an associated set of actions is applied to the packet, typically forwarding the packet to a specific egress port on the switch. The key to hybrid OpenFlow is the NORMAL action:

Optional: NORMAL: Represents the traditional non-OpenFlow pipeline of the switch (see 5.1). Can be used only as an output port and processes the packet using the normal pipeline. If the switch cannot forward packets from the OpenFlow pipeline to the normal pipeline, it must indicate that it does not support this action.

With integrated hybrid OpenFlow, the agent is given a low priority default rule that matches all packets and applies an action to send them to the NORMAL port (i.e. apply forwarding rules determined by the switch's control plan). There are two ways that vendors have chosen to install this rule:

1. Explicit The controller is responsible for installing the default NORMAL rule when the switch connects to it.
2. Implicit The switch is configured to operate in integrated hybrid mode and behaves as if the default NORMAL rule was installed.

HP's OpenDaylight presentation describes enhancements to the OpenDaylight controller required to support the explicit hybrid OpenFlow configuration:

The controller would send a default rule which tells the switch to forward packets to the
NORMAL port. This rule delegates the forwarding decision to the controlled switches, but it means that the controller would receive ZERO packet_in messages if no other rules were pushed. For this reason, we’d put this rule at priority 0 in the last hardware OF table of the pipeline. Without this rule, the default behavior for OF 1.0 is to steal to the controller and the default behavior for OF 1.3 is to drop all packets.

Note: Integrated hybrid OpenFlow control of HP switches provides a simple example demonstrating integration between InMon's controller and HP switches.

Explicit configuration requires that the controller understand each vendor's forwarding pipeline and deploy an appropriate default rule. The implicit method supported by other vendors (e.g. Brocade, Alcatel Lucent Enterprise) is much simpler since the vendor takes responsibility for applying the default NORMAL rule at the appropriate point in the pipeline.

The implicit method also has a number of operational advantages:

1. The rule exists at startup In the implicit case the switch will forward normally before the switch connects to a controller and the switch will successfully forward packets if the controller is down or fails. In the explicit case the switch will drop all traffic on startup and continue to drop traffic if it can't connect to the controller and get the NORMAL rule. 
2. The rule cannot be deleted In the implicit case the default NORMAL isn't visible to the controller and can't be accidentally deleted (which would disable all forwarding on the switch). In the explicit case, the OpenFlow controller must add the rule and it may be accidentally deleted by an SDN application.
3. The agent knows its in hybrid mode In the implicit case the switch is responsible for adding the default rule and knows its in hybrid mode. In the explicit case, there switch would need to examine the rules that the controller had inserted and try and infer the correct behavior. As we'll see later, the switch must be able to differentiate between hybrid mode and pure OpenFlow mode in order to trigger more intelligent behavior.

However, even in the implicit case, there are significant challenges with integrated hybrid OpenFlow as it is currently implemented. The main problem is that the demarcation of responsibility between the NORMAL forwarding logic and the OpenFlow controller isn't clearly specified. For example, a use case described in Mechanisms for Optimizing LAG/ECMP Component Link Utilization in Networks:

Within a LAG/ECMP group, the member component links with least average port utilization are identified.  Some large flow(s) from the heavily loaded component links are then moved to those lightly-loaded member component links using a policy-based routing (PBR) rule in the ingress processing element(s) in the routers.

Figure 2, from the OpenDaylight Dynamic Flow Management proposal expands on the SDN controller architecture for global large flow load balancing:

Figure 2: Large Flow Global Load Balancing

Suppose that the controller has detected a large flow collision and constructs the following OpenFlow rule to direct one of the flows to a different port:

node:{id:'00:00:00:00:00:00:00:01', type:'OF'},
etherType:'0x0800',
nwSrc: '10.0.0.1', nwDst: '10.1.10.2',
protocol: '6', tpSrc: '42344', tpDst: '80'
actions:['OUTPUT=2']

The rule will fail to have the desired effect because the NORMAL control plane in this network is ECMP routing. Successfully sending the packet on port 2 so that it reaches its destination and doesn't interfere with the NORMAL forwarding protocols requires that the layer 2 headers be rewritten to set the VLAN to match port 2's VLAN, set the destination MAC address to match the next hop router's MAC address, the source MAC address to match port 2's MAC address, and finally decrementing the IP TTL.

node:{id:'00:00:00:00:00:00:00:01', type:'OF'},
etherType:'0x0800',
nwSrc: '10.0.0.1', nwDst: '10.1.10.2',
protocol: '6', tpSrc: '42344', tpDst: '80'
actions:[
'setDlSrc='00:04:00:00:00:02',
'setDlDst='00:04:00:00:02:02',
'setVLAN='1',
'decNwTTL',
'OUTPUT=2']

These additional actions involve information that is already known to the NORMAL control plane and which is difficult for the SDN controller to know. It gets even more complicated if you want to take routing and link state into account. The selected port may not represent a valid route, or the link may be down. In addition, routes may change and a rule that was once valid may become invalid and so must be removed (see 6.2. Handling Route Changes above).

Exposing hardware details makes sense if the external controller is responsible for all forwarding decisions (i.e. a pure OpenFlow environment). However, in a hybrid environment the NORMAL control plane is already populating the tables and the external controller should not need to concern itself with the hardware details.

Figure 3: Super NORMAL hybrid OpenFlow switch

Figure 3 proposes an alternative model for implementing integrated hybrid OpenFlow. It is referred to as "Super NORMAL" because it recognizes that the switch's forwarding agent is already managing the physical resources in the data plane and that the goal of integrated hybrid OpenFlow is integration with the forwarding agent, not direct control of the forwarding hardware. In this model a single OpenFlow table is exposed by the forwarding agent with keys and actions that can be composed with the existing control plane. In essence, the OpenFlow protocol is being used to manage forwarding policy, expressed as an OpenFlow table,  that is read by the Forwarding Agent and used to influence forwarding behavior.

Figure 4: SDN fabric controller for commodity data center switches

This model fits well with the hardware architecture, shown in Figure 4, of merchant silicon ASICs used in most current generation data center switches. The NORMAL control plane populates most of the tables in the ASIC and the forwarding agent can apply OpenFlow rules to the ACL Policy Flow Table to override default behavior. Many existing OpenFlow implementations are already very close to this model, but lack the integration needed to compose the OpenFlow rules with their forwarding method. The following enhancements to the hybrid OpenFlow agent would greatly improve the utility of hybrid OpenFlow:

1. Implement implicit default NORMAL behavior
2. Never generate Packet-In events (a natural result of implementing 1. above)
3. Support NORMAL output action
4. Expose a single table with matches and actions that are valid and compose with the configured forwarding protocol(s)
5. Reject rules that are not valid options according to the NORMAL control plane:
   • if the NORMAL output would send a packet to a LAG and the specified port is not a member of the LAG, then the rule must be rejected.
   • if the NORMAL output would send a packet to an ECMP group and the specified port is not a member of the group then the rule must be rejected.
   • if the specified port is down then the rule must be rejected
   • if the rule cannot be fully implemented in the hardware data plane, then the rule must be rejected
6. Remove rules that are no longer valid and send a flow removed message to the controller. A flow is not valid if it would be rejected (e.g. if a port goes down, rules directing traffic to that port must be immediately removed)
7. Automatically add any required details needed to forward the traffic (e.g. rewrite source and destination mac addresses and decrement IP TTL if the packet is being routed)

Hybrid control of forwarding is the most complex operation and requires Super NORMAL functionality. Simpler operations such as blocking traffic or QoS marking are easily handled by the output DROP and NORMAL actions and solutions based on hybrid OpenFlow have been demonstrated:

• Alcatel-Lucent Enterprise: ALUE Demonstrates Practical SDN Use Cases, Joins sFlow.org
• Brocade: Brocade Crowned Winner of SDN Idol 2014 at Open Networking Summit 2014

Understanding the distinct architectural differences between hybrid and pure OpenFlow implementations is essential to get the most out of each approach to SDN. Pure OpenFlow is still an immature technology with limited applications. On the other hand, Hybrid OpenFlow works well with commodity switch hardware, leverages mature control plane protocols, and delivers added value in production networks.

HP proposes hybrid OpenFlow discussion at Open Daylight design forum

Hewlett-Packard, an Open Daylight platinum member, is proposing a discussion of integrated hybrid OpenFlow at the upcoming Open Daylight Developer Design Forum, September 29 - 30, 2014, Santa Clara.

Topics for ODL Design Summit from HP contains the following proposal, making the case for integrated hybrid OpenFlow:

We would like to share our experiences with Customer SDN deployments that require OpenFlow hybrid mode. Why it matters, implementation considerations, and how to achieve better support for it in ODL

OpenFlow-compliant switches come in two types: OpenFlow-only, and OpenFlow-hybrid. OpenFlow-only switches support only OpenFlow operation, in those switches all packets are processed by the OpenFlow pipeline, and cannot be processed otherwise. OpenFlow-hybrid switches support both OpenFlow operation and normal Ethernet switching operation, i.e. traditional L2 Ethernet switching, VLAN isolation, L3 routing (IPv4 routing, IPv6 routing...), ACL and QoS processing

The rationale for supporting hybrid mode is twofold:

1. Controlled switches have decades of embedded traditional networking logic. The controller does not add value to a solution if it replicates traditional forwarding logic. One alternative controller responsibility is that provides forwarding decisions when it wants to override the traditional data-plane forwarding decision.
2. Controllers can be gradually incorporated into a traditional network. The common approach to enterprise SDN assumes a 100% pure SDN-controlled solution from the ground-up. This approach is expensive in terms of actual cost of new switches and in terms of downtime of the network. By providing a controller that can gradually migrate to an SDN solution, the hybrid approach enables customers to start seeing the value of having an SDN controller without requiring them to make a huge leap in replacing their existing network.

The Open Networking Foundation (ONF), the body behind the OpenFlow standard, released Outcomes of the Hybrid Working Group in March 2013, concluding:

On the whole, the group determined that industry can address many of the issues related to the hybrid switch. ONF does not plan or intend to incorporate details of legacy protocols in OpenFlow. The priority of ONF in this context is to explore the migration of networks to OpenFlow.

OpenDaylight has broad industry participation and should be a good forum to discuss integrated hybrid OpenFlow use cases, enhance open source controller support, and address multi-vendor interoperability. HP should find support for integrated hybrid OpenFlow among Open Daylight members:

• Brocade - Brocade has been shipping products supporting hybrid OpenFlow for over a year, see The Practical Path to SDN: Brocade OpenFlow Hybrid Port Mode. Brocade won the 2014 ONS SDN Idol competition with a solution for DDoS mitigation that combined their hybrid OpenFlow switches with the Open Daylight controller and InMon's real-time sFlow analytics software to address the challenge of mitigating large DDoS flood attacks - see ONS2014 SDN Idol finalist demonstrations
• Red Hat - Red Hat's Brent Salisbury is an OpenDaylight software developer and advocate for hybrid OpenFlow, see Hybrid OpenFlow Using The Normal Action
• Juniper - advocated for integrated hybrid OpenFlow at the 2012 ONF Summit, see Hybrid Programmable Forwarding Plane

SDN fabric controller for commodity data center switches discusses a number of use cases where an SDN controller can leverage the hardware capabilities of commodity switches through industry standard sFlow and hybrid OpenFlow protocols.

Integrated hybrid OpenFlow is a practical method for rapidly creating and deploying compelling SDN solutions at scale in production networks. It's encouraging to see HP engaging the Open Daylight community to deliver solutions based on hybrid OpenFlow - hopefully their proposal will find the broad support it deserves and accelerate market adoption of hybrid OpenFlow based SDN.

Update October 8, 2014: Slides from the summit are available, OpenFlow-hybrid Mode

Mininet integrated hybrid OpenFlow testbed

Figure 1: Hybrid Programmable Forwarding Planes

Integrated hybrid OpenFlow combines OpenFlow and existing distributed routing protocols to deliver robust software defined networking (SDN) solutions. Performance optimizing hybrid OpenFlow controller describes how the sFlow and OpenFlow standards combine to deliver visibility and control to address challenges including: DDoS mitigation, ECMP load balancing, LAG load balancing, and large flow marking.

A number of vendors support sFlow and integrated hybrid OpenFlow today, examples described on this blog include: Alcatel-Lucent, Brocade, and Hewlett-Packard. However, building a physical testbed is expensive and time consuming. This article describes how to build an sFlow and hybrid OpenFlow testbed using free Mininet network emulation software. The testbed emulates ECMP leaf and spine data center fabrics and provides a platform for experimenting with analytics driven feedback control using the sFlow-RT hybrid OpenFlow controller.

First build an Ubuntu 13.04 / 13.10 virtual machine then follow instructions for installing Mininet - Option 3: Installation from Packages.

Next, install an Apache web server:

sudo apt-get install apache2

Install the sFlow-RT integrated hybrid OpenFlow controller, either on the Mininet virtual machine, or on a different system (Java 1.6+ is required to run sFlow-RT):

wget http://www.inmon.com/products/sFlow-RT/sflow-rt.tar.gz
tar -xvzf sflow-rt.tar.gz

Copy the leafandspine.py script from the sflow-rt/extras directory to the Mininet virtual machine.

The following options are available:

./leafandspine.py --help
Usage: leafandspine.py [options]

Options:
  -h, --help            show this help message and exit
  --spine=SPINE         number of spine switches, default=2
  --leaf=LEAF           number of leaf switches, default=2
  --fanout=FANOUT       number of hosts per leaf switch, default=2
  --collector=COLLECTOR
                        IP address of sFlow collector, default=127.0.0.1
  --controller=CONTROLLER
                        IP address of controller, default=127.0.0.1
  --topofile=TOPOFILE   file used to write out topology, default topology.txt

Figure 2 shows a simple leaf and spine topology consisting of four hosts and four switches:

Figure 2: Simple leaf and spine topology

The following command builds the topology and specifies a remote host (10.0.0.162) running sFlow-RT as the hybrid OpenFlow controller and sFlow collector:

sudo ./leafandspine.py --collector 10.0.0.162 --controller 10.0.0.162 --topofile /var/www/topology.json

Note: All the links are configured to 10Mbit/s and the sFlow sampling rate is set to 1-in-10. These settings are equivalent to a 10Gbit/s network with a 1-in-10,000 sampling rate - see Large flow detection.

The network topology is written to  /var/www/topology.json making it accessible through HTTP. For example, the following command retrieves the topology from the Mininet VM (10.0.0.61):

curl http://10.0.0.61/topology.json
{"nodes": {"s3": {"ports": {"s3-eth4": {"ifindex": "392", "name": "s3-eth4"}, "s3-eth3": {"ifindex": "390", "name": "s3-eth3"}, "s3-eth2": {"ifindex": "402", "name": "s3-eth2"}, "s3-eth1": {"ifindex": "398", "name": "s3-eth1"}}, "tag": "edge", "name": "s3", "agent": "10.0.0.61", "dpid": "0000000000000003"}, "s2": {"ports": {"s2-eth1": {"ifindex": "403", "name": "s2-eth1"}, "s2-eth2": {"ifindex": "405", "name": "s2-eth2"}}, "name": "s2", "agent": "10.0.0.61", "dpid": "0000000000000002"}, "s1": {"ports": {"s1-eth1": {"ifindex": "399", "name": "s1-eth1"}, "s1-eth2": {"ifindex": "401", "name": "s1-eth2"}}, "name": "s1", "agent": "10.0.0.61", "dpid": "0000000000000001"}, "s4": {"ports": {"s4-eth2": {"ifindex": "404", "name": "s4-eth2"}, "s4-eth3": {"ifindex": "394", "name": "s4-eth3"}, "s4-eth1": {"ifindex": "400", "name": "s4-eth1"}, "s4-eth4": {"ifindex": "396", "name": "s4-eth4"}}, "tag": "edge", "name": "s4", "agent": "10.0.0.61", "dpid": "0000000000000004"}}, "links": {"s2-eth1": {"ifindex1": "403", "ifindex2": "402", "node1": "s2", "node2": "s3", "port2": "s3-eth2", "port1": "s2-eth1"}, "s2-eth2": {"ifindex1": "405", "ifindex2": "404", "node1": "s2", "node2": "s4", "port2": "s4-eth2", "port1": "s2-eth2"}, "s1-eth1": {"ifindex1": "399", "ifindex2": "398", "node1": "s1", "node2": "s3", "port2": "s3-eth1", "port1": "s1-eth1"}, "s1-eth2": {"ifindex1": "401", "ifindex2": "400", "node1": "s1", "node2": "s4", "port2": "s4-eth1", "port1": "s1-eth2"}}}

Don't start sFlow-RT yet, it should only be started after Mininet has finished building the topology.

Verify connectivity before starting sFlow-RT:

mininet> pingall
*** Ping: testing ping reachability
h1 -> h2 h3 h4 
h2 -> h1 h3 h4 
h3 -> h1 h2 h4 
h4 -> h1 h2 h3 
*** Results: 0% dropped (12/12 received)

This test demonstrates that the Mininet topology has been constructed with a set of default forwarding rules that provide connectivity without the need for an OpenFlow controller - emulating the behavior of  a network of integrated hybrid OpenFlow switches.

The following sFlow-RT script ecmp.js demonstrates ECMP load balancing in the emulated network:

// Define large flow as greater than 1Mbits/sec for 1 second or longer
var bytes_per_second = 1000000/8;
var duration_seconds = 1;

var top = JSON.parse(http("http://10.0.0.61/topology.json"));
setTopology(top);

setFlow('tcp',
 {keys:'ipsource,ipdestination,tcpsourceport,tcpdestinationport',
  value:'bytes', t:duration_seconds}
);

setThreshold('elephant',
 {metric:'tcp', value:bytes_per_second, byFlow:true, timeout:2}
);

setEventHandler(function(evt) {
 var rec = topologyInterfaceToLink(evt.agent,evt.dataSource);
 if(!rec || !rec.linkname) return;
 var link = topologyLink(rec.linkname);
 logInfo(link.node1 + "-" + link.node2 + " " + evt.flowKey);
},['elephant']);

Modify the sFlow-RT start.sh script to include the following arguments:

RT_OPTS="-Dopenflow.start=yes -Dopenflow.flushRules=no"
SCRIPTS="-Dscript.file=ecmp.js"

Some notes on the script:

1. The topology is retrieved by making an HTTP request to the Mininet VM (10.0.0.61)
2. The 1Mbits/s threshold for large flows was selected because it represents 10% of the bandwidth of the 10Mbits/s links in the emulated network
3. The event handler prints the link the flow traversed - identifying the link by the pair of switches it connects

Start sFlow-RT:

./start.sh

Now generate some large flows between h1 and h3 using the Mininet iperf command:

mininet> iperf h1 h3
*** Iperf: testing TCP bandwidth between h1 and h3
*** Results: ['9.58 Mbits/sec', '10.8 Mbits/sec']
mininet> iperf h1 h3
*** Iperf: testing TCP bandwidth between h1 and h3
*** Results: ['9.58 Mbits/sec', '10.8 Mbits/sec']
mininet> iperf h1 h3
*** Iperf: testing TCP bandwidth between h1 and h3
*** Results: ['9.59 Mbits/sec', '10.3 Mbits/sec']

The following results were logged by sFlow-RT:

2014-04-21T19:00:36-0700 INFO: ecmp.js started
2014-04-21T19:01:16-0700 INFO: s1-s3 10.0.0.1,10.0.1.1,49240,5001
2014-04-21T19:01:16-0700 INFO: s1-s4 10.0.0.1,10.0.1.1,49240,5001
2014-04-21T20:53:19-0700 INFO: s2-s4 10.0.0.1,10.0.1.1,49242,5001
2014-04-21T20:53:19-0700 INFO: s2-s3 10.0.0.1,10.0.1.1,49242,5001
2014-04-21T20:53:29-0700 INFO: s1-s3 10.0.0.1,10.0.1.1,49244,5001
2014-04-21T20:53:29-0700 INFO: s1-s4 10.0.0.1,10.0.1.1,49244,5001

The results demonstrate that the emulated leaf and spine network is performing equal cost multi-path (ECMP) forwarding - different flows between the same pair of hosts take different paths across the fabric (the highlighted lines correspond to the paths shown in Figure 2).

Open vSwitch in Mininet is the key to this emulation, providing sFlow and multi-path forwarding support 

The following script implements the large flow marking example described in Performance optimizing hybrid OpenFlow controller:

include('extras/leafandspine-hybrid.js');

// Define large flow as greater than 1Mbits/sec for 1 second or longer
var bytes_per_second = 1000000/8;
var duration_seconds = 1;

var idx = 0;

var top = JSON.parse(http("http://10.0.0.61/topology.json"));
setTopology(top);

setFlow('tcp',
 {keys:'ipsource,ipdestination,tcpsourceport,tcpdestinationport',
  value:'bytes', t:duration_seconds}
);

setThreshold('elephant',
 {metric:'tcp', value:bytes_per_second, byFlow:true, timeout:4}
);

setEventHandler(function(evt) {
 var agent = evt.agent;
 var ds = evt.dataSource;
 if(topologyInterfaceToLink(agent,ds)) return;

 var port = ofInterfaceToPort(agent,ds);
 if(port) {
  var dpid = port.dpid;
  var id = "mark" + idx++;
  var k = evt.flowKey.split(',');
  var rule= {
    priority:1000, idleTimeout:2,
    match:{eth_type:2048, ip_proto:6, ip_src:k[0], ip_dst:k[1],
           tcp_src:k[2], tcp_dst:k[3]},
    actions:["set_ip_dscp=32","output=normal"]
  };
  setOfRule(dpid,id,rule);
 }
},['elephant']);

setFlow('tos0',{value:'bytes',filter:'ipdscp=0',t:1});
setFlow('tos128',{value:'bytes',filter:'ipdscp=32',t:1});

Some notes on the script:

1. The topologyInterfaceToLink() function looks up link information based on agent and interface. The event handler uses this function to exclude inter-switch links, applying controls to ingress ports only.
2. The OpenFlow rule priority for rules created by controller scripts must be greater than 500 to override the default rules created by leafandspine.py
3. The tos0 and tos128 flow definitions have been added to so that the re-marking can be seen.

Restart sFlow-RT with the new script and use a web browser to view the default tos0 and the re-marked tos128 traffic.

Figure 3: Marking large flows

Use iperf to generate traffic between h1 and h3 (the traffic needs to cross more than one switch so it can be observed before and after marking). The screen capture in figure 3 demonstrates that the controller immediately detects and marks large flows.

Integrated hybrid OpenFlow control of HP switches

Performance optimizing hybrid OpenFlow controller describes InMon's sFlow-RT controller. The controller makes use of the sFlow and OpenFlow standards and is optimized for real-time traffic engineering applications that managing large traffic flows, including: DDoS mitigation, ECMP load balancing, LAG load balancing, large flow marking etc.

The previous article provided an example of large flow marking using an Alcatel-Lucent OmniSwitch 6900 switch. This article discusses how to replicate the example using HP Networking switches.

At present, the following HP switch models are listed as having OpenFlow support:

• FlexFabric 12900 Switch Series
• 12500 Switch Series
• FlexFabric 11900 Switch Series
• 8200 zl Switch Series
• HP FlexFabric 5930 Switch Series
• 5920 Switch Series
• 5900 Switch Series
• 5400 zl Switch Series
• 3800 Switch Series
• HP 3500 and 3500 yl Switch Series
• 2920 Switch Series 

Note: All of the above HP switches (and many others) support the sFlow standard - see sFlow Products: Network Equipment @ sFlow.org.

HP's OpenFlow implementation supports integrated hybrid mode - provided the OpenFlow controller pushes a default low priority OpenFlow rule that matches all packets and applies the NORMAL action (i.e. instructs the switch to apply default switching / routing forwarding to the packets).

In this example, an HP 5400 zl switch is used to run a slightly modified version of the sFlow-RT controller JavaScript application described in Performance optimizing hybrid OpenFlow controller:

// Define large flow as greater than 100Mbits/sec for 0.2 seconds or longer
var bytes_per_second = 100000000/8;
var duration_seconds = 0.2;

var idx = 0;

setFlow('tcp',
 {keys:'ipsource,ipdestination,tcpsourceport,tcpdestinationport',
  value:'bytes', filter:'direction=ingress', t:duration_seconds}
);

setThreshold('elephant',
 {metric:'tcp', value:bytes_per_second, byFlow:true, timeout:2, 
  filter:{ifspeed:[1000000000]}}
);

setEventHandler(function(evt) {
 var agent = evt.agent;
 var ports = ofInterfaceToPort(agent);
 if(ports && ports.length == 1) {
  var dpid = ports[0].dpid;
  var id = "mark" + idx++;
  var k = evt.flowKey.split(',');
  var rule= {
   priority:500, idleTimeout:20,
   match:{dl_type:2048, nw_proto:6, nw_src:k[0], nw_dst:k[1],
          tp_src:k[2], tp_dst:k[3]},
   actions:["set_nw_tos=128","output=normal"]
  };
  setOfRule(dpid,id,rule);
 }
},['elephant']);

The idleTimeout was increased from 2 to 20 seconds since the switch has a default Probe Interval of 10 seconds (the interval between OpenFlow counter updates). If the OpenFlow rule idleTimeout is set shorter than the Probe Interval the switch will remove the OpenFlow rule before the flow ends.

Mar. 27, 2014 Update: The  HP Switch Software OpenFlow Administrator's Guide K/KA/WB 15.14, Appendix B Implementation Notes, describes the effect of the probe interval on idle timeouts and describes how to change the default (using openflow hardware statistics refresh rate) but warns that shorter refresh rates will increase CPU load on the switch.

The following command line arguments load the script and enable OpenFlow on startup:

-Dscript.file=ofmark.js \
-Dopenflow.start=yes \
-Dopenflow.addNormal=yes

The additional highlighted argument instructs the sFlow-RT controller to install the wild card OpenFlow NORMAL rule automatically when the switch connects.

The screen capture at the top of the page shows a mixture of small flows "mice" and large flows "elephants" generated by a server connected to the HP 5406 zl switch. The graph at the bottom right shows the mixture of unmarked traffic being sent to the switch. The sFlow-RT controller receives a stream of sFlow measurements from the switch and detects each elephant flows in real-time, immediately installing an OpenFlow rule that matches the flow and instructing the switch to mark the flow by setting the IP type of service bits. The traffic upstream of the switch is shown in the top right chart and it can be clearly seen that each elephant flow has been identified and marked, while the mice have been left unmarked.

Note: While this demonstration only used a single switch, the solution easily scales to hundreds of switches and thousands of edge ports.

The results from the HP switch are identical to those obtained with the Alcatel-Lucent switch, demonstrating the multi-vendor interoperability provided by the sFlow and OpenFlow standards. In addition, sFlow-RT's support for an open, standards based, programming environment (JavaScript / ECMAScript) makes it an ideal platform for rapidly developing and deploying traffic engineering SDN applications in existing networks.

Performance optimizing hybrid OpenFlow controller

The latest release of InMon's sFlow-RT controller adds integrated hybrid OpenFlow support - optimized for real-time traffic engineering applications that manage large traffic flows, including: DDoS mitigation, ECMP load balancing, LAG load balancing, large flow marking etc.

This article discusses the evolving architecture of software defined networking (SDN) and the role of analytics and traffic engineering. InMon's sFlow-RT controller is used to provide practical examples of the architecture.

Figure 1: Fabric: A Retrospective on Evolving SDN

The article, Fabric: A Retrospective on Evolving SDN by Martin Casado, Teemu Koponen, Scott Shenker, and Amin Tootoonchian, makes the case for a two tier software defined networking (SDN) architecture; comprising a smart edge and an efficient core. The article, Pragmatic software defined networking on this blog, examines how the edge is moving into virtual switches, with tunneling (VxLAN, NVGRE, GRE, STT) used to virtualize the network and decouple the edge from the core. As complex policy decisions move to the network edge, the core fabric is left with the task of efficiently managing physical resources in order to deliver low latency, high bandwidth connectivity between edge switches.

First generation SDN controllers were designed before the edge / core split became apparent and contain a complex set of features that limit their performance and scaleability. The sFlow-RT controller represents a second generation design, targeted specifically at the challenge of optimizing the performance of the physical network fabric.

Equal cost multi-path routing (ECMP), link aggregation (LAG) and multi-chassis link aggregation (MLAG) provide methods for spreading traffic across multiple paths in data center network fabrics in order to deliver the bandwidth needed to support big data and cloud workloads. However, these methods of load balancing do not handle all types of traffic well - in particular, long duration, high bandwidth "elephant" flows - see Of Mice and Elephants by Martin Casado and Justin Pettit with input from Bruce Davie, Teemu Koponen, Brad Hedlund, Scott Lowe, and T. Sridhar. The article SDN and large flows on this blog reviews traffic studies and discusses the benefits of actively managing large flows using an SDN controller.

The sFlow-RT controller makes use of two critical technologies that allow it to actively manage large flows in large scale production networks:

1. sFlow - The sFlow standard is widely supported by data center switch vendors and merchant silicon and provides the scaleability and rapid detection of large flows needed for effective real-time control.
2. Integrated hybrid OpenFlow - With integrated hybrid OpenFlow, the normal forwarding mechanisms in switches are the default for handling traffic (ECMP, LAG, MLAG), i.e. packets are never sent to the controller to make forwarding decisions. OpenFlow is used to selectively override default forwarding in order to improve performance. This approach is extremely scaleable and robust - there is minimal overhead associated with maintaining the OpenFlow connections between the controller and the switches, and the network will still continue to forward traffic if the controller fails. 

A demonstration using an older version of sFlow-RT with an embedded JavaScript DDoS mitigation application and an external OpenFlow controller (Open Daylight) won the SDN Idol competition at the 2014 Open Networking Summit - see Real-time SDN Analytics for DDoS mitigation for a video of the live demonstration and details of the competition.

Much of the complexity in developing SDN control applications involves sharing and distributing state between the analytics engine, application, and OpenFlow controller. Modifying the DDoS mitigation scripts presented on this blog to use sFlow-RT's embedded OpenFlow controller is straightforward and delivers the performance, scaleability and robustness needed to move the DDoS mitigation applications into production.

However, in this article, the interesting problem of detecting and marking elephant flows described in Of Mice and Elephants and Marking large flows will be used to demonstrate the sFlow-RT controller. Before looking at how sFlow-RT handles large flow marking, it is worth looking at existing approaches. The article, Elephant Flow Mitigation via Virtual-Physical Communication, on VMware's Network Virtualization blog describes how Open vSwitch was modified to include an "NSX Elephant agent" to detect large flows. When a large flow is detected, a notification is sent to the HP VAN SDN Controller that is running a special application that responds to the large flow notifications. The industry’s first east-west federated solution on HP's Networking blog further describes the solution, providing a demonstration in which large flows are given a different priority marking to small flows.

In contrast, the following sFlow-RT controller JavaScript application implements large flow marking:

// Define large flow as greater than 100Mbits/sec for 0.2 seconds or longer
var bytes_per_second = 100000000/8;
var duration_seconds = 0.2;

var idx = 0;

setFlow('tcp',
 {keys:'ipsource,ipdestination,tcpsourceport,tcpdestinationport',
  value:'bytes', filter:'direction=ingress', t:duration_seconds}
);

setThreshold('elephant',
 {metric:'tcp', value:bytes_per_second, byFlow:true, timeout:2, 
  filter:{ifspeed:[1000000000]}}
);

setEventHandler(function(evt) {
 var agent = evt.agent;
 var ports = ofInterfaceToPort(agent);
 if(ports && ports.length == 1) {
  var dpid = ports[0].dpid;
  var id = "mark" + idx++;
  var k = evt.flowKey.split(',');
  var rule= {
   priority:500, idleTimeout:2,
   match:{dl_type:2048, nw_proto:6, nw_src:k[0], nw_dst:k[1],
          tp_src:k[2], tp_dst:k[3]},
   actions:["set_nw_tos=128","output=normal"]
  };
  setOfRule(dpid,id,rule);
 }
},['elephant']);

The following command line arguments load the script and enable OpenFlow on startup:

-Dscript.file=ofmark.js -Dopenflow.start=yes

Some notes on the script:

1. The 100Mbits/s threshold for large flows was selected because it represents 10% of the bandwidth of the 1Gigabit access ports on the network
2. The setFlow filter specifies ingress flows since the goal is to mark flows as they enter the network
3. The setThreshold filter specifies that thresholds are only applied to 1Gigabit access ports
4. The OpenFlow rule generated in setEventHandler exactly matches the addresses and ports in each TCP connection and includes an idleTimeout of 2 seconds. This means that OpenFlow rules are automatically removed by the switch when the flow becomes idle without any further intervention from the controller.

The iperf tool can be used to generate a sequence of large flows to test the controller:

while true; do iperf -c 10.100.10.152 -i 20 -t 20; sleep 20; done

The following screen capture shows a basic test setup and results:

The screen capture shows a mixture of small flows "mice" and large flows "elephants" generated by a server connected to an edge switch (in this case an Alcatel-Lucent OmniSwitch 6900). The graph at the bottom right shows the mixture of unmarked traffic being sent to the switch. The sFlow-RT controller receives a stream of sFlow measurements from the switch and detects each elephant flows in real-time, immediately installing an OpenFlow rule that matches the flow and instructing the switch to mark the flow by setting the IP type of service bits. The traffic upstream of the switch is shown in the top right chart and it can be clearly seen that each elephant flow has been identified and marked, while the mice have been left unmarked.

Note: While this demonstration only used a single switch, the solution easily scales to hundreds of switches and thousands of edge ports.

The sFlow-RT large flow marking application is a much simpler than the large flow marking solution demonstrated by HP and VWware - which requires special purpose instrumentation embedded in a customized virtual switch and complex inter-controller communication. In contrast, the sFlow-RT controller leverages standard sFlow instrumentation built into commodity network devices to detect large flows in real time. Over 40 network vendors support the sFlow standard, including most of the NSX network gateway services partners and the HP 5930AF used in the HP / VMware demo. In addition, sFlow is widely available in virtual switches, including: Open vSwitch, Hyper-V Virtual Switch, IBM Distributed Virtual Switch 5000v, and HP FlexFabric Virtual Switch 5900v.

While it is possible to mark large flows based on a simple notification - as was demonstrated by HP/VMware - load balancing requires complete visibility into all links in the fabric. Using sFlow instrumentation embedded in the network devices, sFlow-RT has the real-time view of the utilization on all links and the information on all large flows and the their paths across the fabric needed for load balancing.

Why sFlow? What about other measurement protocols like Cisco NetFlow, IPFIX, SNMP, or using OpenFlow counters? The sFlow standard is uniquely suited to real-time traffic engineering because it provides the low latency, scaleability, and flexibility needed to support SDN traffic engineering applications. For a detailed discussion of the requirements for analytics driven control, see Performance Aware SDN.

Basing the sFlow-RT fabric controller on widely supported sFlow and OpenFlow standards and including an open, standards based, programming environment (JavaScript / ECMAScript) makes sFlow-RT an ideal platform for rapidly developing and deploying traffic engineering SDN applications in existing networks.

ONS2014 SDN Idol finalist demonstrations

The video of the ONS 2014 SDN Idol final demonstrations has been released (the demonstrations were presented live at the Open Networking Summit on Monday, March 3, 02:30P - 04:00P).

The first demo presented is Real-time SDN Analytics for DDoS mitigation, a joint Brocade / InMon solution that combines real-time sFlow analytics and OpenFlow with SDN so that service providers can deliver large scale distributed denial of service (DDoS) attack mitigation services to their enterprise customers using their existing network infrastructure. DDoS mitigation is particularly topical, two weeks ago, a large attack was targeted at CloudFlare, DDoS Attack Hits 400 Gbit/s, Breaks Record, and this past week, Meetup.com has been hit with a large persistent attack, Meetup Suffering Significant DDoS Attack, Taking It Offline For Days. The SDN DDoS mitigation solution can address these large attacks by leveraging the multi-Terabit, line-rate, monitoring and filtering capabilities in the network switches.

ONS2014 Announces Finalists for SDN Idol 2014 provides some sFlow related trivia relating to the finalists. 

An expert panel of judges selected the finalists:

The finalists were selected based on the following criteria:

Voting is open to ONS delegates and will occur during this evenings reception and the winner will be announced tomorrow.

March 5, 2014 Update: The Brocade/InMon DDoS mitigation as a service solution was voted SDN Idol winner - Brocade Crowned Winner of SDN Idol 2014 at Open Networking Summit 2014 - Real-Time SDN Analytics for DDoS Mitigation Bags Top Honors

ONS2014 Announces Finalists for SDN Idol 2014

Today the Open Networking Summit announced the five finalists for the SDN Idol 2014 competition:

• Brocade and InMon: Real-time SDN Analytics for DDoS mitigation
• HP: HP SDN App Store and Open SDN Ecosystem 
• OpenDaylight: Hydrogen 
• Pica8: Pica8’s Open SDN Starter Kit 
• Radware: Adaptive Network, Application and APT Protection

Real-time SDN Analytics for DDoS mitigation is an example of a performance aware SDN controller that combines sFlow and OpenFlow for the visibility and control needed to build self optimizing networks that automatically adapt to changing traffic conditions. A number of other use cases were outlined by Brocade at the recent OpenDaylight Summit - see Flow-aware Real-time SDN Analytics (FRSA)

There are interesting links with other finalists:

• OpenDaylight Hydrogen The Brocade is a Platinum member of the OpenDaylight project, and the Brocade/InMon DDoS mitigation solution employs OpenDaylight Hydrogen as an OpenFlow controller. Like Brocade, many of the OpenDaylight project members also support sFlow in their networking equipment, including: Brocade, Cisco, IBM, Juniper, NEC, A10 Networks, Arista, Dell, HP, Huawei, Intel, and ZTE. One might expect to see other vendors start to build traffic aware solutions on OpenDaylight in the coming months.
• HP SDN App Store and Open SDN Ecosystem Every OpenFlow enabled switch in HP's SDN Ecosystem supports the sFlow standard. Future versions of HP's SDN controller could leverage the sFlow capabilities of HP switches to deliver network visibility, allowing the controller platform to support scaleable performance aware SDN applications.
• Pica8 Open SDN Starter Kit The switch contained in the starter kit supports sFlow, making the starter kit a great way to experiment with combined sFlow and OpenFlow solutions. There are a number of examples on this blog that could be tried with the starter kit - see OpenFlow.

The five finalists cover a broad spectrum of SDN solutions - it will be interesting to see them demonstrated live at the Open Networking Summit on Monday, March 3, 02:30P - 04:00P

#NFD7 Real Time SDN and NFV Analytics for DDoS Mitigation

Today, at Networking Field Day 7, Ramki Krishnan of Brocade Networks demonstrated how the sFlow and OpenFlow standards can be combined to deliver DDoS mitigation as a service. Ramki is a co-author of related Internet Drafts: Large Flow Use Cases for I2RS PBR and QoS and Mechanisms for Optimal LAG/ECMP Component Link Utilization in Networks.

The talk starts by outlining the growing problem of DDoS attacks and the market opportunity for mitigation solutions, referencing the articles, Prolexic Publishes Top 10 DDoS Attack Trends for 2013, World's largest DDoS strikes US, Europe.

The diagram shows the unique position occupied by Internet Service Provider (ISP) and Internet Exchange (IX) networks, allowing them to filter large flood attacks and prevent them from overwhelming Enterprise customer connections - provided they can use their network to efficiently detect attacks and automatically filter traffic for their customers.

This diagram shows how standard sFlow enabled in the switches and routers provides a continuous stream of measurement data to InMon sFlow-RT, which provided real-time detection and notification of DDoS attacks to the DDoS Mitigation SDN Application. The DDoS Mitigation SDN Application selects a mitigation action and instructs the SDN Controller to push the action to selected switches (for example using a standard OpenFlow rules to drop traffic associated with the DDoS attack).

The key making this solution scale is the use of hybrid port OpenFlow. By default, all traffic is handled by the switch's normal hardware switching and routing function without any intervention from the controller. The OpenFlow rules are used to override the normal forwarding behavior for the selected flow. The solution uses a software controller to leverages the standard sFlow and OpenFlow capabilities of existing network hardware to provide a scaleable, automated, cost effective solution that allows ISP/IX networks to effectively mitigate flood attacks.

The live demo shows a continuous stream of NTP reflection attacks created by a traffic generator, each attack lasting 20 seconds. The chart at the top right shows the attack traffic in red and the normal traffic in green. The Brocade MLXe switch sends a continuous stream of sFlow measurements to InMon's sFlow-RT analytics engine.

The sFlow-RT software performs a number of functions:

1. Provides a REST API allowing the customer to set thresholds and mitigation policies
2. Detects the DDoS attack
3. Extracts attributes that characterize the attack traffic - UDP source port (123) and destination IP address (12.12.1.2) in this example
4. Constructs a filter to drop the attack
5. Makes a call to OpenDaylight's Flow Programmer REST API to instruct OpenDaylight to send the filter as an OpenFlow rule the MLXe
6. Continues to monitor the DDoS traffic
7. Makes a call to OpenDaylight to remove the rule once the attack subsides
8. Provides statistics to drive the demo dashboard - which in a real deployment would be the customer portal

The chart at the bottom right of the screen shows the traffic after it has been filtered by the controller. As each new attack is launched, it is immediately detected and removed so that the link is protected and the normal traffic gets to the customer network. While the demonstration shows one switch and one protected 10Gigabit link, the solution easily scales to hundreds of switches, tens of thousands links and 100Gigabit link speeds.

This demonstration of DDoS mitigation is only one application of this architecture - Ramki's OpenDaylight Summit talk Flow-aware Real-time SDN Analytics (FRSA)  presented a number of others.