Thursday, November 15, 2018

Mininet, ONOS, and segment routing

Leaf and spine traffic engineering using segment routing and SDN and CORD: Open-source spine-leaf Fabric describe a demonstration at the 2015 Open Networking Summit using the ONOS SDN controller and a physical network of 8 switches.

This article will describe how to emulate a leaf and spine network using Mininet and configure the ONOS segment routing application to provide equal cost multi-path (ECMP) routing of flows across the fabric. The Mininet Dashboard application running on the sFlow-RT real-time analytics platform is used to provide visibility into traffic flows across the emulated network.

First, run ONOS using Docker:
docker run --name onos --rm -p 6653:6653 -p 8181:8181 -d onosproject/onos
Use the graphical interface, http://onos:8181, to enable the OpenFlow Provider Suite, Network Config Host Provider, Network Config Link Provider, and Segment Routing applications. The screen shot above shows the resulting set of enabled services.

Next, install sFlow-RT and the Mininet Dashboard application on host with Mininet:
tar -xvzf sflow-rt.tar.gz
./sflow-rt/ sflow-rt mininet-dashboard
Start sFlow-RT:
Download the script:
Start Mininet:
sudo env ONOS= mn --custom,sflow-rt/extras/ \
--link tc,bw=10 --topo=sr '--controller=remote,ip=$ONOS,port=6653'
The script is used to create a leaf and spine topology in Mininet and send the network configuration to the ONOS controller. The script enables sFlow monitoring of the switches and sends the network topology to sFlow-RT.

The leaf and spine topology will appear in the ONOS web interface.
The topology will also appear in the Mininet Dashboard application:
Run an iperf test using the Mininet cli:
mininet> iperf h1 h3
The path that the traffic takes is highlighted on the Mininet Dashboard topology:
In this case the traffic flowed between leaf1 and leaf2 via spine1. Since ONOS segment routing uses equal cost multi-path routing, subsequent iperf tests may take the alternative via spine2.
Switch to the Charts tab to see traffic trend charts. In this case, the trend charts show the results of six iperf tests. The Traffic chart shows the top flows and the Topology charts show the busy links and the network diameter.

See Writing Applications for an introduction to programming sFlow-RT's analytics engine. Mininet flow analytics provides a simple example of detecting large (elephant) flows.


  1. Hello.
    Thank you for your post. Can we amend the (to accommodate a custom topology) or use a custom script to work with onos controller and still use sFlow?

  2. ONOS traffic analytics and ONOS measurement based control provide additional examples of using Mininet with ONOS and sFlow / sFlow-RT.

    Segment routing requires quite a bit of configuration. You could use the script as a basis for building your own custom topology. Enabling sFlow is independent of the topology script, just include the sflow-rt/extras/ script in the list of custom topologies to enable sFlow and push the Mininet topology to sFlow-RT (sFlow-RT must be running before you start Mininet).

  3. hello Peter, thank you for your response. Where can I find help or guidance to create a custom topology bench-marking Any help would be much appreciated. Thank you in advance

  4. Hi Peter,

    SFlow-RT seems to work really well with software switches in a Mininet environment. Can I also use SFlow-RT on OF-DPA switches? Do you have any experience testing Segment Routing and ONOS with physical switches based on OF-DPA?

    1. Leaf and spine traffic engineering using segment routing and SDN used physical, Broadcom based, switches with ONOS and segment routing. The results closely matched Mininet emulations. In this example the switches were running a version of FTOS.

      Broadcom ASICs supported by OF-DPA include hardware support for sFlow. However, I don't believe OF-DPA includes APIs needed to configure the sFlow instrumentation.

      SAI, SDKLT, and OpenNSL provide methods to configure sFlow instrumentation in the ASIC.

      The open source Host sFlow has been ported to a number of network operating systems to export the measurements as standard sFlow.

  5. Hi Peter. I followed your tutorial and catched a problem.
    I can use Iperf to send the packets between 2 hosts, but I cannot replay the packets in a pcap file between 2 hosts (Using tcpreplay). Do you know why?
    Thank you in advance

    1. I wouldn't expect replaying a pcap to work. Mininet is emulating a real network and protocol interactions matter (ARP, tcp connection establishment, etc).

      A better approach is to automate tests using iperf, etc. to create real traffic patterns that can be replayed as needed.