Thursday, November 15, 2018

Mininet, ONOS, and segment routing

Leaf and spine traffic engineering using segment routing and SDN and CORD: Open-source spine-leaf Fabric describe a demonstration at the 2015 Open Networking Summit using the ONOS SDN controller and a physical network of 8 switches.

This article will describe how to emulate a leaf and spine network using Mininet and configure the ONOS segment routing application to provide equal cost multi-path (ECMP) routing of flows across the fabric. The Mininet Dashboard application running on the sFlow-RT real-time analytics platform is used to provide visibility into traffic flows across the emulated network.

First, run ONOS using Docker:
docker run --name onos --rm -p 6653:6653 -p 8181:8181 -d onosproject/onos
Use the graphical interface, http://onos:8181, to enable the OpenFlow Provider Suite, Network Config Host Provider, Network Config Link Provider, and Segment Routing applications. The screen shot above shows the resulting set of enabled services.

Next, install sFlow-RT and the Mininet Dashboard application on host with Mininet:
tar -xvzf sflow-rt.tar.gz
./sflow-rt/ sflow-rt mininet-dashboard
Start sFlow-RT:
Download the script:
Start Mininet:
sudo env ONOS= mn --custom,sflow-rt/extras/ \
--link tc,bw=10 --topo=sr '--controller=remote,ip=$ONOS,port=6653'
The script is used to create a leaf and spine topology in Mininet and send the network configuration to the ONOS controller. The script enables sFlow monitoring of the switches and sends the network topology to sFlow-RT.

The leaf and spine topology will appear in the ONOS web interface.
The topology will also appear in the Mininet Dashboard application:
Run an iperf test using the Mininet cli:
mininet> iperf h1 h3
The path that the traffic takes is highlighted on the Mininet Dashboard topology:
In this case the traffic flowed between leaf1 and leaf2 via spine1. Since ONOS segment routing uses equal cost multi-path routing, subsequent iperf tests may take the alternative via spine2.
Switch to the Charts tab to see traffic trend charts. In this case, the trend charts show the results of six iperf tests. The Traffic chart shows the top flows and the Topology charts show the busy links and the network diameter.

See Writing Applications for an introduction to programming sFlow-RT's analytics engine. Mininet flow analytics provides a simple example of detecting large (elephant) flows.


  1. Hello.
    Thank you for your post. Can we amend the (to accommodate a custom topology) or use a custom script to work with onos controller and still use sFlow?

  2. ONOS traffic analytics and ONOS measurement based control provide additional examples of using Mininet with ONOS and sFlow / sFlow-RT.

    Segment routing requires quite a bit of configuration. You could use the script as a basis for building your own custom topology. Enabling sFlow is independent of the topology script, just include the sflow-rt/extras/ script in the list of custom topologies to enable sFlow and push the Mininet topology to sFlow-RT (sFlow-RT must be running before you start Mininet).

  3. hello Peter, thank you for your response. Where can I find help or guidance to create a custom topology bench-marking Any help would be much appreciated. Thank you in advance

  4. Hi Peter,

    SFlow-RT seems to work really well with software switches in a Mininet environment. Can I also use SFlow-RT on OF-DPA switches? Do you have any experience testing Segment Routing and ONOS with physical switches based on OF-DPA?

    1. Leaf and spine traffic engineering using segment routing and SDN used physical, Broadcom based, switches with ONOS and segment routing. The results closely matched Mininet emulations. In this example the switches were running a version of FTOS.

      Broadcom ASICs supported by OF-DPA include hardware support for sFlow. However, I don't believe OF-DPA includes APIs needed to configure the sFlow instrumentation.

      SAI, SDKLT, and OpenNSL provide methods to configure sFlow instrumentation in the ASIC.

      The open source Host sFlow has been ported to a number of network operating systems to export the measurements as standard sFlow.

  5. Hi Peter. I followed your tutorial and catched a problem.
    I can use Iperf to send the packets between 2 hosts, but I cannot replay the packets in a pcap file between 2 hosts (Using tcpreplay). Do you know why?
    Thank you in advance

    1. I wouldn't expect replaying a pcap to work. Mininet is emulating a real network and protocol interactions matter (ARP, tcp connection establishment, etc).

      A better approach is to automate tests using iperf, etc. to create real traffic patterns that can be replayed as needed.

  6. Hai sir,

    I tried the above sections, but i had a problem, when i running this command :

    sudo env ONOS= mn --custom,sflow-rt/extras/ \
    --link tc,bw=10 --topo=sr '--controller=remote,ip=$ONOS,port=6653'

    Its simply shows the cursor blinking
    Then i tried to change the onos ip to localhost
    Then its works,and create toplogy on mininet and those output shows in mininet-dashboard
    But the onos Gui shows no devices are connected.
    how to solve this problem??

    1. If you are running ONOS on the local machine then ONOS=localhost is the correct setting.

      Did you enable the OpenFlow Provider Suite, Network Config Host Provider, Network Config Link Provider, and Segment Routing applications in ONOS?

    2. Thank you sir,
      Now its working fine.Thank you again for sharing this informative blog.Its highly help my project works.

  7. hi sir, thank you for sharing this. I have tried to follow the steps. The devices visible in mininet dashboard but not in onos. Only hosts are registered but not devices. I have enabled all related apps. Have tried on onos 2.5, 2.0, and 1.15. Is there any other setting on onos that I need to enable?

    1. Did you enable enable the OpenFlow Provider Suite, Network Config Host Provider, Network Config Link Provider, and Segment Routing applications shown in the ONOS screen capture? ONOS 1.15 was used in this example.

    2. I just tried the example using the ONOS 2.5 docker image and the examples work. The network topology appears in the ONOS UI. You do need to start the ONOS applications listed above.