Thursday, May 19, 2016

Mininet flow analytics

Mininet is free software that creates a realistic virtual network, running real kernel, switch and application code, on a single machine (VM, cloud or native), in seconds. Mininet is useful for development, teaching, and research. Mininet is also a great way to develop, share, and experiment with OpenFlow and Software-Defined Networking systems.

This article shows how standard sFlow instrumentation built into Mininet can be combined with sFlow-RT analytics software to provide real-time traffic visibility for Mininet networks. Augmenting Mininet with sFlow telemetry realistically emulates the instrumentation built into most vendor's switch hardware, provides visibility into Mininet experiments, and opens up new areas of research (e.g. SDN and large flows).

The following papers are a small selection of projects using sFlow-RT:
In order to make it easier to get started, the latest release of sFlow-RT includes a Mininet helper script sflow.py that automates sFlow configuration. The following example shows how to use the script and build a simple application in Python.

Install sFlow-RT on the Mininet host:
wget https://inmon.com/products/sFlow-RT/sflow-rt.tar.gz
tar -xvzf sflow-rt.tar.gz
cd sflow-rt
./start.sh
In a second terminal, add the --custom argument to the Mininet command line. For example, the following command builds a depth 2 tree topology with link bandwidths of 10Mbit/s
cd sflow-rt
sudo mn --custom extras/sflow.py --link tc,bw=10 --topo tree,depth=2,fanout=2
The sflow.py script extends Mininet, automatically enabling sFlow on each of the switches in the topology, and posting a JSON representation of the Mininet topology using sFlow-RT's REST API.

Traffic engineering of large "Elephant" flows is an active area of research. The following Python script, elephant.py, demonstrates how Elephant flows can be detected using sFlow-RT REST API calls:
#!/usr/bin/env python
import requests
import json

rt = 'http://127.0.0.1:8008'

flow = {'keys':'link:inputifindex,ipsource,ipdestination','value':'bytes'}
requests.put(rt+'/flow/pair/json',data=json.dumps(flow))

threshold = {'metric':'pair','value':1000000/8,'byFlow':True,'timeout':1}
requests.put(rt+'/threshold/elephant/json',data=json.dumps(threshold))

eventurl = rt+'/events/json?thresholdID=elephant&maxEvents=10&timeout=60'
eventID = -1
while 1 == 1:
  r = requests.get(eventurl + "&eventID=" + str(eventID))
  if r.status_code != 200: break
  events = r.json()
  if len(events) == 0: continue

  eventID = events[0]["eventID"]
  events.reverse()
  for e in events:
    print e['flowKey']
Some notes on the script:
  • The link:inputifindex function in the flow definition identifies the link in the topology associated with the ingress port on the Mininet switch, see Defining Flows
  • The script defines an Elephant flow as a flow that consumes 10% of the link bandwidth. In this example Mininet was configured with a link bandwidth of 10Mbit/s so an Elephant is a flow that exceeds 1Mbit/s. Since the specified flow measures traffic in bytes/second the threshold needs to be converted by bytes/second (dividing by 8).
  • The sFlow-RT REST API uses long-polling as a method of asynchronously pushing events to the client. The events HTTP request blocks until there are new events or a timeout occurs. The client immediately reconnects after receiving a response to wait for further events. 
See Writing Applications for additional information.

Start the script:
$ ./elephant.py
Run an iperf test using the Mininet CLI:
mininet> iperf h1 h3
*** Iperf: testing TCP bandwidth between h1 and h3
*** Results: ['9.06 Mbits/sec', '9.98 Mbits/sec']
The following results should appear as soon as the flow is detected:
$ ./elephant.py 
s1-s2,10.0.0.1,10.0.0.3
s1-s3,10.0.0.1,10.0.0.3
The output identifies the links carrying the flow between h1 and h3 and shows the IP addresses of the hosts.
The sFlow-RT web interface provides basic charting capabilities. The chart above shows a sequence of iperf tests.

The Python script can easily be modified to address a number of interesting use cases. Instead of simply printing events, a REST call can be made to an OpenFlow controller (POX, OpenDaylight, Floodlight, ONOS, etc) to apply controls to mark, mirror, load balance, rate limit, or block flows.

22 comments:

  1. Please Help me, i face this error " Caught exception.Cleaning up.... SyntaxError: invalid syntax (sflow.py, line 25) , when i run "sudo mn --custom extras/sflow.py --link tc,bw=10 --topo tree,depth=2,fanout=2" .
    Regreds..

    ReplyDelete
    Replies
    1. I just tried it with sFlow-RT version 2.0-r1121 on an Ubuntu 14.04 system running Mininet 2.1.0 and Python 2.7.6 without any error. What is your setup?

      Delete
  2. How does I make REST calls to POX for adding a flow??

    ReplyDelete
    Replies
    1. POX is written in Python, so you should be able to modify the example to run as a POX application and access native POX APIs.

      The following project looks like it could be extended to add/remove flows via REST API:
      https://github.com/festradasolano/pox-jsonrest

      Delete
  3. Hi Peter,
    Where i can find the sflow.py?

    Thank you,

    ReplyDelete
    Replies
    1. It's in the sflow-rt extras directory.

      Delete
  4. I don't see it ...

    floodlight@floodlight:~/sflow-rt/extras$ ls
    README init.d leafandspine.py tail_log.py
    demo.pcap leafandspine-hybrid.js tail_flows.py topflows.py
    floodlight@floodlight:~/sflow-rt/extras$ ls -l
    total 4772

    ReplyDelete
    Replies
    1. Which version of sFlow-RT are you using? The script is in http://www.inmon.com/products/sFlow-RT/sflow-rt.tar.gz, not the DEB or RPM distributions.

      Delete
  5. how to use sflow with pox controller for load balancing?

    ReplyDelete
  6. Hi Peter, how do I get the switch dpid from the inputifindex ?

    ReplyDelete
    Replies
    1. There are a number of methods. For example, the following query gets the dpid of ifindex 2 on agent 10.0.0.202:
      /metric/10.0.0.202/2.of_dpid/json

      The following query finds the OpenFlow port:
      /metric/10.0.0.202/2.of_port/json

      If you are writing an embedded JavaScript application, then the following function returns the dpid and openflow port:
      ofInterfaceToPort(agent,ifIndex)

      Delete
  7. Hi Peter
    How can i check the flow table entries of Mininet ovs using sflow-rt?

    ReplyDelete
    Replies
    1. sFlow-RT cannot report the flow table entries. You can use the "ovs-ofctl dump-flows" command to get the active flows.

      Delete
  8. Hi Peter, How to get all metrics of one switch ?

    ReplyDelete
    Replies
    1. Normally you would use the following REST API query:
      http://localhost:8008/metric/10.0.0.20/json
      or the JavaScript equivalent, metrics('10.0.0.20')

      However, with Mininet all the logical switches are represented by a single instance of Open vSwitch and so you need to query the interfaces associated with the selected Mininet switch. The sflow.py script posts the Mininet topology to sFlow-RT. You can query the topology using the /topology/json REST API call, or the getTopology() JavaScript equivalent.

      The nodes section in the topology lists the Mininet switches and their ports. Each port has an ifIndex and you can use the ifIndex numbers to identify switch specific metrics, e.g. 1.ifinoctets is a counter associated with ifIndex=1.

      Delete
  9. Hi Peter,
    I was working on routing the flows after detection. Can you suggest me any tutorial? I am struggling finding one. I wrote an algorithm and need to implement it now. Thanks in advance.

    ReplyDelete
    Replies
    1. What are you using as your controller? How you re-route depends on the available APIs.

      Delete
    2. I am planning to use POX or RYU controller. So do you mean there are API's which are already built for re-route?

      Delete
  10. I am planning to use RYU or POX controller. So do you mean there are already API's for the controllers which can help me to re-route the flows?

    ReplyDelete
    Replies
    1. I don't believe RYU or POX has a REST API for programming flows. ONOS measurement based control describes how to use the ONOS controller.

      You could write your own Python extension to RYU or POX and access sFlow-RT's REST API to retrieve flow metrics, see Writing Applications for information on sFlow-RT REST APIs.

      Delete
  11. Hi Peter,
    When I change the fanout value to more than 2 then elephant.py does not detect or print any output. Is it because the script is unable to read the topology?

    sudo mn --custom extras/sflow.py --link tc,bw=10 --topo tree,depth=2,fanout=2

    ReplyDelete
    Replies
    1. I just tested the script and it appears to work. Are you using the latest version of sFlow-RT? You might want to load the Mininet Dashboard application. It can now display topology as well, see Mininet weathermap.

      If you run your iperf test between h1 and h2 then the traffic doesn't cross any inter-switch links and so the elephant.py script won't generate any events. Try to do the iperf test between h1 and h3

      Delete