Saturday, May 21, 2016

Mininet dashboard

Mininet Dashboard has been released on GitHub, https://github.com/sflow-rt/mininet-dashboard. Follow the steps in Mininet flow analytics to install sFlow-RT and configure sFlow instrumentation in Mininet.

The following steps install the dashboard and start sFlow-RT:
cd sflow-rt
./get-app.sh sflow-rt mininet-dashboard
./start.sh
The dashboard web interface shown in the screen shot should now be accessible. Run a test to see data in the dashboard. The following test created the results shown:
cd sflow-rt
sudo mn --custom extras/sflow.py --link tc,bw=10 --topo tree,depth=2,fanout=2 --test iperf
The dashboard has three time series charts that update every second and show five minutes worth of data. From top to bottom, the charts are:
  1. Top Flows - Click on a peak in the chart to see the flows that were active at that time.
  2. Top Ports - Click on a peak in the chart to see the ingress ports that were active at that time.
  3. Topology Diameter - The diameter of the topology.
The dashboard application is easily modified to add additional metrics, generate events, or implement controls. For example, adding the following code to the end of the sflow-rt/app/mininet-dashboard/scripts/metrics.js file implements equivalent functionality to the large flow detection Python script described in Mininet flow analytics:
setFlow('pair',
  {'keys':'link:inputifindex,ipsource,ipdestination','value':'bytes'});

setThreshold('elephant',
  {'metric':'pair','value':1000000/8,'byFlow':true,'timeout':1});

setEventHandler(function(evt) {
  logInfo(evt.flowKey);
},['elephant']);
Restart sFlow-RT and repeat the iperf test and the following events should be logged:
$ ./start.sh 
2016-05-21T18:00:03-0700 INFO: Listening, sFlow port 6343
2016-05-21T18:00:03-0700 INFO: Starting the Jetty [HTTP/1.1] server on port 8008
2016-05-21T18:00:03-0700 INFO: Starting com.sflow.rt.rest.SFlowApplication application
2016-05-21T18:00:03-0700 INFO: Listening, http://localhost:8008
2016-05-21T18:00:03-0700 INFO: app/mininet-dashboard/scripts/metrics.js started
2016-05-21T18:00:12-0700 INFO: s1-s2,10.0.0.1,10.0.0.4
2016-05-21T18:00:12-0700 INFO: s1-s3,10.0.0.1,10.0.0.4
See Writing Application for more information.

Update April 20, 2020: The following articles provide additional examples: Mininet weathermap, Mininet flow analytics with custom scripts, Ryu measurement based controlONOS traffic analytics, and Mininet, ONOS, and segment routing.

43 comments:

  1. cd sflow-rt
    ./get-app<.sh> sflow-rt mininet-dashboard
    ./start.sh

    Thank you for the article!

    ReplyDelete
    Replies
    1. Thank you for the correction. I have updated the instructions in the article.

      Delete
  2. what if i have my own topology, in example customtopo.py and i want to run in sflow.py ??

    Thank you

    ReplyDelete
    Replies
    1. I believe that the latest versions of Mininet allow multiple comma separated entries in the --custom argument, i.e. --custom extras/sflow.py,customtopo.py

      You might also be interested in the Mininet dashboard application for sFlow-RT.

      Delete
    2. Thanks Peter.
      I have same question with it.
      Now it works :)

      Delete
  3. Replies
    1. In this example sFlow-RT is running on the command line, so just type CNTRL+C to stop it and then run ./start.sh again.

      If you are installed sFlow-RT as a package then type:
      sudo systemctl restart sflow-rt.service

      Delete
  4. Hello Peter, I have problem with detecting elephant flow in running mininet-dashboard app. I must edited topology command, because I am working with ryu rest QoS. Problem is after iperf test between h1 and h2, I do not get events about detected elephant flow.

    edited topology: sudo mn --custom extras/sflow.py --topo tree,depth=1,fanout=2 --controller remote --switch ovsk,protocols=OpenFlow13

    link to screens from mininet screen: https://imgur.com/a/bGhvGQS
    link to my metrics.js file: https://pastebin.com/x5XNWzrk

    ReplyDelete
    Replies
    1. Your topology doesn't have any inter-switch links and so the link:inputifindex expression in your flow definition won't return a value and so the flow will not be tracked.

      I created the following elephant.js file (it's cleaner to keep your changes in a separate file, rather than edit the mininet-dashboard application files):

      setFlow('pair',
       {'keys':'inputifindex,ipsource,ipdestination','value':'bytes'});

      setThreshold('elephant',
       {'metric':'pair','value':1000000/8,'byFlow':true,'timeout':1});

      setEventHandler(function(evt) {
       var [inputifindex,ipsource,ipdestination] = evt.flowKey.split(',');
       var {node,port} = topologyInterfaceToPort(evt.agent,inputifindex);
       logInfo(port + " " + ipsource + " " + ipdestination);
      },['elephant']);

      And started sFlow-RT with the following command:
      env RTPROP="-Dscript.file=elephant.js" ./start.sh

      When you get an elephant flow it will print the switch port along with the flow:
      s1-eth1 10.0.0.1 10.0.0.2

      Delete
    2. Thanks for your reply Peter. I have one more problem with flowHandler. Here is link to stackoverflow: https://stackoverflow.com/questions/49965853/handle-flow-records-in-mininet-dashboard/49968342

      Delete
    3. I ran the script (with the setFlowHandler function you added) from stackoverflow. The script works. You may need to run a second iperf test since sFlow-RT is data driven and requires a stream of packet samples to flush flow records.

      Delete
  5. Can I ask you one more question? I am trying to execute curl command from elephant.js file, but every try fails. Is there a way how to execute curl command? Here is my edited elephant.js file: https://pastebin.com/GmwsrvyK

    ReplyDelete
    Replies
    1. sFlow-RT has a built-in http client that you can use to interact with REST APIs. ONOS measurement based control provides an example.

      Delete
  6. hi,,, thanks for the post,,, does sflow just sampling packet header for the end-to-end switch ? i have a topology with mininet and i just can get header sampling(flow sample) from ene-to-end switch that connected to my hosts not for the central switches. i can get counter sample from all switches but cant get flow sample from center switch. is there any problem with my mininet?

    thanks in advance

    ReplyDelete
    Replies
    1. Could you explain what you mean by "end-to-end switch" and "central switch"?

      Did you use one of the standard Mininet topologies?

      Mininet weathermap

      Delete
  7. hello what different between sflow-RT and sflow ?

    ReplyDelete
    Replies
    1. sFlow is a standard set of measurements transmitted in a standard format from network devices. sFlow-RT is software that collects and analyzes sFlow measurements.

      Delete
  8. hello i have question why when i restart the sflow_rt i see the bandwidth is become high fron the previous bandwidth and i noticed when i wrote the iperf without restart the sflow-rt the bandwidth is become low please can somebody explain me the reason?

    ReplyDelete
    Replies
    1. If you restart sFlow-RT is loses the topology that the sflow.py script sends when Mininet was started. The topology is used to de-duplicate traffic crossing multiple links in the topology and so the reported values will be inaccurate. It is best to leave sFlow-RT running, but if you must restart it, restart Mininet afterward.

      Delete
    2. @ peter thanks , but what you are meaning loses the topology ? please can you explain to me

      Delete
    3. The sflow.py script posts a JSON representation of the Mininet topology to sFlow-RT when Mininet starts up. sFlow-RT does not store the topology to disk, so it will be lost if you restart.

      Delete
  9. hello again if i want to detect the large flows (elephant flows) with sflow-rt and ryu and mininet can i ust this script elephant.py and where i put this script and is i run the simple_switch13.py and ofctl_rest.py in another terminal please can somebody guide me or help me thanks

    ReplyDelete
  10. hello
    how i can get a dataset of an SDN datacenter ?

    thank you in advance

    ReplyDelete
    Replies
    1. Data sets from production networks tend to be confidential. I am not aware of any publicly available sources. Mininet, ONOS, and segment routing describes how to emulate and monitor a data center SDN leaf-spine fabric.

      Delete
  11. Hello

    when I tried to run the above command
    sudo mn --custom extras/sflow.py --link tc,bw=10 --topo tree,depth=2,fanout=2 --test iperf

    I am getting an exception couldnot find sflow-rt/extras/sflow.py file.
    I have even tried with the absolute path but always giving the same error
    and if I try to run via home directory i am getting a syntax error on line 42.

    ReplyDelete
    Replies
    1. The extras/sflow.py script is distributed as part of the sflow-rt tarball distribution. If you installed sFlow-RT as a .deb or .rpm, then you won't have the file. The instructions assume you unpacked the tarball under your home directory.

      Delete
    2. Hi Peter sflow.py is available but when i try to execute it. I get this error/

      Delete
    3. The syntax error identifies line 42, a print statement, as the problem. The script was written for Python 2.7 but it appears you are using Python 3. Print statements need to use function notation, so you can change the three lines 42, 46, and 50 to use the syntax print(). I will fix this and push a new release.

      Delete
    4. Hi Peter

      Let me know if you have already pushed the new release? Can I download and install it again?

      Delete
    5. I have tried again. after reinstalling now I get
      Caught exception
      Type Error: a byte-like object is required, not 'str'

      Delete
    6. If you download the latest release, 3.0-1461, it should fix the issue.

      Delete
    7. I'll take a look at the new error.

      Delete
    8. I just posted 3.0-1461, please give it a try and let me know if it fixes the Python 3 issues.

      Delete
    9. Hi Peter,

      I am trying to run sudo mn --custom extras/sflow.py --link tc,bw=10 --topo tree,depth=2,fanout=2

      but i am getting the similar issue posted in thread

      eepak@deepak-virtual-machine:~$ sudo bash
      root@deepak-virtual-machine:/home/deepak# cd ..
      root@deepak-virtual-machine:/home# sudo mn --custom extras/sflow.py --link tc,bw=10 --topo tree,depth=2,fanout=2
      --------------------------------------------------------------------------------
      Caught exception. Cleaning up...

      Exception: could not find custom file: extras/sflow.py
      --------------------------------------------------------------------------------

      However i am running the current version of 3.0-1545

      2021-01-02T13:14:48+05:30 INFO: Starting sFlow-RT 3.0-1545
      2021-01-02T13:14:50+05:30 INFO: Version check, running latest
      2021-01-02T13:14:50+05:30 INFO: Listening, sFlow port 6343
      2021-01-02T13:14:51+05:30 INFO: Listening, HTTP port 8008
      2021-01-02T13:14:51+05:30 INFO: extras/mark.js started
      2021-01-02T13:14:51+05:30 WARNING: extras/mark.js File not found extras/mark.js
      2021-01-02T13:14:51+05:30 INFO: extras/mark.js stopped

      Please guide if possible

      Delete
    10. Since you are running the mn command in your home directory you need to specify the path as sflow-rt/extras/sflow.py

      Delete
  12. When i execute custom topology im getting error
    Im not able to keep my customtopology.py file inside sflow-rt/extras folder.

    ReplyDelete
    Replies
    1. What is the error you are getting? You don't need to put your custom topology in the sflow-rt/extras directory.

      The article Mininet flow analytics with custom scripts should help.

      Delete
  13. This comment has been removed by the author.

    ReplyDelete
  14. hi peter,
    please i want to know how can i use the output of the EventHander(ip source and ip destination) as an input in another python script and how can i combine with them ..
    for example : when the sflow-rt detect an elephant flow in the topology and receive an event, the python script must run simultaneously (the python script need the ip source and ipdestination of the elephant flow to run)
    is it possible ?
    thanks

    ReplyDelete
    Replies
    1. Mininet flow analytics includes the Python equivalent of the JavaScript large flow detection script in this article.

      Writing Applications describes sFlow-RT REST APIs using Python examples. There are examples of defininig flows, settings thresholds, and receiving events.

      Delete

  15. Caught exception. Cleaning up...

    Exception: could not find custom file: extras/sflow.py

    ReplyDelete
    Replies
    1. The sflow.py script is in the sflow-rt/extras/ directory. If you are running the mininet command from your home directory you should use the command:

      sudo mn --custom sflow-rt/extras/sflow.py ...

      Delete