Friday, June 14, 2019

Mininet flow analytics with custom scripts

Mininet flow analytics describes how to use the sflow.py helper script that ships with the sFlow-RT analytics engine to enable sFlow telemetry, e.g.
sudo mn --custom sflow-rt/extras/sflow.py --link tc,bw=10 \
--topo tree,depth=2,fanout=2
Mininet, ONOS, and segment routing provides an example using a Custom Topology, e.g.
sudo env ONOS=10.0.0.73 mn --custom sr.py,sflow-rt/extras/sflow.py \
--link tc,bw=10 --topo=sr '--controller=remote,ip=$ONOS,port=6653'
This article describes how to incorporate sFlow monitoring in a fully custom Mininet script. Consider the following simpletest.py script based on Working with Mininet:
#!/usr/bin/python                                                                            
                                                                                             
from mininet.topo import Topo
from mininet.net import Mininet
from mininet.util import dumpNodeConnections
from mininet.log import setLogLevel

class SingleSwitchTopo(Topo):
    "Single switch connected to n hosts."
    def build(self, n=2):
        switch = self.addSwitch('s1')
        # Python's range(N) generates 0..N-1
        for h in range(n):
            host = self.addHost('h%s' % (h + 1))
            self.addLink(host, switch)

def simpleTest():
    "Create and test a simple network"
    topo = SingleSwitchTopo(n=4)
    net = Mininet(topo)
    net.start()
    print "Dumping host connections"
    dumpNodeConnections(net.hosts)
    print "Testing bandwidth between h1 and h4"
    h1, h4 = net.get( 'h1', 'h4' )
    net.iperf( (h1, h4) )
    net.stop()

if __name__ == '__main__':
    # Tell mininet to print useful information
    setLogLevel('info')
    simpleTest()
Add the highlighted lines to incorporate sFlow telemetry:
#!/usr/bin/python                                                                            
                                                                                             
from mininet.topo import Topo
from mininet.net import Mininet
from mininet.util import dumpNodeConnections
from mininet.log import setLogLevel
from mininet.util import customClass
from mininet.link import TCLink

# Compile and run sFlow helper script
# - configures sFlow on OVS
# - posts topology to sFlow-RT
execfile('sflow-rt/extras/sflow.py') 

# Rate limit links to 10Mbps
link = customClass({'tc':TCLink}, 'tc,bw=10')

class SingleSwitchTopo(Topo):
    "Single switch connected to n hosts."
    def build(self, n=2):
        switch = self.addSwitch('s1')
        # Python's range(N) generates 0..N-1
        for h in range(n):
            host = self.addHost('h%s' % (h + 1))
            self.addLink(host, switch)

def simpleTest():
    "Create and test a simple network"
    topo = SingleSwitchTopo(n=4)
    net = Mininet(topo,link=link)
    net.start()
    print "Dumping host connections"
    dumpNodeConnections(net.hosts)
    print "Testing bandwidth between h1 and h4"
    h1, h4 = net.get( 'h1', 'h4' )
    net.iperf( (h1, h4) )
    net.stop()

if __name__ == '__main__':
    # Tell mininet to print useful information
    setLogLevel('info')
    simpleTest()
When running the script the highlighted output confirms that sFlow has been enabled and the topology has been posted to sFlow-RT:
pp@mininet:~$ sudo ./simpletest.py
*** Creating network
*** Adding controller
*** Adding hosts:
h1 h2 h3 h4 
*** Adding switches:
s1 
*** Adding links:
(10.00Mbit) (10.00Mbit) (h1, s1) (10.00Mbit) (10.00Mbit) (h2, s1) (10.00Mbit) (10.00Mbit) (h3, s1) (10.00Mbit) (10.00Mbit) (h4, s1) 
*** Configuring hosts
h1 h2 h3 h4 
*** Starting controller
c0 
*** Starting 1 switches
s1 ...(10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) 
*** Enabling sFlow:
s1
*** Sending topology
Dumping host connections
h1 h1-eth0:s1-eth1
h2 h2-eth0:s1-eth2
h3 h3-eth0:s1-eth3
h4 h4-eth0:s1-eth4
Testing bandwidth between h1 and h4
*** Iperf: testing TCP bandwidth between h1 and h4 
*** Results: ['6.32 Mbits/sec', '6.55 Mbits/sec']
*** Stopping 1 controllers
c0 
*** Stopping 4 links
....
*** Stopping 1 switches
s1 
*** Stopping 4 hosts
h1 h2 h3 h4 
*** Done
Mininet dashboard and Mininet weathermap describe the sFlow-RT Mininet Dashboard application shown at the top of this article. The tool provides a real-time visualization of traffic flowing over the Mininet network. Writing Applications describes how to develop custom analytics applications for sFlow-RT.

30 comments:

  1. Hi Peter,
    Above lines aren't working with my case.

    mininet@mininet-vm:~/mininet/examples$ sudo ./2.py
    *** Adding controller
    *** Add switches
    *** Add hosts
    *** Add links
    *** Starting network
    *** Configuring hosts
    h1 h5 h2 h6 h3 h4
    *** Starting controllers
    *** Starting switches
    *** Post configure switches and hosts
    *** Starting CLI:

    ReplyDelete
  2. The execfile('sflow-rt/extras/sflow.py') statement contains a relative path. It looks like you are running your mininet script in the the ~/mininet/examples directory, so you would need to change the path so that sflow.py can be found (e.g. '../../sflow-rt/extras/sflow.py' if you installed sflow-rt in your home directory).

    ReplyDelete
  3. I've used this path in code :
    execfile('/home/mininet/sflow-rt/extras/sflow.py'). Network gets started without sflow.

    If I try to do some changes like below, it gives:

    execfile('home/mininet/sflow-rt/extras/sflow.py')
    IOError: [Errno 2] No such file or directory: 'home/mininet/sflow-rt/extras/sflow.py'

    ReplyDelete
    Replies
    1. It looks like the first version is loading the sflow.py code. Is this the first line in your custom script? Does your script run net.start()? Loading the sflow.py library modifies net.start() to enable sFlow functionality.

      Delete
    2. This is my code for your review please:

      #!/usr/bin/python

      from mininet.net import Mininet
      from mininet.node import Controller, RemoteController, OVSController
      from mininet.node import CPULimitedHost, Host, Node
      from mininet.node import OVSKernelSwitch, UserSwitch
      from mininet.node import IVSSwitch
      from mininet.cli import CLI
      from mininet.log import setLogLevel, info
      from mininet.link import TCLink, Intf
      from subprocess import call
      from mininet.util import customClass
      from mininet.link import TCLink

      # Compile and run sFlow helper script
      # - configures sFlow on OVS
      # - posts topology to sFlow-RT
      execfile('/home/mininet/sflow-rt/extras/sflow.py')

      # Rate limit links to 10Mbps
      link = customClass({'tc':TCLink}, 'tc,bw=10')

      def myNetwork():

      net = Mininet( topo=None,
      build=False,
      ipBase='10.0.0.0/8',
      link=link)

      info( '*** Adding controller\n' )
      c0=net.addController(name='c0',
      controller=RemoteController,
      ip='192.168.15.141',
      protocol='tcp',
      port=6633)

      info( '*** Add switches\n')
      s1 = net.addSwitch('s1', cls=OVSKernelSwitch)
      s2 = net.addSwitch('s2', cls=OVSKernelSwitch)

      info( '*** Add hosts\n')
      h1 = net.addHost('h1', cls=Host, ip='10.0.0.1', defaultRoute=None)
      h5 = net.addHost('h5', cls=Host, ip='10.0.0.5', defaultRoute=None)
      h2 = net.addHost('h2', cls=Host, ip='10.0.0.2', defaultRoute=None)
      h6 = net.addHost('h6', cls=Host, ip='10.0.0.6', defaultRoute=None)
      h3 = net.addHost('h3', cls=Host, ip='10.0.0.3', defaultRoute=None)
      h4 = net.addHost('h4', cls=Host, ip='10.0.0.4', defaultRoute=None)

      info( '*** Add links\n')
      net.addLink(s1, h5)
      net.addLink(s1, h1)
      net.addLink(s1, h2)
      net.addLink(s2, h6)
      net.addLink(s2, h3)
      net.addLink(s2, h4)
      net.addLink(s1, s2)

      info( '*** Starting network\n')
      net.build()
      info( '*** Starting controllers\n')
      for controller in net.controllers:
      controller.start()

      info( '*** Starting switches\n')
      net.get('s1').start([c0])
      net.get('s2').start([c0])

      info( '*** Post configure switches and hosts\n')

      CLI(net)
      net.stop()

      if __name__ == '__main__':
      setLogLevel( 'info' )
      myNetwork()

      Delete
    3. Why are you starting each switch individually? You are bypassing the net.start() function that would enable sFlow. Can you replace the two statements starting each switch with net.start()?

      Delete
    4. You should probably remove the net.build() and the controller.start() code since I believe net.start() will perform these functions.

      Delete
    5. Please find this code and it's output but graphs and resources are not being populated on sFlow.

      Code:
      #!/usr/bin/python

      from mininet.net import Mininet
      from mininet.node import Controller, RemoteController, OVSController
      from mininet.node import CPULimitedHost, Host, Node
      from mininet.node import OVSKernelSwitch, UserSwitch
      from mininet.node import IVSSwitch
      from mininet.cli import CLI
      from mininet.log import setLogLevel, info
      from mininet.link import TCLink, Intf
      from subprocess import call
      from mininet.util import customClass
      from mininet.link import TCLink

      # Compile and run sFlow helper script
      # - configures sFlow on OVS
      # - posts topology to sFlow-RT
      execfile('/home/mininet/sflow-rt/extras/sflow.py')

      # Rate limit links to 10Mbps
      link = customClass({'tc':TCLink}, 'tc,bw=10')

      def myNetwork():

      net = Mininet( topo=None,
      build=False,
      ipBase='10.0.0.0/8',
      link=link)

      info( '*** Adding controller\n' )
      c0=net.addController(name='c0',
      controller=RemoteController,
      ip='192.168.15.141',
      protocol='tcp',
      port=6633)

      info( '*** Add switches\n')
      s1 = net.addSwitch('s1', cls=OVSKernelSwitch)
      s2 = net.addSwitch('s2', cls=OVSKernelSwitch)

      info( '*** Add hosts\n')
      h1 = net.addHost('h1', cls=Host, ip='10.0.0.1', defaultRoute=None)
      h5 = net.addHost('h5', cls=Host, ip='10.0.0.5', defaultRoute=None)
      h2 = net.addHost('h2', cls=Host, ip='10.0.0.2', defaultRoute=None)
      h6 = net.addHost('h6', cls=Host, ip='10.0.0.6', defaultRoute=None)
      h3 = net.addHost('h3', cls=Host, ip='10.0.0.3', defaultRoute=None)
      h4 = net.addHost('h4', cls=Host, ip='10.0.0.4', defaultRoute=None)

      info( '*** Add links\n')
      net.addLink(s1, h5)
      net.addLink(s1, h1)
      net.addLink(s1, h2)
      net.addLink(s2, h6)
      net.addLink(s2, h3)
      net.addLink(s2, h4)
      net.addLink(s1, s2)

      info( '*** Starting network\n')
      info( '*** Starting controllers\n')
      for controller in net.controllers:
      net.start()

      info( '*** Starting switches\n')
      net.get('s1').start([c0])
      net.get('s2').start([c0])

      info( '*** Post configure switches and hosts\n')

      CLI(net)
      net.stop()

      if __name__ == '__main__':
      setLogLevel( 'info' )
      myNetwork()


      Output:
      mininet@mininet-vm:~/mininet/examples$ sudo ./3.py
      *** Adding controller
      *** Add switches
      *** Add hosts
      *** Add links
      (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) *** Starting network
      *** Starting controllers
      *** Configuring hosts
      h1 h5 h2 h6 h3 h4
      *** Starting controller
      c0
      *** Starting 2 switches
      s1 (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) s2 (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) ...(10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit)
      *** Enabling sFlow:
      s1 s2
      *** Sending topology
      *** Starting switches
      (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) *** Post configure switches and hosts
      *** Starting CLI:

      Delete
    6. You should delete the following lines (since net.start() has already started the switches):
      net.get('s1').start([c0])
      net.get('s2').start([c0])

      The topology should be showing up in the sFlow-RT Mininet-Dashboard application. You need to generate some traffic. Try running an iperf test, e.g.
      mininet> iperf h1 h4

      Delete
    7. Thanks Peter, You're a champion..
      Removing these lines solved the problem finally by doing step by step as you said..
      net.get('s1').start([c0])
      net.get('s2').start([c0])..

      Great Thanks Man :)

      Well, could you help regarding the configuration of OpenFlow for the same network? Thanks in advance.

      Delete
  4. It's the script generated through MiniEdit export level 2 script. Can you please elaborate in detail? Or you may do the required change please.

    ReplyDelete
  5. Hello everyone
    I just installed sFlow-rt to monitor some outputs. I really want to connect it with mininet-wifi but I get an error.
    I also tried to connect it with mininet (sudo mn --custom extras/sflow.py --link tc,bw=10 --topo tree,depth=2,fanout=2 --test iperf) but I get an error too.

    -----------------------------------------------------------------------------------------
    ~/Downloads/sflow-rt$ sudo mn --custom extras/sflow.py --link tc,bw=10 --topo tree,depth=2,fanout=2 --test iperf
    --------------------------------------------------------------------------------
    Caught exception. Cleaning up...

    ImportError: No module named requests
    --------------------------------------------------------------------------------

    On the other hand, the developers of mininet-wifi have also created a script as example to use sflow with stations and access points and measure all kind of tests. But I get an error too.
    ---------------------------------------------------------------------------------------------------------------------
    ~/mininet-wifi$ sudo python mn-wifi-book-en/codes/cap4/sflow-rt.py
    Traceback (most recent call last):
    File "mn-wifi-book-en/codes/cap4/sflow-rt.py", line 14, in
    from requests import put
    ImportError: No module named requests
    ------------------------------------------------------------------------------------------------------------------------
    this is the sflow-rt.py code
    https://github.com/ramonfontes/mn-wifi-book-en/blob/master/codes/cap4/sflow-rt.py

    I would appreciate any comments

    ReplyDelete
  6. i am getting this error.please suggest me a solution

    --------------------------------------------------------------------------------
    Caught exception. Cleaning up...

    NameError: name 'execfile' is not defined

    ReplyDelete
  7. Hi Peter, i face some problem. I follow the script from Mr. Unknown and the coding works perfectly. However i face this problem, would you help me to solve this problem?

    Problem: raise ConnectionError(e, request=request)
    requests.exceptions.ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=8008): Max retries exceeded with url: /topology/json (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused',))

    Code:
    #!/usr/bin/python

    from mininet.net import Mininet
    from mininet.node import Controller, RemoteController, OVSController
    from mininet.node import CPULimitedHost, Host, Node
    from mininet.node import OVSKernelSwitch, UserSwitch
    from mininet.node import IVSSwitch
    from mininet.cli import CLI
    from mininet.log import setLogLevel, info
    from mininet.link import TCLink, Intf
    from subprocess import call
    from mininet.util import customClass
    from mininet.link import TCLink

    # Compile and run sFlow helper script
    # - configures sFlow on OVS
    # - posts topology to sFlow-RT
    execfile('/home/mininet/sflow-rt/extras/sflow.py')

    # Rate limit links to 10Mbps
    link = customClass({'tc':TCLink}, 'tc,bw=10')

    def myNetwork():

    net = Mininet( topo=None,
    build=False,
    ipBase='10.0.0.0/8',
    link=link)

    info( '*** Adding controller\n' )
    c0=net.addController(name='c0',
    controller=RemoteController,
    ip='127.0.0.1',
    protocol='tcp',
    port=6633)

    info( '*** Add switches\n')
    s1 = net.addSwitch('s1', cls=OVSKernelSwitch)

    info( '*** Add hosts\n')
    h1 = net.addHost('h1', cls=Host, ip='10.0.0.1', defaultRoute=None)
    h5 = net.addHost('h5', cls=Host, ip='10.0.0.5', defaultRoute=None)
    h2 = net.addHost('h2', cls=Host, ip='10.0.0.2', defaultRoute=None)
    h6 = net.addHost('h6', cls=Host, ip='10.0.0.6', defaultRoute=None)
    h3 = net.addHost('h3', cls=Host, ip='10.0.0.3', defaultRoute=None)
    h4 = net.addHost('h4', cls=Host, ip='10.0.0.4', defaultRoute=None)

    info( '*** Add links\n')
    net.addLink(s1, h1)
    net.addLink(h2, s1)
    net.addLink(h3, s1)
    net.addLink(h4, s1)
    net.addLink(h5, s1)
    net.addLink(h6, s1)

    info( '*** Starting network\n')
    info( '*** Starting controllers\n')
    for controller in net.controllers:
    net.start()

    info( '*** Starting switches\n')


    info( '*** Post configure switches and hosts\n')

    CLI(net)
    net.stop()

    if __name__ == '__main__':
    setLogLevel( 'info' )
    myNetwork()


    ReplyDelete
    Replies
    1. Was sFlow-RT running? You need to start sFlow-RT before you start Mininet. The error looks like it is the sflow.py plugin trying to send the Mininet topology to sFlow-RT.

      Delete
    2. How do i run sFlow-RT?
      Is it needed to run the ./start.sh in new terminal?

      Delete
    3. Yes - run ./start.sh in a new terminal.

      You might want to also install the sFlow-RT Mininet Dashboard application

      Delete
    4. Wow. It works! It is amazing!
      Thanks Peter!

      Delete
    5. Hi, i am using ur code but i face a trouble that, when i m running ur code, it shows
      Traceback (most recent call last):
      File "./coba.py", line 3, in
      from mininet.net import Mininet
      ImportError: No module named mininet.net

      and i already run the Sflow-rt

      thanks for answering

      Delete
    6. Are you shure you have installed Mininet on the server? What happens if you type:

      sudo mn

      Delete
  8. Im getting an nameerror saying name TCLink is not defined

    ReplyDelete
    Replies
    1. from mininet.link import TCLink

      Did you include the above statement? Can you provide a detailed error report?

      Delete
  9. hallo sir, i use mr.peter code, but i have problem, which is ImportError: No module named mininet.net
    how to figure it out sir? thankyou

    ReplyDelete
  10. hello, Im getting a error while running sflow.py. The code gives error like " KeyError: s3-eth1". When examined the /sys/devices/virtual/net, I see only lo interface. How can I solce this?

    ReplyDelete
    Replies
    1. Could you provide some more details? What version of Linux, Python, Mininet, and sFlow-RT are you using? Was there are specific line number given in the error message?

      Delete
  11. Hi Peter, I created a custom mininet script & placed it in ~/mininet/custom directory. I have also installed floodlight as a controller and sflow-rt with mininet -dashboard. I am running Mininet version 2.3.1b1, python3 version 3.8.10

    from mininet.net import Mininet
    from mininet.cli import CLI
    from mininet.log import setLogLevel, info
    from mininet.topo import Topo
    from mininet.node import RemoteController
    import pdb
    import time
    import sys

    sys.path.insert(0,'/home/rishabh/sflow-rt/extras')
    import sflow
    from mininet.util import customClass
    from mininet.link import TCLink

    execfile('/home/rishabh/sflow-rt/extras/sflow.py')
    link = customClass({'tc':TCLink}, 'tc,bw=10')

    def myNet():

    net = Mininet(topo=None,build=False,link=link)

    info( '*** Adding controller\n')
    net.addController('c0', controller=RemoteController,ip="127.0.0.1",port=6653)

    info( '*** Adding hosts\n')
    h1=net.addHost('h1')
    h2=net.addHost('h2')
    h3=net.addHost('h3')
    h4=net.addHost('h4')
    h5=net.addHost('h5')
    h6=net.addHost('h6')
    h7=net.addHost('h7')
    h8=net.addHost('h8')
    h9=net.addHost('h9')
    h10=net.addHost('h10')

    info( '*** Adding switch\n' )
    s1=net.addSwitch('s1')
    s2=net.addSwitch('s2')
    s3=net.addSwitch('s3')
    s4=net.addSwitch('s4')
    s5=net.addSwitch('s5')

    info( '*** Creating links\n' )
    net.addLink(s1,s2)
    net.addLink(s2,s3)
    net.addLink(s3,s4)
    net.addLink(s4,s5)

    net.addLink(h1,s1)
    net.addLink(h2,s1)

    net.addLink(h3,s2)
    net.addLink(h4,s2)

    net.addLink(h5,s3)
    net.addLink(h6,s3)

    net.addLink(h7,s4)
    net.addLink(h8,s4)

    net.addLink(h9,s5)
    net.addLink(h10,s5)

    info( '*** Starting network\n')
    net.start()

    info( '*** Running CLI\n' )
    CLI( net )

    info( '*** Stopping network' )
    net.stop()


    if __name__ == '__main__':
    setLogLevel( 'info' )
    myNet()

    sflow-rt is running on a new tab and when i run the command "sudo python3 ./second.py" i get the error message as:
    Traceback (most recent call last):
    File "./second.py", line 1, in
    from mininet.net import Mininet
    ModuleNotFoundError: No module named 'mininet'

    ReplyDelete
    Replies
    1. you run mininet with root privilege you should install mininet to use it with root privilege
      use this commande to install mininet and use it with root privilege
      sudo pip install mininet

      Delete
  12. This comment has been removed by the author.

    ReplyDelete