Friday, June 14, 2019

Mininet flow analytics with custom scripts

Mininet flow analytics describes how to use the sflow.py helper script that ships with the sFlow-RT analytics engine to enable sFlow telemetry, e.g.
sudo mn --custom sflow-rt/extras/sflow.py --link tc,bw=10 \
--topo tree,depth=2,fanout=2
Mininet, ONOS, and segment routing provides an example using a Custom Topology, e.g.
sudo env ONOS=10.0.0.73 mn --custom sr.py,sflow-rt/extras/sflow.py \
--link tc,bw=10 --topo=sr '--controller=remote,ip=$ONOS,port=6653'
This article describes how to incorporate sFlow monitoring in a fully custom Mininet script. Consider the following simpletest.py script based on Working with Mininet:
#!/usr/bin/python                                                                            
                                                                                             
from mininet.topo import Topo
from mininet.net import Mininet
from mininet.util import dumpNodeConnections
from mininet.log import setLogLevel

class SingleSwitchTopo(Topo):
    "Single switch connected to n hosts."
    def build(self, n=2):
        switch = self.addSwitch('s1')
        # Python's range(N) generates 0..N-1
        for h in range(n):
            host = self.addHost('h%s' % (h + 1))
            self.addLink(host, switch)

def simpleTest():
    "Create and test a simple network"
    topo = SingleSwitchTopo(n=4)
    net = Mininet(topo)
    net.start()
    print "Dumping host connections"
    dumpNodeConnections(net.hosts)
    print "Testing bandwidth between h1 and h4"
    h1, h4 = net.get( 'h1', 'h4' )
    net.iperf( (h1, h4) )
    net.stop()

if __name__ == '__main__':
    # Tell mininet to print useful information
    setLogLevel('info')
    simpleTest()
Add the highlighted lines to incorporate sFlow telemetry:
#!/usr/bin/python                                                                            
                                                                                             
from mininet.topo import Topo
from mininet.net import Mininet
from mininet.util import dumpNodeConnections
from mininet.log import setLogLevel
from mininet.util import customClass
from mininet.link import TCLink

# Compile and run sFlow helper script
# - configures sFlow on OVS
# - posts topology to sFlow-RT
execfile('sflow-rt/extras/sflow.py') 

# Rate limit links to 10Mbps
link = customClass({'tc':TCLink}, 'tc,bw=10')

class SingleSwitchTopo(Topo):
    "Single switch connected to n hosts."
    def build(self, n=2):
        switch = self.addSwitch('s1')
        # Python's range(N) generates 0..N-1
        for h in range(n):
            host = self.addHost('h%s' % (h + 1))
            self.addLink(host, switch)

def simpleTest():
    "Create and test a simple network"
    topo = SingleSwitchTopo(n=4)
    net = Mininet(topo,link=link)
    net.start()
    print "Dumping host connections"
    dumpNodeConnections(net.hosts)
    print "Testing bandwidth between h1 and h4"
    h1, h4 = net.get( 'h1', 'h4' )
    net.iperf( (h1, h4) )
    net.stop()

if __name__ == '__main__':
    # Tell mininet to print useful information
    setLogLevel('info')
    simpleTest()
When running the script the highlighted output confirms that sFlow has been enabled and the topology has been posted to sFlow-RT:
pp@mininet:~$ sudo ./simpletest.py
*** Creating network
*** Adding controller
*** Adding hosts:
h1 h2 h3 h4 
*** Adding switches:
s1 
*** Adding links:
(10.00Mbit) (10.00Mbit) (h1, s1) (10.00Mbit) (10.00Mbit) (h2, s1) (10.00Mbit) (10.00Mbit) (h3, s1) (10.00Mbit) (10.00Mbit) (h4, s1) 
*** Configuring hosts
h1 h2 h3 h4 
*** Starting controller
c0 
*** Starting 1 switches
s1 ...(10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) 
*** Enabling sFlow:
s1
*** Sending topology
Dumping host connections
h1 h1-eth0:s1-eth1
h2 h2-eth0:s1-eth2
h3 h3-eth0:s1-eth3
h4 h4-eth0:s1-eth4
Testing bandwidth between h1 and h4
*** Iperf: testing TCP bandwidth between h1 and h4 
*** Results: ['6.32 Mbits/sec', '6.55 Mbits/sec']
*** Stopping 1 controllers
c0 
*** Stopping 4 links
....
*** Stopping 1 switches
s1 
*** Stopping 4 hosts
h1 h2 h3 h4 
*** Done
Mininet dashboard and Mininet weathermap describe the sFlow-RT Mininet Dashboard application shown at the top of this article. The tool provides a real-time visualization of traffic flowing over the Mininet network. Writing Applications describes how to develop custom analytics applications for sFlow-RT.

13 comments:

  1. Hi Peter,
    Above lines aren't working with my case.

    mininet@mininet-vm:~/mininet/examples$ sudo ./2.py
    *** Adding controller
    *** Add switches
    *** Add hosts
    *** Add links
    *** Starting network
    *** Configuring hosts
    h1 h5 h2 h6 h3 h4
    *** Starting controllers
    *** Starting switches
    *** Post configure switches and hosts
    *** Starting CLI:

    ReplyDelete
  2. The execfile('sflow-rt/extras/sflow.py') statement contains a relative path. It looks like you are running your mininet script in the the ~/mininet/examples directory, so you would need to change the path so that sflow.py can be found (e.g. '../../sflow-rt/extras/sflow.py' if you installed sflow-rt in your home directory).

    ReplyDelete
  3. I've used this path in code :
    execfile('/home/mininet/sflow-rt/extras/sflow.py'). Network gets started without sflow.

    If I try to do some changes like below, it gives:

    execfile('home/mininet/sflow-rt/extras/sflow.py')
    IOError: [Errno 2] No such file or directory: 'home/mininet/sflow-rt/extras/sflow.py'

    ReplyDelete
    Replies
    1. It looks like the first version is loading the sflow.py code. Is this the first line in your custom script? Does your script run net.start()? Loading the sflow.py library modifies net.start() to enable sFlow functionality.

      Delete
    2. This is my code for your review please:

      #!/usr/bin/python

      from mininet.net import Mininet
      from mininet.node import Controller, RemoteController, OVSController
      from mininet.node import CPULimitedHost, Host, Node
      from mininet.node import OVSKernelSwitch, UserSwitch
      from mininet.node import IVSSwitch
      from mininet.cli import CLI
      from mininet.log import setLogLevel, info
      from mininet.link import TCLink, Intf
      from subprocess import call
      from mininet.util import customClass
      from mininet.link import TCLink

      # Compile and run sFlow helper script
      # - configures sFlow on OVS
      # - posts topology to sFlow-RT
      execfile('/home/mininet/sflow-rt/extras/sflow.py')

      # Rate limit links to 10Mbps
      link = customClass({'tc':TCLink}, 'tc,bw=10')

      def myNetwork():

      net = Mininet( topo=None,
      build=False,
      ipBase='10.0.0.0/8',
      link=link)

      info( '*** Adding controller\n' )
      c0=net.addController(name='c0',
      controller=RemoteController,
      ip='192.168.15.141',
      protocol='tcp',
      port=6633)

      info( '*** Add switches\n')
      s1 = net.addSwitch('s1', cls=OVSKernelSwitch)
      s2 = net.addSwitch('s2', cls=OVSKernelSwitch)

      info( '*** Add hosts\n')
      h1 = net.addHost('h1', cls=Host, ip='10.0.0.1', defaultRoute=None)
      h5 = net.addHost('h5', cls=Host, ip='10.0.0.5', defaultRoute=None)
      h2 = net.addHost('h2', cls=Host, ip='10.0.0.2', defaultRoute=None)
      h6 = net.addHost('h6', cls=Host, ip='10.0.0.6', defaultRoute=None)
      h3 = net.addHost('h3', cls=Host, ip='10.0.0.3', defaultRoute=None)
      h4 = net.addHost('h4', cls=Host, ip='10.0.0.4', defaultRoute=None)

      info( '*** Add links\n')
      net.addLink(s1, h5)
      net.addLink(s1, h1)
      net.addLink(s1, h2)
      net.addLink(s2, h6)
      net.addLink(s2, h3)
      net.addLink(s2, h4)
      net.addLink(s1, s2)

      info( '*** Starting network\n')
      net.build()
      info( '*** Starting controllers\n')
      for controller in net.controllers:
      controller.start()

      info( '*** Starting switches\n')
      net.get('s1').start([c0])
      net.get('s2').start([c0])

      info( '*** Post configure switches and hosts\n')

      CLI(net)
      net.stop()

      if __name__ == '__main__':
      setLogLevel( 'info' )
      myNetwork()

      Delete
    3. Why are you starting each switch individually? You are bypassing the net.start() function that would enable sFlow. Can you replace the two statements starting each switch with net.start()?

      Delete
    4. You should probably remove the net.build() and the controller.start() code since I believe net.start() will perform these functions.

      Delete
    5. Please find this code and it's output but graphs and resources are not being populated on sFlow.

      Code:
      #!/usr/bin/python

      from mininet.net import Mininet
      from mininet.node import Controller, RemoteController, OVSController
      from mininet.node import CPULimitedHost, Host, Node
      from mininet.node import OVSKernelSwitch, UserSwitch
      from mininet.node import IVSSwitch
      from mininet.cli import CLI
      from mininet.log import setLogLevel, info
      from mininet.link import TCLink, Intf
      from subprocess import call
      from mininet.util import customClass
      from mininet.link import TCLink

      # Compile and run sFlow helper script
      # - configures sFlow on OVS
      # - posts topology to sFlow-RT
      execfile('/home/mininet/sflow-rt/extras/sflow.py')

      # Rate limit links to 10Mbps
      link = customClass({'tc':TCLink}, 'tc,bw=10')

      def myNetwork():

      net = Mininet( topo=None,
      build=False,
      ipBase='10.0.0.0/8',
      link=link)

      info( '*** Adding controller\n' )
      c0=net.addController(name='c0',
      controller=RemoteController,
      ip='192.168.15.141',
      protocol='tcp',
      port=6633)

      info( '*** Add switches\n')
      s1 = net.addSwitch('s1', cls=OVSKernelSwitch)
      s2 = net.addSwitch('s2', cls=OVSKernelSwitch)

      info( '*** Add hosts\n')
      h1 = net.addHost('h1', cls=Host, ip='10.0.0.1', defaultRoute=None)
      h5 = net.addHost('h5', cls=Host, ip='10.0.0.5', defaultRoute=None)
      h2 = net.addHost('h2', cls=Host, ip='10.0.0.2', defaultRoute=None)
      h6 = net.addHost('h6', cls=Host, ip='10.0.0.6', defaultRoute=None)
      h3 = net.addHost('h3', cls=Host, ip='10.0.0.3', defaultRoute=None)
      h4 = net.addHost('h4', cls=Host, ip='10.0.0.4', defaultRoute=None)

      info( '*** Add links\n')
      net.addLink(s1, h5)
      net.addLink(s1, h1)
      net.addLink(s1, h2)
      net.addLink(s2, h6)
      net.addLink(s2, h3)
      net.addLink(s2, h4)
      net.addLink(s1, s2)

      info( '*** Starting network\n')
      info( '*** Starting controllers\n')
      for controller in net.controllers:
      net.start()

      info( '*** Starting switches\n')
      net.get('s1').start([c0])
      net.get('s2').start([c0])

      info( '*** Post configure switches and hosts\n')

      CLI(net)
      net.stop()

      if __name__ == '__main__':
      setLogLevel( 'info' )
      myNetwork()


      Output:
      mininet@mininet-vm:~/mininet/examples$ sudo ./3.py
      *** Adding controller
      *** Add switches
      *** Add hosts
      *** Add links
      (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) *** Starting network
      *** Starting controllers
      *** Configuring hosts
      h1 h5 h2 h6 h3 h4
      *** Starting controller
      c0
      *** Starting 2 switches
      s1 (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) s2 (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) ...(10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit)
      *** Enabling sFlow:
      s1 s2
      *** Sending topology
      *** Starting switches
      (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) *** Post configure switches and hosts
      *** Starting CLI:

      Delete
    6. You should delete the following lines (since net.start() has already started the switches):
      net.get('s1').start([c0])
      net.get('s2').start([c0])

      The topology should be showing up in the sFlow-RT Mininet-Dashboard application. You need to generate some traffic. Try running an iperf test, e.g.
      mininet> iperf h1 h4

      Delete
    7. Thanks Peter, You're a champion..
      Removing these lines solved the problem finally by doing step by step as you said..
      net.get('s1').start([c0])
      net.get('s2').start([c0])..

      Great Thanks Man :)

      Well, could you help regarding the configuration of OpenFlow for the same network? Thanks in advance.

      Delete
  4. It's the script generated through MiniEdit export level 2 script. Can you please elaborate in detail? Or you may do the required change please.

    ReplyDelete
  5. Hello everyone
    I just installed sFlow-rt to monitor some outputs. I really want to connect it with mininet-wifi but I get an error.
    I also tried to connect it with mininet (sudo mn --custom extras/sflow.py --link tc,bw=10 --topo tree,depth=2,fanout=2 --test iperf) but I get an error too.

    -----------------------------------------------------------------------------------------
    ~/Downloads/sflow-rt$ sudo mn --custom extras/sflow.py --link tc,bw=10 --topo tree,depth=2,fanout=2 --test iperf
    --------------------------------------------------------------------------------
    Caught exception. Cleaning up...

    ImportError: No module named requests
    --------------------------------------------------------------------------------

    On the other hand, the developers of mininet-wifi have also created a script as example to use sflow with stations and access points and measure all kind of tests. But I get an error too.
    ---------------------------------------------------------------------------------------------------------------------
    ~/mininet-wifi$ sudo python mn-wifi-book-en/codes/cap4/sflow-rt.py
    Traceback (most recent call last):
    File "mn-wifi-book-en/codes/cap4/sflow-rt.py", line 14, in
    from requests import put
    ImportError: No module named requests
    ------------------------------------------------------------------------------------------------------------------------
    this is the sflow-rt.py code
    https://github.com/ramonfontes/mn-wifi-book-en/blob/master/codes/cap4/sflow-rt.py

    I would appreciate any comments

    ReplyDelete