Wednesday, October 3, 2018

Ryu measurement based control

ONOS measurement based control describes how real-time streaming telemetry can be used to automatically trigger SDN controller actions. The article uses DDoS mitigation as an example.

This article recreates the demonstration using the Ryu SDN framework and emulating a network using Mininet. Install both pieces of software on a Linux server or virtual machine in order to follow this example.

Start Ryu with the simple_switch and ryu.app.ofctl_rest applications loaded:
ryu-manager ryu.app.simple_switch,ryu.app.ofctl_rest
Note: The simple_switch and ofctl_rest scripts are part of a standard Ryu installation.
This demonstration uses the sFlow-RT real-time analytics engine to process standard sFlow streaming telemetry from the network switches.

Download sFlow-RT:
wget https://inmon.com/products/sFlow-RT/sflow-rt.tar.gz
tar -xvzf sflow-rt.tar.gz
Install the Mininet Dashboard application:
sflow-rt/get-app.sh sflow-rt mininet-dashboard
The following script, ryu.js, implements the DDoS mitigation function described in the previous article:
var ryu = '127.0.0.1';
var controls = {};

setFlow('udp_reflection',
 {keys:'ipdestination,udpsourceport',value:'frames'});
setThreshold('udp_reflection_attack',
 {metric:'udp_reflection',value:100,byFlow:true,timeout:2});

setEventHandler(function(evt) {
 // don't consider inter-switch links
 var link = topologyInterfaceToLink(evt.agent,evt.dataSource);
 if(link) return;

 // get port information
 var port = topologyInterfaceToPort(evt.agent,evt.dataSource);
 if(!port) return;

 // need OpenFlow info to create Ryu filtering rule
 if(!port.dpid || !port.ofport) return;

 // we already have a control for this flow
 if(controls[evt.flowKey]) return;

 var [ipdestination,udpsourceport] = evt.flowKey.split(',');
 var msg = {
  priority:40000,
  dpid:parseInt(port.dpid,16),
  match: {
   in_port:port.ofport,
   dl_type:0x800,
   nw_dst:ipdestination+'/32',
   nw_proto:17,
   tp_src:udpsourceport 
  }
 };

 var resp = http2({
  url:'http://'+ryu+':8080/stats/flowentry/add',
  headers:{'Content-Type':'application/json','Accept':'application/json'},
  operation:'post',
  body: JSON.stringify(msg)
 });

 controls[evt.flowKey] = {
  time:Date.now(),
  threshold:evt.thresholdID,
  agent:evt.agent,
  metric:evt.dataSource+'.'+evt.metric,
  msg:msg
 };

 logInfo("blocking " + evt.flowKey);
},['udp_reflection_attack']);

setIntervalHandler(function() {
 var now = Date.now();
 for(var key in controls) {
  let rec = controls[key];

  // keep control for at least 10 seconds
  if(now - rec.time < 10000) continue;
  // keep control if threshold still triggered
  if(thresholdTriggered(rec.threshold,rec.agent,rec.metric,key)) continue;

  var resp = http2({
   url:'http://'+ryu+':8080/stats/flowentry/delete',
   headers:{'Content-Type':'application/json','Accept':'application/json'},
   operation:'post',
   body: JSON.stringify(rec.msg)
  });

  delete controls[key];

  logInfo("unblocking " + key);
 }
});
Some notes on the script:
  1. The Ryu ryu.app.ofctl_rest is used to add/remove filters that block the DDoS traffic
  2. The udp_reflection flow definition is designed to detect UDP amplification attacks, e.g. DNS amplification attacks
  3. Controls are applied to the switch port where traffic enters the network
  4. The controls structure is used to keep track of state associated with deployed configuration changes so that they can be undone
  5. The intervalHandler() function is used to automatically release controls after 10 seconds - the timeout is short for the purposes of demonstration, in practical deployments the timeout would be much measured in hours
  6. For simplicity, this script is missing the error handling needed for production use.
  7. See Writing Applications for more information.
Run the following command to start sFlow-RT and run the ryu.js script:
./sflow-rt/start.sh -Dscript.file=../ryu.js
We are going to use hping3 to simulate a DDoS attack, so install the software using the following command:
sudo apt install hping3
Next, start Mininet:
sudo mn --custom sflow-rt/extras/sflow.py --link tc,bw=10 \
--controller=remote,ip=127.0.0.1 --topo tree,depth=2,fanout=2
Generate normal traffic between hosts h1 and h3:
mininet> iperf h1 h3
The weathermap view shows the flow crossing the network from switch s2 to s3 via s1.
Generate an attack:
mininet> h1 hping3 --flood --udp -k -s 53 h3
The weathermap view verifies that the attack has been successfully blocked since none of the traffic is seen traversing the network.

The chart at the top of this article shows the iperf test followed by the simulated attack. The top chart shows the top flows entering the network, showing the DNS amplification attack traffic in blue. The middle chart shows traffic broken out by switch port. Here, the blue line shows the attack traffic arriving at switch s2 port s2-eth1 while the red line shows that only a small amount of traffic is forwarded to switch s3 port s3-eth3 before the attack is blocked at switch s2 by the controller.

Mininet with Ryu and sFlow-RT is a great way to rapidly develop and test SDN applications, avoiding the time and expense involved in setting up a physical network. The application is easily moved from the Mininet virtual network to a physical network since it is based on the same industry standard sFlow telemetry generated by physical switches. In this case, using commodity switch hardware to cost effectively detect and filter massive (100's of Gbit/s) DDoS attacks.

Note: Northbound Networks Zodiac GX is an inexpensive gigabit switch that provides a convenient way to transition from an emulated Mininet environment to a physical network handling real traffic.

33 comments:

  1. Thanks for the post. I got this message when running ryu.js

    019-02-10T00:28:55+0100 WARNING: /home/aymen/Desktop/ryu.js IO exception /home/aymen/Desktop/ryu.js
    2019-02-10T00:28:55+0100 INFO: /home/aymen/Desktop/ryu.js stopped

    ReplyDelete
  2. I am facing a problem with the ryu.js script while running.

    var [ipdestination,udpsourceport] = evt.flowKey.split(',');
    ^
    SyntaxError: Unexpected token [
    at Module._compile (module.js:439:25)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)
    at node.js:902:3

    This might be causing the ryu.js to be stopped while entering the command : env "RTPROP=-Dscript.file=$PWD/ryu.js" sflow-rt/start.sh

    ReplyDelete
    Replies
    1. The ryu.js script is run by sFlow-RT, it appears you are trying to run it using node.js. Use the following command to start sFlow-RT with the ryu.js script:

      env "RTPROP=-Dscript.file=$PWD/ryu.js" sflow-rt/start.sh

      Delete
    2. Hi, Please could you follow my issue above, I am using the right command but I am getting IO exception while running ryu.js!

      Delete
    3. You need to start RYU (with ryu.app.ofctl_rest) before you start sFlow-RT (with the ryu.js). The IO exception looks like a failed http request to the RYU rest API because RYU wasn't running.

      Delete
  3. hi peter,
    I want to know how ONOS AND Ryu's controllers mitigate DDoS attacks. Controllers filter the DDoS traffic or block the attacker. We can see action =Drop.

    Please help me to understand the mitigation solution. Are controllers using the typical (RTBH) blackhole strategy to mitigate DDoS? I mean forward the packet to a specific destination or null addresses? If yes, then what is the destination IP for the blackhole (OR FILTER) traffic on these controllers?

    ReplyDelete
    Replies
    1. The example in this article emulates the type of control you could implement with BGP Flowspec (see Real-time DDoS mitigation using sFlow and BGP FlowSpec). In this case the simulated DNS amplification attack is filtered based on UDP source port and destination IP address.

      One of the advantages of using BGP RTBH / Flowspec for mitigation is that the controls can be propagated to upstream neighbor routers (via BGP) to drop the traffic close to its source.

      Delete
  4. hi peter when I run the attack command hping... my ryu.js automatically stopped and is giving error
    server returned http response code 400 for http 127.0.0.1:8080.
    Before attack it is running accurately but after attack it stops with giving this error.

    ReplyDelete
    Replies
    1. The HTTP 400 error indicates a bad request. It's likely that the ryu.app.ofctl_rest API has changed since this article was written. Have you checked to see if Ryu is logging any information about why it is rejecting the request?

      Delete
    2. Have you checked that Mininet is connected to Ryu (--controller=remote,ip=127.0.0.1)? If Ryu isn't the controller, then it will reject the REST call since it doesn't know about the switch.

      Delete
    3. onos example is running perfectly fine for me. But this ryu.js stopped when the attack called. And if I change the call like stats/flow/add/1 it is also running, just this flowentry/add is not running. I have checked the addresses at API site the addresses are looking fine. Can you plz double check for me? thanks

      Delete
    4. Can you check the Ryu logs? There should be information explaining why the HTTP request was rejected.

      Delete
    5. Hi~Peter:I have the same problem, RYU responded as follows,can you help me,thanks.

      Invalid dpid: 0000000000000002
      Traceback (most recent call last):
      File "/root/ryu/ryu/app/ofctl_rest.py", line 258, in wrapper
      dp = self.dpset.get(int(str(dpid), 0))
      ValueError: invalid literal for int() with base 0: '0000000000000002'
      127.0.0.1 - - [02/Dec/2020 23:28:20] "POST /stats/flowentry/add HTTP/1.1" 400 143 0.057866

      Delete
    6. What version of RYU are you using? I haven't had any problems with 4.15. Given the error you are seeing, you might want to make the following change to your ryu.js script.

      Change the line:
      dpid:port.dpid,

      To:
      dpid:parseInt(port.dpid),

      Delete
    7. Actually, that should be:
      dpid:parseInt(port.dpid,16),

      Delete
    8. It works!!, thank you very much~ ^^

      Delete
  5. When i am trying to create this custom topology
    sudo mn --custom sflow-rt/extras/sflow.py --link tc,bw=10 --controller=remote,ip=127.0.0.1 --topo tree,depth=2,fanout=2

    I am getting the following error;

    *** Starting controller
    c0
    *** Starting 3 switches
    s1 s2 s3 ...(10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit)
    --------------------------------------------------------------------------------
    Caught exception. Cleaning up...

    UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc0 in position 60: invalid start byte
    --------------------------------------------------------------------------------

    ReplyDelete
    Replies
    1. That looks like a Mininet / Python error. What version of Python are you using? I don't think Mininet supports Python3 at this time.

      The following article might be useful:
      https://blog.sflow.com/2020/11/multipass.html.

      Even if you aren't going to use Multipass to build a virtual machine, it gives the steps needed to build an Ubuntu 18.04 VM with Mininet and Ryu. Ubuntu 18 was chosen because it still has Python 2.7 as the default.

      Delete
  6. I could solve the error by changing namestr = namestr.decode('uitf-8') to namestr = namestr.decode('cp1252') in the sflow.py file

    ReplyDelete
    Replies
    1. Thanks for the information. It helped identify a bug which I believe is now fixed in the latest, 3.0-1538, release.

      Delete
  7. Hi Peter, I am facing another issue.
    I am following the exact steps as mentioned in this article,but when i generate ping attack, i get the following error on the sflow-rt application.
    2020-11-29T19:59:21-08:00 WARNING: /home/sneha/sflow-rt/ryu.js /home/sneha/sflow-rt/ryu.js#37 IO error java.io.IOException: Server returned HTTP response code: 400 for URL: http://127.0.0.1:8080/stats/flowentry/add

    ReplyDelete
  8. Hi~Peter
    I have the same problem(http response code 400), RYU controller responded as follows, can you help me? Thanks.

    Invalid dpid: 0000000000000002
    Traceback (most recent call last):
    File "/root/ryu/ryu/app/ofctl_rest.py", line 258, in wrapper
    dp = self.dpset.get(int(str(dpid), 0))
    ValueError: invalid literal for int() with base 0: '0000000000000002'
    127.0.0.1 - - [02/Dec/2020 23:28:20] "POST /stats/flowentry/add HTTP/1.1" 400 143 0.057866

    ReplyDelete
  9. Hi Peter
    When I use the hping3 (DDoS) commands, it only flood the destination IP with 60K bits per second (from sflow dashboard).
    I try to generate some normal traffic flow to modify the environment and i am able to get the code from GitHub (https://github.com/stainleebakhla/mininet-flow-generator). However, the normal traffic flows have much higher bits per second sent through than the hping3 command.
    What should i do to overcome this strange phenomenon ?

    ReplyDelete
  10. Hi Peter I need only collect data from switch and send it to controller to used latter in migration process how can I do this?

    ReplyDelete
  11. hello sir,
    I followed all the steps that you have written but i am facing 'ImportError: No module named requests' error! while creating mininet topology.
    But, requests module is already been installed in the system. I tried by reinstalling requests module but i am facing same error all the time. what should i do? please help!

    ReplyDelete
    Replies
    1. Are you running Mininet using Python 2 or Python 3? Did you install the matching version of requests, i.e. "pip3 install requests" for Python3

      Delete
  12. Hi, Peter.
    Do you have any other attack examples with RYU controller?

    ReplyDelete
    Replies
    1. I am sorry I don't have other examples with RYU. You should be able to modify the example in this article to simulate and mitigate other types of attack. ddos-protect has signatures for a variety of volumetric attacks that you could look at.

      Delete
    2. I actually wanted to try with TCP attack(like SYN Attack), but I can't figure out where to change of the code.

      Am I right if I change nw_proto:17 to nw_proto:6 ?

      I really need your help. Thank you.

      Delete
    3. You would also need to modify the setFlow() options to filter on TCP flags, see Defining Flows

      Delete
    4. I was wondering if you have successfully done this alteration as im having a hard time to modify the code to both detect and mitigate ICMP attacks as well along with the UDP attacks that are already implemented in the script.

      note - The ryu script is working fine but it only mitigates UDP attacks

      Delete
    5. DDoS Protect is an example that mitigates multiple attack types. You need to add a setFlow() and setThreshold() to monitor for the ICMP attack. You would also need to modify the setEventHandler to apply different controls depending on the evt.thresholdID.

      Delete