Wednesday, October 3, 2018

Ryu measurement based control

ONOS measurement based control describes how real-time streaming telemetry can be used to automatically trigger SDN controller actions. The article uses DDoS mitigation as an example.

This article recreates the demonstration using the Ryu SDN framework and emulating a network using Mininet. Install both pieces of software on a Linux server or virtual machine in order to follow this example.

Start Ryu with the simple_switch and ryu.app.ofctl_rest applications loaded:
ryu-manager ryu.app.simple_switch,ryu.app.ofctl_rest
Note: The simple_switch and ofctl_rest scripts are part of a standard Ryu installation.
This demonstration uses the sFlow-RT real-time analytics engine to process standard sFlow streaming telemetry from the network switches.

Download sFlow-RT:
wget https://inmon.com/products/sFlow-RT/sflow-rt.tar.gz
tar -xvzf sflow-rt.tar.gz
Install the Mininet Dashboard application:
sflow-rt/get-app.sh sflow-rt mininet-dashboard
The following script, ryu.js, implements the DDoS mitigation function described in the previous article:
var ryu = '127.0.0.1';
var controls = {};

setFlow('udp_reflection',
 {keys:'ipdestination,udpsourceport',value:'frames'});
setThreshold('udp_reflection_attack',
 {metric:'udp_reflection',value:100,byFlow:true,timeout:2});

setEventHandler(function(evt) {
 // don't consider inter-switch links
 var link = topologyInterfaceToLink(evt.agent,evt.dataSource);
 if(link) return;

 // get port information
 var port = topologyInterfaceToPort(evt.agent,evt.dataSource);
 if(!port) return;

 // need OpenFlow info to create Ryu filtering rule
 if(!port.dpid || !port.ofport) return;

 // we already have a control for this flow
 if(controls[evt.flowKey]) return;

 var [ipdestination,udpsourceport] = evt.flowKey.split(',');
 var msg = {
  priority:40000,
  dpid:port.dpid,
  match: {
   in_port:port.ofport,
   dl_type:0x800,
   nw_dst:ipdestination+'/32',
   nw_proto:17,
   tp_src:udpsourceport 
  }
 };

 var resp = http2({
  url:'http://'+ryu+':8080/stats/flowentry/add',
  headers:{'Content-Type':'application/json','Accept':'application/json'},
  operation:'post',
  body: JSON.stringify(msg)
 });

 controls[evt.flowKey] = {
  time:Date.now(),
  threshold:evt.thresholdID,
  agent:evt.agent,
  metric:evt.dataSource+'.'+evt.metric,
  msg:msg
 };

 logInfo("blocking " + evt.flowKey);
},['udp_reflection_attack']);

setIntervalHandler(function() {
 var now = Date.now();
 for(var key in controls) {
  let rec = controls[key];

  // keep control for at least 10 seconds
  if(now - rec.time < 10000) continue;
  // keep control if threshold still triggered
  if(thresholdTriggered(rec.threshold,rec.agent,rec.metric,key)) continue;

  var resp = http2({
   url:'http://'+ryu+':8080/stats/flowentry/delete',
   headers:{'Content-Type':'application/json','Accept':'application/json'},
   operation:'post',
   body: JSON.stringify(rec.msg)
  });

  delete controls[key];

  logInfo("unblocking " + key);
 }
});
Some notes on the script:
  1. The Ryu ryu.app.ofctl_rest is used to add/remove filters that block the DDoS traffic
  2. The udp_reflection flow definition is designed to detect UDP amplification attacks, e.g. DNS amplification attacks
  3. Controls are applied to the switch port where traffic enters the network
  4. The controls structure is used to keep track of state associated with deployed configuration changes so that they can be undone
  5. The intervalHandler() function is used to automatically release controls after 10 seconds - the timeout is short for the purposes of demonstration, in practical deployments the timeout would be much measured in hours
  6. For simplicity, this script is missing the error handling needed for production use.
  7. See Writing Applications for more information.
Run the following command to start sFlow-RT and run the ryu.js script:
./sflow-rt/start.sh -Dscript.file=../ryu.js
We are going to use hping3 to simulate a DDoS attack, so install the software using the following command:
sudo apt install hping3
Next, start Mininet:
sudo mn --custom sflow-rt/extras/sflow.py --link tc,bw=10 \
--controller=remote,ip=127.0.0.1 --topo tree,depth=2,fanout=2
Generate normal traffic between hosts h1 and h3:
mininet> iperf h1 h3
The weathermap view shows the flow crossing the network from switch s2 to s3 via s1.
Generate an attack:
mininet> h1 hping3 --flood --udp -k -s 53 h3
The weathermap view verifies that the attack has been successfully blocked since none of the traffic is seen traversing the network.

The chart at the top of this article shows the iperf test followed by the simulated attack. The top chart shows the top flows entering the network, showing the DNS amplification attack traffic in blue. The middle chart shows traffic broken out by switch port. Here, the blue line shows the attack traffic arriving at switch s2 port s2-eth1 while the red line shows that only a small amount of traffic is forwarded to switch s3 port s3-eth3 before the attack is blocked at switch s2 by the controller.

Mininet with Ryu and sFlow-RT is a great way to rapidly develop and test SDN applications, avoiding the time and expense involved in setting up a physical network. The application is easily moved from the Mininet virtual network to a physical network since it is based on the same industry standard sFlow telemetry generated by physical switches. In this case, using commodity switch hardware to cost effectively detect and filter massive (100's of Gbit/s) DDoS attacks.

Note: Northbound Networks Zodiac GX is an inexpensive gigabit switch that provides a convenient way to transition from an emulated Mininet environment to a physical network handling real traffic.

14 comments:

  1. Thanks for the post. I got this message when running ryu.js

    019-02-10T00:28:55+0100 WARNING: /home/aymen/Desktop/ryu.js IO exception /home/aymen/Desktop/ryu.js
    2019-02-10T00:28:55+0100 INFO: /home/aymen/Desktop/ryu.js stopped

    ReplyDelete
  2. I am facing a problem with the ryu.js script while running.

    var [ipdestination,udpsourceport] = evt.flowKey.split(',');
    ^
    SyntaxError: Unexpected token [
    at Module._compile (module.js:439:25)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)
    at node.js:902:3

    This might be causing the ryu.js to be stopped while entering the command : env "RTPROP=-Dscript.file=$PWD/ryu.js" sflow-rt/start.sh

    ReplyDelete
    Replies
    1. The ryu.js script is run by sFlow-RT, it appears you are trying to run it using node.js. Use the following command to start sFlow-RT with the ryu.js script:

      env "RTPROP=-Dscript.file=$PWD/ryu.js" sflow-rt/start.sh

      Delete
    2. Hi, Please could you follow my issue above, I am using the right command but I am getting IO exception while running ryu.js!

      Delete
    3. You need to start RYU (with ryu.app.ofctl_rest) before you start sFlow-RT (with the ryu.js). The IO exception looks like a failed http request to the RYU rest API because RYU wasn't running.

      Delete
  3. hi peter,
    I want to know how ONOS AND Ryu's controllers mitigate DDoS attacks. Controllers filter the DDoS traffic or block the attacker. We can see action =Drop.

    Please help me to understand the mitigation solution. Are controllers using the typical (RTBH) blackhole strategy to mitigate DDoS? I mean forward the packet to a specific destination or null addresses? If yes, then what is the destination IP for the blackhole (OR FILTER) traffic on these controllers?

    ReplyDelete
    Replies
    1. The example in this article emulates the type of control you could implement with BGP Flowspec (see Real-time DDoS mitigation using sFlow and BGP FlowSpec). In this case the simulated DNS amplification attack is filtered based on UDP source port and destination IP address.

      One of the advantages of using BGP RTBH / Flowspec for mitigation is that the controls can be propagated to upstream neighbor routers (via BGP) to drop the traffic close to its source.

      Delete
  4. hi peter when I run the attack command hping... my ryu.js automatically stopped and is giving error
    server returned http response code 400 for http 127.0.0.1:8080.
    Before attack it is running accurately but after attack it stops with giving this error.

    ReplyDelete
    Replies
    1. The HTTP 400 error indicates a bad request. It's likely that the ryu.app.ofctl_rest API has changed since this article was written. Have you checked to see if Ryu is logging any information about why it is rejecting the request?

      Delete
    2. Have you checked that Mininet is connected to Ryu (--controller=remote,ip=127.0.0.1)? If Ryu isn't the controller, then it will reject the REST call since it doesn't know about the switch.

      Delete
    3. onos example is running perfectly fine for me. But this ryu.js stopped when the attack called. And if I change the call like stats/flow/add/1 it is also running, just this flowentry/add is not running. I have checked the addresses at API site the addresses are looking fine. Can you plz double check for me? thanks

      Delete
    4. Can you check the Ryu logs? There should be information explaining why the HTTP request was rejected.

      Delete
  5. When i am trying to create this custom topology
    sudo mn --custom sflow-rt/extras/sflow.py --link tc,bw=10 --controller=remote,ip=127.0.0.1 --topo tree,depth=2,fanout=2

    I am getting the following error;

    *** Starting controller
    c0
    *** Starting 3 switches
    s1 s2 s3 ...(10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit) (10.00Mbit)
    --------------------------------------------------------------------------------
    Caught exception. Cleaning up...

    UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc0 in position 60: invalid start byte
    --------------------------------------------------------------------------------

    ReplyDelete
    Replies
    1. That looks like a Mininet / Python error. What version of Python are you using? I don't think Mininet supports Python3 at this time.

      The following article might be useful:
      https://blog.sflow.com/2020/11/multipass.html.

      Even if you aren't going to use Multipass to build a virtual machine, it gives the steps needed to build an Ubuntu 18.04 VM with Mininet and Ryu. Ubuntu 18 was chosen because it still has Python 2.7 as the default.

      Delete