Flow smoothing

The sFlow-RT real-time analytics engine includes statistical smoothing. The chart above illustrates the effect of different levels of smoothing when analyzing real-time sFlow telemetry.

The traffic generator in this example creates an alternating pattern: 1.25Mbytes/second for 30 seconds followed by a pause of 30 seconds. Smoothing time constants between 1 second and 500 seconds have been applied to generate the family of charts. The blue line is the result of 1 second smoothing and closely tracks the traffic pattern. At the other extreme, the dark red line is the result of 500 second smoothing, showing a constant 625Kbytes/second (the average of the waveform).

There is a tradeoff between responsiveness and variability (noise) when selecting the level of smoothing. Selecting a suitable smoothing level depends on the flow analytics application.

Low smoothing values are appropriate when fast response is required, for example:

• DDoS detection
• Leaf and spine traffic engineering using segment routing and SDN
• Internet router using merchant silicon

Higher smoothing values are appropriate when less variability is desirable, for example:

• Real-time traffic visualization using Netflix Vizceral
• Real-time web analytics
• Real-time visibility and control of campus networks

Generating the chart

The results described in this article are easily reproduced using the testbed described in Mininet flow analytics.

The following, smooting.js, script defines a set of flows with different smoothing periods:

var times = [1,2,5,10,20,50,100,200,500];
for(var i = 0; i < times.length; i++) {
  setFlow('t_'+times[i],{value:'bytes',t:times[i]});
}

Start sFlow-RT:

env RTPROP=-Dscript.file=smoothing.js ./start.sh

Start Mininet:

sudo mn --custom extras/sflow.py --link tc,bw=10

Type the following Mininet command to open terminals on simulated hosts, h1 and h2:

mininet> xterm h1 h2

In h2 terminal window:

iperf -s

In h1 terminal window:

while true; do iperf -c 10.0.0.2 -t 30; sleep 30; done

Plot the chart by opening the sFlow-RT URL:

http://sflow-rt:8008/metric/ALL/t_1,t_2,t_5,t_10,t_20,t_50,t_100,t_200,t_500/html

See Writing Applications for more information.