Wednesday, August 31, 2022

DDoS Sonification

Sonification presents data as sounds instead of visual charts. One of the best known examples of sonification is the representation of radiation level as a click rate in a Geiger counter. This article describes ddos-sonify, an experiment to see if sound can be usefully employed to represent information about Distributed Denial of Service (DDoS) attacks. The DDoS attacks and BGP Flowspec responses testbed was used to create the video demonstration at the top of this page in which a series of simulated DDoS attacks are detected and mitigated. Play the video to hear the results.

The software uses the Tone.js library to control Web Audio sound generation functionality in a web browser.

var voices = {};
var loop;
var loopInterval = '4n';
$('#sonify').click(function() {
  if($(this).prop("checked")) {
    voices.synth = new Tone.PolySynth(Tone.Synth).toDestination();
    voices.metal = new Tone.PolySynth(Tone.MetalSynth).toDestination();
    voices.pluck = new Tone.PolySynth(Tone.PluckSynth).toDestination();
    voices.membrane = new Tone.PolySynth(Tone.MembraneSynth).toDestination();
    voices.am = new Tone.PolySynth(Tone.AMSynth).toDestination();
    voices.fm = new Tone.PolySynth(Tone.FMSynth).toDestination();
    voices.duo = new Tone.PolySynth(Tone.DuoSynth).toDestination();
    Tone.Transport.bpm.value=80;
    loop = new Tone.Loop((now) => {
      sonify(now);
    },loopInterval).start(0);
    Tone.Transport.start();
  } else {
    loop.stop();
    loop.dispose();
    Tone.Transport.stop();
  }
});
Clicking on the Convert charts to sound checkbox on the web page initializes the different sound synthesizers that will be used to create sounds and starts a timed loop that will periodically call the sonify() function convert current values of each of the metrics into sounds.
var metrics = [
  {name:'top-5-ip-flood', threshold:'threshold_ip_flood', voice:'synth'},
  {name:'top-5-ip-fragmentation', threshold:'threshold_ip_fragmentation', voice:'duo'},
  {name:'top-5-icmp-flood', threshold:'threshold_icmp_flood', voice:'pluck'},
  {name:'top-5-udp-flood', threshold:'threshold_udp_flood', voice:'membrane'},
  {name:'top-5-udp-amplification', threshold:'threshold_udp_amplification', voice:'metal'},
  {name:'top-5-tcp-flood', threshold:'threshold_tcp_flood', voice:'am'},
  {name:'top-5-tcp-amplification', threshold:'threshold_tcp_amplification', voice:'fm'}
];
var notes = ['C4','D4','E4','F4','G4','A4','B4','C5'];
function sonify(now) {
  var sounds = {};
  var max = {};
  metrics.forEach(function(metric) {
    let vals = db.trend.trends[metric.name];
    let topn = vals[vals.length - 1];
    let thresh = db.trend.values[metric.threshold];
    let chord = sounds[metric.voice];
    if(!chord) {
      chord = {};
      sounds[metric.voice] = chord;
    }
    for(var key in topn) {
      let [tgt,group,port] = key.split(',');
      let note = notes[port % notes.length];
      chord[note] = Math.max(chord[note] || 0, Math.min(1,topn[key] / thresh));
      max[metric.voice] = Math.max(max[metric.voice] || 0, chord[note]);
    };
  });
  var interval = Tone.Time(loopInterval).toSeconds();
  var delay = 0;
  for(let voice in sounds) {
    let synth = voices[voice];
    let chord = sounds[voice];
    let maxval = max[voice];
    if(maxval) {
      let volume = Math.min(0,(maxval - 1) * 20);
      synth.volume.value=volume;
      let note_array = [];
      for(let note in chord) {
        let val = chord[note];
        if((val / maxval) < 0.7) continue;
        note_array.push(note);
      }
      let duration = Tone.Time(maxval*interval).quantize('64n');
      if(duration > 0) synth.triggerAttackRelease(note_array,duration,now+delay);
    }
    delay += Tone.Time('16n').toSeconds();
  }
}
The metrics array identifies individual DDoS metrics and their related thresholds and associates them with a sound (voice). The sonify() function retrieves current values of each of the metrics and scales them by their respective threshold. Each metric value is mapped to a musical note based on the TCP/UDP port used in the attack. Different attack types are mapped to different voices, for example, a udp_amplification attack will have a metallic sound while a udp_flood attack will have a percussive sound. Volume and duration of notes are proportional to the intensity of the attack.

The net effect in a production network is of a quiet rythm of instruments. When a DDoS attack occurs, the notes associated with the particular attack become much louder and drown out the background sounds. Over time it is possible to recoginize the distinct sounds on each type of DDoS attack.

Tuesday, August 23, 2022

NVIDIA ConnectX SmartNICs

NVIDIA ConnectX SmartNICs offer best-in-class network performance, serving low-latency, high-throughput applications with one, two, or four ports at 10, 25, 40, 50, 100, 200, and up to 400 gigabits per second (Gb/s) Ethernet speeds.

This article describes how use the instrumentation built into ConnectX SmartNICs for data center wide network visibility. Real-time network telemetry for automation provides some background, giving an overview of the sFlow industry standard with an example of troubleshooting a high performance GPU compute cluster.

Linux as a network operating system describes how standard Linux APIs are used in NVIDIA Spectrum switches to monitor data center network performance. Linux Kernel Upstream Release Notes v5.19 describes recent driver enhancements for ConnectX SmartNICs that extend visibility to servers for end-to-end visibility into the performance of high performance distributed compute infrastructure.

The open source Host sFlow agent uses standard Linux APIs to configure instrumentation in switches and hosts, streaming the resulting measurements to analytics software in real-time for comprehensive data center wide visibility.

Packet sampling provides detailed visibility into traffic flowing across the network. Hardware packet sampling makes it possible to monitor 400 gigabits per second interfaces on the server at line rate with minimal CPU/memory overhead.
psample { group=1 egress=on }
dent { sw=off switchport=^eth[0-9]+$ }
The above Host sFlow configuration entries enable packet sampling on the host. Linux 4.11 kernel extends packet sampling support describes the Linux PSAMPLE netlink channel used by the network adapter to send packet samples to the Host sFlow agent. The dent module automatically configures hardware packet sampling on network interfaces matching the switchport pattern (eth0, eth1, .. in this example) using the Linux tc-sample API, directing the packet samples to be sent to the specified psample group
Visibility into dropped packets offers significant benefits for network troubleshooting, providing real-time network-wide visibility into the specific packets that were dropped as well the reason the packet was dropped. This visibility instantly reveals the root cause of drops and the impacted connections.
dropmon { group=1 start=on sw=on hw=on }
The above Host sFlow configuration entry enables hardware dropped packet monitoring on the network adapter hardware.
sflow {
  collector { ip=10.0.0.1 }
  psample { group=1 egress=on }
  dropmon { group=1 start=on sw=on hw=on }
  dent { sw=off switchport=^eth[0-9]+$ }
}
The above /etc/hsflowd.conf file shows a complete configuration. The centralized collector, 10.0.0.1, receives sFlow from all the servers and switches in the network to provide comprehensive end-to-end visibility.
The sFlow-RT analytics engine receives and analyzes the sFlow telemetry stream, providing real-time analytics to visibility and automation systems (e.g. Flow metrics with Prometheus and Grafana).

Tuesday, August 9, 2022

DDoS detection with advanced real-time flow analytics

The diagram shows two high bandwidth flows of traffic to the Customer Network, the first (shown in blue) is a bulk transfer of data to a big data application, and the second (shown in red) is a distributed denial of service (DDoS) attack in which large numbers of compromised hosts attempt to flood the link connecting the Customer Network to the upstream Transit Provider. Industry standard sFlow telemetry from the customer router streams to an instance of the sFlow-RT real-time analytics engine which is programmed to detect (and mitigate) the DDoS attack.

This article builds on the Docker testbed to demonstrate how advanced flow analytics can be used to separate the two types of traffic and detect the DDoS attack.

docker run --rm -d -e "COLLECTOR=host.docker.internal" -e "SAMPLING=100" \
--net=host -v /var/run/docker.sock:/var/run/docker.sock:ro \
--name=host-sflow sflow/host-sflow
First, start a Host sFlow agent using the pre-built sflow/host-sflow image to generate the sFlow telemetry that would stream from the switches and routers in a production deployment. 
setFlow('ddos_amplification', {
  keys:'ipdestination,udpsourceport',
  value: 'frames',
  values: ['count:ipsource']
});
setThreshold('ddos_amplification', {
  metric:'ddos_amplification',
  value: 10000,
  byFlow:true,
  timeout: 2
});
setEventHandler(function(event) {
  var [ipdestination,udpsourceport] = event.flowKey.split(',');
  var [sourcecount] = event.values;
  if(sourcecount === 1) {
    logInfo("bulk transfer to " + ipdestination);
  } else {
    logInfo("DDoS port " + udpsourceport + " against " + ipdestination);
  }
},['ddos_amplification']);
The ddos.js script above provides a simple demonstration of sFlow-RT's advanced flow analytics. The setFlow() function defines the a flow signature for detecting UDP amplification attacks, identifying the targetted IP address and the amplification protocol. In addition to the primary value of frames per second, a secondary value counting the number of ipsource addresses has been included. The setThreshold() function causes an event to by generated whenever a flow exceeds 10,000 frames per second. Finally, the setEventHandler() function defines how the events will be processed. See Writing Applications for more information on developing sFlow-RT applications.
docker run --rm -v $PWD/ddos.js:/sflow-rt/ddos.js \
-p 8008:8008 -p 6343:6343/udp --name sflow-rt \
sflow/prometheus -Dscript.file=ddos.js
Start sFlow-RT using pre-built sflow/prometheus image.
docker run --rm -it sflow/hping3 --flood --udp -k \
-p 443 host.docker.internal
In a separate window, simulate a bulk tranfer using pre-built sflow/hping3 image (use CTL+C to stop the attack).
2022-08-09T00:03:20Z INFO: bulk transfer to 192.168.65.2
The transfer will be immediately detected and logged in the sFlow-RT window.
docker run --rm -it sflow/hping3 --flood --udp -k \
--rand-source -s 53 host.docker.internal
Simulate a UDP amplification attack.
2022-08-09T00:05:19Z INFO: DDoS port 53 against 192.168.65.2
The attack will be immmediately detected and logged in the sFlow-RT window.

The open source sFlow-RT ddos-protect application is a full featured DDoS mitigation solution that uses the advanced flow analytics features described in this article to detect a wide range of volumetric attacks. In addition, ddos-protect can automatically mitigate attacks using BGP remotely triggered blackhole (RTBH) or BGP Flowspec actions. DDoS protection quickstart guide describes how to test, deploy, and monitor the DDoS mitigation solution with examples using Arista, Cisco, and Juniper routers.