SONiC

SONiC is part of the Open Compute Project (OCP), creating "an open source network operating system based on Linux that runs on switches from multiple vendors and ASICs." The latest SONiC.201911 release of the open source SONiC network operating system adds sFlow support.

SONiC: sFlow High Level Design

The diagram shows the elements of the implementation.

1. The open source Host sFlow agent running in the sFlow container monitors the Redis database (in the Database container) for sFlow related configuration changes.
2. The syncd container monitors the configuration database and pushes hardware settings (packet sampling) to the ASIC using the SAI (Switch Abstraction Inteface) driver (see SAI 1.5).
3. The ASIC driver hands sampled packet headers and associated metadata captured by the ASIC to user space via the Linux PSAMPLE netlink channel (see Linux 4.11 kernel extends packet sampling support).
4. The Host sFlow agent receives the PSAMPLE messages and forwards them to configured sFlow collector(s) as standard sFlow packet samples.
5. In addition, the Host sFlow agent streams telemetry (interface counters and host metrics gathered from the Redis database and Linux kernel) to the collector(s) as standard sFlow counter records.

The following CLI commands enable sFlow on all switch ports and send to sFlow collector 10.0.0.70:

sflow collector add 10.0.0.70
sflow polling-interval 30
sflow interface enable all
sflow enable

The open source sflowtool command line utility is a simple way of confirming that sFlow is being received at the collector host (10.0.0.70). The Docker sflow/sflowtool image provides a convenient method of running sflowtool:

docker run -p 6343:6343/udp sflow/sflowtool

If sFlow is arriving at the host, you will see the contents of the sFlow records printed in the console.

Running the Docker sflow/prometheus image exposes network-wide telemetry and flow data in the Prometheus export format so that it can be imported into a time series database:

docker run -p 8008:8008 -p 6343:6343/udp -d sflow/prometheus

See InfluxDB 2.0, Prometheus exporter and Flow metrics with Prometheus and Grafana for examples.

The Prometheus exporter is an application running on the sFlow-RT real-time analytics engine, which has the scaleability to handle large SONiC data center deployments. In addition to delivering metrics, sFlow-RT applications are available that address trouble-shooting, traffic engineering, DDoS mitigation, and security challenges of managing the large scale leaf and spine networks where SONiC is typically deployed.

Real-time DDoS mitigation using BGP RTBH and FlowSpec

DDoS Protect is a recently released open source application running on the sFlow-RT real-time analytics engine. The software uses streaming analytics to rapidly detect and characterize DDoS flood attacks and automatically applies BGP remote triggered black hole (RTBH) and/or FlowSpec controls to mitigate their impact. The total time to detect and mitigate an attack is in the order of a second.

The combination of multi-vendor standard telemetry (sFlow) and control (BGP FlowSpec) provide the real-time visibility and control needed to quickly and automatically adapt the network to address a range of challenging problems, including: DDoS, traffic engineering, and security.

Solutions are deployable today: Arista BGP FlowSpec describes the recent addition of BGP FlowSpec support to Arista EOS (EOS has long supported sFlow), and sFlow available on Juniper MX series routers describes the release of sFlow support on Juniper MX routers (which have long had BGP FlowSpec support). This article demonstrates DDoS mitigation using Arista EOS. Similar configurations should work with any router that supports sFlow and BGP FlowSpec.

The diagram shows a typical deployment scenario in which an instance of sFlow-RT (running the DDoS Protect application) receives sFlow from the site router (ce-router). A DNS amplification attack is launched against the site. Analysis of the sFlow telemetry immediately recognizes the amplification attack, identifying UDP source port (53) and targeted IP address (192.0.2.129). This information is used to create a BGP FlowSpec rule which propagates via the site router to the transit provider router (sp-router) where the traffic is quickly dropped, ensuring that WAN link bandwidth is protected from the denial of service attack.

There are a number of possible variations on this example. FlowSpec controls can be implemented on the site router to filter out smaller attacks and RTBH controls sent to the transit provider to block large attacks. If more than one site router is involved, an instance of sFlow-RT can be associated with each of the routers, or a route reflector can be set up to distribute controls from sFlow-RT to all the site routers. If your are a service provider, you can use the software to provide DDoS mitigation as a service to your customers.

The following partial configuration enables sFlow and BGP on an Arista EOS device (EOS 4.22 or later):

!
service routing protocols model multi-agent
!
sflow sample 16384
sflow polling-interval 30
sflow destination 10.0.0.70
sflow run
!
interface Ethernet1
   flow-spec ipv4 ipv6
!
interface Management1
   ip address 10.0.0.96/24
!
ip routing
!
ipv6 unicast-routing
!
router bgp 64497
   router-id 192.0.2.1
   neighbor 10.0.0.70 remote-as 65070
   neighbor 10.0.0.70 transport remote-port 1179
   neighbor 10.0.0.70 allowas-in 3
   neighbor 10.0.0.70 send-community extended
   neighbor 10.0.0.70 maximum-routes 12000 
   !
   address-family flow-spec ipv4
      neighbor 10.0.0.70 activate
   !
   address-family flow-spec ipv6
      neighbor 10.0.0.70 activate
   !
   address-family ipv4
      neighbor 10.0.0.70 activate
   !
   address-family ipv6
      neighbor 10.0.0.70 activate
!

DDoS Protect is packaged with sFlow-RT in the sflow/ddos-protect Docker image. Running the following command on host 10.0.0.70 launches the controller:

% docker run --net=host sflow/ddos-protect \
-Dddos_protect.router=10.0.0.96 \
-Dddos_protect.as=65070 \
-Dddos_protect.enable.ipv6=yes \
-Dddos_protect.enable.flowspec=yes \
-Dddos_protect.enable.flowspec6=yes
2020-02-14T13:54:58-08:00 INFO: Starting sFlow-RT 3.0-1466
2020-02-14T13:54:59-08:00 INFO: Version check, running latest
2020-02-14T13:54:59-08:00 INFO: License installed, swkey.json
2020-02-14T13:55:00-08:00 INFO: Listening, BGP port 1179
2020-02-14T13:55:00-08:00 INFO: Listening, sFlow port 7343
2020-02-14T13:55:00-08:00 INFO: Listening, HTTP port 8008
2020-02-14T13:55:00-08:00 INFO: DNS server 8.8.8.8
2020-02-14T13:55:00-08:00 INFO: DNS server 8.8.4.4
2020-02-14T13:55:00-08:00 INFO: app/flow-trend/scripts/top.js started
2020-02-14T13:55:00-08:00 INFO: app/ddos-protect/scripts/ddos.js started
2020-02-14T13:55:37-08:00 INFO: BGP open 10.0.0.96 33917

The last log line confirms that the router has successfully opened the BGP connection to the controller. Now it's time to configure the controller.

In this example, the controller is configured to detect UDP amplification attacks, applying a FlowSpec filter when traffic exceeds 10,000 packets per second (this threshold has been set low so we can test the controller using a simulated attack). In addition, a new Address Group, site, has been created containing the CIDR to be protected, 192.0.2.0/24.

sudo hping3 --flood --udp --rand-source -k -s 53 192.0.2.129

The above command simulates DNS amplification attack using hping3.

The DDoS Protect Charts tab provides an up to the second trend chart for each of the attack types being monitored (see screen capture at top of this article). In this case, the udp_amplification chart shows the simulated attack targeting 192.0.2.129 exceeded the 10,000 Packets per Second threshold, triggering an automated response.

localhost>show bgp flow-spec ipv4
BGP Flow Specification rules for VRF default
Router identifier 10.0.0.98, local AS number 65096
Rule status codes: # - not installed, M - received from multiple peers

   Matching Rule                                                Actions
   192.0.2.129/32;*;IP:17;SP:53;                                Drop

Command line output from the site router shown above verifies that a FlowSpec control blocking the amplification attack has been received. The control will remain in place for 60 minutes (the configured timeout), after which it will be automatically withdrawn. If the attack is still in progress it will be immediately detected and the control reapplied.

DDoS Protect can mitigate a wide range of common attacks, including: NTP, DNS, Memcached, SNMP, and SSDP amplification attacks; IP, UDP, ICMP and TCP flood attacks; and IP fragmentation attacks. Mitigation options include: remote triggered black hole (RTBH), filtering, rate limiting, and DSCP marking. IPv6 is fully supported in detection and mitigation of each of these attack types.

The standard sFlow/BGP support built into routers provides a low cost, simple to deploy, method of efficiently removing DDoS traffic. Follow the steps described in this article to try out the solution on your network.