Real-time DDoS mitigation using BGP RTBH and FlowSpec

DDoS Protect is a recently released open source application running on the sFlow-RT real-time analytics engine. The software uses streaming analytics to rapidly detect and characterize DDoS flood attacks and automatically applies BGP remote triggered black hole (RTBH) and/or FlowSpec controls to mitigate their impact. The total time to detect and mitigate an attack is in the order of a second.

The combination of multi-vendor standard telemetry (sFlow) and control (BGP FlowSpec) provide the real-time visibility and control needed to quickly and automatically adapt the network to address a range of challenging problems, including: DDoS, traffic engineering, and security.

Solutions are deployable today: Arista BGP FlowSpec describes the recent addition of BGP FlowSpec support to Arista EOS (EOS has long supported sFlow), and sFlow available on Juniper MX series routers describes the release of sFlow support on Juniper MX routers (which have long had BGP FlowSpec support). This article demonstrates DDoS mitigation using Arista EOS. Similar configurations should work with any router that supports sFlow and BGP FlowSpec.

The diagram shows a typical deployment scenario in which an instance of sFlow-RT (running the DDoS Protect application) receives sFlow from the site router (ce-router). A DNS amplification attack is launched against the site. Analysis of the sFlow telemetry immediately recognizes the amplification attack, identifying UDP source port (53) and targeted IP address (192.0.2.129). This information is used to create a BGP FlowSpec rule which propagates via the site router to the transit provider router (sp-router) where the traffic is quickly dropped, ensuring that WAN link bandwidth is protected from the denial of service attack.

There are a number of possible variations on this example. FlowSpec controls can be implemented on the site router to filter out smaller attacks and RTBH controls sent to the transit provider to block large attacks. If more than one site router is involved, an instance of sFlow-RT can be associated with each of the routers, or a route reflector can be set up to distribute controls from sFlow-RT to all the site routers. If your are a service provider, you can use the software to provide DDoS mitigation as a service to your customers.

The following partial configuration enables sFlow and BGP on an Arista EOS device (EOS 4.22 or later):

!
service routing protocols model multi-agent
!
sflow sample 16384
sflow polling-interval 30
sflow destination 10.0.0.70
sflow run
!
interface Ethernet1
   flow-spec ipv4 ipv6
!
interface Management1
   ip address 10.0.0.96/24
!
ip routing
!
ipv6 unicast-routing
!
router bgp 64497
   router-id 192.0.2.1
   neighbor 10.0.0.70 remote-as 65070
   neighbor 10.0.0.70 transport remote-port 1179
   neighbor 10.0.0.70 allowas-in 3
   neighbor 10.0.0.70 send-community extended
   neighbor 10.0.0.70 maximum-routes 12000 
   !
   address-family flow-spec ipv4
      neighbor 10.0.0.70 activate
   !
   address-family flow-spec ipv6
      neighbor 10.0.0.70 activate
   !
   address-family ipv4
      neighbor 10.0.0.70 activate
   !
   address-family ipv6
      neighbor 10.0.0.70 activate
!

DDoS Protect is packaged with sFlow-RT in the sflow/ddos-protect Docker image. Running the following command on host 10.0.0.70 launches the controller:

% docker run --net=host sflow/ddos-protect \
-Dddos_protect.router=10.0.0.96 \
-Dddos_protect.as=65070 \
-Dddos_protect.enable.ipv6=yes \
-Dddos_protect.enable.flowspec=yes \
-Dddos_protect.enable.flowspec6=yes
2020-02-14T13:54:58-08:00 INFO: Starting sFlow-RT 3.0-1466
2020-02-14T13:54:59-08:00 INFO: Version check, running latest
2020-02-14T13:54:59-08:00 INFO: License installed, swkey.json
2020-02-14T13:55:00-08:00 INFO: Listening, BGP port 1179
2020-02-14T13:55:00-08:00 INFO: Listening, sFlow port 7343
2020-02-14T13:55:00-08:00 INFO: Listening, HTTP port 8008
2020-02-14T13:55:00-08:00 INFO: DNS server 8.8.8.8
2020-02-14T13:55:00-08:00 INFO: DNS server 8.8.4.4
2020-02-14T13:55:00-08:00 INFO: app/flow-trend/scripts/top.js started
2020-02-14T13:55:00-08:00 INFO: app/ddos-protect/scripts/ddos.js started
2020-02-14T13:55:37-08:00 INFO: BGP open 10.0.0.96 33917

The last log line confirms that the router has successfully opened the BGP connection to the controller. Now it's time to configure the controller.

In this example, the controller is configured to detect UDP amplification attacks, applying a FlowSpec filter when traffic exceeds 10,000 packets per second (this threshold has been set low so we can test the controller using a simulated attack). In addition, a new Address Group, site, has been created containing the CIDR to be protected, 192.0.2.0/24.

sudo hping3 --flood --udp --rand-source -k -s 53 192.0.2.129

The above command simulates DNS amplification attack using hping3.

The DDoS Protect Charts tab provides an up to the second trend chart for each of the attack types being monitored (see screen capture at top of this article). In this case, the udp_amplification chart shows the simulated attack targeting 192.0.2.129 exceeded the 10,000 Packets per Second threshold, triggering an automated response.

localhost>show bgp flow-spec ipv4
BGP Flow Specification rules for VRF default
Router identifier 10.0.0.98, local AS number 65096
Rule status codes: # - not installed, M - received from multiple peers

   Matching Rule                                                Actions
   192.0.2.129/32;*;IP:17;SP:53;                                Drop

Command line output from the site router shown above verifies that a FlowSpec control blocking the amplification attack has been received. The control will remain in place for 60 minutes (the configured timeout), after which it will be automatically withdrawn. If the attack is still in progress it will be immediately detected and the control reapplied.

DDoS Protect can mitigate a wide range of common attacks, including: NTP, DNS, Memcached, SNMP, and SSDP amplification attacks; IP, UDP, ICMP and TCP flood attacks; and IP fragmentation attacks. Mitigation options include: remote triggered black hole (RTBH), filtering, rate limiting, and DSCP marking. IPv6 is fully supported in detection and mitigation of each of these attack types.

The standard sFlow/BGP support built into routers provides a low cost, simple to deploy, method of efficiently removing DDoS traffic. Follow the steps described in this article to try out the solution on your network.