Saturday, February 15, 2020

Real-time DDoS mitigation using BGP RTBH and FlowSpec

DDoS Protect is a recently released open source application running on the sFlow-RT real-time analytics engine. The software uses streaming analytics to rapidly detect and characterize DDoS flood attacks and automatically applies BGP remote triggered black hole (RTBH) and/or FlowSpec controls to mitigate their impact. The total time to detect and mitigate an attack is in the order of a second.

The combination of multi-vendor standard telemetry (sFlow) and control (BGP FlowSpec) provide the real-time visibility and control needed to quickly and automatically adapt the network to address a range of challenging problems, including: DDoS, traffic engineering, and security.

Solutions are deployable today: Arista BGP FlowSpec describes the recent addition of BGP FlowSpec support to Arista EOS (EOS has long supported sFlow), and sFlow available on Juniper MX series routers describes the release of sFlow support on Juniper MX routers (which have long had BGP FlowSpec support). This article demonstrates DDoS mitigation using Arista EOS. Similar configurations should work with any router that supports sFlow and BGP FlowSpec.
The diagram shows a typical deployment scenario in which an instance of sFlow-RT (running the DDoS Protect application) receives sFlow from the site router (ce-router). A DNS amplification attack is launched against the site. Analysis of the sFlow telemetry immediately recognizes the amplification attack, identifying UDP source port (53) and targeted IP address (192.0.2.129). This information is used to create a BGP FlowSpec rule which propagates via the site router to the transit provider router (sp-router) where the traffic is quickly dropped, ensuring that WAN link bandwidth is protected from the denial of service attack.

There are a number of possible variations on this example. FlowSpec controls can be implemented on the site router to filter out smaller attacks and RTBH controls sent to the transit provider to block large attacks. If more than one site router is involved, an instance of sFlow-RT can be associated with each of the routers, or a route reflector can be set up to distribute controls from sFlow-RT to all the site routers. If your are a service provider, you can use the software to provide DDoS mitigation as a service to your customers.

The following partial configuration enables sFlow and BGP on an Arista EOS device (EOS 4.22 or later):
!
service routing protocols model multi-agent
!
sflow sample 16384
sflow polling-interval 30
sflow destination 10.0.0.70
sflow run
!
interface Ethernet1
   flow-spec ipv4 ipv6
!
interface Management1
   ip address 10.0.0.96/24
!
ip routing
!
ipv6 unicast-routing
!
router bgp 64497
   router-id 192.0.2.1
   neighbor 10.0.0.70 remote-as 65070
   neighbor 10.0.0.70 transport remote-port 1179
   neighbor 10.0.0.70 allowas-in 3
   neighbor 10.0.0.70 send-community extended
   neighbor 10.0.0.70 maximum-routes 12000 
   !
   address-family flow-spec ipv4
      neighbor 10.0.0.70 activate
   !
   address-family flow-spec ipv6
      neighbor 10.0.0.70 activate
   !
   address-family ipv4
      neighbor 10.0.0.70 activate
   !
   address-family ipv6
      neighbor 10.0.0.70 activate
!
DDoS Protect is packaged with sFlow-RT in the sflow/ddos-protect Docker image. Running the following command on host 10.0.0.70 launches the controller:
% docker run --net=host sflow/ddos-protect \
-Dddos_protect.router=10.0.0.96 \
-Dddos_protect.as=65070 \
-Dddos_protect.enable.ipv6=yes \
-Dddos_protect.enable.flowspec=yes \
-Dddos_protect.enable.flowspec6=yes
2020-02-14T13:54:58-08:00 INFO: Starting sFlow-RT 3.0-1466
2020-02-14T13:54:59-08:00 INFO: Version check, running latest
2020-02-14T13:54:59-08:00 INFO: License installed, swkey.json
2020-02-14T13:55:00-08:00 INFO: Listening, BGP port 1179
2020-02-14T13:55:00-08:00 INFO: Listening, sFlow port 7343
2020-02-14T13:55:00-08:00 INFO: Listening, HTTP port 8008
2020-02-14T13:55:00-08:00 INFO: DNS server 8.8.8.8
2020-02-14T13:55:00-08:00 INFO: DNS server 8.8.4.4
2020-02-14T13:55:00-08:00 INFO: app/flow-trend/scripts/top.js started
2020-02-14T13:55:00-08:00 INFO: app/ddos-protect/scripts/ddos.js started
2020-02-14T13:55:37-08:00 INFO: BGP open 10.0.0.96 33917
The last log line confirms that the router has successfully opened the BGP connection to the controller. Now it's time to configure the controller.
In this example, the controller is configured to detect UDP amplification attacks, applying a FlowSpec filter when traffic exceeds 10,000 packets per second (this threshold has been set low so we can test the controller using a simulated attack). In addition, a new Address Group, site, has been created containing the CIDR to be protected, 192.0.2.0/24.
sudo hping3 --flood --udp --rand-source -k -s 53 192.0.2.129
The above command simulates DNS amplification attack using hping3.
The DDoS Protect Charts tab provides an up to the second trend chart for each of the attack types being monitored (see screen capture at top of this article). In this case, the udp_amplification chart shows the simulated attack targeting 192.0.2.129 exceeded the 10,000 Packets per Second threshold, triggering an automated response.
localhost>show bgp flow-spec ipv4
BGP Flow Specification rules for VRF default
Router identifier 10.0.0.98, local AS number 65096
Rule status codes: # - not installed, M - received from multiple peers

   Matching Rule                                                Actions
   192.0.2.129/32;*;IP:17;SP:53;                                Drop
Command line output from the site router shown above verifies that a FlowSpec control blocking the amplification attack has been received. The control will remain in place for 60 minutes (the configured timeout), after which it will be automatically withdrawn. If the attack is still in progress it will be immediately detected and the control reapplied.

DDoS Protect can mitigate a wide range of common attacks, including: NTP, DNS, Memcached, SNMP, and SSDP amplification attacks; IP, UDP, ICMP and TCP flood attacks; and IP fragmentation attacks. Mitigation options include: remote triggered black hole (RTBH), filtering, rate limiting, and DSCP marking. IPv6 is fully supported in detection and mitigation of each of these attack types.

The standard sFlow/BGP support built into routers provides a low cost, simple to deploy, method of efficiently removing DDoS traffic. Follow the steps described in this article to try out the solution on your network.

6 comments:

  1. Hi,

    Can you explain how to advertise the bgp flowspec route to two routers at the same time during attack detection please?

    ReplyDelete
    Replies
    1. The current implementation of ddos-protect only supports a single BGP peer. You could use FRR or Bird as a route reflector. Connect sFlow-RT to the route reflector use it to propagate the Flowspec filters and blackhole routes to your two routers.

      Delete
  2. Thank you for your reply.

    I'm trying to advertise bgp flowspec route to Bird.

    I have only "Import withdraws" although I send updates routes.

    bird.conf :

    router id 192.168.10.2;
    flow4 table flowtab4;

    protocol bgp client1 {
    debug all;
    local 192.168.10.2 as 65000;
    neighbor 192.168.10.1 as 65000;
    flow4 {
    import all;
    export all;
    };

    ipv4 {
    import all;
    export all;

    };

    }

    protocol bgp sflow {
    debug all;
    local 192.168.10.2 as 65000;
    neighbor 192.168.10.3 port 1179 as 65000;
    flow4 {
    import all;
    export all;
    };

    ipv4 {
    import all;
    export all;

    };
    }

    Do you have an idea please?

    ReplyDelete
    Replies
    1. I haven't used Bird's implementation of Flowspec.

      Looking at your config, don't you need to include table references in the flow4{} and ipv4{} sections for client1 and flow?

      In the protocol bgp sflow {} config, you should set export none in the flow4 and ipv4 sections.

      If you get it working, please post with the working configuration.

      Delete
    2. The latest version of ddos-protect supports multiple routers. Set ddos_protect.router to a comma separated list of router IP addresses, e.g.
      -Dddos_protect.route=10.0.0.96,10.0.0.97

      Delete
  3. The ddos-protect application only supports filtering / dropping based on destination IP addresses (and TCP/UDP ports, ICMP type, fragmentation). Typically in a DDoS attack there are large numbers of attackers so it is impractical to filter based on sources. However, it is possible with sFlow-RT/BGP Flowspec. You would need to write your own controller, or modify ddos-protect, see Real-time DDoS mitigation using sFlow and BGP FlowSpec

    ReplyDelete