Monday, March 1, 2021

DDoS Mitigation with Juniper, sFlow, and BGP Flowspec

Real-time DDoS mitigation using BGP RTBH and FlowSpec, DDoS protection of local address space, Pushing BGP Flowspec rules to multiple routersMonitoring DDoS mitigation, and Docker DDoS testbed demonstrate how sFlow and BGP Flowspec are combined by the DDoS Protect application running on the sFlow-RT real-time analytics engine to automatically detect and block DDoS attacks.

This article discusses how to deploy the DDoS Protect application in a Juniper Networks environment. Juniper has a long history of supporting BGP Flowspec on their routing platforms and Juniper has added support for sFlow to their entire product range, see sFlow available on Juniper MX series routers.

First, Junos doesn't provide a way to connect to the non-standard BGP port (1179) that sFlow-RT uses by default. Allowing sFlow-RT to open the standard BGP port (179) requires that the service be given additional Linux capabilities. 

docker run --rm --net=host --sysctl net.ipv4.ip_unprivileged_port_start=0 \
sflow/ddos-protect -Dbgp.port=179

The above command launches the prebuilt sflow/ddos-protect Docker image. Alternatively, if sFlow-RT has been installed as a deb / rpm package, then the required permissions can be added to the service.

sudo systemctl edit sflow-rt.service
Type the above command to edit the service configuration and add the following lines:
[Service]
AmbientCapabilities=CAP_NET_BIND_SERVICE
Next, edit the sFlow-RT configuration file for the DDoS Protect application:
sudo vi /usr/local/sflow-rt/conf.d/ddos-protect.conf
and add the line:
bgp.port=179
Finally, restart sFlow-RT:
sudo systemctl restart sflow-rt
The application is now listening for BGP connections on TCP port 179.

Now configure the router to send sFlow telemetry to sFlow-RT - see Junos: sFlow Monitoring Technology
set protocols sflow collector 192.168.65.2 udp-port 6343
set protocols sflow polling-interval 20
set protocols sflow sample-rate ingress 1000
set protocols sflow interfaces ge-0/0/0
set protocols sflow interfaces ge-0/0/1
...
For example, the above commands enable sFlow monitoring on a Juniper MX router. See sFlow-RT Agents for recommended sFlow configuration settings.

Also configure a BGP Flowspec session with sFlow-RT - see Junos: Multiprotocol BGP.
policy-options {
    policy-statement ACCEPT_ALL {
        from protocol bgp;
        then accept;
    }
}
routing-options {
    router-id 1.1.1.1;
    autonomous-system 65000;
}
protocols {
    bgp {
        group sflow-rt {
            type internal;
            local-address 172.17.0.2;
            family inet {
                unicast;
                flow {
                    no-validate ACCEPT_ALL;
                }
            }
            family inet6 {
                unicast;
                flow {
                    no-validate ACCEPT_ALL;
                }
            }
            neighbor 192.168.65.2 {
                import ACCEPT_ALL;
                peer-as 65000;
            }
        }
    }
}
The above configuration establishes the BGP Flowspec session with sFlow-RT.

Real-time DDoS mitigation using BGP RTBH and FlowSpec describes how to simulate a DDoS UDP amplification attack in order to test the automated detection and control functionality.  
root@07358a106c21> show route table inetflow.0 detail    

inetflow.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
192.0.2.129,*,proto=17,srcport=53/term:N/A (1 entry, 0 announced)
        *BGP    Preference: 170/-101
                Next hop type: Fictitious, Next hop index: 0
                Address: 0x55653aae979c
                Next-hop reference count: 1
                Next hop: 
                State: <Active Int Ext SendNhToPFE>
                Local AS: 65000 Peer AS: 65000
                Age: 6 
                Validation State: unverified 
                Task: BGP_65000.192.168.65.2
                AS path: I 
                Communities: traffic-rate:0:0
                Accepted
                Localpref: 100
                Router ID: 0.6.6.6
Command line output from the router shown above verifies that a Flowspec control blocking the amplification attack has been received. The control will remain in place for 60 minutes (the configured timeout), after which it will be automatically withdrawn. If the attack is still in progress it will be immediately detected and the control reapplied.

DDoS Protect can mitigate a wide range of common attacks, including: NTP, DNS, Memcached, SNMP, and SSDP amplification attacks; IP, UDP, ICMP and TCP flood attacks; and IP fragmentation attacks. Mitigation options include: remote triggered black hole (RTBH), filtering, rate limiting, DSCP marking, and redirection. IPv6 is fully supported in detection and mitigation of each of these attack types.

Monday, January 25, 2021

Topology


Real-time network and system metrics as a service describes how to use data captured from the network shown above to explore the functionality of sFlow-RT real-time analytics software. This article builds on the previous article to show how knowledge of network topology can be used to enhance analytics, see Topology for documentation.

First, follow the instructions in the previous example and start an instance of sFlow-RT using the captured sFlow.  
curl -O https://raw.githubusercontent.com/sflow-rt/fabric-view/master/demo/topology.json
Then, download the topology file for the example.
curl -X PUT -H "Content-Type: application/json" -d @topology.json \
http://localhost:8008/topology/json
Install the topology using the sFlow-RT REST API.
curl http://localhost:8008/topology/json
Retrieve the topology.
{
 "version": 0,
 "links": {
  "L1": {
   "node2": "spine1",
   "node1": "leaf1",
   "port1": "swp1s0",
   "port2": "swp49"
  },
  "L2": {
   "node2": "spine1",
   "node1": "leaf1",
   "port1": "swp1s1",
   "port2": "swp50"
  },
  "L3": {
   "node2": "spine2",
   "node1": "leaf1",
   "port1": "swp1s2",
   "port2": "swp51"
  },
  "L4": {
   "node2": "spine2",
   "node1": "leaf1",
   "port1": "swp1s3",
   "port2": "swp52"
  },
  "L5": {
   "node2": "spine2",
   "node1": "leaf2",
   "port1": "swp1s0",
   "port2": "swp49"
  },
  "L6": {
   "node2": "spine2",
   "node1": "leaf2",
   "port1": "swp1s1",
   "port2": "swp50"
  },
  "L7": {
   "node2": "spine1",
   "node1": "leaf2",
   "port1": "swp1s2",
   "port2": "swp51"
  },
  "L8": {
   "node2": "spine1",
   "node1": "leaf2",
   "port1": "swp1s3",
   "port2": "swp52"
  }
 }
}
The JSON topology structure contains the eight links connecting the leaf and spine switches in the diagram, identifying the nodes and ports associated with each link.
curl -H "Content-Type:application/json" -X PUT \
--data '{"keys":"ipsource,ipdestination",value:"bytes"}' \
http://localhost:8008/flow/srcdst/json
Now define the srcdst flow metric described in the previous article.
curl "http://localhost:8008/activeflows/TOPOLOGY/srcdst/json?aggMode=edge"
Knowledge of topology opens up additional options when querying for flows. For example, the above command only considers devices that are part of the topology and sums flows entering edge device access ports, i.e. traffic entering the the leaf switches from the servers.
[
 {
  "flowN": 1,
  "value": 248800.14028768288,
  "key": "10.4.3.2,10.4.4.2"
 },
 {
  "flowN": 1,
  "value": 176879.3798722214,
  "key": "10.4.1.2,10.4.2.2"
 },
 {
  "flowN": 1,
  "value": 526.0366052656848,
  "key": "10.4.4.2,10.4.3.2"
 },
 {
  "flowN": 1,
  "value": 375.06686598182193,
  "key": "10.4.2.2,10.4.1.2"
 }
]
The result accurately reports the amount of traffic being exchanged between the servers, discarding duplicate data reported as traffic flows traverse the links between switches.
-Dbrowse-flows.agents=TOPOLOGY -Dbrowse-flows.aggMode=edge
Adding the above arguments to the end of the command line used to start sFlow-RT configures the Flow Browser application to use the topology de-duplication method.
Click on the link below to plot a graph of the top IP Protocols using the browse-flows application (screen capture shown above):
http://localhost:8008/app/browse-flows/html/index.html?keys=ipprotocol&value=bps
Note: No data will be shown until the topology is posted to sFlow-RT.
function print(label,obj) {
  logInfo(label+"="+JSON.stringify(obj));
}

setFlow('protocol',{keys:'ipprotocol',value:'bytes'});

setIntervalHandler(function() {
  print("locate_mac",topologyLocateHostMac('000AF725C062'));
  print("locate_ip",topologyLocateHostIP('10.4.3.2'));
  print("flow_max",activeFlows('ALL','protocol',5,0,'max'));
  print("flow_sum",activeFlows('ALL','protocol',5,0,'sum'));
  print("flow_edge",activeFlows('TOPOLOGY','protocol',5,0,'edge'));
});
The demo.js script shown above uses sFlow-RT's embedded scripting API, see Writing Applications. The script defines the flow called protocol that tracks top IP Protocols and prints out the top flows with different aggregation methods. The script also demonstrates an additional capability made possible when topology is known. The topologyLocateHostMac() and topologyLocateHostIP() function locates an addresses to the edge port connecting them to the network.
-Dscript.file=$PWD/demo.js
Run the script by adding the above argument to the end of the command line used to run sFlow-RT.
2021-01-22T17:08:35-08:00 INFO: locate_mac=[{"ipaddress":"10.4.3.2","node":"leaf1","agent":"192.168.0.11","ifindex":"38","port":"swp32s1","mac":"000AF725C062"}]
2021-01-22T17:08:35-08:00 INFO: locate_ip=[{"ipaddress":"10.4.3.2","node":"leaf1","agent":"192.168.0.11","ifindex":"38","port":"swp32s1","mac":"000AF725C062"}]
2021-01-22T17:08:35-08:00 INFO: flow_max=[{"flowN":16,"agent":"192.168.0.14","value":1208583376.782055,"dataSource":"54","key":"6"}]
2021-01-22T17:08:35-08:00 INFO: flow_sum=[{"flowN":16,"value":6615963204.827695,"key":"6"}]
2021-01-22T17:08:35-08:00 INFO: flow_edge=[{"flowN":4,"value":2104039983.2917378,"key":"6"}]
The output from the script shows that the addresses were located to leaf1 port swp32s1. The flow_max and flow_sum queries don't use the topology and combine data from all 16 data sources (switch ports) that are reporting traffic. The sum mode returns the largest value since traffic is added for every data source. The max mode finds the data source reporting the largest value for the flow and reports that value (agent: 192.168.0.14, dataSource: 54). The edge mode is equivalent to the REST query used earlier.

Mininet is a network emulator that you can run on your laptop in a virtual machine (e.g. using Multipass) that provides a useful platform for building virtual topologies and exploring topology related analytics. Mininet dashboardMininet weathermap, and Mininet, ONOS, and segment routing provide examples.


Ideally the network configuration and topology will be available in a centralized repository that can be queried to generate the information required by sFlow-RT. Alternatively, Link Layer Discovery Protocol (LLDP) data retrieved from network devices can be used to construct the topology. Fabric Visibility, Arista EOS CloudVision, and Fabric visibility with Cumulus Linux provide examples.