BGP FlowSpec is a method of distributing access control lists (ACLs) using the BGP protocol. Distributed denial of service (DDoS) mitigation is an important use case for the technology, allowing a targeted network to push filters to their upstream provider to selectively remove the attack traffic.
Unfortunately, FlowSpec is currently only available on high end routing devices and so experimenting with the technology is expensive. Looking for an alternative, Cumulus Linux is an open Linux platform that allows users to install Linux packages and develop their own software.
This article describes a proof of concept implementation of basic FlowSpec functionality using
ExaBGP installed on a free
Cumulus VX virtual machine. The same solution can be run on inexpensive commodity white box hardware to deliver terabit traffic filtering in a production network.
First, install latest version of
ExaBGP on the Cumulus Linux switch:
curl -L https://github.com/Exa-Networks/exabgp/archive/4.0.0.tar.gz | tar zx
Now define the handler,
acl.py, that will convert BGP FlowSpec updates into standard Linux
netfilter/iptables entries used by Cumulus Linux to specify hardware ACLs (see
Netfilter - ACLs):
#!/usr/bin/python
import json
import re
from os import listdir,remove
from os.path import isfile
from subprocess import Popen,STDOUT,PIPE
from sys import stdin, stdout, stderr
id = 0
acls = {}
dir = '/etc/cumulus/acl/policy.d/'
priority = '60'
prefix = 'flowspec'
bld = '.bld'
suffix = '.rules'
def commit():
Popen(["cl-acltool","-i"],stderr=STDOUT,stdout=PIPE).communicate()[0]
def aclfile(name):
global dir,priority,prefix,suffix
return dir+priority+prefix+name+suffix
def handleSession(state):
if "down" == state:
for key,rec in acls.items():
fn = aclfile(str(rec['id']))
if isfile(fn):
remove(fn)
del acls[key]
commit()
def buildACL(flow,action):
acl = "[iptables]\n-A FORWARD --in-interface swp+"
if flow.get('protocol'):
acl = acl + " -p " + re.sub('[!<>=]','',flow['protocol'][0])
if flow.get('source-ipv4'):
acl = acl + " -s " + flow['source-ipv4'][0]
if flow.get('destination-ipv4'):
acl = acl + " -d " + flow['destination-ipv4'][0]
if flow.get('source-port'):
acl = acl + " --sport " + re.sub('[!<>=]','',flow['source-port'][0])
if flow.get('destination-port'):
acl = acl + " --dport " + re.sub('[!<>=]','',flow['destination-port'][0])
acl = acl + " -j DROP\n"
return acl
def handleFlow(add,flow,action):
global id
key = flow['string']
if add:
acl = buildACL(flow,action)
id = id + 1
acls[key] = {"acl":acl,"id":id}
fn = aclfile(str(id))
f = open(fn,'w')
f.write(acl)
f.close()
commit()
elif key in acls:
rec = acls[key]
fn = aclfile(str(rec['id']))
if isfile(fn):
remove(fn)
del acls[key]
commit()
while True:
try:
line = stdin.readline().strip()
msg = json.loads(line)
type = msg["type"]
if "state" == type:
state = msg["neighbor"]["state"]
handleSession(state)
elif "update" == type:
update = msg["neighbor"]["message"]["update"]
if update.get('announce'):
flow = update["announce"]["ipv4 flow"]["no-nexthop"][0]
community = update["attribute"]["extended-community"][0]
handleFlow(True,flow,community)
elif update.get('withdraw'):
flow = update["withdraw"]["ipv4 flow"][0]
handleFlow(False,flow,None)
else:
pass
except IOError:
pass
Note: This script is a simple demonstration of the concept that has significant limitations: there is no error handling, the only action is to drop traffic, and FlowSpec comparison operators are ignored. The script is is based on the article
RESTful control of Cumulus Linux ACLs.
Update July 6, 2017: An improved version of the acl.py script is now available in the ExaBGP repository on GitHub, see
flow.py
Next, create the
exabgp.conf file:
process acl {
run ./acl.py;
encoder json;
}
template {
neighbor controller {
family {
ipv4 flow;
}
api speaking {
processes [ acl ];
neighbor-changes;
receive {
parsed;
update;
}
}
}
}
neighbor 10.0.0.70 {
inherit controller;
router-id 10.0.0.140;
local-as 65140;
peer-as 65070;
local-address 0.0.0.0;
connect 1179;
}
Finally, run ExaBGP:
sudo env exabgp.daemon.user=root exabgp-4.0.0/sbin/exabgp exabgp.conf
This configuration instructs ExaBGP to connect to the controller, 10.0.0.162, and prepare to receive BGP FlowSpec messages. When a BGP message is received, ExaBGP decodes the message and passes it on in the form of a JSON encoded string to the
acl.py program. For example, the following FlowSpec message was sent to block an NTP reflection DDoS attack (traffic from UDP port 123) targeting host 192.168.0.1:
{
"exabgp": "4.0.0",
"time": 1498868955.31,
"host": "tor-router",
"pid": 3854,
"ppid": 3849,
"counter": 6,
"type": "update",
"neighbor": {
"address": {
"local": "0.0.0.0",
"peer": "10.0.0.70"
},
"asn": {
"local": "65140",
"peer": "65070"
},
"direction": "receive",
"message": {
"update": {
"attribute": {
"origin": "igp",
"as-path": [
65140
],
"confederation-path": [],
"local-preference": 100,
"extended-community": [
9225060886715040000
]
},
"announce": {
"ipv4 flow": {
"no-nexthop": [
{
"destination-ipv4": [
"192.168.0.1/32"
],
"protocol": [
"=udp"
],
"source-port": [
"=123"
],
"string": "flow destination-ipv4 192.168.0.1/32 protocol =udp source-port =123"
}
]
}
}
}
}
}
}
The
acl.py script wrote the file
/etc/cumulus/acl/policy.d/60flowspec1.rules with an iptables representation of the FlowSpec message:
[iptables]
-A FORWARD --in-interface swp+ -p udp -d 192.168.0.1/32 --sport 123 -j DROP
The
acl.py script also invoked the
cl-acltool command to install the new rule, which can be verified using iptables:
sudo iptables --list FORWARD
Chain FORWARD (policy ACCEPT)
target prot opt source destination
DROP all -- 240.0.0.0/5 anywhere
DROP all -- loopback/8 anywhere
DROP all -- base-address.mcast.net/4 anywhere
DROP all -- 255.255.255.255 anywhere
DROP udp -- anywhere 192.168.0.1 udp spt:ntp
The attack traffic is now dropped by the switch. In a hardware switch, this entry would be pushed by Cumulus Linux into the hardware, filtering the traffic at line rate to provide Terabit traffic filtering.
