Why use sFlow for packet analysis? To rephrase the Heineken slogan, sFlow reaches the parts of the network that other technologies cannot reach. Industry standard
sFlow is widely supported by
switch vendors, embedding wire-speed packet monitoring throughout the network. With sFlow, any link or group of links can be remotely monitored. The alternative approach of physically attaching a probe to a
SPAN/Mirror port is becoming much less feasible with increasing network sizes (10's of thousands of switch ports) and link speeds (10, 100, and
400 Gigabits). Using sFlow for packet capture doesn't replace traditional packet analysis, instead sFlow extends the capabilities of existing packet capture tools into the high speed switched network.
This article describes the
sflow/tcpdump and
sflow/tshark Docker images, which provide a convenient way to analyze packets captured using sFlow.
Run the following command to analyze packets using
tcpdump:
$ docker run -p 6343:6343/udp -p 8008:8008 sflow/tcpdump
19:06:42.000000 ARP, Reply 10.0.0.254 is-at c0:ea:e4:89:b0:98 (oui Unknown), length 64
19:06:42.000000 IP 10.0.0.236.548 > 10.0.0.70.61719: Flags [P.], seq 3380015689:3380015713, ack 515038158, win 41992, options [nop,nop,TS val 1720029042 ecr 904769627], length 24
19:06:42.000000 IP 10.0.0.236.548 > 10.0.0.70.61719: Flags [P.], seq 149816:149832, ack 510628, win 41992, options [nop,nop,TS val 1720029087 ecr 904770068], length 16
19:06:42.000000 IP 10.0.0.236.548 > 10.0.0.70.61719: Flags [P.], seq 149816:149832, ack 510628, win 41992, options [nop,nop,TS val 1720029087 ecr 904770068], length 16
The normal
tcpdump options can be used. For example, to select DNS packets:
$ docker run -p 6343:6343/udp -p 8008:8008 sflow/tcpdump -vv port 53
reading from file -, link-type EN10MB (Ethernet)
19:08:49.000000 IP (tos 0x0, ttl 64, id 22316, offset 0, flags [none], proto UDP (17), length 65)
10.0.0.70.43801 > dns.google.53: [udp sum ok] 35941+ A? clients2.google.com. (37)
19:09:00.000000 IP (tos 0x0, ttl 255, id 16813, offset 0, flags [none], proto UDP (17), length 66)
10.0.0.64.50675 > 10.0.0.1.53: [udp sum ok] 57874+ AAAA? p49-imap.mail.me.com. (38)
The following command selects TCP SYN packets:
$ docker run -p 6343:6343/udp sflow/tcpdump 'tcp[tcpflags] == tcp-syn'
reading from file -, link-type EN10MB (Ethernet)
19:10:37.000000 IP 10.0.0.30.46786 > 10.0.0.162.1179: Flags [S], seq 2993962362, win 29200, options [mss 1460,sackOK,TS val 20531427 ecr 0,nop,wscale 9], length 0
Capture 10 packets to a file and then exit:
$ docker run -v $PWD:/pcap -p 6343:6343/udp sflow/tcpdump -w /pcap/packets.pcap -c 10
reading from file -, link-type EN10MB (Ethernet)
A tcpdump Tutorial with Examples — 50 Ways to Isolate Traffic provides an overview of the capabilities of tcpdump with useful examples.
Run the following command to analyze packets using
tshark - a terminal based version of Wireshark:
$ docker run -p 6343:6343/udp -p 8008:8008 sflow/tshark
Capturing on '-'
1 0.000000 10.0.0.236 → 10.0.0.70 AFP 1518 [Reply without query?]
2 0.000000 10.0.0.236 → 10.0.0.70 AFP 1518 [Reply without query?]
3 0.000000 10.0.0.114 → 10.0.0.72 SSH 1518 Server: Encrypted packet (len=1448)
Packets can be filtered using
Display Filters. For example, the following command selects DNS traffic:
$ docker run -p 6343:6343/udp -p 8008:8008 sflow/tshark -Y 'dns'
Capturing on '-'
328 22.000000 8.8.8.8 → 10.0.0.70 DNS 136 Standard query response 0xfce4 AAAA img.youtube.com CNAME ytimg.l.google.com AAAA
472 36.000000 10.0.0.52 → 10.0.0.1 DNS 79 Standard query 0x173e AAAA www.nytimes.com
Print ip source, destination, protocol and packet lengths:
$ docker run -p 6343:6343/udp -p 8008:8008 sflow/tshark -T fields -e ip.src -e ip.dst -e ip.proto -e ip.len
Capturing on '-'
10.0.0.70 10.0.0.236 6 1500
10.0.0.236 10.0.0.70 6 52
10.0.0.70 10.0.0.236 6 1500
10.0.0.236 10.0.0.70 6 52
10.0.0.70 10.0.0.236 6 1500
Capture 100 packets and print summary of the protocols:
$ docker run -p 6343:6343/udp -p 8008:8008 sflow/tshark -q -z io,phs -c 100
Capturing on '-'
100 packets captured
===================================================================
Protocol Hierarchy Statistics
Filter:
eth frames:100 bytes:85721
ip frames:99 bytes:85657
tcp frames:97 bytes:85119
dsi frames:61 bytes:82122
_ws.short frames:54 bytes:77180
afp frames:6 bytes:4856
_ws.short frames:5 bytes:4766
_ws.short frames:15 bytes:1050
http frames:1 bytes:499
_ws.short frames:1 bytes:499
iscsi frames:1 bytes:118
iscsi.flags frames:1 bytes:118
scsi frames:1 bytes:118
_ws.short frames:1 bytes:118
ipv6 frames:2 bytes:538
tcp frames:2 bytes:538
tls frames:2 bytes:538
_ws.short frames:2 bytes:538
arp frames:1 bytes:64
_ws.short frames:1 bytes:64
===================================================================
Capture 100 packets and print a summary of the IP traffic by address:
$ docker run -p 6343:6343/udp -p 8008:8008 sflow/tshark -q -z endpoints,ip -c 100
Capturing on '-'
100 packets captured
================================================================================
IPv4 Endpoints
Filter:
| Packets | | Bytes | | Tx Packets | | Tx Bytes | | Rx Packets | | Rx Bytes |
10.0.0.70 95 81713 44 25507 51 56206
10.0.0.236 91 80820 50 55956 41 24864
10.0.0.30 6 2369 2 1508 4 861
10.0.0.16 1 587 1 587 0 0
10.0.0.28 1 587 0 0 1 587
10.0.0.160 1 1258 0 0 1 1258
10.0.0.172 1 218 1 218 0 0
================================================================================
The following command prints packet decodes as JSON:
$ docker run -p 6343:6343/udp -p 8008:8008 sflow/tshark -T json
Capturing on '-'
[
{
"_index": "packets-2019-09-06",
"_type": "pcap_file",
"_score": null,
"_source": {
"layers": {
"frame": {
"frame.interface_id": "0",
"frame.interface_id_tree": {
"frame.interface_name": "-"
},
"frame.encap_type": "1",
"frame.time": "Sep 6, 2019 19:41:12.000000000 UTC",
"frame.offset_shift": "0.000000000",
"frame.time_epoch": "1567798872.000000000",
"frame.time_delta": "0.000000000",
"frame.time_delta_displayed": "0.000000000",
"frame.time_relative": "0.000000000",
"frame.number": "1",
"frame.len": "64",
"frame.cap_len": "60",
"frame.marked": "0",
"frame.ignored": "0",
"frame.protocols": "eth:ethertype:arp"
},
"eth": {
"eth.dst": "70:10:6f:d8:13:30",
"eth.dst_tree": {
"eth.dst_resolved": "HewlettP_d8:13:30",
"eth.addr": "70:10:6f:d8:13:30",
"eth.addr_resolved": "HewlettP_d8:13:30",
"eth.lg": "0",
"eth.ig": "0"
},
"eth.src": "98:4b:e1:03:4a:61",
"eth.src_tree": {
"eth.src_resolved": "HewlettP_03:4a:61",
"eth.addr": "98:4b:e1:03:4a:61",
"eth.addr_resolved": "HewlettP_03:4a:61",
"eth.lg": "0",
"eth.ig": "0"
},
"eth.type": "0x00000806",
"eth.padding": "00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00"
},
"arp": {
"arp.hw.type": "1",
"arp.proto.type": "0x00000800",
"arp.hw.size": "6",
"arp.proto.size": "4",
"arp.opcode": "1",
"arp.src.hw_mac": "98:4b:e1:03:4a:61",
"arp.src.proto_ipv4": "10.0.0.30",
"arp.dst.hw_mac": "00:00:00:00:00:00",
"arp.dst.proto_ipv4": "10.0.0.232"
},
"_ws.short": "[Packet size limited during capture: Ethertype truncated]"
}
}
},
The tshark
-T ek option formats the JSON output as a single line per packet making the output easy to parse in scripts. For example, the following emerging.py script downloads the
Emerging Threats compromised IP address database, parses the JSON records, checks to see if source and destination addresses can be found in the database, and prints out information on any matches:
#!/usr/bin/env python
from sys import stdin
from json import loads
from requests import get
blacklist = set()
r = get('https://rules.emergingthreats.net/blockrules/compromised-ips.txt')
for line in r.iter_lines():
blacklist.add(line)
for line in stdin:
msg = loads(line)
try:
time = msg['timestamp']
layers = msg['layers']
ip = layers["ip"]
src = ip["ip_ip_src"]
dst = ip["ip_ip_dst"]
if src in blacklist or dst in blacklist:
print "%s %s %s" % (time,src,dst)
except KeyError:
pass
The following command runs the script:
$ docker run -p 6343:6343/udp -p 8008:8008 sflow/tshark -T ek | ./tshark.py
See the
TShark man page for more options.
Forwarding using sFlow-RT describes how to set up and tear down sFlow streams using the sFlow-RT analytics engine. This is a simple way to direct a stream of sFlow to a desktop running sflowtool. For example, suppose sflowtool is running on host 10.0.0.30 and sFlow-RT is running on host 10.0.0.1, the following command would start a session:
curl -H "Content-Type:application/json" -X PUT --data '{"address":"10.0.0.30"}' \
http://10.0.0.1:8008/forwarding/tcpdump/json
and the following command would end the session:
curl -X DELETE http://10.0.0.1:8008/forwarding/tcpdump/json
Note: The
sflow/sflow-rt Docker image is a convenient way to run sFlow-RT:
docker run -p 8008:8008 -p 6343:6343/udp sflow/sflow-rt
Finally,
Triggered remote packet capture using filtered ERSPAN, shows how the broad visibility provided by sFlow can be combined with hardware filtering to trigger full packet capture of selected traffic.
