sflowtool

The sflowtool command line utility is used to convert standard sFlow records into a variety of different formats. While there are a large number of native sFlow analysis applications, familiarity with sflowtool is worthwhile since it allows a wide variety of additional tools to analyze sFlow data as well as opening up the data to custom scripting.

First download, compile and install sflowtool using the following commands:

[root@xenvm4 ~]# wget http://www.inmon.com/bin/sflowtool-3.22.tar.gz
[root@xenvm4 ~]# tar -xvzf sflowtool-3.22.tar.gz
[root@xenvm4 ~]# cd sflowtool-3.22
[root@xenvm4 sflowtool-3.22]# ./configure
[root@xenvm4 sflowtool-3.22]# make
[root@xenvm4 sflowtool-3.22]# make install

Update 14 August 2015: Download the latest version of sflowtool from GitHub, https://github.com/sflow/sflowtool/archive/master.zip

The default behavior of sflowtool is to convert sFlow into ASCII text:

[root@xenvm4 ~]# sflowtool
startDatagram =================================
datagramSourceIP 10.0.0.111
datagramSize 144
unixSecondsUTC 1321922602
datagramVersion 5
agentSubId 0
agent 10.0.0.20
packetSequenceNo 3535127
sysUpTime 270660704
samplesInPacket 1
startSample ----------------------
sampleType_tag 0:2
sampleType COUNTERSSAMPLE
sampleSequenceNo 228282
sourceId 0:14
counterBlock_tag 0:1
ifIndex 14
networkType 6
ifSpeed 100000000
ifDirection 0
ifStatus 3
ifInOctets 4839078
ifInUcastPkts 15205
ifInMulticastPkts 0
ifInBroadcastPkts 4294967295
ifInDiscards 0
ifInErrors 0
ifInUnknownProtos 4294967295
ifOutOctets 149581962744
ifOutUcastPkts 158884229
ifOutMulticastPkts 4294967295
ifOutBroadcastPkts 4294967295
ifOutDiscards 101
ifOutErrors 0
ifPromiscuousMode 0
endSample   ----------------------
endDatagram   =================================

The text output of flowtool is easily processed using scripts. The following example provides a basic skeleton for processing the output of sflowtool in Perl:

#!/usr/bin/perl -w
use strict;
use POSIX;

open(PS, "/usr/local/bin/sflowtool|") || die "Failed: $!\n";
while( <PS> ) {  
  my ($attr,$value) = split;
 
  # process attribute  
}

close(PS);

Examples of scripts using sflowtool on this blog include Memcached hot keys and Memcached missed keys. Other examples include converting sFlow for Graphite and RRDtool.

The sFlow standard extends to application layer monitoring, including visibility into HTTP performance. Implementations of sFlow for popular web servers, including Apache, NGINX, Tomcat and node.js offer real-time visibility into large web farms.

The -H option causes sflowtool to output the HTTP request samples using the combined log format, making the data accessible to most log analyzers.

[root@xenvm4 ~]# sflowtool -H
10.0.0.70 - - [22/Nov/2011:12:36:32 -0800] "GET http://sflow.org/images/h-photo.jpg HTTP/1.1" 304 0 "http://sflow.org/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_2) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.120 Safari/535.2"
10.0.0.70 - - [22/Nov/2011:12:36:32 -0800] "GET http://sflow.org/inc/nav.js HTTP/1.1" 304 0 "http://sflow.org/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_2) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.120 Safari/535.2"
10.0.0.70 - - [22/Nov/2011:12:36:32 -0800] "GET http://sflow.org/images/participant-foundry.gif HTTP/1.1" 304 0 "http://sflow.org/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_2) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.120 Safari/535.2"

For example, the following commands use sflowtool and webalizer to create reports:

/usr/local/bin/sflowtool -H | rotatelogs log/http_log &
webalizer -o report log/*

The resulting webalizer report shows top URLs:

The sFlow standard operates by randomly sampling packet headers. The sflowtool -t option allows sFlow to be used for remote packet capture, converting packet header information from sFlow to standard pcap format that can be used with packet analysis applications.

The following example uses sflowtool and tcpdump to display a packet trace:

[root@xenvm4 ~]# sflowtool -t | tcpdump -r - -vv
reading from file -, link-type EN10MB (Ethernet)
10:30:01.000000 arp who-has 10.0.0.66 tell 10.0.0.220
10:30:07.000000 IP (tos 0x0, ttl  64, id 49952, offset 0, flags [DF], proto: TCP (6), length: 1500) xenserver1.sf.inmon.com.39120 > openfiler.sf.inmon.com.iscsi-target: . 2757963136:2757964584(1448) ack 4136690254 win 3050 
10:30:07.000000 IP (tos 0x0, ttl  64, id 49953, offset 0, flags [DF], proto: TCP (6), length: 1500) xenserver1.sf.inmon.com.39120 > openfiler.sf.inmon.com.iscsi-target: . 1448:2896(1448) ack 1 win 3050 
10:30:07.000000 IP (tos 0x0, ttl  64, id 49954, offset 0, flags [DF], proto: TCP (6), length: 1500) xenserver1.sf.inmon.com.39120 > openfiler.sf.inmon.com.iscsi-target: . 2896:4344(1448) ack 1 win 3050

The Wireshark article describes how to use sflowtool and Wireshark to graphically display packet information.

sflowtool can also be used to convert sFlow to NetFlow version 5. The following command converts sFlow records into NetFlow records and sends them to UDP port 9991 on netflow.inmon.com:

[root@xenvm4 ~]# sflowtool -c netflow.inmon.com -d 9991

Converting sFlow to NetFlow provides compatibility with NetFlow analyzers.  However, converting sFlow to NetFlow results in a significant loss of information and it is better to use a native sFlow analyzer to get the full value of sFlow. In many cases traffic analysis software supports both sFlow and NetFlow, so conversion is unnecessary.

Finally, sFlow provides information on network, server, virtual machine and application performance and the sflowtool source code offers developers a useful starting point for adding sFlow support to network, server and application performance monitoring software - see Developer resources for additional information.