The node-sflow-module project is an open source implementation of sFlow monitoring for node.js, an open source event-based environment environment for creating network applications that built on Google's high performance V8 JavaScript Engine.
The advantage of using sFlow is the scalability it offers for monitoring the performance of large web server clusters or load balancers where request rates are high and conventional logging solutions generate too much data or impose excessive overhead. Real-time monitoring of HTTP provides essential visibility into the performance of large-scale, complex, multi-layer services constructed using Representational State Transfer (REST) architectures. In addition, monitoring HTTP services using sFlow is part of an integrated performance monitoring solution that provides real-time visibility into applications, servers and switches (see sFlow Host Structures).
The node-sflow-module software (sflow.js) is designed to integrate with the Host sFlow agent to provide a complete picture of server performance. Download, install and configure Host sFlow before proceeding to install node.js - see Installing Host sFlow on a Linux Server. There are a number of options for analyzing cluster performance using Host sFlow, including Ganglia and sFlowTrend.
Next, download the sflow.js file from http://node-sflow-module.googlecode.com/. Copy the sflow.js file into the same directory as your node.js application. Including the sFlow instrumentation is a one line change to the application, see the simple Hello World example below:
var http = require("http");
require("./sflow.js").instrument(http);
http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello World\n');
}).listen(1337, "127.0.0.1");
console.log('Server running at http://127.0.0.1:1337/');
Once installed, the sflow.js will stream measurements to a central sFlow Analyzer. Currently the only software that can decode HTTP sFlow is sflowtool. Download, compile and install the latest sflowtool sources on the system your are using to receive sFlow from the servers in the node.js cluster.
Running sflowtool will display output of the form:
[pp@pcentos ~]$ sflowtool startDatagram ================================= datagramSourceIP 10.0.0.112 datagramSize 116 unixSecondsUTC 1314458638 datagramVersion 5 agentSubId 8124 agent 10.0.0.112 packetSequenceNo 1 sysUpTime 22002 samplesInPacket 1 startSample ---------------------- sampleType_tag 0:2 sampleType COUNTERSSAMPLE sampleSequenceNo 1 sourceId 3:8124 counterBlock_tag 0:2201 http_method_option_count 0 http_method_get_count 2 http_method_head_count 0 http_method_post_count 0 http_method_put_count 0 http_method_delete_count 0 http_method_trace_count 0 http_methd_connect_count 0 http_method_other_count 0 http_status_1XX_count 0 http_status_2XX_count 2 http_status_3XX_count 0 http_status_4XX_count 0 http_status_5XX_count 0 http_status_other_count 0 endSample ---------------------- endDatagram ================================= startDatagram ================================= datagramSourceIP 10.0.0.112 datagramSize 236 unixSecondsUTC 1314458652 datagramVersion 5 agentSubId 8124 agent 10.0.0.112 packetSequenceNo 2 sysUpTime 35729 samplesInPacket 1 startSample ---------------------- sampleType_tag 0:1 sampleType FLOWSAMPLE sampleSequenceNo 0 sourceId 3:8124 meanSkipCount 6 samplePool 6 dropEvents 0 inputPort 0 outputPort 1073741823 flowBlock_tag 0:2201 flowSampleType http http_method 2 http_protocol 1001 http_uri / http_host 10.0.0.112:8124 http_useragent Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_1) AppleWebKit/534. http_bytes 0 http_duration_uS 0 http_status 200 flowBlock_tag 0:2100 extendedType socket4 socket4_ip_protocol 6 socket4_local_ip 10.0.0.112 socket4_remote_ip 10.1.1.60 socket4_local_port 8124 socket4_remote_port 52609 endSample ---------------------- endDatagram =================================
The -H option causes sflowtool to output the HTTP request samples using the combined log format:
[pp@pcentos ~]$ sflowtool -H 10.1.1.60 - - [27/Aug/2011:08:26:52 -0700] "GET / HTTP/1.1" 200 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_1) AppleWebKit/534."
Converting sFlow to combined logfile format allows existing log analyzers to be used to analyze the sFlow data. For example, the following commands use sflowtool and webalizer to create reports:
The resulting webalizer report shows top URLs:
Finally, the real potential of HTTP sFlow is as part of a broader performance management system providing real-time visibility into applications, servers, storage and networking across the entire data center.
