Loggly

Loggly is a cloud logging and and analysis platform. This article will demonstrate how to integrate network events generated from industry standard sFlow instrumentation build into network switches.

Loggly offers a free 14 day evaluation, so you can try this example at no cost.

ICMP unreachable describes how monitoring ICMP destination unreachable messages can help identify misconfigured hosts and scanning behavior. The article uses the sFlow-RT real-time analytics software to process the raw sFlow and report on unreachable messages.

The following script, loggly.js, modifies the sFlow-RT script from the article to send events to the Loggly HTTP/S Event Endpoint:

var token = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx';
  
var url = 'https://logs-01.loggly.com/inputs/'+token+'/tag/http/';

var keys = [
  'icmpunreachablenet',
  'icmpunreachablehost',
  'icmpunreachableprotocol',
  'icmpunreachableport'
];

for (var i = 0; i < keys.length; i++) {
  var key = keys[i];
  setFlow(key, {
    keys:'macsource,ipsource,macdestination,ipdestination,' + key,
    value:'frames',
    log:true,
    flowStart:true
  });
}

setFlowHandler(function(rec) {
  var keys = rec.flowKeys.split(',');
  var msg = {
    flow_type:rec.name,
    src_mac:keys[0],
    src_ip:keys[1],
    dst_mac:keys[2],
    dst_ip:keys[3],
    unreachable:keys[4]
  };

  try { http(url,'post','application/json',JSON.stringify(msg)); }
  catch(e) { logWarning(e); };
}, keys);

Some notes on the script:

1. Modify the script to use the correct token for your Loggly account.
2. Including MAC addresses can help identify hosts even if they spoof IP addresses
3. See Writing Applications for more information.

Run the script using the sflow/sflow-rt docker image:

docker run -p 6343:6343/udp -v $PWD/loggly.js:/loggly.js \
sflow/sflow-rt -Dscript.file=/loggly.js

Events should now start appearing in Loggly.

The Loggly Live Tail page can be used to verify that the logs are being received. The screen capture at the start of this article shows a chart trending events by the host that triggered them, identifying 10.0.0.30 as the source of the network scan.

The loggly.js script can easily be modified to track and log different types of network activity. For example, Blacklists describes how to download a set of blacklisted addresses, match traffic against the blacklist and generate events for the matches.

Intranet DDoS attacks describes the threats posed by IoT (Internet of Things) devices and the need for visibility throughout the network in order to tackle these threats. Incorporating sFlow in the monitoring strategy extends visibility beyond the firewalls to the entire network.

In addition to generating events, sFlow analytics can be used to deliver performance metrics. The article, Cloud analytics, describes how to use sFlow-RT to send performance metrics to the Librato cloud service - also part of Solarwinds.