Real-time trending of dropped packets

Discard Browser is a recently released open source application running on the sFlow-RT real-time analytics engine. The software uses streaming analytics to trend dropped packets.

Using sFlow to monitor dropped packets describes the recently added packet drop monitoring functionality added to the open source Host sFlow agent. The article describes how to install and configure the agent on Linux-based platforms and stream industry standard sFlow telemetry to an sFlow collector.

Visibility into dropped packets describes instrumentation, recently added to the Linux kernel, that provides visibility into packets dropped by the kernel data path on a host, or dropped by a switch ASIC when packets are forwarded in hardware.  Extending sFlow to provide visibility into dropped packets offers significant benefits for network troubleshooting, providing real-time network-wide visibility into the specific packets that were dropped as well the reason the packet was dropped. This visibility instantly reveals the root cause of drops and the impacted connections.

Packet discard monitoring complements sFlow's existing counter polling and packet sampling mechanisms and shares a common data model so that all three sources of data can be correlated.  For example, if packets are being discarded because of buffer exhaustion, the discard records don't necessarily tell the whole story. The discarded packets may represent mice flows that are victims of an elephant flow. Packet samples will reveal the traffic that isn't being dropped and provide a more complete picture. Counter data adds additional information such as CPU load, interface speed, link utilization, packet and discard rates that further completes the picture.

A simple way to run Discard Browser, pre-packaged with sFlow-RT collector, is to use the Docker sflow/prometheus image:

docker run --rm -p 6343:6343/udp -p 8008:8008 sflow/prometheus

Access the web interface at http://localhost:8008/.

The screen shot at the top of the page shows dropped connection attempts to the Linux host running the sFlow agent. The connection attempt were generated using curl:

curl http://192.168.1.87:8080

The Docker image also includes Metric Browser and Flow Browser applications that allow the full range of sFlow telemetry to be explored - see Real-time network and system metrics as a service for examples.

The included Prometheus exporter allows data to be pulled into a time series database to be included in dashboards. The Grafana sFlow-RT Network Interfaces, sFlow-RT Countries and Networks, sFlow-RT DDoS Protect, and sFlow-RT Health dashboards provide examples.