sFlow Monitoring for AI

A Proposal towards sFlow Monitoring Dashboards for AI-controlled NRENs is a recent talk by Mariam Kiran (Esnet) presented at the recent GÉANT Telemetry and Big Data Workshop.

In the talk, Miram describes the set open source tools (Netdata, Prometheus, Zabbix, Ntopng, and PerfSONAR) that they attempted to synthesize a complete picture of the network.

A number of tools were combined since each tool provides a different subset of the measurements needed to drive the AI controller. 

Integrating the data from the different sources was a challenge, but they were able to pull the data together into a single Grafana dashboard. Unfortunately, there was a lot of noise in legacy measurement schemes, making the data set unsuitable for training the AI controller.

The team decided to go toward sFlow, replacing the legacy monitoring tools with sFlow enabled devices, in order to generate the very clean data needed for machine learning.

For background, the talk, Real-time network telemetry for automation, describes why sFlow is uniquely suited to automation, providing the comprehensive, real-time, system-wide, visibility needed to make networked systems observable.