SONiC developments for visibility into AI/ML networks in 2026

SONiC sFlow High Level Design (HLD) v1.4 was recently published. This is the latest in a series of revisions bringing support for sFlow extensions that enhance network visibility for AI / ML traffic flows.

v1.3 Egress sFlow support

RoCEv2 / Ultra Ethernet host adapters bypass the Linux kernel and transfer data directly to GPU memory, rendering traditional host-based network monitoring tools ineffective (tcpdump, Wireshark, eBPF etc.). Ingress/egress packet sampling on the top of rack switch offloads monitoring from the host to the switch to provide visibility into host traffic.

In addition, some measurements may only be possible for egress sampled packets. For example, the v1.3 HLD describes how SONiC SAI drivers can support the sFlow Delay and Transit Structures extension:

Depending on platform capabilities, SAI driver may report additional attributes defined in https://github.com/torvalds/linux/blob/master/include/uapi/linux/psample.h. For example, PSAMPLE_ATTR_OUT_TC (egress queue), PSAMPLE_ATTR_OUT_TC_OCC (egress queue depth), and PSAMPLE_ATTR_LATENCY (transit delay) populate the sFlow Transit Delay Structures (https://sflow.org/sflow_transit.txt).

Typically this data is only known when packets egress the switch and may only be available for egress sampled packets.

Transit delay and queuing describes the measurements and provides an example. The sFlow transit delay and queue depth extension adds additional metadata to each packet sample. The combination of delay/queue measurement and packet header makes it clear where queues are filling, why the queues are filling, and who is sending the traffic during microbursts.

v1.4 Dropped packet notification (Mirror-on-Drop) support

RoCEv2 / Ultra Ethernet performance is severely impacted by packet loss, so the visibility into lost packets is essential for real-time detection and remediation of packet loss events.

The sFlow dropped packet notification feature immediately reports every dropped packet. Drop events are not sampled, but instead a rate limit is used to ensure that burst of dropped packets don’t flood the monitoring system. Each dropped packet notification shows the switch and port where the packet was dropped, the reason why it was dropped, and the header of the dropped packet reveals who was affected and what they are trying to do.

Dropped packet notifications and packet sampling in sFlow are a powerful combination. Packet sampling provides detailed visibility into the traffic successfully flowing through the network and the drop notifications provide details on the failures. Correlating the two measurements allows you to see the traffic filling buffers at the time packets are dropped. This detail is needed to tune network settings (like ECN/PFC buffer utilization thresholds, RoCEv2 credits, etc.) to avoid packet loss.

Configuration

Enabling sFlow on SONiC for monitoring production RoCEv2 / Ultra Ethernet traffic requires a small number of commands to monitor all the ports on the switch. The same configuration is applied to every switch in the fabric for comprehensive visibility.

sflow collector add sflow-rt 192.0.2.129 --vrf mgmt
sflow sample-direction both
sflow drop-monitor-limit 50
sflow enable

In this case sampling-direction both ensures that both ingress and egress packets are sampled for complete visibility. Setting drop-monitor-limit enables dropped packet notifications and sets the rate limit, in packets per second, for dropped packet notification messages. Finally, sending sFlow over the out of band management network to the collector ensures that monitoring traffic cannot interfere with production traffic and adversely affect RoCEv2 performance.

Availability

SONiC sFlow version 1.3 features are available in current SONiC releases, and dropped packet notifications should be available as part of the upcoming SONiC 202605 Release.

AI Metrics with Prometheus and Grafana provides step by step instructions for monitoring a backend AI/ML network.

To see an example, the live SDSC Expanse cluster live AI/ML metrics dashboard can be accessed by clicking on the dashboard link. The San Diego Supercomputer Center (SDSC) Expanse cluster in the example has the following specifications: 5 Pflop/s peak; 93,184 CPU cores; 208 NVIDIA GPUs; 220 TB total DRAM; 810 TB total NVMe.