The linear topology shown in the diagram was created using Containerlab with FRR nodes implementing IPv6 segment routing (SRv6) using compact uSID (micro-SID) encoding of path segments. Each router node is instrumented using Host sFlow agents. FRR and Host sFlow are used in SONiC open source network operating system so the lab is a realistic reflection the data you would get from a production network. The sFlow telemetry is sent to an instance of the sFlow-RT real-time analytics engine in order to demonstrate visibility into SRv6 traffic flows.
![]() |
| Click on image to enlarge |
The open source browse-flows application is an easy way to explore flows. The following command uses the sflow/prometheus Docker image, configured with the SRv6 uSID scheme and prefix:
docker run -p 6343:6343/udp -p 8008:8008 sflow/prometheus \ -Ddecode.srv6usid.scheme=f3216 \ -Ddecode.srv6usid.prefix=fcbb:bb00::
In this case, the chart displays the ingress segment list seen by each router along the path. The segment list is assembled by examining the destination IPv6 address (which can encode up to 6 uSID segments). For example, r1 encodes the path in the IPv6 address fcbb:bb00:102:103:104:105:106:107. The remaining segments are encoded in the IPv6 segment routing option fcbb:bb00:108:109:10a:10b:10c:d000. When the initial 6 segments are exhausted at router r7, the remaining segments are copied from the router option into the destination address and the ip6routesegsleft counter decrements to zero indicating that this is the final set of segments. See Defining Flows for the full set of packet attributes that can be used to filter and characterize flows.
Multipath Reliable Connection (MRC) is an extension to RoCEv2 that uses source routing to spray packets across a large number of separate paths across an AI / ML fabric in order to avoid congestion caused by large flow collisions. SRv6 uSID encoding allows MRC host adapters to explicitly select packet paths across the fabric and respond quickly to congestion or link failure. Enabling industry standard sFlow telemetry on network switches complements MRC by exposing traffic distribution, path utilization, congestion, packet loss, and failure behavior.
The two protocols are completely independent, complementary, and non-interfering:
- MRC is implemented in the host network adapters to make an AI training network resilient and performant through packet spraying, adaptive load balancing, and rapid recovery from failures.
-
sFlow telemetry is a standard feature of the multi-plane switch
fabric connecting the hosts and consumes no host resources that could
interfere with MRC performance. sFlow provides independent, network-wide,
visibility into how the fabric is actually behaving.
Note: Traditional network flow based technologies (IPFIX, NetFlow, ...) struggle with packet spraying (which spreads the packets in each flow over many disjoint paths - rendering flow based monitoring in the routers ineffective). In contrast, sFlow's stateless packet oriented (rather than flow oriented) sampling and packet header export mechanism provides detailed real-time visibility without burdening the network devices making the measurements.


No comments:
Post a Comment