Processing active test data can detect that packets are being lost within approximately 20 seconds, but doesn't provide the precise location where packets are dropped. A custom multi-path traceroute tool (fbtracert) is used to follow up and narrow down the location of the packet loss.
While described as measuring packet loss, the test system is really measuring path loss. For example, if there are 64 ECMP paths in a pod, then the loss of one path would result in a packet loss of approximately 1 in 64 packets in traffic flows that cross the ECMP group.
Black hole detection describes an alternative approach. Industry standard sFlow instrumentation embedded within most vendor's switch hardware provides visibility into the paths that packets take across the network - see Packet paths. In some ways the sFlow telemetry is very similar to the traceroute tests, each measurement identifies the specific location a packet was seen.
The passive sFlow monitoring approach has significant benefits:
- Eliminates active test traffic since production traffic exercises network paths.
- Eliminates traffic generators and test targets required to perform the active tests.
- Simplifies analysis since sFlow measurements provides a direct indication of anomaly location.
- Reduced operation complexity and associated costs.