Thursday, November 21, 2019

Real-time monitoring at terabit speeds

The Flow Trend chart above shows a real-time, up to the second, view of nearly 3 terabits per second of traffic flowing across the SCinet network, described as the fastest, most powerful volunteer-built network in the world. The network is build each year to support The International Conference for High Performance Computing, Networking, Storage, and Analysis. The SC19 conference is currently underway in Denver, Colorado.
The diagram shows the Joint Big Data Testbed generating the traffic in the chart. The Caltech demonstration is described in NRE-19: SC19 Network Research Exhibition: Caltech Booth 543 Demonstrations Hosting NRE-13, NRE-19, NRE-20, NRE-22, NRE-23, NRE-24, NRE-35:
400GE First Data Networks: Caltech, Starlight/NRL, USC, SCinet/XNET, Ciena, Mellanox, Arista, Dell, 2CRSI, Echostreams, DDN and Pavilion Data, as well as other supporting optical, switch and server vendor partners will demonstrate the first fully functional 3 X400GE local ring network as well as 400GE wide area network ring, linking the Starlight and Caltech booths and Starlight in Chicago. This network will integrate storage using NVMe over Fabric, the latest high throughput methods, in-depth monitoring and realtime flow steering. As part of these demonstrations, we will make use of the latest DWDM, Waveserver Ai, and 400GE as well as 200GE switch and network interfaces from Arista, Dell, Mellanox and Juniper as part of this core set of demonstrations.
Industry standard sFlow telemetry from the Arista, Dell, Mellanox, and Extreme switches in the testbed is being processed by an instance of the sFlow-RT real-time analytics engine running the embedded Flow Trend application (as well as a number of other application, including:  SC19 SCinet: Grafana network traffic dashboard).

This example demonstrates the scalability of sFlow monitoring, leveraging instrumentation built into switch ASICs to deliver comprehensive line rate visibility into the 400 Gigabit per second traffic flows generated by the testbed.

No comments:

Post a Comment