Visualizing real-time network traffic flows at scale

Particle has been released on GitHub, https://github.com/sflow-rt/particle. The application is a real-time visualization of network traffic in which particles flow between hosts arranged around the edges of the screen. Particle colors represent different types of traffic.

Particles provide an intuitive representation of network packets transiting the network from source to destination. The animation slows time so that the particle takes 10 seconds (instead of milliseconds) to transit the network. Groups of particles traveling the same path represent flows of packets between the hosts. Particle size and frequency are used to indicate the intensity of the traffic flowing on a path.

Particles don't follow straight lines, instead following quadratic Bézier curves around the center of the screen. Warping particle paths toward the center of the screen ensures that all paths are of similar length and visible - even if the start and end points are on the same axis.

The example above is from a site with over 500 network switches carrying hundreds of Gigabits of traffic. Internet, Customer, Site and Datacenter hosts have been assigned to the North, East, South and West sides respectively.

The screen is updated 60 times per second for smooth animation. Active flow metrics are updated every second. Hovering over the screen freezes the animation, highlights the nearest particle, and displays details.

To try out the software, first create a configuration file to label axes and assign addresses for your network.

particle.axisN=Internet
particle.cidrN=0.0.0.0/0
particle.axisS=Site
particle.cidrS=10.1.1.0/24,10.1.2.0/24
particle.axisE=Datacenter
particle.cidrE=10.2.0.0/16
particle.axisW=Remote
particle.cidrW=10.3.0.0/16

The above, particle.conf file, provides an example.

The simplest way to run the software is to use the pre-built Docker image.

docker run -p 8008:8008 -p 6343:6343/udp \
-v $PWD/particle.conf:/sflow-rt/particle.conf \
-e "RTPROP=-Dsystem.propertyFiles=particle.conf" \
sflow/particle

Access the web interface on port 8008.

The Docker image also contains a random simulation of flows to demonstrate the software:

docker run -e "RTPROP=-Dparticle.demo=yes" \
-p 6343:6343/udp -p 8008:8008 sflow/particle

This particle visualization was inspired by experiments with Vizceral, see Real-time traffic visualization using Netflix Vizceral. Vizceral focuses on interactions between layered microservices.

Visualizing network traffic unique challenges that needed to be addressed. For example, in these examples the North, Internet, axis (0.0.0.0/0) represents over 4 billion hosts - a number far greater than the number of pixels available on the screen. Instead of trying to represent each host individually, hosts are assigned a position proportional to their location in the range. For example, host 120.0.0.0 is assigned a position half way along the axis. Assigning fixed positions to each host ensures that traffic between the hosts will always take the same path across the screen, making it easier to recognize patterns and identify changes.

Chances are you have network equipment that supports sFlow telemetry since the standard is widely supported by vendors, including: A10, Aerohive, ALUe, Allied Telesis, Arista, Aruba, Big Switch, Cisco, Cumulus, Dell, D-Link, Edge-Core, Extreme, F5, Fortinet, Huawei, IP Infusion, Juniper, Netgear, OpenSwitch, Pica8, Proxim, Quanta, SMC, ZTE, and ZyXEL. Give Particle a try and see how traffic flows on your network.