Leaf and spine network emulation on Mac OS M1/M2 systems

The GitHub sflow-rt/containerlab project contains example network topologies for the Containerlab network emulation tool that demonstrate real-time streaming telemetry in realistic data center topologies and network configurations. The examples use the same FRRouting (FRR) engine that is part of SONiC, NVIDIA Cumulus Linux, and DENT network operating systems. Containerlab can be used to experiment before deploying solutions into production. Examples include: tracing ECMP flows in leaf and spine topologies, EVPN visibility, and automated DDoS mitigation using BGP Flowspec and RTBH controls.

The Containerlab project currently has limited support for Mac OS, stating "ARM-based Macs (M1/2) are not supported, and no binaries are generated for this platform. This is mainly due to the lack of network images built for arm64 architecture as of now." However, this argument doesn't apply to the Linux based images used in these examples.

First install Docker Desktop on your Apple silicon based Mac (select the Apple Chip option).

mkdir clab
cd clab
docker run --rm -it --privileged \
  --network host --pid="host" \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /run/netns:/run/netns \
  -v $(pwd):$(pwd) -w $(pwd) \
  sflow/clab bash

Run Containerlab by typing the above commands in a terminal. This command uses a pre-built multi-architecture sflow/clab image. If you are running on an x86 platform, follow the official Containerlab Installation instructions.

git clone https://github.com/sflow-rt/containerlab.git

Download the Containerlab topologies from the sflow-rt/containerlab project.

containerlab deploy -t containerlab/clos5.yml

Start the 5 stage leaf and spine topology shown at the top of this page. The initial launch may take a couple of minutes as the container images are downloaded for the first time. Once the images are downloaded, the topology deploys in around 10 seconds.

An instance of the sFlow-RT real-time analytics engine receives industry standard sFlow telemetry from all the switches in the network. All of the switches in the topology are configured to send sFlow to the sFlow-RT instance. In this case, Containerlab is running the pre-built sflow/prometheus image which packages sFlow-RT with useful applications for exploring the data.

Connect to the web interface, http://localhost:8008. The sFlow-RT dashboard verifies that telemetry is being received from 10 agents (the 10 switches in the Clos fabric). See the sFlow-RT Quickstart guide for more information.

The screen capture shows a real-time view of traffic flowing across the network during a series iperf3 tests. Click on the sFlow-RT Apps menu and select the browse-flows application, or click here for a direct link to a chart with the settings shown above.

docker exec -it clab-clos5-h1 iperf3 -c 172.16.4.2

Each of the hosts in the network has an iperf3 server, so running the above command will test bandwidth between h1 and h4.

docker exec -it clab-clos5-leaf1 vtysh

Linux with open source routing software (FRRouting) is an accessible alternative to vendor routing stacks (no registration / license required, no restriction on copying means you can share images on Docker Hub, no need for virtual machines). FRRouting is popular in production network operating systems (e.g. Cumulus Linux, SONiC, DENT, etc.) and the VTY shell provides an industry standard CLI for configuration, so labs built around FRR allow realistic network configurations to be explored.

containerlab destroy -t containerlab/clos5.yml

When you are finished, run the above command to stop the containers and free the resources associated with the emulation. Try out other topologies from the project to explore topics such as DDoS mitigation, BGP Flowspec, and EVPN.

Moving the monitoring solution from Containerlab to production is straightforward since sFlow is widely implemented in datacenter equipment from vendors including: A10, Arista, Aruba, Cisco, Edge-Core, Extreme, Huawei, Juniper, NEC, Netgear, Nokia, NVIDIA, Quanta, and ZTE. In addition, the open source Host sFlow agent makes it easy to extend visibility beyond the physical network into the compute infrastructure.