Real-time flow analytics with Containerlab templates

The GitHub sflow-rt/containerlab project contains example network topologies for the Containerlab network emulation tool that demonstrate real-time streaming telemetry in realistic data center topologies and network configurations. The examples use the same FRRouting (FRR) engine that is part of SONiC, NVIDIA Cumulus Linux, and DENT. Containerlab can be used to experiment before deploying solutions into production. Examples include: tracing ECMP flows in leaf and spine topologies, EVPN visibility, and automated DDoS mitigation using BGP Flowspec and RTBH controls.

This article describes an experiment with Containerlab's advanced Generated topologies capability, taking the 3 stage Clos topology shown above and creating a template that can be used to generate topologies with any number of leaf and spine switches.

The clos3.yml topology file specifies the 2 leaf 2 spine topology shown above:

name: clos3
mgmt:
  network: fixedips
  ipv4_subnet: 172.100.100.0/24
  ipv6_subnet: 2001:172:100:100::/80

topology:
  defaults:
    env:
      COLLECTOR: 172.100.100.8
  nodes:
    leaf1:
      kind: linux
      image: sflow/clab-frr
      mgmt_ipv4: 172.100.100.2
      mgmt_ipv6: 2001:172:100:100::2
      env:
        LOCAL_AS: 65001
        NEIGHBORS: eth1 eth2
        HOSTPORT: eth3
        HOSTNET: "172.16.1.1/24"
        HOSTNET6: "2001:172:16:1::1/64"
      exec:
        - touch /tmp/initialized
    leaf2:
      kind: linux
      image: sflow/clab-frr
      mgmt_ipv4: 172.100.100.3
      mgmt_ipv6: 2001:172:100:100::3
      env:
        LOCAL_AS: 65002
        NEIGHBORS: eth1 eth2
        HOSTPORT: eth3
        HOSTNET: "172.16.2.1/24"
        HOSTNET6: "2001:172:16:2::1/64"
      exec:
        - touch /tmp/initialized
    spine1:
      kind: linux
      image: sflow/clab-frr
      mgmt_ipv4: 172.100.100.4
      mgmt_ipv6: 2001:172:100:100::4
      env:
        LOCAL_AS: 65003
        NEIGHBORS: eth1 eth2
      exec:
        - touch /tmp/initialized
    spine2:
      kind: linux
      image: sflow/clab-frr
      mgmt_ipv4: 172.100.100.5
      mgmt_ipv6: 2001:172:100:100::5
      env:
        LOCAL_AS: 65003
        NEIGHBORS: eth1 eth2
      exec:
        - touch /tmp/initialized
    h1:
      kind: linux
      image: sflow/clab-iperf3
      mgmt_ipv4: 172.100.100.6
      mgmt_ipv6: 2001:172:100:100::6
      exec:
        - ip addr add 172.16.1.2/24 dev eth1
        - ip route add 172.16.2.0/24 via 172.16.1.1
        - ip addr add 2001:172:16:1::2/64 dev eth1
        - ip route add 2001:172:16:2::/64 via 2001:172:16:1::1
    h2:
      kind: linux
      image: sflow/clab-iperf3
      mgmt_ipv4: 172.100.100.7
      mgmt_ipv6: 2001:172:100:100::7
      exec:
        - ip addr add 172.16.2.2/24 dev eth1
        - ip route add 172.16.1.0/24 via 172.16.2.1
        - ip addr add 2001:172:16:2::2/64 dev eth1
        - ip route add 2001:172:16:1::/64 via 2001:172:16:2::1
    sflow-rt:
      kind: linux
      image: sflow/prometheus
      mgmt_ipv4: 172.100.100.8
      mgmt_ipv6: 2001:172:100:100::8
      ports:
        - 8008:8008
  links:
    - endpoints: ["leaf1:eth1","spine1:eth1"]
    - endpoints: ["leaf1:eth2","spine2:eth1"]
    - endpoints: ["leaf2:eth1","spine1:eth2"]
    - endpoints: ["leaf2:eth2","spine2:eth2"]
    - endpoints: ["h1:eth1","leaf1:eth3"]
    - endpoints: ["h2:eth1","leaf2:eth3"]

The new clos3.clab.gotmpl file is a templated version of the topology:

name: clos3
mgmt:
  network: fixedips
  ipv4_subnet: 172.100.100.0/24
  ipv6_subnet: 2001:172:100:100::/80
topology:
  defaults:
    kind: linux
    env:
      COLLECTOR: 172.100.100.{{ add $.spines.num $.leaves.num $.leaves.num 2 }}
  nodes:
{{- range $leafIndex := seq 1 $.leaves.num }}
    leaf{{ $leafIndex }}:
        image: sflow/clab-frr
        mgmt_ipv4: 172.100.100.{{ add $leafIndex 1 }}
        mgmt_ipv6: 2001:172:100:100::{{ add $leafIndex 1 }}
        env:
            LOCAL_AS: {{ add 65000 $leafIndex }}
            NEIGHBORS:{{- range $spineIndex := seq 1 $.spines.num }} eth{{ $spineIndex}}{{- end }}
            HOSTPORT: eth{{ add $.spines.num 1 }}
            HOSTNET: 172.16.{{ $leafIndex }}.1/24
            HOSTNET6: 2001:172:16:{{ $leafIndex }}::1/64
        exec:
            - touch /tmp/initialized
{{- end }}
{{- range $spineIndex := seq 1 $.spines.num }}
    spine{{ $spineIndex }}:
        image: sflow/clab-frr  
        mgmt_ipv4: 172.100.100.{{ add $.leaves.num $spineIndex 1 }}      
        mgmt_ipv6: 2001:172:100:100::{{ add $.leaves.num $spineIndex 1 }}
        env:
            LOCAL_AS: {{ add 65000 $.leaves.num 1 }}
            NEIGHBORS:{{- range $leafIndex := seq 1 $.leaves.num }} eth{{ $leafIndex }}{{- end }}
        exec:
            - touch /tmp/initialized
{{- end }}
{{- range $leafIndex := seq 1 $.leaves.num }}
    h{{ $leafIndex }}:   
        image: sflow/clab-iperf3
        mgmt_ipv4: 172.100.100.{{ add $.spines.num $.leaves.num $leafIndex 1 }}
        mgmt_ipv6: 2001:172:100:100::{{ add $.spines.num $.leaves.num $leafIndex 1 }}
        exec:
            - ip addr add 172.16.{{ $leafIndex }}.2/24 dev eth1
            - ip route add 172.16.0.0/16 via 172.16.{{ $leafIndex }}.1
            - ip addr add 2001:172:16:{{ $leafIndex }}::2/64 dev eth1
            - ip route add 2001:172:16::/48 via 2001:172:16:{{ $leafIndex }}::1             
{{- end }}
    sflow-rt:
        image: sflow/prometheus
        mgmt_ipv4: 172.100.100.{{ add $.spines.num $.leaves.num $.leaves.num 2 }}
        mgmt_ipv6: 2001:172:100:100::{{ add $.spines.num $.leaves.num $.leaves.num 2 }}
        ports:
            - 8008:8008
  links:
{{- range $spineIndex := seq 1 $.spines.num }}
  {{- range $leafIndex := seq 1 $.leaves.num }}
    - endpoints: ["spine{{ $spineIndex }}:eth{{ $leafIndex }}", "leaf{{ $leafIndex }}:eth{{ $spineIndex }}"]
  {{- end }}
{{- end }}
{{- range $leafIndex := seq 1 $.leaves.num }}
    - endpoints: ["leaf{{ $leafIndex }}:eth{{ add $.spines.num 1 }}", "h{{ $leafIndex }}:eth1"]
{{- end }}

The template makes uses of settings in the corresponsing clos3.clab_vars.yml file:

spines:
  num: 2
leaves:
  num: 2

While creating a template involves some work, the result is a more compact representation of the configuration since repetative leaf and spine configurations are captures in iterative blocks. The advantage becomes clear with larger topologies since a 4 leaf 4 spine explicit configuration would be twice as large, but the tempate remains unchanged.

docker run --rm -it --privileged --network host --pid="host" \
  -v /var/run/docker.sock:/var/run/docker.sock -v /run/netns:/run/netns \
  -v $(pwd):$(pwd) -w $(pwd) \
  ghcr.io/srl-labs/clab bash

Run the above command to start Containerlab.

wget https://raw.githubusercontent.com/sflow-rt/containerlab/master/clos3.clab.gotmpl
wget https://raw.githubusercontent.com/sflow-rt/containerlab/master/clos3.clab_vars.yml

Download the template and settings files.

containerlab deploy -t clos3.clab.gotmpl

Create the emulated network.

docker exec -it clab-clos3-leaf1 vtysh -c "show running-config"

See configuration of leaf1 router.

Connect to the web interface, http://localhost:8008. The sFlow-RT dashboard verifies that telemetry is being received from the four (two leaf and two spine) switches in the topology. See the sFlow-RT Quickstart guide for more information.

The screen capture shows a real-time view of traffic flowing across the network during an iperf3 tests. Click on the sFlow-RT Apps menu and select the browse-flows application, or click here for a direct link to a chart with the settings shown above.

docker exec -it clab-clos3-h1 iperf3 -c 172.16.2.2

Each of the hosts in the network has an iperf3 server, so running the above command will test bandwidth between h1 and h2.

containerlab destroy -t clos3.clab.gotmpl

When you are finished, run the above command to stop the containers and free the resources associated with the emulation.

Finally, try editing the clos3.clab_vars.yml file and increase the number of leaf switches to 12 and the number of spine switches to 5 and repeat the tests with a more realistic topology. A big advantage of using containers to emulate network devices is that they are extremely lightweight, allowing realistic production networks to be emulated on a laptop. Try the other sflow-rt/containerlab examples to experiment with DDoS mitigation, EVPN monitoring, and flow tracing.