Host, Docker, Swarm and Kubernetes monitoring

The open source Host sFlow agent incorporates technologies that address the challenges of microservice monitoring; leveraging recent enhancements to Berkeley Packet Filter (BPF) in the Linux kernel to randomly sample packets, and  Asynchronous Docker metrics to track rapidly changing workloads. The continuous stream of real-time telemetry from all compute nodes, transported using the industry standard sFlow protocol, provides comprehensive real-time cluster-wide visibility into all services and the traffic flowing between them.

The Host sFlow agent is available as pre-packaged rpm/deb files that can be downloaded and installed on each node in a cluster.

sflow {
  collector { ip=10.0.0.70 }
  docker { }
  pcap { dev=docker0 }
  pcap { dev=docker_gwbridge } 
}

The above /etc/hsflowd.conf file, see Configuring Host sFlow for Linux via /etc/hsflowd.conf, enables the docker {} and pcap {} modules for detailed visibility into container metrics and network traffic flows, and streams telemetry to an sFlow collector (10.0.0.70). The configuration is the same for every node making it simple to install and configure Host sFlow on all nodes using orchestration software such as Puppet, Chef, Ansible, etc.

The agent is also available as the pre-build sflow/host-sflow image, providing a simple method of instrumenting nodes running container workloads.

docker run \
--detach \
--name=host-sflow \
--env "COLLECTOR=10.0.0.70" \
--net=host \
--volume /var/run/docker.sock:/var/run/docker.sock:ro \
sflow/host-sflow

Execute above command to install and run the Host sFlow agent on a Docker node.

docker service create \
--mode global \
--name host-sflow \
--env "COLLECTOR=10.0.0.70" \
--network host \
--mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock,readonly \
sflow/host-sflow

Install and run an instance of the Host sFlow agent on each node in a Docker Swarm cluster.

Deploying Host sFlow under Kubernetes is a little more complicated.

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: host-sflow
spec:
  selector:
    matchLabels:
      name: host-sflow
  template:
    metadata:
      labels:
        name: host-sflow
    spec:
      hostNetwork: true
      containers:
      - name: host-sflow
        image: sflow/host-sflow:latest
        env:
          - name: COLLECTOR
            value: "10.0.0.70"
          - name: NET
            value: "host"
        volumeMounts:
          - mountPath: /var/run/docker.sock
            name: docker-sock
            readOnly: true
      volumes:
        - name: docker-sock
          hostPath:
            path: /var/run/docker.sock

First, create a deployment description file like the host-sflow.yml file above.

kubectl apply -f host-sflow.yml

Install and run an instance of the Host sFlow agent on each node in the Kubernetes cluster.

docker run -p 6343:6343/udp sflow/sflowtool

Run the command above on the collector (10.0.0.70) to verify that sFlow is arriving, see Running sflowtool using Docker.

docker run -p 6343:6343/udp -p 8008:8008 sflow/sflow-rt

Run the sflow/sflow-rt image to access real-time cluster performance metrics and network traffic flows through a REST API. Forwarding using sFlow-RT describes how to copy sFlow telemetry streams for additional tools.

Install sFlow-RT applications to export metrics to Prometheus, block DDoS attacks, visualize flows, etc. Writing Applications describes how to use APIs to build your own applications to integrate analytics with automation and monitoring tools.