Choosing an sFlow analyzer

sFlow achieves network-wide visibility by shifting complexity away from the switches to the sFlow analysis application. Simplifying the monitoring task for the switch makes it possible to implement sFlow in hardware, providing wire-speed performance, without increasing the cost of the switch. However, the shift of complexity to the sFlow analysis application makes the selection of the sFlow analyzer a critical factor in realizing the full benefits of sFlow monitoring.

To illustrate some of the features that you should look for in an sFlow analyzer, consider the following basic question, "Which hosts are generating the most traffic on the network?" The chart provides information that answers the question, displaying the top traffic sources and the amount of traffic that they generate. In order to generate this chart, the sFlow analyzer needs to support the following features:

1. Since the busiest hosts in the network could be anywhere, the sFlow analyzer needs to monitor every link in the network to accurately generate the chart.
2. Traffic may traverse a number of monitored switch ports, in the example above, traffic between hosts A and B is monitored by 10 switch ports. In order to correctly report on the amount of traffic by host, the sFlow analyzer needs to combine data from the different switch ports in a way that correctly calculates the traffic totals and avoids under or over counting.
   
3. The sFlow analyzer must fully support sFlow's packet sampling mechanism in order to accurately calculate traffic volumes.
   
4. Notice that the chart contains IPv4, IPv6 and MAC addresses. The sFlow analyzer needs to be able to decode packet headers and report on all the protocols in use on the network, including layer 2 and layer 3 traffic. Traffic on local area networks (LANs) is much more diverse than routed wide area network (WAN) traffic. In addition to the normal TCP/IP traffic seen on the WAN, LAN traffic can include multicast, broadcast, service discovery (Bonjour), host configuration (DHCP), printing, backup and storage traffic not typically seen on the WAN.

When selecting an sFlow analyzer, try to arrange an evaluation and test the product on a full scale production network.  Evaluating scalability and accuracy is not something that is easily performed in a test lab.

Accuracy and packet loss

Traffic records are often lost:

1. A switch must reliably perform it's primary function of forwarding packets, so if there is any contention for resources in the switch, measurement records will be discarded.
   
2. There will inevitably be some loss of measurement records as they are transferred over the network from the switches to the traffic analyzer. Again, measurement traffic is a low priority and may be discarded if the network is busy.
3. Finally, a traffic analyzer may lose traffic records if larger numbers of switches are being monitored and records are arriving faster than they can be processed.
   

The chart shows the effect of lost records on the accuracy of sFlow and NetFlow monitoring:

1. NetFlow has no mechanism to compensate for lost records. If NetFlow records are lost then traffic will be underreported. The greater the number of records lost, the lower the reported traffic. The bursty and unpredictable traffic produced by NetFlow monitoring increases the likelihood that NetFlow records will be lost. The loss of even one NetFlow record can significantly affect accuracy since a single flow record may summarize a large transfer of data and represent a substantial fraction of the overall network traffic.
   
2. sFlow's packet sampling mechanism treats record loss as a decrease in the sampling probability. The sFlow records contain information that allows the traffic analyzer to measure the effective sampling rate, compensate for the packet loss, and generate corrected values. Each sFlow record represents a single packet event and large flows of traffic will generate a number of sFlow records. Thus, the loss of an sFlow record does not represent a significant loss of data and doesn't affect the overall accuracy of traffic measurements.

Underreporting traffic, particularly during peak periods is a serious problem for troubleshooting, congestion management and traffic engineering applications. For usage-based billing applications, underreported traffic represents lost revenue.

When monitoring using NetFlow and sFlow to achieve network-wide visibility, situating the traffic analyzer near the NetFlow sources will help reduce the loss of flow records and improve accuracy.

Measurement overhead

How much extra traffic will network monitoring generate? The goal of network-wide visibility is to improve performance on the network, so the extra traffic generated by monitoring needs to be small and must not degrade performance.

The chart looks at the overhead in terms of measurement records reported per packet on the network. Ideally the overhead associated with monitoring should be small and constant (less than 0.1% of the traffic). Since flow-oriented monitoring (e.g. NetFlow) involves the creation and export of flow records, the overhead is determined by the average number of packets in a flow. If there are a large number of packets in a flow, the overhead will be low. However, if the number of packets per flow is small then the overhead will be high and in the worst case may result in a flow record being generated and exported for every packet on the network.

In practice, the number packets per flow can vary enormously depending on the type of traffic being monitored. DNS traffic is one packet per flow, web traffic will typically have 5-10 packets per flow and video streams may have thousands of packets per flow.

The overhead generated by flow monitoring can become acute during a worm outbreak or when the network is subjected to a denial of service attack (DoS attack). In both cases large numbers of single packet flows are created and the additional overhead created by flow monitoring is likely to exacerbate the problem. The impact of this increased measurement traffic on the network is made worse by the traffic bursts that flow monitoring creates. It is precisely during these times that network visibility is most needed so that the threat can be identified and controlled.

Since sFlow is not a flow-based protocol, the overhead is completely unaffected by the number of packets per flow. sFlow's use of packet sampling limits the overhead of traffic monitoring and ensures accurate, timely, network-wide visibility without impacting network performance - even during extreme traffic situations like a denial of service attack.

Measurement traffic

The charts, based on measurements from switches in a production environment, compare NetFlow and sFlow in terms of the load that they generate on the network. The following observations can be made based on this data:

• NetFlow monitoring generates periodic bursts of traffic; the periodicity is confirmed by the sharp spikes in the frequency chart. This behavior is typical of flow-based traffic monitoring protocols (see Exporting IP flows using IPFIX) since flow generation involves maintaining a cache of active flows on the switch and the use timers to trigger flow export.
• sFlow monitoring generates a random pattern of traffic with no periodicity and no bursts. The randomness is confirmed by the flat frequency chart.

Network-wide visibility involves collecting traffic data from large numbers of switches and routers. The bursts of traffic generated by flow monitoring can cause problems with delay, packet loss and jitter that will effect other traffic on the network. The periodicity observed in flow monitoring creates the risk that the different streams of monitoring traffic will synchronize and reinforce each other as large numbers of devices are monitored.

It is essential that the technology used to manage network traffic does not itself cause traffic problems. The random, low-level, background traffic that sFlow generates ensures that large networks can be safely monitored without any adverse effects. This behavior is no accident, sFlow was designed to be scalable and the random packet sampling mechanism in sFlow is one of the reasons that its traffic is well behaved.

Control

Control theory is an area of engineering and applied mathematics dealing with the behavior and control of dynamic systems. Many of the concepts can usefully be applied to network visibility and control.

The diagram shows the basic elements of a feedback controller. When controlling a network, the network would be the "System", the "Sensor" takes observations of the system (sFlow) and converts them into an estimate of the current network state (link utilizations, traffic flows etc.). The measured network state is compared to a Reference (usage policies, thresholds etc.) and any deviations from the desired behavior is used to trigger a control action (blocking a port, setting a rate limit etc.), changing the behavior of the network and restoring service levels.

Control theory has concepts of stability, observability, controllability and robustness that are very general and worth thinking about in the context of network management:

• Stability is a way of describing how well behaved a system is. If you make a small change and the system's behavior oscillates wildly then it isn't stable (routing instability and congestion are examples of instability in a network setting).
• Observability is a way of saying, "You can't control what you can't see." If you don't incorporate traffic measurement into the network design (by specifying switches with built-in traffic monitoring) then traffic will not be observable. Every device needs to have built-in traffic monitoring if you want to ensure that the whole network is observable.
  
• Controllability is something that should be considered when designing the network; deploying managed switches in each layer of the network with appropriate control capabilities (e.g. access control lists, rate-limiting, priorities etc.) ensures controllability.
• Robustness is a measure of how resilient the control system is. The managed network should degrade gracefully during unexpected situations (failures, DoS, Slashdot etc.).

sFlow was designed to provide the network-wide visibility needed for effective traffic control. sFlow has the attributes, described in Control Systems Design, that the measurement component of a control system requires: reliability, accuracy, responsiveness, noise immunity, linearity and non-intrusiveness.

In describing the responsiveness requirement, the author states, "Slow responding measurements can not only affect the quality of control but can actually make the feedback loop unstable." sFlow's timely reporting of link utilization data and packet samples provides the responsive visibility into network traffic needed to make the information actionable. While flow-based measurements provide useful usage data for traffic accounting and reporting, they are by their nature less responsive than sFlow and less useful for control.

sFlow and Netflow

If you are interested in network-wide visibility it is easy to be confused by the different types of traffic monitoring available. Technologies such as sFlow, Cisco NetFlow®, Juniper J-Flow, NetStream and IPFIX all appear to perform similar functions, but are supported by different network equipment vendors.

The situation is simpler than it appears, in reality there are only two basic types of traffic monitoring available:

1. Layer-2 (L2) packet, sFlow is designed to provide network-wide visibility. Monitoring all the way to the layer-2 access ports requires a protocol that scales well and can easily be implemented on a layer-2 switch. Because sFlow is packet-based, it is able to report in detail on all types of traffic on the network.
   
2. Layer-3 (L3) flow, NetFlow, J-Flow, NetStream and IPFIX are all very similar technologies. Flow monitoring is typically implemented on routers and provide information about TCP/IP connections with limited visibility into other types of traffic.
   

To be scalable and cost effective, traffic monitoring needs to be built into switches and routers. Start by taking an inventory of the devices in your network and see what traffic monitoring they provide, you will probably find that your network is represented by one of the three diagrams above.

The diagrams show three typical scenarios based on the type of equipment in the network:

1. All the switches and routers support sFlow and a central sFlow analyzer provides network-wide visibility. Most vendors support sFlow, so it is possible to build an sFlow capable network to meet almost any requirement.
   
2. The switches support sFlow and the routers support L3 flow monitoring. In this case a traffic analyzer that supports sFlow and L3 flow monitoring will also be able to provide network-wide visibility. This situation typically occurs in multi-vendor environments where sFlow is supported by the switch vendor and flow monitoring is supported by the router vendor.
3. The routers support L3 flow monitoring and the switches have no built-in traffic monitoring capability. In this case, only traffic through the routers is monitored providing very limited visibility into the data center and campus. This situation is typical of single vendor networks where the vendor exclusively supports L3 flow monitoring.

The first step to improved network visibility is to select a traffic analysis tool and enable whatever traffic monitoring is available from existing network equipment.

Making traffic monitoring a selection requirement for future network upgrades will allow you to increase network visibility over time. Selecting network equipment from one of the many vendors that support sFlow does not add to the network cost. Adding traffic monitoring later is likely to be prohibitively expensive.

Scalability and accuracy of packet sampling

This chart from Packet Sampling Basics is useful for explaining why sFlow's packet sampling mechanism provides the accuracy and scalability needed for network-wide visibility. The chart shows that the accuracy of a traffic measurement (e.g. How much bandwidth is being consumed by backup traffic?) increases rapidly as the number of samples contributing to the measurement increases.

The chart shows that the percentage accuracy is independent of the number of packets on the network. This independence is the key to sFlow's scalability.  For example, a measurement will have a 5% accuracy as long as it is based on at least 1,500 samples.  Only 1,500 samples are required whether the network contains one switch or 1,000 switches,  10Mbps links or 100Gbps links.

The accuracy of sampled data is also independent of the type of traffic: traffic can consist of a small number of large connections, many small connections, traffic can arrive in bursts or spread out over time. In all cases the accuracy is determined only by the number of samples.

The packet sampling mechanism in sFlow is implemented in hardware, providing wire-speed performance.  When a switch samples a packet, the sampled packet header and packet path information is immediately sent to the central traffic analyzer. Promptly sending the sFlow data reduces the amount of memory on the switch and provides the sFlow collector with a real-time view of network activity.

Using sFlow to monitor all the switches in the network provides a robust and accurate means of monitoring traffic suitable for exacting applications such as network billing and charge-back.  The redundancy that end-to-end monitoring provides ensures that very little data is lost, even when switches fail or are taken down for maintenance.