Scalability

The problem with networks, and many other types of system, is that as they get larger they also become more complex to manage. What makes the management challenge even harder is that complexity doesn't simply grow in proportion to network size, but tends to grow exponentially.

The reason for the exponential growth in complexity is that the various components of the system interact. The diagram above illustrates this effect with a very simple example. Imagine a network consisting of just two connected machines, A and B. The two possible interactions are A talks to B, or B talks to A.

Now consider the effect of adding two more machines, C and D. The additional interactions C talks to D and D talks to C doubles the complexity, however, A talks to C, C talk to A, B talk to D, D talks to B, A talks to D, D talks to A, B talks to C and C talk to B brings the total number of direct interactions to 12.

In this example, doubling the network size increased the number of possible interactions and the complexity by a factor of 6. For many systems this is an underestimate of the increase in complexity, we didn't take into account indirect interactions such as A talks to D via B etc.

To manage risk, many companies use small scale trials as a way to pilot new systems. However, while pilot implementations can be a useful way to test basic functionality, they do not guarantee that the solution will work when deployed at full scale. Many costly information system failures result because the challenge of managing large scale system complexity was not properly addressed (see Understanding Information System Failures from the Complexity Perspective).

Network-wide visibility provides a powerful means of reducing complexity. While network complexity results from the large number of possible interactions, only a tiny fraction of the possible interactions actually occur at an given moment. Traffic visibility reduces complexity by revealing the active paths so that resources can be applied where they are needed.

In order to be effective, the measurement system itself must be scalable, delivering the complete, timely, actionable information needed to manage complexity. The sFlow standard was designed specifically for scalable, network-wide visibility and control and enjoys broad multi-vendor support. Products incorporating the sFlow standard deliver visibility throughout the physical switch, virtual switch, virtual router and cloud layers, ensuring effective management of complexity in large, dynamic, virtualized environments.