Showing posts with label sFlowTrend. Show all posts
Showing posts with label sFlowTrend. Show all posts

Thursday, February 22, 2024

VyOS 1.4 LTS released

Protectli Vault - 4 Port

The VyOS 1.4.0 (Sagitta) LTS release announcement is exciting news! VyOS is an open source router operating system based on Linux that can be installed on commodity PC hardware - for optimal performance at least 1GB RAM and 4GB of storage space is recommended.

The new 1.4 LTS release includes a significantly enhanced implementation of industry standard sFlow telemetry based on the open source Host sFlow agent.

set system sflow interface eth0
set system sflow interface eth1
set system sflow interface eth2
set system sflow interface eth3
set system sflow polling 30
set system sflow sampling-rate 1000
set system sflow drop-monitor-limit 50
set system sflow server 192.0.2.100
Enter the commands above to enable sFlow monitoring on interfaces eth0, eth1, eth2, and eth3. Interface counters will be exported every 30 seconds, packets will be sampled with probability 1/1000, and up to 50 packet headers (and drop reasons) per second will collected from packets dropped by the router. The sFlow telemetry stream will be sent to an sFlow collector at 192.0.2.100.

Running Docker on the sFlow collector makes it easy to run a variety of sFlow analytics tools.

docker run --rm -p 6343:6343/udp sflow/sflowtool
Run the sflow/sflowtool image to decode and print the contents of the sFlow telemetry stream and verify receipt of data.
docker run --rm -p 6343:6343/udp sflow/tcpdump tcp port 80
Run the sflow/tcpdump image to decode and filter sampled packet headers. For more complex packet analysis tasks, try the sflow/tshark image.
Run the sflow/sflowtrend image to trend interface counters and top flows.
Deploy real-time network dashboards using Docker compose describes how to configure Prometheus and Grafana to capture time series data and create custom dashboards.
Dropped packet reason codes in VyOS describes how the new Linux kernel in VyOS 1.4 provides detailed visibility into every dropped packet (including the reason it was dropped). This cabability is used by the new sFlow agent implement the sFlow Dropped Packet Notification Structures extension to provide network-wide visibility into dropped packets.

Download VyOS today to try out the new features. Pre-built LTS images are available with paid support, but anyone can build an image from sources or download the latest rolling release.

Thursday, March 19, 2020

SFMIX San Francisco shelter in place

A shelter in place order restricted San Francisco residents to their homes beginning at 12:01 a.m. on March 17, 2020. Many residents work for Bay Area technology companies such as Salesforce, Facebook, Twitter, Google, Netflix and Apple. Employees from these companies are able to, and have been instructed to, work remotely from their homes. In addition, other housebound residents are making use of social networking to keep in touch with friends and family as well as streaming media and online gaming for entertainment.

The traffic trend chart above from the San Francisco Metropolitan Internet Exchange (SFMIX) shows the change in network traffic that has resulted from the shelter in place order. Peak traffic has increased by around 10Gbit/s (a 25% increase) and continues throughout the day (whereas peaks previously occurred in the evenings).

The SFMIX network directly connects a number of data centers in the Bay Area and the member organizations that peer from those data centers.  Peering through the exchange network keeps traffic local by directly connecting companies with their employees and customers and avoiding potentially congested service provider networks.
SFMIX recently finished a network upgrade to 100Gbit/s Arista switches and all fiber optic connections and so is easily able to handle the increase in traffic.

Network visibility is critical to being able to quickly respond to unexpected changes in network usage. The sFlow measurement technology built into high speed switches is a standard method of monitoring Internet Exchanges (IXs) - Internet Exchange (IX) Metrics is a measurement tool developed with SFMIX.

Every organization depends on their networks and visibility is critical to manage the challenges posed by a rapidly changing environment. sFlow is an industry standard that is widely implemented by network vendors.  Enable sFlow telemetry from existing network equipment and deploy an sFlow analytics tool to gain visibility into your network traffic. sFlowTrend is a free tool that can be installed and running in minutes. Flow metrics with Prometheus and Grafana describes how to integrate network traffic visibility into existing operational dashboards.

Thursday, August 30, 2018

Northbound Networks Zodiac GX

Mininet is widely used to emulate software defined networks (SDNs). Mininet flow analytics describes how standard sFlow telemetry, from Open vSwitch used by Mininet emulate the network, provides feedback to an SDN controller, allowing the controller to adapt the network to changing traffic, for example, to mitigate a distributed denial of service (DDoS) attack.

Northbound Networks Zodiac GX is an inexpensive open source software based switch that is ideal for experimenting with software defined networking (SDN) in a physical network setting. The small fanless package makes the switch an attractive option for desktop use. The Zodiac GX is also based on Open vSwitch, making it easy to take SDN control strategies developed on Mininet.
Enabling sFlow on the Zodiac GX is easy, navigate to the System>Startup page and add the following line to the end of the startup script (before the exit 0 line):
ovs-vsctl -- --id=@sflow create sflow agent=$OVS_BR target=$IP_CONTROLLER_1 sampling=100 polling=10 -- set bridge $OVS_BR sflow=@sflow
Reboot the switch for the changed to take effect.

Use sflowtool to verify that sFlow is arriving at the controller host and to examine the contents of the telemetry stream. Running sflowtool using Docker is a simple alternative to building the software from sources:
docker run --rm -p 8008:8008 -p 6343:6343/udp sflow/sflowtool
The text output from sflowtool can be piped into scripts to perform basic sFlow analysis.
A graphical sFlow analyzer performs the analysis tasks for you. The screen shot above shows sFlowTrend, a free sFlow analyzer that displays traffic trends. The software can be downloaded and installed or run using Docker:
docker run --rm -p 6343:6343/udp -p 8087:8087 -p 8443:8443 sflow/sflowtrend
The sFlowTrend charts update every minute. This is generally fast enough for human consumption, but real-time, up to the second, visibility is critical for SDN use cases.
The screen shot from Flow Trend shows an up to the second view of traffic. The spike in traffic is due to a 4K video being streamed from YouTube. The following command runs the software:
docker run --rm -p 6343:6343/udp -p 8008:8008 sflow/flow-trend
Flow Trend is an application running on the sFlow-RT real-time analytics platform.
Applications running on the sFlow-RT platform deliver real-time visibility to SDN, DevOps and Orchestration stacks, enabling new classes of performance aware application such as load balancing, DDoS mitigation, and workload placement.

RYU provides a framework that can be used to develop SDN applications in Python. For example, the following command runs the simple learning bridge application that ships with RYU:
docker run -it --rm -p 6633:6633 osrg/ryu ryu-manager --verbose ryu/ryu/app/simple_switch_13.py
As soon as the switch connects to the controller, you should see a flurry of events as the controller programs flows on the Zodiac GX switch.

Faucet is an SDN controller for production networks implemented using RYU. Before we can use Faucet, we need to gather basic OpenFlow information from the switch.
$ ssh -t admin@10.0.0.230 "sudo ovs-ofctl show ovslan"
admin@10.0.0.230's password: 
Password: 
OFPT_FEATURES_REPLY (xid=0x2): dpid:000044d1fa6291b2
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst
 1(eth0.1): addr:44:d1:fa:62:91:b2
     config:     0
     state:      STP_FORWARD
     current:    1GB-FD AUTO_NEG
     speed: 1000 Mbps now, 0 Mbps max
 2(eth0.2): addr:44:d1:fa:62:91:b2
     config:     0
     state:      STP_FORWARD
     current:    1GB-FD AUTO_NEG
     speed: 1000 Mbps now, 0 Mbps max
 3(eth0.3): addr:44:d1:fa:62:91:b2
     config:     0
     state:      STP_FORWARD
     current:    1GB-FD AUTO_NEG
     speed: 1000 Mbps now, 0 Mbps max
 4(eth0.4): addr:44:d1:fa:62:91:b2
     config:     0
     state:      STP_FORWARD
     current:    1GB-FD AUTO_NEG
     speed: 1000 Mbps now, 0 Mbps max
 5(eth0.5): addr:44:d1:fa:62:91:b2
     config:     0
     state:      STP_FORWARD
     current:    1GB-FD AUTO_NEG
     speed: 1000 Mbps now, 0 Mbps max
 LOCAL(ovslan): addr:44:d1:fa:62:91:b2
     config:     0
     state:      0
     current:    1GB-FD AUTO_NEG
     speed: 1000 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
Connection to 10.0.0.230 closed.
Update August 30, 2018: The only piece of information needed to construct the faucet config file below is the switch dpid. The Zodiac GX uses the MAC address of the switch as the dpid, so you can simply read the MAC address printed on a label on the bottom of the switch.
Now create a directory called faucet that contains the initial Faucet configuration file, faucet.yaml:
vlans:
    office:
        vid: 100
        description: "office network"

dps:
    zodiac:
        dp_id: 0x000044d1fa6291b2
        hardware: "ZodiacGX"
        interfaces:
            1:
                name: "eth0.1"
                description: "port1"
                native_vlan: office
            2:
                name: "eth0.2"
                description: "port2"
                native_vlan: office
            3:
                name: "eth0.3"
                description: "port3"
                native_vlan: office
            4:
                name: "eth0.4"
                description: "port4"
                native_vlan: office
            5:
                name: "eth0.5"
                description: "port5"
                native_vlan: office
            0xfffffffe:
                name: "ovslan"
                description: "local"
                native_vlan: office
Now run Faucet:
docker run -it --rm -v $PWD/faucet/:/etc/faucet/ -v $PWD/faucet/:/var/log/faucet/ -p 6633:6653 -p 9302:9302 faucet/faucet
As soon as the switch connects to the controller, you should see events logged to the faucet.log file in the same directory as the faucet.yaml configuration file.

Faucet Documentation describes how to extend the configuration to include firewall, routing, segmentation, and network function virtualization (NFV) rules to the configuration.

The next step is integrating sFlow analytics with the controller. Writing Applications describes how write sFlow-RT applications use REST API and embedded JavaScript API. The document includes Python examples that could be embedded in RYU controller applications. Alternatively, sFlow-RT's embedded HTTP client can be used to push control actions to an SDN controller, see ONOS measurement based control for an example.

The sFlow telemetry stream contains detailed Open vSwitch performance metrics in addition to flow and interface counter data. The sFlow-RT analytics pipeline can be programmed to generate and push statistics to time series databases and dashboards, see Prometheus and Grafana and InfluxDB and Grafana.

Exporting events using syslog describes how sFlow-RT can be programmed to detect and report on traffic anomalies, sending events to Security Information and Event Management (SIEM) tools, using Splunk and Logstash as examples.

The sflow/sflow-rt Docker image provides a convenient means of developing and deploying sFlow-RT applications alongside the SDN controllers demonstrated in this article.

An important benefit of sFlow telemetry is that it decouples monitoring from the control plane. You are free to change SDN controllers, use distributed routing / switching protocols, move between network operating systems, or build your own control plane while maintaining the same level of visibility. Industry standard sFlow is widely supported by vendors, including: A10, Aerohive, ALUe, Allied Telesis, Arista, Aruba, Big Switch, Broadcom, Cisco, Cumulus, Dell, D-Link, Edge-Core, Extreme, F5, Fortinet, Huawei, IP Infusion, Juniper, Mellanox, Netgear, OpenSwitch, Pica8, Proxim, Quanta, SMC, ZTE, and ZyXEL.

Thursday, March 22, 2018

OCP Summit 2018

Network telemetry was a popular topic at the recent OCP U.S. Summit 2018 in San Jose, California, with an entire afternoon track of the two day conference devoted to the subject. Videos of the talks should soon be posted on the conference web site.

The following articles on this blog cover related topics:
In addition, there were a couple of live sFlow telemetry demonstration in the conference exhibit hall.
The first was a demonstration of leaf and spine fabric visibility using white box switches running the open source Linux Foundation OpenSwitch network operating system. OpenSwitch describes how the open source Host sFlow agent enables standard sFlow instrumentation in merchant silicon based white box switches using OpenSwitch Control Plane Services (CPS), which in turn programs the silicon using the OCP Switch Abstraction Interface (SAI).

The rack in the booth contains a two spine, five leaf network. Each of the switches in the network is streaming real-time sFlow telemetry to an instance of Fabric View which is displaying real-time (up to the second) flow analytics on the monitor to the right of the picture.
The second demonstration shows a Marvell top of rack (ToR) switch with an ARM based management CPU running Linux, DPDK and SONiC. In this example, the switch is streaming sFlow telemetry to a free version of sFlowTrend which is displaying top flows in the trend chart.

Monday, March 20, 2017

Nutanix

Maximum Performance from Acropolis Hypervisor and Open vSwitch describes the network architecture within a Nutanix converged infrastructure appliance - see diagram above. This article will explore how the Host sFlow agent can be deployed to enable sFlow instrumentation in the Open vSwitch (OVS)  and deliver streaming network and system telemetry from nodes in a Nutanix cluster.
This article is based on a single hardware node running Nutanix Community Edition (CE), built following the instruction in Part I: How to setup a three-node NUC Nutanix CE cluster. If you don't have hardware readily available, the article, 6 Nested Virtualization Resources To Get You Started With Community Edition, describes how to run Nutanix CE as a virtual machine.
The sFlow standard is widely supported by network equipment vendors, which combined with sFlow from each Nutanix appliance, delivers end to end visibility in the Nutanix cluster. The following screen captures from the free sFlowTrend tool are representative examples of the data available from the Nutanix appliance.
The Network > Top N chart displays the top flows traversing OVS. In this case an HTTP connection is responsible for most of the traffic. Inter-VM and external traffic flows traverse OVS and are efficiently monitored by the embedded sFlow instrumentation.
The Hosts > CPU utilization chart shows an increase in CPU utilization due to the increased traffic.
The Hosts > Disk IO shows the Write operations associated with connection.

Installing Host sFlow agent on Nutanix appliance

The following steps install Host sFlow on a Nutanix device:

First log into the Nutanix host as root.
Update June 19, 2019: When you log in as root you will see a warning that installing software on the hypervisor is not supported. See comment below.
Next, find the latest version of the Centos 7 RPM on sFlow.net and use the following commands to download and install the software:
wget https://github.com/sflow/host-sflow/releases/download/v2.0.8-1/hsflowd-centos7-2.0.8-1.x86_64.rpm
rpm -ivh hsflowd-centos7-2.0.8-1.x86_64.rpm
rm hsflowd-centos7-2.0.8-1.x86_64.rpm
Edit the /etc/hsflowd.conf file to direct sFlow telemetry to collector 10.0.0.50, enable KVM monitoring (virtual machine stats), and push sFlow configuration to OVS (network stats):
sflow {
  ...
  # collectors:
  collector { ip=10.0.0.50 udpport=6343 }
  ...
  # Open vSwitch sFlow configuration:
  ovs { }
  # KVM (libvirt) hypervisor and VM monitoring:
  kvm { }
  ...
}
Now start the Host sFlow daemon:
systemctl enable hsflowd.service
systemctl start hsflowd.service
Data will immediately start to appear in sFlowTrend.

Thursday, April 28, 2016

Cisco SF250, SG250, SF350, SG350, SG350XG, and SG550XG series switches

Software version 2.1.0 adds sFlow support to Cisco 250 Series Smart Switches, 350 Series Smart Switches and 550X Series Stackable Managed Switches.
Cisco network engineers might not be familiar with the multi-vendor sFlow technology since it is a relatively new addition to Cisco products. The article, Cisco adds sFlow support, describes some of the key features of sFlow and contrasts them to Cisco NetFlow.
Configuring sFlow on the switches is straightforward. For example, The following commands configure a switch to sample packets at 1-in-1024, poll counters every 30 seconds and send sFlow to an analyzer (10.0.0.50) over UDP using the default sFlow port (6343):
sflow receiver 1 10.0.0.50
For each interface:
sflow flow-sampling 1024 1
sflow counter-sampling 30 1
A previous posting discussed the selection of sampling rates. Additional information can be found on the Cisco web site.

Trying out sFlow offers suggestions for getting started with sFlow monitoring and reporting. The article recommends the sFlowTrend analyzer as a way to get started since it is a free, purpose built sFlow analyzer that delivers the full capabilities of sFlow instrumentation in the Cisco switches.

Tuesday, October 9, 2012

sFlowTrend adds web server monitoring

This chart was generated using the free sFlowTrend application to monitor an Apache web server using the sFlow standard. The chart shows a real-time, minute by minute view of Top URIs by Operations/s for a busy web server. What is interesting about the chart is the sudden drop off in total operations per second over the last few minutes.
The drop in throughput can be verified by examining the standard HTTP performance counters that are exported using sFlow's efficient push mechanism. The Counters chart above shows the same drop in throughput.

There are a couple of possible explanations that come to mind, the first is that the size of pages has increased, possibly because large images were added.
The Top URI extensions by Bytes/s chart shown above makes it clear that the proportion of image data hasn't changed and that the overall data rate has fallen, so the drop in throughput doesn't appear to be a bandwidth problem.
Another possibility is that there has been an increase in server latency. The Top URIs by Duration chart above shows a recent increase in the latency of the http://10.0.0.150/login.php page.

At this point the problem can probably be resolved by talking with the application team to see if they have made any recent changes to the login page. However, there is additional information available that might help further diagnose the problem.
Host sFlow agents installed on the servers provides a scaleable way of monitoring performance. The CPU utilization chart above shows a drop in CPU load on the web server that coincides with the reduced web throughput. It appears that the performance problem isn't related to web server CPU, but is likely the result of requests to a slow backend system.

Note: If it had been a CPU related issue, we might have expected that the latence would have increased for all URIs, not just the login.php page.

Network visibility is a critical component of application performance monitoring. In this case, network traffic data can help by identifying the backend systems that the web server is depends on. Fortunately, most switch vendors support the sFlow standard and the traffic data is readily accessible in sFlowTrend.
The Top servers chart above shows the top services and servers by Frames/s. The drop in traffic to the web server, 10.0.0.150 is readily apparent, as is a drop in traffic to the Memcached server, 10.0.0.151 (TCP:11211). The Memcached server is used to cache the results of database queries in order to improve site performance and scaleability, but the performance problem doesn't seem to be directly related to the Memcached performance since the amount of Memcache traffic has dropped proportionally with the HTTP traffic (if there had been an increase in Memcache traffic, this might have indicated that the Memcached server was overloaded).
A final piece of information available through sFlow is the link utilization trend which confirms that there is the drop in performance isn't due to a lack of network capacity.

At this point we have a pretty thorough understanding of the impact of the problem on application, server and network resources. Talking to the developers reveals a recent update to the login.php script that introduced a software bug that failed to properly cache information. The resulting increase in load to the database was causing the login page to load slowly and resulted in the drop in site throughput. Fixing the bug returned site performance to normal levels.

Note: This example is a recreation of a typical performance problem using real servers and switches generating sFlow data. However, the load is artificially generated using Apache JMeter since actual production data can't be shown.

Trying out sFlow monitoring on your own site is easy. The sFlowTrend application is a free download. There are open source sFlow modules available for popular web servers, including: Apache, NGINX, Tomcat and node.js. The open source Host sFlow agent runs on most operating systems and enabling sFlow on switches is straightforward (see sFlow.org for a list of switches supporting the sFlow standard). The article, Choosing an sFlow analyzer, provides additional information for large scale deployments.

Thursday, March 1, 2012

Windows Server 8 beta


The public beta of Windows Server 8 is now available for download. Windows Server 8 contains significant enhancements to support virtualization and cloud computing. This article describes how sFlow standard monitoring can be installed and configured to manage the performance of large scale Windows Server 8 cloud deployments.

With sFlow monitoring, each server sends performance measurements to a central analysis application - in this example the sFlow analyzer has address 10.0.0.50 and each Windows 8 server will be configured to send sFlow to this address.

First download the hsflowd-win8-X.XX.X-x64.msi file from sflow.net. Double click on the file to launch the installer:


Click on the Next button.


Click the check box to accept the license and then click on the Next button.


Click the Next button to accept the default installation location.


Enter the IP address of the sFlow collector, a Counter polling interval and a Sampling rate.  In this case the sFlow collector has address 10.0.0.50. The default polling interval of 20 seconds and default sampling rate of 1 in 256 should be suitable for most installations, see Sampling rates for more information on choosing sampling and polling setting.

Click on the Next button to accept the settings.


Click on the Install button to start the installation. After a few moments you will be prompted to install the network monitoring driver, see Hyper-V extensible virtual switch.


Click to check the Always trust software from "InMon Corp." box and click on the Install button to install the driver. Once the installation is complete, you should see the following window.


Click on the Finish button to complete the installation.

There is one final step to configure monitoring of the virtual switch. On the Start screen, type hyper-v manager to launch the Hyper-V Manager application.


Click on the Virtual Switch Manager... in the Actions column on the right to launch the Virtual Switch Manager.


Expand each virtual switch, click on Extensions and enable the sFlow packet sampling filter (in this case for vSwitch1). At this point the sFlow software installation and configuration steps are complete and you should start seeing data in your sFlow analyzer.

Use the Registry Editor if you need to change the sFlow configuration settings: the IP address of the sFlow analyzer, the polling interval, or the packet sampling rate. On the Start screen, type regedit to launch the Registry Editor.


Double-click on a setting to edit its value. Once the settings have been changed, the sFlow agent needs to be restarted before the changes will take effect.

On the Start screen type services to run the Services manager.


Select the Host sFlow Agent service and click on the Restart link (or use the right button menu option).

The following charts use the free sFlowTrend tool to demonstrate some of the performance metrics reported from Windows Server 8. However, there are many other open source and commercial sFlow analysis applications, see sFlow Collectors.

The following chart shows the top network connections flowing through the virtual switch.


It is easy to see that the bulk of the traffic is associated with Windows file sharing. The chart provides a minute-by-minute view of network activity based on measurements made by the driver installed in the virtual switch.

The next chart shows the CPU utilization on the server.


The processor utilization is averaging at just under 15%, with a brief peak of 40%. Based on this chart, it looks like there is plenty of capacity to run additional virtual machines.

The final chart in this example shows disk activity for the server and each virtual machine.


The physical servers and the virtual machines are all shown, along with key disk performance and usage metrics. Again, the numbers suggest that additional virtual machines could be added without impacting performance - the disk is only 26% used, there are only 3 virtual machines deployed, and there is very little disk IO activity.

The sFlow standard is widely supported by switch vendors. Using sFlow to monitor the Windows Server 8 extensible virtual switch simplifies management by allowing a common set of tools to monitor physical and virtual network performance. Performance management is further simplified since sFlow also provides an efficient way of monitoring server and virtual machine statistics. Unifying network and system monitoring provides the integrated view of performance needed for the coordinated management that is essential in virtualized environments. In addition to monitoring data center resources, sFlow can also be used to monitor performance within public clouds (see Rackspace cloudservers and Amazon EC2), providing the integrated visibility needed to optimize workload placement and minimise operating costs in hybrid cloud environments.

For public or private cloud operators, support for sFlow in the Hyper-V extensible switch and the Open vSwitch embeds visibility in leading commercial hypervisors (Microsoft Hyper-V and Citrix XenServer), open source hypervisors (Xen Cloud Platform (XCP), Xen, KVM, Proxmox VE and VirtualBox) and cloud management systems (OpenStack, OpenQRM and OpenNebula), providing the scalable visibility needed for accounting and billing, operational control and cost management.

Friday, June 24, 2011

Five W's

The Five W's are the set of questions that a news report must answer in order to be considered complete:
  • What happened (what is the story)?
  • Who is it about?
  • When did it take place?
  • Where did it take place?
  • Why did it happen?
  • How did it happen?
These questions provide a good framework for solving performance management problems. The following example demonstrates how the network-wide visibility provided by the sFlow standard makes it easy to quickly answer each question in order to detect, diagnose and eliminate performance problems.

Note: The free sFlowTrend tool is used to demonstrate problem solving using sFlow, but there are many other tools to choose from.

What?

What happened? Threshold violations on interface counters provide the notification that there is a problem. sFlow provides an extremely efficient method of collecting interface counters, monitoring every interface in the network, allowing prompt detection of performance problems.


This screen capture of the sFlowTrend dashboard shows that a problem with excessive unicast packets has been detected. There are many devices and interfaces in this network, the next question is who reported the problem? Clicking on the bar provides the following answer.

Who?

Who is reporting the problem? The following table sorts the switches to show which ones are seeing excessive unicast traffic. Comparing switches provides a baseline making it easy to see whether the problem is widespread, or localized to specific devices. 


Note: Many monitoring systems are hierarchical, counters are polled locally and notifications of threshold violations are sent to the central management system. The problem with this approach is that the underlying data needed to put the event into context is lost. The sFlow architecture centralizes monitoring - performance counters from all devices are centrally collected and thresholds calculations are performed by the collector. sFlow makes it simple to drill down and compare the statistics underlying any notification, making it much easier to troubleshoot problems. 

Drilling down further, the following table shows individual interfaces sorted to show which interface is seeing excessive traffic. 


Now that we know where the problem is, the next question is when did it start?

When?

Again, because sFlow centralizes all the critical performance data, follow up is straightforward. Counter trends on any link can be displayed and the following chart was obtained by drilling down on the interface highlighted in the screen above.


This chart shows that a 2 minute spike in traffic occurred around 10 minutes ago. The chart shows that link utilization has returned to normal levels so there is no need for immediate action. However, it is worth identifying why the spike occurred and assess if it is likely to occur again.

Why? How?

Interface counters are only one type of data exported by sFlow. sFlow agents also export real-time traffic information. The two types of data complement one another, counters allow performance anomalies to be quickly detected and traffic information provides the detail needed to identify the root cause of the problem.

By flipping the chart from Utilization to a Top Connections, sFlowTrend uses the sFlow traffic measurements to break down the traffic into the top connections responsible for the traffic.


The chart shows why the traffic spiked. Multiple TCP connections to port 80 (web) from ganglia.sf.inmon.com were responsible for the spike in traffic. The top two connections are to fedora.mirrors.tds.net, providing a clue as to the type of traffic.

How did the spike happen? It seems likely that a system update was run on the ganglia.sf.inmon.com server. Given the timing of the traffic, this looks like an unscheduled update. It would be a good idea to talk to the system administrator for the server and suggest off-peak times to schedule the updates so that they don't interfere with peak business hours traffic.

Where?

What if the spike was ongoing and we couldn't contact the system administrator of the server to shut down the update? In this case it is very important to be able to locate the server on the network in order to take action.

When an sFlow agent reports on network traffic it also includes information on the packet path across the device. Combining data from all the switches allows an sFlow analyzer to discover and track network topology and the location of each host on the network.

Clicking on an address in sFlowTrend provides the current location.


In this case, the host ganglia.sf.inmon.com is located on port A13 on switch 10.0.0.244. Knowing where the server is located allows the network administrator to log into the switch and take corrective action, blocking or rate limiting the traffic.

The article, Network edge, described how the process of detecting traffic problems and applying controls to the switches can be fully automated. Automation is particularly important in large scale environments where manual intervention is labor intensive and slow.

Finally, the sFlow standard is widely supported by network equipment vendors, providing simple, scalable, end-to-end monitoring of wired and wireless networking as well as servers, virtual machines and applications running on the network. Comprehensive, integrated visibility is the key to simplifying management and controlling costs in networked environments.