Linux 3.3 released

Today, Linus Torvalds announced the official release of Linux 3.3. Included in this release is kernel support for the Open vSwitch.

The inclusion of Open vSwitch support in the mainline Linux kernel integrates the advanced network visibility and control capabilities (through support of sFlow and OpenFlow) needed for virtualizing networking in cloud environments. Open vSwitch is already the default switch in the Xen Cloud Platform (XCP) and Citrix XenServer 6 and inclusion within the Linux kernel will help to further unify networking across open source virtualization systems, including: Xen, KVM, Proxmox VE and VirtualBox. In addition, integrated sFlow and OpenFlow support has also been demonstrated for the upcoming Windows 8 version of Microsoft's Hyper-V virtualization platform and sFlow and OpenFlow are also widely supported by network equipment vendors.

Broad support for open standards like sFlow and OpenFlow is critical, integrating the visibility and control capabilities within physical and virtual network elements that allows orchestration systems such as OpenStack, openQRM, and OpenNebula to automate and optimize management of network and server resources in cloud data centers.

System boundary

Credit: Wikimedia

A critical step in developing a control strategy for a system is deciding where to draw the system boundary. For example, in the data center, network management typically draws the boundary to include all the networking devices and their interconnections. The boundary is drawn to include the access switch ports, but exclude the servers and network adapters. System management is the mirror image, drawing the boundary to include servers and network adapters and exclude the network.

Given the system boundary drawn around the network, network planning and design treats the demand on the network as immutable and focusses on creating a network topology that will handle projected demand. Similarly, system capacity management is concerned with providing enough computational resources to handle application workloads while largely ignoring the network - treating it as a uniform "cloud" that will support the whatever demand is placed on it.

The move to scale-out applications, virtualization and convergence aims to improve efficiency by creating a flexible, scalable computing infrastructure that can adapt to changing demand. However, these architectural changes challenge the traditional separation between network and system management. For example, the following diagram shows how moving a virtual machine can dramatically alter network traffic.

Figure 1 Moving a virtual machine can increase network traffic

Changes in network traffic patterns can also have a profound effect on application performance. With convergence, networked storage increases demand for bandwidth and also makes applications sensitive to network congestions.

Redrawing the system boundary to include the network, servers, storage and applications provides the increased span of control needed to deliver efficient cloud services. For example, instead of treating the demand on the network as immutable, it becomes possible to move virtual machines in order to reshape network traffic patterns.

Figure 2 Moving a virtual machine to reduce network traffic

Generally, moving computation closer to data improves application performance by reducing latency and increasing bandwidth as well as reducing the overall load on the network. For example, the Hadoop distributed computing software is rack aware, since "network traffic between different nodes with in the same rack is much more desirable than network traffic across the racks."

The sFlow standard is well suited to automation, providing a unified measurement system that includes network, system and application performance. The Data center convergence, visibility and control presentation describes the critical role that measurement plays in managing costs and optimizing performance.

Windows Server 8 beta

The public beta of Windows Server 8 is now available for download. Windows Server 8 contains significant enhancements to support virtualization and cloud computing. This article describes how sFlow standard monitoring can be installed and configured to manage the performance of large scale Windows Server 8 cloud deployments.

With sFlow monitoring, each server sends performance measurements to a central analysis application - in this example the sFlow analyzer has address 10.0.0.50 and each Windows 8 server will be configured to send sFlow to this address.

First download the hsflowd-win8-X.XX.X-x64.msi file from sflow.net. Double click on the file to launch the installer:

Click on the Next button.

Click the check box to accept the license and then click on the Next button.

Click the Next button to accept the default installation location.

Enter the IP address of the sFlow collector, a Counter polling interval and a Sampling rate.  In this case the sFlow collector has address 10.0.0.50. The default polling interval of 20 seconds and default sampling rate of 1 in 256 should be suitable for most installations, see Sampling rates for more information on choosing sampling and polling setting.

Click on the Next button to accept the settings.

Click on the Install button to start the installation. After a few moments you will be prompted to install the network monitoring driver, see Hyper-V extensible virtual switch.

Click to check the Always trust software from "InMon Corp." box and click on the Install button to install the driver. Once the installation is complete, you should see the following window.

Click on the Finish button to complete the installation.

There is one final step to configure monitoring of the virtual switch. On the Start screen, type hyper-v manager to launch the Hyper-V Manager application.

Click on the Virtual Switch Manager... in the Actions column on the right to launch the Virtual Switch Manager.

Expand each virtual switch, click on Extensions and enable the sFlow packet sampling filter (in this case for vSwitch1). At this point the sFlow software installation and configuration steps are complete and you should start seeing data in your sFlow analyzer.

Use the Registry Editor if you need to change the sFlow configuration settings: the IP address of the sFlow analyzer, the polling interval, or the packet sampling rate. On the Start screen, type regedit to launch the Registry Editor.

Double-click on a setting to edit its value. Once the settings have been changed, the sFlow agent needs to be restarted before the changes will take effect.

On the Start screen type services to run the Services manager.

Select the Host sFlow Agent service and click on the Restart link (or use the right button menu option).

The following charts use the free sFlowTrend tool to demonstrate some of the performance metrics reported from Windows Server 8. However, there are many other open source and commercial sFlow analysis applications, see sFlow Collectors.

The following chart shows the top network connections flowing through the virtual switch.

It is easy to see that the bulk of the traffic is associated with Windows file sharing. The chart provides a minute-by-minute view of network activity based on measurements made by the driver installed in the virtual switch.

The next chart shows the CPU utilization on the server.

The processor utilization is averaging at just under 15%, with a brief peak of 40%. Based on this chart, it looks like there is plenty of capacity to run additional virtual machines.

The final chart in this example shows disk activity for the server and each virtual machine.

The physical servers and the virtual machines are all shown, along with key disk performance and usage metrics. Again, the numbers suggest that additional virtual machines could be added without impacting performance - the disk is only 26% used, there are only 3 virtual machines deployed, and there is very little disk IO activity.

The sFlow standard is widely supported by switch vendors. Using sFlow to monitor the Windows Server 8 extensible virtual switch simplifies management by allowing a common set of tools to monitor physical and virtual network performance. Performance management is further simplified since sFlow also provides an efficient way of monitoring server and virtual machine statistics. Unifying network and system monitoring provides the integrated view of performance needed for the coordinated management that is essential in virtualized environments. In addition to monitoring data center resources, sFlow can also be used to monitor performance within public clouds (see Rackspace cloudservers and Amazon EC2), providing the integrated visibility needed to optimize workload placement and minimise operating costs in hybrid cloud environments.

For public or private cloud operators, support for sFlow in the Hyper-V extensible switch and the Open vSwitch embeds visibility in leading commercial hypervisors (Microsoft Hyper-V and Citrix XenServer), open source hypervisors (Xen Cloud Platform (XCP), Xen, KVM, Proxmox VE and VirtualBox) and cloud management systems (OpenStack, OpenQRM and OpenNebula), providing the scalable visibility needed for accounting and billing, operational control and cost management.