Thursday, June 26, 2014

Docker performance monitoring

IT’S HERE: DOCKER 1.0 recently announced the first production release of the Docker Linux container platform. Docker is seeing explosive growth and has already been embraced by IBM, RedHat and RackSpace. Today the open source Host sFlow project released support for Docker, exporting standard sFlow performance metrics for Linux containers and unifying Linux containers with the broader sFlow ecosystem.
Visibility and the software defined data center
Host sFlow Docker support simplifies data center performance management by unifying monitoring of Linux containers with monitoring of virtual machines (Hyper-V, KVM/libvirt, Xen/XCP/XenServer), virtual switches (Open vSwitch, Hyper-V Virtual Switch, IBM Distributed Virtual Switch, HP FlexFabric Virtual Switch), servers (Linux, Windows, Solaris, AIX, FreeBSD), and physical networks (over 40 vendors, including: A10, Arista, Alcatel-Lucent, Arista, Brocade, Cisco, Cumulus, Extreme, F5, Hewlett-Packard, Hitachi, Huawei, IBM, Juniper, Mellanox, NEC, ZTE). In addition, standardizing metrics allows allows measurements to be shared among different tools, further reducing operational complexity.


The talk provides additional background on the sFlow standard and case studies. The remainder of this article describes how to use Host sFlow to monitor a Docker server pool.

First, download, compile and install the Host sFlow agent on a Docker host (Note: The agent needs to be built from sources since Docker support is currently in the development branch):
svn checkout http://svn.code.sf.net/p/host-sflow/code/trunk host-sflow-code
cd host-sflow-code
make DOCKER=yes
make install
make schedule
service hsflowd start
Next, if SE Linux is enabled, run the following commands to allow Host sFlow to retrieve network stats (or disable SE Linux):
audit2allow -a -M hsflowd
semodule -i hsflowd.pp
See Installing Host sFlow on a Linux server for additional information on configuring the agent.


The slide presentation describes how Docker can be used with Open vSwitch to create virtual networks connecting containers. In addition to providing advanced SDN capabilities, the Open vSwitch includes sFlow instrumentation, providing detailed visibility into network traffic between containers and to the outside network.

The Host sFlow agent makes it easy to enable sFlow on Open vSwitch. Simply enable the sflowovd daemon and Host sFlow configuration settings will be automatically applied to the Open vSwitch.
service sflowovsd start
There are a number of tools that consume and report on sFlow data and these should be able to report on Docker since the metrics being reported are the same standard set reported for virtual machines. Here are a few examples from this blog:
Looking at the big picture, the comprehensive visibility of sFlow combined with the agility of SDN and Docker lays the foundation for optimized workload placement, resource allocation, and scaling by the orchestration system, maximizing the utility of the physical network, storage and compute infrastructure.

3 comments:

  1. You'll probably need to install the package "libcap-dev" (or libcap-devel) before this will compile.

    ReplyDelete
  2. Please could you help please I managed to svn to the link and install all packages successfully and then cd to the right place, but all other commands don't work and I can't find anything to help. Thanks

    ReplyDelete
    Replies
    1. The Host sFlow project moved from SourceForge to GitHub so these instructions are no longer correct. Linux bridge, macvlan, ipvlan, adapters describes how to build from sources using the new repository. The INSTALL.Docker instructions, distributed with the sources, gives more details.

      Delete