Networked storage

A previous posting discussed how sFlow is used to provide visibility in the data center. This post looks more closely at the challenge posed by networked storage.

There are many good reasons to use networked storage: the storage resources can be shared, replicated and backed up independently of the systems that use them. In a virtual server environment, using networked storage for the virtual machine images simplifies the replication of virtual machines and the migration of virtual machines between servers (e.g. VMWare vMotion, Citrix XenMotion or Xen Live Migration).

In addition, the migrating of storage from a dedicated storage area network (SAN) to a single Converged Enhanced Ethernet (CEE) network promises to reduce cost and create a more flexible data center infrastructure. However, this migration also places additional demands on the LAN infrastructure.

Regardless of the type of networked storage (iSCSI, NFS, AoE or FCoE), the management of network bandwidth is critical to successful deployment and operation. For example, the chart above shows site-wide traffic from a large campus network broken out by protocol. The storage traffic (iSCSI) is clearly the largest load on the network, dwarfing the amount of web (HTTP) traffic.

The visibility into network traffic provided by sFlow is critical to effectively managing network resources. If the network is poorly provisioned, congestion associated with storage traffic will degrade quality of service (QoS) for other applications on the network and impair system performance since network congestion will also manifest itself as slow disk performance.

Network visibility in the data center

Current trends toward Virtualization, Converged Enhanced Ethernet (CEE), Fibre Channel over Ethernet (FCoE), Service Oriented Architectures (SOA) and Cloud Computing are part of a broader re-architecture of the data center in which enterprise applications are decomposed into simpler elements that can be deployed, moved, replicated and connected using high-speed switched Ethernet.

The following example, shown in the diagram, illustrates the management challenges faced in this new environment. A system manager decides to move a virtual machine from one server to another. The system management tools show that there is plenty of capacity on the destination server and this looks like a safe move. Unfortunately, the move causes the storage traffic, which had previously been confined to a single switch, to congest links across the data center causing system wide performance problems.

In this new environment the traditional siloed approach in which different teams manage the network, storage and servers does not work. An integrated approach to management is needed if the full benefits of a converged data center are to be achieved. Ensuring network-wide visibility into the storage, network and services running in the data center, their traffic volumes and dependencies is a critical component of an integrated management strategy.

In order to achieve data center wide visibility, every layer of the data center network, including the core, distribution, top of rack and blade server switches, needs to be instrumented. This might seem like a daunting (and expensive!) challenge. However, most vendors have integrated sFlow into their switch products. The sFlow standard provides a proven solution that is available in network products from the leading computer and network vendors, including HP, IBM, Dell, Brocade, BLADE, Juniper, Force10 and 3Com (for a complete list, see sFlow.org). Making sFlow a requirement when building out a new data center is a sound investment, adding very little to the cost of the network, but ensuring that the visibility needed safely deploy, optimize and scale up new services is available to the operations team.

Voice quality of service

Complaints were coming in about poor voice quality and dropped calls.

The challenge in managing VoIP (Voice over IP) deployments is that voice traffic can take a variety of paths across the network and problems anywhere along the path can affect call quality. In the diagram, the voice traffic is shown by the RTP (Real-time Transport Protocol) connections between the phones and computers.

Fortunately, the network switches support sFlow and the network management team had recently installed an sFlow analyzer (Traffic Sentinel), providing them with network-wide visibility. The sFlow analyzer alerted the team to excessive broadcast traffic affecting a large part of the network (see Link utilization to see how sFlow provides network-wide interface statistics). The packet header and packet path information provided by sFlow allowed the analyzer to rapidly locate the source of the broadcasts to a single server. A call to the server administrator identified the cause of the recent broadcast activity; they were testing a new application for distributing software on the network and the application was generating the broadcast traffic. Shutting down the application resolved the problem.

This example demonstrates how sFlow monitoring can assist in ensuring quality of service (QoS). Proactively managing traffic allows network managers to act before service levels deteriorate to the point where users are complaining. The paper, Managing Quality of Service Using sFlow, provides a detailed discussion of this approach.

Finally, the example illustrates the challenges created by network convergence and virtualization. Typically, different teams manage voice, data, computing and storage services using their own management tools. The example shows how the "siloed" approach to management fails as services converge to share the same network resources. As well as creating challenges, convergence offers the opportunity to dramatically simplify management. Instrumenting the converged network, by selecting network equipment with sFlow support, reduces the number of tools needed to provide visibility into each of these areas (voice, data, computing, storage) and delivers the shared visibility into the network infrastructure essential for avoiding conflicts.

Configuring Extreme switches

The following commands configure an Extreme Networks switch (10.0.0.246), sampling packets at 1-in-512, polling counters every 30 seconds and sending the sFlow to an analyzer (10.0.0.50) over UDP using the default sFlow port (6343):

enable sflow
configure sflow agent 10.0.0.246
configure sflow collector 10.0.0.50 port 6343
configure sflow sample-rate 512
configure sflow poll-interval 30
enable sflow backoff-threshold
configure sflow backoff-threshold 100
enable sflow ports all

A previous posting discussed the selection of sampling rates. Additional information can be found on the Extreme Networks web site.

See Trying out sFlow for suggestions on getting started with sFlow monitoring and reporting.

Note: Extreme Networks switches support automatic backoff of sampling rates based on a settable samples-per-second threshold. This mechanism ensures that a poorly selected sampling rate will not generate excessive numbers of samples (see sFlow Version 5, section 4.2.2).

Configuring Force10 switches

The following commands configure a Force10 switch (10.0.0.245), sampling packets at 1-in-512, polling counters every 30 seconds and sending the sFlow to an analyzer (10.0.0.50) over UDP using the default sFlow port (6343):

config> sflow collector 10.0.0.50 agent-addr 10.0.0.245
config> sflow sample-rate 512
config> sflow polling 30
config> sflow enable

Then for each interface:

interface> sflow enable

You can also use the following command to list the configuration settings:

show sflow
A previous posting discussed the selection of sampling rates. Additional information can be found on the Force10 Networks web site.
See Trying out sFlow for suggestions on getting started with sFlow monitoring and reporting.

Configuring Brocade switches

The following configuration enables sFlow monitoring of all interfaces on a Brocade (formerly Foundry) FGS switch, sampling packets at 1-in-512, polling counters every 30 seconds and sending the sFlow to an analyzer (10.0.0.50) on UDP port 6343 (the default sFlow port):

fgs(config)# int e 0/1/1 to 0/1/24
fgs(config-mif-0/1/1-0/1/24)# sflow forwarding
fgs(config-mif-0/1/1-0/1/24)# exit
fgs(config)# sflow destination 10.0.0.50 6343
fgs(config)# sflow sample 512
fgs(config)# sflow polling-interval 30
fgs(config)# sflow enable

You can also use the following command to list the configuration settings:

fgs# show sflow

A previous posting discussed the selection of sampling rates. Additional information can be found on the Brocade web site.

See Trying out sFlow for suggestions on getting started with sFlow monitoring and reporting.

Configuring Juniper switches

The following configuration enables sFlow monitoring of all interfaces on a Juniper EX3200 switch, sampling packets at 1-in-500, polling counters every 30 seconds and sending the sFlow to an analyzer (10.0.0.50) on UDP port 6343 (the default sFlow port).

protocols {
 sflow {
  polling-interval 30;
  sample-rate 500;
  collector 10.0.0.50 {
   udp-port 6343;
  }
  interfaces ge-0/0/0.0;
  interfaces ge-0/0/1.0;
  interfaces ge-0/0/2.0;
  interfaces ge-0/0/3.0;
  interfaces ge-0/0/4.0;
  interfaces ge-0/0/5.0;
  interfaces ge-0/0/6.0;
  interfaces ge-0/0/7.0;
  interfaces ge-0/0/8.0;
  interfaces ge-0/0/9.0;
  interfaces ge-0/0/10.0;
  interfaces ge-0/0/11.0;
  interfaces ge-0/0/12.0;
  interfaces ge-0/0/13.0;
  interfaces ge-0/0/14.0;
  interfaces ge-0/0/15.0;
  interfaces ge-0/0/16.0;
  interfaces ge-0/0/17.0;
  interfaces ge-0/0/18.0;
  interfaces ge-0/0/19.0;
  interfaces ge-0/0/20.0;
  interfaces ge-0/0/21.0;
  interfaces ge-0/0/22.0;
  interfaces ge-0/0/23.0;
 }
}

A previous posting discussed the selection of sampling rates. Additional information on configuring Juniper switches can be found on the Juniper Networks web site.

See Trying out sFlow for suggestions on getting started with sFlow monitoring and reporting.