How do you identify the network traffic associated with a physical or virtual server? The need to answer this type of question becomes critical as data center convergence demands close coordination between the network, storage and system teams (see Management silos). For example, suppose a system manager is considering migrating a virtual machine and wants to make sure that the move won't be disruptive to the network (see Network visibility in the data center). Providing an answer to this question can be extremely difficult without the right tools.
The diagram shows the complex relationship between virtual machines, virtual network adapters, virtual switches, physical adapters and the data center network of physical switches. A single virtual machine may have more than one virtual network adapter, each of which is connected to a different virtual switch. Each virtual switch may be connected by one or more physical network adapters to more than one physical switch. Each physical and virtual machine may have an IP address for each of its network adapters. In addition, there may be large numbers of additional IP addresses allocated to physical or virtual machines in order to support applications (for example a web server may have an IP address associated with each web site that it hosts). Finally, the task is made even more complicated by virtual machine mobility since traffic data from multiple locations needs to be combined to report on the virtual machine.
The first step in meeting this challenge must be addressed at the time the data center network is designed. Data center networks are currently being upgraded in order to provide the flat, high-speed, layer 2 fabric needed to facilitate virtual machine mobility. As part of the network refresh, selecting switches with embedded traffic monitoring adds little to the cost of the network, but provides the instrumentation needed for network-wide visibility.
The sFlow standard is specifically designed for embedded monitoring of layer 2 devices. Selecting switches with embedded sFlow provides a low cost, scalable means of obtaining layer 2-7 visibility into all traffic flowing over the switched network (including storage traffic). Building a network visibility strategy around the sFlow standard maximizes the choice of vendors and ensures interoperable monitoring in mixed vendor environments, eliminating vendor lock-in and facilitating "best in class" product selection.
However, even with complete visibility into the network traffic, it is still challenging to identify which traffic belongs to a particular virtual machine. Looking at network traffic you can see MAC addresses and IP addresses, but you don't know which addresses belong to which virtual machine. Complicating matters further, the set of addresses associated with a particular virtual machine may change over time. What is needed is a single identifier that is uniquely associated with the virtual machine.
System administrators face this challenge every day. By associating a UUID (Universally Unique Identifier) with each virtual and physical machine they are able to uniquely identify that machine, even if its network configuration is changed.
Previous articles have looked at the visibility into system performance that Host sFlow provides (see Cluster performance and Top servers). In addition, to the performance counters, Host sFlow exports the UUID corresponding to each physical and virtual machine, along with the MAC addresses associated with each of its physical or virtual network adapters. An sFlow analyzer (see Choosing an sFlow analyzer) can combine sFlow from hosts and switches to provide a complete picture network activity.
From the network managers point of view, the UUID becomes just another attribute of network traffic that can be filtered and reported on. With a common language it is easy for network and system managers to coordinate their activities. This time around, the system manger can ask the question, "What is the network impact of moving virtual machine, UUID:a45a256c-c24b-2976-a931-8f2b90b1e026?"
Given the UUID, the sFlow analyzer can trend traffic for the virtual machine, irrespective of its location and even during migrations. The sFlow monitoring system is always aware of the current location of the virtual machine and the MAC and IP addresses associated with it. The trend chart above shows a real-time picture of traffic associated with the virtual machine. It's easy to see that the bulk of the traffic is NFS storage traffic associated with the storage server (10.0.0.40).
To assess the impact of moving the virtual machine, it is useful to visualize the traffic clustered by physical switch. The chart above shows traffic flows associated with the virtual machine clustered by switch. The virtual machine and NFS server are clearly attached to different switches. Ideally, you would like to minimize the number of network hops that the storage traffic takes (in order to reduce load on the network core and reduce disk IO latency for the virtual machine). Suggesting that the virtual machine by migrated to the same switch as the NFS server, or a location nearby, would accomplish this goal.
The close coordination of network, system and storage management captured in this example is critical for successful management of converged data centers (see Management silos). This level of coordination is only possible because the data center was designed with visibility in mind. The switches, virtual switches and servers have all been instrumented with sFlow in order to provide an integrated, data center wide, view of performance.
