Snowflakes, IPFIX, NetFlow and sFlow

Snow flakes by Wilson Bentley

Each snowflake is unique and beautiful. However, while such immense diversity is attractive in nature, variation in data center management standards results in operational complexity, making it difficult to implement the automation and control needed to effectively manage at scale.

The following table examines the approaches taken by the IPFIX and sFlow standards by contrasting how they handle four basic aspects of measurement.

Note: The IPFIX standard is based on Cisco's NetFlow™ version 9 protocol and most of the points of comparison apply equally to NetFlow.

	IPFIX	sFlow
Packet	IPFIX currently defines over 50 fields relating to packet header (see IP Flow Information Export (IPFIX) Entities): protocolIdentifier ipClassOfService tcpControlBits sourceTransportPort sourceIPv4Address destinationTransportPort destinationIPv4Address sourceIPv6Address destinationIPv6Address flowLabelIPv6 icmpTypeCodeIPv4 igmpType sourceMacAddress vlanId ipVersion ipv6ExtensionHeaders destinationMacAddress icmpTypeCodeIPv6 icmpTypeIPv4 icmpCodeIPv4 icmpTypeIPv6 icmpCodeIPv6 udpSourcePort udpDestinationPort tcpSourcePort tcpDestinationPort tcpSequenceNumber tcpAcknowledgementNumber tcpWindowSize tcpUrgentPointer tcpHeaderLength ipHeaderLength totalLengthIPv4 payloadLengthIPv6 ipTTL nextHeaderIPv6 ipDiffServCodePoint ipPrecedence fragmentFlags ipPayloadLength udpMessageLength isMulticast ipv4IHL ipv4Options tcpOptions ipTotalLength ethernetHeaderLength ethernetPayloadLength ethernetTotalLength dot1qVlanId dot1qPriority dot1qCustomerVlanId dot1qCustomerPriority ethernetType The IPFIX standard does not require vendors to support all the fields, each vendor is free to export any combination of fields that they choose, none of the fields are mandatory. The result is that each vendor and each product produces unique and incompatible data.	The sFlow standard specifies a single way to report packet attributes, the packet header, ensuring that every vendor and product produces compatible results. Every sFlow compatible device deployed since the sFlow standard was published in 2001 provides visibility into every protocol that has ever, or will ever, run over Ethernet. The packet header includes all the protocol fields exported by IPFIX as well as fields associated with emerging protocols such as FCoE, AoE, TRILL, NVGRE and VxLAN that have yet to by defined in IPFIX.
Time	IPFIX has over 30 elements that can be used to represent time (see IP Flow Information Export (IPFIX) Entities): flowEndSysUpTime flowStartSysUpTime flowStartSeconds flowEndSeconds flowStartMilliseconds flowEndMilliseconds flowStartMicroseconds flowEndMicroseconds flowStartNanoseconds flowEndNanoseconds flowStartDeltaMicroseconds flowEndDeltaMicroseconds flowDurationMilliseconds flowDurationMicroseconds observationTimeSeconds observationTimeMilliseconds observationTimeMicroseconds observationTimeNanoseconds monitoringIntervalStartMilliSeconds monitoringIntervalEndMilliSeconds collectionTimeMilliseconds maxExportSeconds maxFlowEndSeconds minExportSeconds minFlowStartSeconds maxFlowEndMicroseconds maxFlowEndMilliseconds maxFlowEndNanoseconds minFlowStartMicroseconds minFlowStartMilliseconds minFlowStartNanoseconds The IPFIX standard allows vendors to report time using these elements in any combination, or to omit timestamps altogether. In order to report time consistently, every agent must have a real-time clock and be time synchronized. Finally, it is left up the vendors to decide how often to export data and so an IPFIX collector must understand each vendor's implementation in order to be certain that it has received all the data and detect data loss.	The sFlow standard requires that data be sent immediately. The stateless nature of the protocol means that data can be combined and timestamps added by the central sFlow collector without any need for timestamps or time synchronization among the agents. Note: The sFlow datagrams do contain a time stamp, the agent uptime in milliseconds at the time the datagram was sent.
Sampling	IPFIX currently defines eight different algorithms for packet sampling (see IANA Packet Sampling Parameters): Systematic count-based Sampling Systematic time-based Sampling Random n-out-of-N Sampling Uniform probabilistic Sampling Property match Filtering Hash based Filtering using BOB Hash based Filtering using IPSX Hash based Filtering using CRC Vendors are not required to implement any of these algorithms and are free to invent their own sampling schemes (see NetFlow-lite). In addition, many of the standard algorithms can be shown to be inaccurate.	The sFlow standard mandates a single, statistically valid, sampling algorithm. All sFlow compliant vendors and products, implement the same algorithm and produce accurate, interoperable results.
URL	There is no-standard IPFIX element for exporting a URL. However, IPFIX does allow vendor extensions, resulting in multiple schemes for exporting URL data. Examples include: nProbe URLs are additional fields that can be included as flow keys when configuring the probe. Dell SonicWall URLs are included in an HTTP specific table and link to flow records. Citrix AppFlow URLs are included in an HTTP request table with links to additional HTTP response and ingress/egress connection tables.  In each case, in addition to the URL element itself being vendor specific, the information model associated with the exported URLs is also unique, reflecting the internal architecture of the exporting device.	The sFlow standard mandates a set of HTTP counters and transaction attributes that ensures consistent reporting from HTTP aware entities such as web servers (Apache, Tomcat, NGINX etc.) and load balancers (F5 etc.), irrespective of vendor or internal architecture. Each URL is exported as part of the standard transaction record that includes: client IP, server IP, referrer, authuser, user-agent, mime-type, status, request-bytes, response-bytes, response time. In addition, the sFlow standard defines a unified data model that links measurements from network devices, servers and application instances to provide a comprehensive, data center wide, view of performance.

From the examples in the table, it is apparent that IPFIX and sFlow standards take two very different approaches. The IPFIX standard is descriptive, defining a standard set of attributes that vendors can use to describe the information that they choose to export. The result is that vendors use IPFIX to differentiate each product, reporting a unique and inconsistent set of measurements based on its internal architecture and product features. In contrast, the sFlow standard is prescriptive, defining a set of measurements that every vendor must implement. While IPFIX provides a way to describe each "snowflake", the sFlow standard results from vendors working together to identifying common measurements and implement them in an interoperable way.

Henry Ford transformed the auto industry by moving from hand-made, custom parts to standardized components and processes that allowed for mass production. The data center is undergoing a similar transformation, from small, static, custom environments to large scale, commoditized, flexible, cloud architectures. The sFlow standard delivers the universal performance measurements needed for automation, enjoys broad vendor support, and along with other disruptive technologies like 10G Ethernet, merchant silicon, Software Defined Networking (SDN), OpenFlow, networked storage and virtualization is enabling this transformation.