Monday, June 6, 2016

Streaming telemetry

The OpenConfig project has been getting a lot of attention lately.  A number of large network operators, lead by Google, are developing "a consistent set of vendor-neutral data models (written in YANG) based on actual operational needs from use cases and requirements from multiple network operators."

The OpenConfig project extends beyond configuration, "Streaming telemetry is a new paradigm for network monitoring in which data is streamed from devices continuously with efficient, incremental updates. Operators can subscribe to the specific data items they need, using OpenConfig data models as the common interface."

Anees Shaikh's Network Field Day talk provides an overview of OpenConfig and includes an example that demonstrates how configuration and state are combined in a single YANG data model. In the example, read/write config attributes used to configure a network interface (name, description, MTU, operational state) are combined with the state attributes needed to verify the configuration (MTU, name, description, oper-status, last-change) and collect metrics (in-octets, in-ucast-pkts, in-broadcast-pkts, ...).

Anees positions OpenConfig streaming telemetry mechanism as an attractive alternative to polling for metrics using Simple Network Management Protocol (SNMP) - see Push vs Pull for a detailed comparison between pushing (streaming) and pulling (polling) metrics.

Streaming telemetry is not unique to OpenConfig. Industry standard sFlow is a streaming telemetry alternative to SNMP that has seen rapid vendor adoption over the last decade. Drivers for growth discusses how the rise of merchant silicon and virtualization have accelerated adoption of sFlow, particularly in data centers.

sFlowOpenConfig TelemetrySNMP
OrganizationsFlow.orgOpenConfig.netIETF
UsersGeneral PurposeLarge Service ProvidersGeneral Purpose
ScopeData Plane, Control PlaneManagement Plane, Control PlaneControl Plane
Vendor Support40+ (see sFlow.org)1 (Cisco IOS XR )Near universal
Modelsstructure definitionsYANG modelsManagement Information Base
EncodingXDR (RFC 4506)protobufs, JSON, NetConfASN.1
TransportUDPUDP, HTTPUDP
ModePushPushPull

The table compares sFlow and OpenConfig Telemetry. There are a number of similarities: sFlow and OpenConfig are both driven by participation based organizations that publish standards to ensure multi-vendor interoperability, and both push standard sets of metrics using standard encodings over widely supported transport protocols.

However, important differences result from OpenConfig's exclusive focus on large service provider configuration and monitoring requirements. Telemetry is tied to the hierarchical YANG configuration models, making it easy to correlate operational and configured state, but limiting the scope of monitoring to the management and control planes of devices that are configured using OpenConfig.

In contrast, sFlow is a management and control plane agnostic monitoring technology (i.e. a device may be configured using CLI, NetConf, JSON RPC, OpenConfig, etc. to use any control plane OpenFlow, BGP, OSPF, TRILL, spanning tree, etc.). In addition, sFlow is primarily concerned with the data and control planes, i.e. capturing information about packets and forwarding actions.
The article, Management, control and data planes in network devices and systems, by Ivan Pepelnjak, provides background on data, control, and management plane terminology.
Gathering data plane telemetry requires hardware support and merchant silicon vendors (Broadcom, Cavium/XPliant, Intel/Fulcrum, Marvell etc.) include sFlow instrumentation in their switching/routing ASICs. Embedded hardware support allows sFlow to efficiently stream standardized data plane telemetry from all devices in a large multi-vendor network.

To conclude, sFlow and OpenConfig shouldn't be viewed as competing technologies. Instead, the complementary capabilities of sFlow and OpenConfig and shared architectural model make it easy to combine sFlow and OpenConfig into an integrated management solution that unifies monitoring and control of management, control, and data planes.

No comments:

Post a Comment