The Control Plane, Data Plane and Forwarding Plane in Networks
The Control Plane, Data Plane and Forwarding Plane in Networks is the heart core DNA in today’s networking hardware to move IP packets from A to Z. The Management plane is another vital component but also widely excepted as user to hardware interaction. These planes of operation are the building blocks of the layered architecture that networks have evolved to today. By abstracting data to conform to these constructs is how the Internet works today. If I could rewind a dozen years I would probably have made myself write the concepts of Control, Data and Forwarding planes on the chalkboard a few thousands times rather than spending the first couple of years of my career trying to understand the incredible complexities of networks backwards with a weak foundation.
Working with some interns this week I was walking through these “planes” with them and as I have in the past got to “Data Plane” and “Forwarding Plane”. It seems a bit nascent from time to time what the difference, if any is between the forwarding and data plane. I thought it might be helpful to cobble together some definitions from some of my favorite books, a couple of RFCs and some drawings, so I don’t have to fumble around every semester with poor explanations. Kicking off with fast path, slow path, BCAM and Trie lookups doesn’t do anyone any good. So lets look at some pictures instead. Disclaimer much of this can be semantics which tend to get tedious and boring so, well sorry.
Control Plane
The control plane is the component to a router that focuses on how that one individual box interacts with its neighbors with state exchange. The Routing Information (data)Base (RIB) and Label Information Base (LIB) are processed in software and used to populate FIB(forwarding information base) and the LFIB. Vendors can implement these in different fashions on how those tables are partitioned between multiple routing instances. For example, a router has a BGP and OSPF adjacencies, those routing protocols have different algorithms to determine what a chosen path to a network would be. Building the topology or global view as that particular router sees it from its point of view. That is fairly important to recognize that its “global view” is from its perspective of either the IGP or EGP.
Figure 1. Juniper’s RIB tables are isolated into well defined groups for each routing instance. This can be helpful if/when you need to import routes from one instances table to another. Another way to think of that is importing or exporting from one MPLS/VPN VRF A to VRF B.
In today’s networks we have taken distribution of state to the extreme. For your average router (minus MLAG, Virtual Chassis proprietary technologies etc) every box has the ability to stand alone. A routers control plane may as well be standalone from it’s neighbor as if each one is in a separate administrative domain. While link state flooding protocols have administrative awareness through things like areas in ISIS and OSPF or Autonomous Systems (ASN) in BGP policy application is pretty rudimentary at best with such a high level of distribution and low levels of abstraction.
Here is an excerpt from a good read, High Performance Switches and Routers (2011):
The control plane functions include the system configuration, management, and exchange of routing table information. These are performed relatively infrequently. The route controller exchanges the topology information with other routers and constructs a routing table based on a routing protocol, for example, RIP (Routing Information Protocol), OSPF (Open Shortest Path Forwarding), or BGP (Border Gateway Protocol). It can also create a forwarding table for the forwarding engine. Since the control functions are not performed on each arriving individual packet, they do not have a strict speed constraint and are implemented in software in general. – High Performance Switches and Routers (2011) Chao & Liuhttp://www.amazon.com/High-Performance-Switches-Routers-Jonathan/dp/0470053674
The Control plane feeds the forwarding/data plane with what it needs to create its forwarding tables and updates topology changes as they occur. Those are pretty low even in large networks single to at most I would speculate double digit per second changes. This is the reason the control plane can often be thought of as the “slow path” in legacy route once switch many packet switching architectures. A list of functions performed in traditional routing engines/route processors are the following:
- Allocates resources to the forwarding engine/plane.
- Routing state
- ARP handling is always processed by general purpose processor located in the routing engine.
- Security functions to secure the control plane access. Telnet, ssh, AAA etc.
- Establishes and maintains management sessions, such as Telnet connections
- Routing state to neighboring network elements.
- Vendor and platform specific stacking, clustering, pairing etc.
Data Plane and Forwarding Plane
I am starting this off with saying I am using data and forwarding planes interchangeably.
Figure 2. The Planes separated and typical packet logic into these information bases.
Figure 3. Lets break the control and data forwarding plane up. Green circle = Control Plane Routing Protocols Purple box = Switches and forwarding data plane.
The data plane is the workhorse of the switching elements in our networks. It has the responsibility of parsing packet headers (or cells, SONET) in high speed search ASICs. It manages QOS, filtering, encapsulations, Queuing, Policing all of the reasons we had and still do in many cases purpose built silicon or custom ASIC designs.
The data/forwarding plane must do those operations in the “Fast Path” to keep up with performance needs in data centers and core networks. Acheiving that sort of performance is often done with varying compoenets of memory types whether Trie based BCAM, TCAM, NPU and even FPGA is starting to post impressive numbers. Merchant silicon manufacturers are really starting to key in on programmability of data center commodity switching providing programmability that allows for vendors to differentiate in programmability of that silicon. Arista and Cisco just this past week released new low latency switches with Arista‘s 7150S leveraging the Alta Intel/Fulcrum chipset while Cisco’s 3358 was a custom spun or likely fabless ASIC.
Figure 4. Slow and Fast path in a software vSwitch (for those that have a Control Plane). In the early days of SDN so far the Control Plane is the special sauce or vendor differentiation.
Fast and Slow path is a pretty reasonable way to look at control and data planes. Control Plane has limited updates to the FIB or LFIB in the case of Label Forwarding (same concepts) while the forwarding plane is looking up prefixes and getting called infinitely more times per second thus the need for very fast search ASICs or similar performance chips.
Forwarding data planes typically come either centralized or distributed. This means the forwaring engine is either centrally located across the ethernet fabric/crossbar or pushed all the way to the edge. The more performance required the more that distributed forwarding is pushed to the edge. How we design networks is merely a macro of what is happening on the board.
Figure 5. Centralized and Decentralized Forwarding Engines.
Search ASICs and NPU are pushing more towards the edge for greater scale and policy application performance. Sounds awfully familiar to whats happening in wireless today. Clearly control(lers) are heading to the edge with intelligence for performance. That said we need to ability to oversubscribe rather than buying unused capacity. That covered in the 2nd part of this post.
Management Plane
Telnet, SNMP, SSH, XML, stone tablet and chisel, thats some of how we orchestrate and operate networks today, well except for the wireless market. It is pretty simple today. I think that says a lot about the industry right now for that matter. If we get anything out of the next few years it will be simplification of network operations and management from abstraction, orchestration and/or automation. There have been some interesting thoughts and IETF draft submissions lately around the operational plane. One submitted by Martin, Pfaff and crew (read blog or raw draft) around OVSDB which looks promising.
Where Is It All Going?
Well the question is do we decouple and if so how much? I am going to fork that off to another Part2 blog post because that will be the interesting guessing game. Do we extract the control plane and if so how central or distributed does it need to be. Stay tuned for that in the next few days. In the meantime if you have not watched Ferro describing Control, Data and Management planes on Ivan’s site at http://demo.ipspace.net. It is one of the most articulate lectures I have ever seen on the topic so check it out here, click the videos tab and it is video 1.
