TechnologyInside - on the web: The Rise and maturity of MPLS

In Networks Part 2: The phenomenon of Ipsilon, it was mentioned that one of the key strengths of IP routing was its resilience. If a particular node broke, then packets would find an alternative route. However, this strength also created a weakness in that individual packets in a stream could arrive at their destination following different paths through the network thus creating additional delay or latency while the packets were reassembled in the correct order.

This is what happens in connectionless IP networks such as the Internet. In practice, this creates unpredictable performance in congested networks as any one who uses the Internet will experience each and every day. MPLS was the white knight that came along to save the world from this fate.

	IP is all dominant - ATM relegated to transmission.
	Multiple media integration will take place at IP layer NOT at the ATM layer - video, data, image, text and voice.
	There are no NATIVE ATM services beyond �cheap bandwidth� cell-relay services.
	IP is the �killer application� for ATM (if there is one!)
	Customers are not asking for ATM, they are asking for IP
	The advent of �label switching� has changed the nature of WAN ATM network architecture for ever.

The advent of Multi-Protocol Label Switching (MPLS) defined a mechanism for packet forwarding in routers and enabled the rollout of connection-oriented networks. Connection-oriented data networks emulated the way that paths were set up in the Public Switched Telephone Network (PSTN) networks. This involves setting up a path prior to sending packets across the network.

Using labels that were added to packet headers, MPLS uses IP addresses to identify end points and intermediate routers on the path from source to destination. This makes MPLS networks IP-compatible and easily integrated with traditional IP networks. However, packets are routed along pre-configured Label Switched Paths (LSPs) using signalling protocol such as LDP, RSVP-TE, or CR-LDP thus making their flow both predictable and manageable.

When a label-headed packet arrives at a router, the router uses this label to identify the LSP. It then looks up the LSP in its own forwarding table to determine the link over which to forward the packet, and the label to use on this next hop. A different label is used for each hop, and it is chosen by the router or switch performing the forwarding operation. This allows the use of very fast and simple forwarding engines, as the router only has to select the label to minimize processing. In a normal router, it has to intercept and process every packet that transits. Ingress routers (called Label Edge Routers) at the edge of the MPLS network use the packet's destination address to determine which LSP to use. Inside the network, the MPLS routers use only the LSP labels to forward the packet to the egress router.

If you would like a more in-depth overview of MPLS, there is no better place to visit than Cisco's web site and read their MPLS Overview

The arrival of the concept of cut through routing enabled a new generation of engineers and companies to focus on these issues and they brought a new over-arching vision for this innovative technology - convergence of services over IP euphemistically called 'Everything over IP). Once most vendors had jumped on the IP bandwagon, following the demise of ATM, convergence became all the rage back in the late 1990s as the next step in IP world domination. It has never looked back.

Convergence of what you may ask. In the 90s, there was a complete separation of voice and data networks in the public service world. Voice services were carried over the highly standardised, understood tightly managed PSTN, while Wide Area Data Networks (WANs) were migrating towards high efficient Frame relay services (as described in 1992). The IP convergence vision forecast that all services would be converged onto a single IP-based network. Of course, there was a tremendous backlash to this idea from the traditional telecommunications engineering community who generally thought that this was all nonsense and could never work (they might still be right!). As with most things in life, both sides of the debate were correct in many areas but everyone was wrong about the timescales that it would take to achieve convergence. It's only now (2007) that we are seeing the convergence vision start to pan out in incumbent carrier consumer services with the possible 'leader' turning out to be our very own British Telecom with their 21C initiative (or here).

The advent of cut-through routing pushed Cisco into launching Tag Switching which morphed into Multi Protocol Label Switching (MPLS) when taken up the IETF Internet standardisation body which addressed a number of key issues that needed to resolved before convergence of certain services could take place over a pure IP network. The principle one of these was Class of Service (CoS).

One of the big strengths of ATM was that it enabled service and network designers to separate traffic into different classes of services that would be treated differently if the ATM network ever became congested. The highest class of service was reserved for real time services (isochronous services) such as voice and video that would take priority over less important store-and-forward services such as email. The modern equivalent to this is Voice over IP (VoIP) services taking precedence over Internet traffic.

CoS is replicated in MPLS by, not surprisingly, the Class of Service (CoS) feature. This enables network designers to provide differentiated types of service across an MPLS network by classifying packets at the edge of the network using Committed access rate (CAR). Packets are classified at the edge of the network before labels are assigned. CAR then uses the type of service (TOS) bits in the IP header to separate traffic that have been tagged with different TOS bits onto different Label Switched Paths.

In addition TOS, there is another feature that carriers can deploy in addition to basic MPLS to ensure QoS in times of congestion - Differentiated services or DiffServ.

On normal router, when more packets are sent through a port than it is able to handle, the queue will fill up and excess packets will be discarded. This will cause jitter, delays and malfunctions in your application software.

If the carrier uses the same categorisation of services as defined in ATM, best effort, assured and real-time services, multiple queues can be set up in ports to ensure that these three classes of service are treated differently in times of congestion. There are two basic queuing/scheduling mechanisms Weighted Round Robin (WRR) and Strict Priority. Strict Priority queue is used for real-time traffic, and WRR for assured and best efforts, what's left is best efforts.

At the end of the day, the use of technology for technologies sake is never justified so just what was it that drove the deployment of MPLS?

(a) Voice over IP (VoIP): The first real deployment of MPLS was not in customer-facing services, but was buried in the core of carrier's networks to carry voice traffic. Before the advent of IP networks, 100% of voice traffic was carried over traditional PSTN networks and was subject to a strong technical and commercial regime defined by the ITU. Carriers needed to agree to use appropriate public technical and interconnect standards driven by the big incumbent monopolies. They also needed to agree to commercial settlement for other carrier's traffic transiting their networks and to pay other carriers to carry their voice traffic. Settlement was both a cost and a revenue and in the 'good 'ol pre-IP days' these were usually in balance to everyone's benefit.

However, if international voice traffic could be buried or hidden in IP connections using VoIP, it was possible to significantly reduce transit costs AND avoid settlement out-payments at the same time! I guess the practical strategy in the growing market of the time was to siphon-off traffic growth onto IP networks to cap regulated out-payments to other carriers. "Not me Guv", was the mantra!

This financial gain led to the wide-scale deployment of VoIP in the late 1990s but was pretty much hidden from the public arena.

Since 2002, VoIP has slowly moved from hidden deployment out to end-user customer services. Initially this took place as enterprise services but a little later in the form of low-cost VoIP international call operators. There are literally hundreds of VoIP service providers today (you can see my list on TechnologySpectra).

(b) MPLS-based IP-VPNs: Another service that drove MPLS deployment was the idea that frame relay based WANs could be replaced by IP based services. These were called IP Virtual Private Networks (IP-VPNs). When this was first mooted, data services engineers familiar with frame relay services dismissed the concept out of hand but eventually came round under pressure.

The first IP VPNs came to market using the public Internet as the transport network and used IP-SEC as the encryption mechanism to keep the content private and are called Internet-VPNs. Unfortunately, these had one big problem - unpredictable performance (down to full stop situations) due to congestion on the Internet. Things are better these days especially if your Internet-based IP-VPN is national, rather than international in nature and they certainly provide low-cost solutions if performance is not of concern.

However, to get adequate and dependable performance carriers needed to provide MPLS-based IP-VPNs on their own networks using MPLS to to deliver appropriate performance using MPLS' Quality of Service (QoS) capabilities described above. The mechanisms were defined in a seminal IETF standard, RFC-2547 published in 1999.

One of the big issues that is still to be faced in 2006 is that if an IP-VPN straddles multiple carrier networks, as it has to do in the real world with global WANs, there are major technical and commercial issues in making QoS seamless across those networks. More on this in a future post.

The deployment of MPLS-based IP-VPNS has been pretty much universal over the last few years and is pushing frame relay WANs to one side (see graph at bottom of the page). However, things are not all rosy as IP-VPNS, like all IP networks, require significantly skilled (and expensive) CCIE engineers to manage them. They are very complex to set up and manage and could not in any sense be considered to follow a Keep it simple, stupid! (KISS) philosophy as expostulated by the Ethernet community [Confusingly, most carriers use MPLS to carry Ethernet services on their networks].

Note: An interesting Ethernet initiative is PBB-TE (Provider Backbone Bridging Traffic Engineering is an emerging IEEE standard that incorporates a set of enhancements to Ethernet known as Provider Backbone Transport (PBT) that allows the use of Ethernet for a carrier class public transport network. Again, more in a later post.

(c) MPLS traffic engineering: What is traffic engineering? My network book defines it thus: The ability to guarantee performance in a network for a certain amount of capacity for a certain amount of time. Also known as "traffic management," it implies the ability to analyze the current traffic load and dynamically make necessary adjustments to accommodate the different types of traffic or changing conditions."

Delivering appropriate Quality of Service to conform to customers' Service Level Agreements (SLAs) for both voice and data is key for a carrier. Traffic engineering as been at the heart of the PSTN for decades but it was a new activity for IP networks - and still is in practice.

In the past, most carriers delivered backbone Internet services used ATM as the bearer for IP traffic by routing it over specific inter-city paths to better manage the uncontrollable nature of IP. MPLS was seen to be the principle way forward in enabling carriers to remove ATM completely from their networks and still be able to traffic engineer their networks. This lead to the IETF standardising this need in a standard called MPLS-TE.

Interestingly, MPLS-TE has not been taken up on the scale originally expected by the carrier community for various reasons. More on this in a future post.

(d) Cost reduction: We have talked about services and engineering so far, but by far the biggest reason that MPLS has been taken up is the promise of reduced network management OPEX costs - the core of many upgrade business plans.

A non-converged legacy network has multiple layers. The lowest being the Wave Division Multiplexing (WDM) fibre layer , above this is the Synchronous Digital Hierarchy (SDH or SONET) layer, above this ATM and finally IP. Each of these layers runs independent of each other and each required a separate software management regime and separate fault management systems. This is extremely costly to operate and can quite easily lead to knock-on effects if a network element breaks.

Convergence to a single network based on IP predicted a wonderful Return on Investment (RoI) by collapsing those multiple layers down to IP using MPLS on a fibre network. Even after ten years, these benefits have yet still to be proven and the world's carriers are looking at BT's 21C initiative to see whether this proves to be the case. Personally, I think that there is no cheaper network to manage for voice services than a fully written off traditional TDM-based PSTN network which can still be be a part of a converged network - I hope this does not make me appear to be a luddite...

Building on the theme of everything over IP, pseudowires is a Cisco standard authored by Luca Martini several years ago and is now on the IETF agenda.

Pseudowires enables the transport of 'legacy' time division multiplexed (TDM) services over an MPLS enabled core network. It does this by emulating the characteristics of those legacy services, such as frame relay or ATM.

Pseudowires has been particularly useful for carriers providing DSL services for Internet access as this rather old technology is based on ATM.

Having read back over what I have written, I think I have raised more questions than I have answered and have certainly triggered several future posts that will talk about some of the issues raised.

So what about the title of this post? Yes, MPLS is now an old protocol. Back in 2000 / 2001 very few carriers had actually deployed it and it is even more recent that MPLS-based IP-VPNs have become common. Technologies really do take decades to penetrate our everyday lives.

Looking at the graph below, services such as ATM and frame relay, that could be described as legacy protocols, are now in terminal decline. However, IP over MPLS and Ethernet are growing rapidly. The point to notice though, is that it could be considered that MPLS growth has matured while Ethernet is still experiencing high growth.

MPLS has provided solutions to the QoS enigma but, as with any IP based service, it has proved to be complex and expensive to manage and the jury is still out about whether its deployment (as with convergence) really saves cost. In this age of ever-accelerating technology and service roll-outs, I think it is fair to say that MPLS itself has is now in a state of maturity. Indeed, maybe there are other solutions that will replace it going forward. But, as I've said several times in this particular post, that is the subject for a future post as well.