Internet Interconnection and Exchange

Internet Interconnection and Exchange

As mentioned earlier, NSPs can interconnect at one of several points. First, four NAPs in the United States are partially funded by the National Science Foundation (Ameritech, PacBell, Sprint, and MFS Datanet), and the reason they get some support from the U.S. government is that they also provide interconnection into Internet 2, the academic research backbone. Second, connection can be made via agreements between ISPs (that is, peering). Third, there are commercial interconnection and exchange points, which can be used through private NAPs.

NAPs

Remember that the local service providers typically connect into the NAPs, or exchange points, through top-tier ISPs rather than directly. So what's the definition of a NAP? A NAP is the place where NSPs and/or ISPs exchange traffic with their counterparts. It's a public meeting point that houses cables, routers, switches, LANs, data communications equipment, and network management and telecommunications links. NAPs enable the NSPs and top-tier ISPs to exchange Internet traffic without having to send the traffic through a main transit link. This translates to decreased costs associated with the transit links, and it reduces congestion on the transit links.

Figure 9.11 shows the inside of one of the NAPs, Pacific Bell's point in California. On the left are the various ISPs and NSPs that are connecting to the NAP; note the various ISPs and their connection over a DSU into the Pac Bell ATM switch. The ATM switches are redundant and connected by redundant optical carrier levels (OC-3 initially, but they are being upgraded all the time). In addition, there are route servers, which maintain the databases of the appropriate path to take, based not just on the routing algorithm but also on the policy that the ISPs want to observe. In other words, there may be relationships about how traffic is passed between two providers not just a next-hop or lowest-cost scenario that determine what the path is. The route servers reside on a LAN, in this case, a fiber-based LAN called Fiber Distributed Data Interface (FDDI). FDDI was the first of the 100Mbps LAN backbones, and today most of these LANs would be Gigabit Ethernet. Resources connect into DSUs that lead to the routes of the selected NSPs. Routing arbiters are in place so that if there's dispute about how traffic has been routed, the arbiters can step in and arbitrate the situation.

Figure 9.11. The Pacific Bell NAP

The NAPs have equipment that's quite similar to what you find inside an Internet POP, which is quite similar to what you find inside a telco POP. This speaks to the convergence between the information types and their supporting infrastructures.

NAPs are becoming more discriminating. For instance, some require that the minimum connections be T-3 or E-3 and that they be redundant as well. That eliminates a lot of the smaller players from connecting to NAPs. If the NAPs were not so discriminating, there would be tremendous congestion at the public exchange points. For any traffic that's time sensitive, such as Internet telephony, VoIP applications, networked interactive games, multimedia, and streaming video, this congestion will cause problems with reliability and predictability. NAPs are also increasing in number; there are about 175 NAPs worldwide today. (For more information on NAPs, go to www.ep.net.)

Besides the four government-funded NAPs in the United States, other NAPs are for profit, and they charge per connection into the switch or router, initially in the US$4,000 to US$6,000 range. The cost varies with the economic times as well as the speed of the connection. And the NAPs, again, demand that you be able to guarantee a level of QoS before you connect at that exchange point.

Despite the fact that NAPs are now becoming more discriminating, they have become a point of congestion. Losses and delays are negatively affecting applications everyone wants to see developed, and therefore other alternatives have been brought about, as discussed in the following sections.

Peering Agreements

An alternative to NAPs is the use of private peering agreements. In a peering solution, operators agree to exchange with one another the same amount of traffic over high-speed lines between their routers so that users on one network can reach addresses on the other. This type of agreement bypasses public congestion points, such as NAPs. It's called peering because there's an assumption that the parties are equal in that they have an equal amount of traffic to exchange with one another. That's an important point because it obviously makes a difference in how money is exchanged. This is an issue of concern at the moment and one that many people are studying to determine ultimately whether there has to be a regulatory mechanism or whether market forces will drive it, but in general, with the first generation of peering agreements there was an understanding that peers were equals. Newer agreements often call for charges to be applied when traffic levels exceed what was agreed to in negotiations.

The most obvious benefit of peering is that because two parties agree on working with one another, and therefore exchanging information about the engineering and performance of their networks, the overall performance of the network is increased, including better availability, the ability to administer service-level agreements, and the ability to provide greater security. Major backbone providers are very selective about international peering, where expensive international private line circuits are used to exchange international routes. Buying transit provides the same benefits as peering, but at a higher price. Exchanging traffic between top-tier providers basically means better performance and fewer routers involved. And again, these types of arrangements are critical to seeing the evolution and growth in IP telephony, VoIP, and multimedia.

One problem with peering is that it can be limited. Under peering arrangements, ISPs often can have access only to each other's networks. In other words, I'll agree to work with you, but you can work only with me; I don't want you working with anyone else. Exclusivity types of demands might arise.

Private NAPs

The alternative to NAPs and peering arrangements is to use a private NAP, also called an overnet. Private NAPs are connected directly, via private lines, to the IP backbones of several of the major NSPs or backbone providers. This design means that customers can get access to the major backbones without having to peer at the congested NAPs. The private exchange enables second-tier providers to connect to several first-tier providers, all in one location as well. Examples of private NAPs are InterNAP, Savvis Communications, and Digital Island. Savvis opened a private exchange point in London, and it expects to connect two first-tier ISPs there. InterNAP is opening exchanges in London and Amsterdam, and it also has plans for Frankfurt and Paris, expecting to connect with six first-tier ISPs, as well as two major in-country providers. Thus, InterNAP currently has some 50 NAPs that are live or about to open. Both Savvis and InterNAP run exchanges in the United States, and each also claims to get great deals on transit relationships or even to negotiate peering relationships that are unavailable to smaller ISPs.

Considerations for the Future

We need to be careful what we consider to be ready for primetime business, especially when it comes to the public Internet, that uncontrolled subcollection of networks that number more than 150,000. For example, one of the U.S. NAPs, MAE East, which handles something like 33% of the world's Internet traffic, for a long time resided in an underground parking garage in an unprotected space. Since that information was published, the problem surely has been resolved, but this was a condition whereby a person who was going in reverse with a very heavy foot could have taken down a third of the Internet.

There's a NAP in California as well: MAE West. MAE West is one of the busiest of the NAPs, connecting points between various NSPs. It's so crowded with equipment that the air conditioning can't keep up at times.

There was an instance not too long ago, in a small Florida ISP, in which a command entry error during router setup redirected 25% of all Internet traffic across a single T-1 link, which caused a two-hour brownout on the global Internet. The ISP hadn't used BGP filtering, but if it had, that message may have been caught and so much of the world's traffic would not have been redirected to what amounted to a local service provider backbone.

Finally, there was a software glitch at Network Solutions, which managed the top-level domain names for the entire Internet. A file containing more than one million Internet addresses got corrupted. Worse yet, a technician ignored the alarm that was issued and allowed the files to be replicated across the Internet, thereby blacking out Web sites for several hours.

So, I very much appreciate the candor of Mike O'Dell, chief scientist for UUnet Technologies, who used to say about the public Internet, "If you're not scared, you just don't understand."

But the Internet is not all bad and scary. The problems described here mainly affect business-class applications, where you need stringent controls. We also see a great many benefits and cost-efficiencies as a result of the public Internet.

 

For more learning resources, quizzes, and discussion forums on concepts related to this chapter, see www.telecomessentials.com/learningcenter.