In a very short period, many of the top enterprise applications have moved into the cloud.  Most of these applications are served up by highly available architectures that include geographically diverse data centers.   Many of the cloud-based applications can be categorized as critical to operations since any WAN outage would have an immediate impact on operations.

The shift to the cloud brings into focus the importance of connectivity to high quality internet transport and along with this, the access diversity required to ensure that uptime objectives are met.  When making diversity decisions cost is always a consideration, so it’s important to know the cost of downtime on the business and to classify the business sites according to their importance.  Classification of site type is also very helpful when assigning sites for SDWAN policy management.  Site types might include the following:

  • Platinum – Data center facility
  • Gold – Major office site, highly productive plant
  • Silver- Typical office site type
  • Bronze – Small office

Understanding all diversity attributes is very important step to building a highly available architecture.  Simply ordering internet circuits from two different ISPs and not considering path diversity is likely to put the decision maker in a tough spot when both circuits (that you have been paying for) both go down simultaneously for an extended outage.  Experienced telecom consultants have the technical expertise, carrier design experience, and the mapping tools to ensure that all options are properly vetted.

Below are some of the key considerations when designing for ISP Diversity:

  • Entrance Facility Diversity
  • Local Access Diversity/Path Diversity
  • PoP Diversity
  • PE (Provider Edge) Diversity
  • Circuit Tagging
  • Wireless Considerations

Entrance Facility Diversity

Sites that are or will be housing “platinum” level infrastructure should have dual entrances, ideally at opposite ends of the building.  If dual entrances are not available, the business case should be made to present the costs of a building a diverse entrance.  The cost of burying conduit outside of the building to align with a local access provider’s design should be accounted for.  Although the costs of building a diverse entrance facility might seem high, management must understand the costs to their business if a backhoe takes out “ISP A” and the estimated time to repair is quoted in days.

Local Access Diversity/Path Diversity

Engaging with an Internet Service Provider (ISP) to get into route details is a critical step if you are seeking to optimize path diversity.  You will quickly find that certain competitive local exchange carriers (CLECs) are more likely to provide you with path details than are larger providers and incumbent local exchange providers (LECs).  In many cases, a large LEC (i.e. Verizon) must follow a cumbersome internal process to generate a .KMZ (map file) and that this process will not begin until an order is entered into the system and the route design process begins.  Therefore, you will need to define your design strategy accordingly and carefully consider the providers that you seek to engage.

PoP Diversity

Depending on the metro in which you are designing a solution, there may or may not be multiple Points of Presence where the ISPs locate their edge routers or PE (Provider Edge).  In most major cities there will be multiple carrier “hotels” that are in proximity to the major fiber routes.  When selecting the ISPs for the bidding process, its important to list out where their CE routers are physically located so that you can be certain that your final design and vendor down selection allows for PoP diversity.

Provider Edge Diversity

Provider Edge or PE diversity is an important term to understand when designing for diversity.  PE diversity ensures that access circuits from the same carrier are not terminated on the same edge router.  You might think that PoP diversity will cover this concern, however, you need to ensure that the ISP properly tags the requested circuits for PE diversity even if PoP diversity has been ordered.  The reason is that over time carriers may seek to consolidate PoPs and groom circuits back to a single PoP.  By ensuring that your circuits are tagged for PE diversity, the carrier systems should notify network engineers to maintain at least PE diversity when making grooming decisions.  Further, PE diversity will help to ensure uptime when PE router upgrades are pushed through a maintenance window.

Circuit Tagging

As mentioned above, it is critical to communicate to the Sales and Sales Engineering teams that you are working with that you intend to design for diversity.  Be aware that sales engineering teams often need to follow a specific internal process to design for diversity and log the designs prior to moving into the pricing process since often, diverse designs have added cost components.  Be certain that your sales team acknowledges that they understand what you are seeking to accomplish and that they are fully aware of the internal steps necessary to design, price, and validate diversity tagging for each component of the final solution.

Wireless Considerations

With the expansion of 5G and advancements in fixed wireless internet solutions it is easy to think that selecting wireless as a solution for your diverse link is a simple solution for ISP diversity.  The first step to evaluating wireless options is to fully understand the realities of performance metrics from your physical location.  As you can read in our article on 5G, performance varies widely based on physical location and may only be fully tested by placing wireless equipment and testing the performance metrics.  Fixed wireless ISP service providers can complete a site survey and test their “shot” to calculate accurate performance prior to installing edge equipment on your roof.  Obviously, you will want to be highly confident in the performance of the selected solution, especially when critical applications require specific performance metrics.  Lastly, you need to understand where the wireless ISP provider edge resides to understand any risks in the complete solution.

Summary

As noted in the article, simply ordering circuits from two different ISPs is not designing for diversity.  Consult with an experienced team of network design engineers who can work with you in an unbiased manner to select the best suppliers with which to engage, and to fully document the design as part of the sourcing process.  The team at Macronet Services has extensive experience in the design of some of the largest global telecom networks.  We also have access to fiber and wireless tools as well as a full portfolio of over 300 suppliers from which to choose.  Contact us anytime to discuss what you are looking to accomplish.