Enterprise Internet Architecture: More Connections Don’t Always Mean More Capacity

When an enterprise starts seeing high utilization on its Internet connections, a common suggestion is to add another provider or another BGP peering relationship. That may be the right decision, but I would want to understand exactly what we’re buying and which traffic will use it before enabling anything.

Let’s take a practical example. We already have paid Internet transit from Lumen and Cogent, plus a single 10G connection to Equinix Internet Exchange (IX). Now we’re evaluating Hurricane Electric (HE, AS6939). Should we provision a dedicated HE circuit, or peer directly with HE over the Equinix IX connection we already have?

Both approaches are legitimate, but they are not the same architecture. One can introduce additional physical capacity and a new failure domain; the other uses existing IX capacity to establish another direct BGP relationship.

Start With What We Already Have

For this example, assume our Internet edge looks like this:

                         [ Enterprise INET Router ]
                                    |
               +--------------------+-------------------+
               |                    |                   |
         [ Lumen ISP ]        [ Cogent ISP ]      [ Equinix IX ]
          IP Transit            IP Transit         Single 10G
                                                     |
                                               IX participants
                                               + MLPE peers

We advertise our authorized public Internet prefixes to Lumen and Cogent, and we also advertise them to the Equinix MLPE route servers where appropriate. Other networks can reach us through whichever paths their BGP policies select. Our own outbound traffic likewise follows the routes and local policy configured on our Internet edge.

One distinction matters before going any further: Equinix IX is a Layer 2 exchange fabric, not an IP transit provider. Its route servers exchange BGP information, while actual traffic crosses the IX fabric between participants’ routers. Bilateral peering with HE on the IX would use that same physical 10G connection.

Option 1: Give Hurricane Electric Its Own Circuit

In the first design, we provision a new, separate connection to HE and establish an agreed bilateral BGP peering relationship on it. We keep the existing single 10G Equinix IX connection for our other IX peers and MLPE traffic.

             [ Enterprise INET Router ]
                         |
           +-------------+-------------+-------------+
           |             |             |             |
     [ New HE ]      [ Lumen ]     [ Cogent ]   [ Equinix IX ]
    Dedicated link   IP Transit   IP Transit       10G
      eBGP to HE                                   |
      AS6939                                  Other IX peers
                                              + MLPE

Path A: A dedicated HE peering connection for routes learned from HE. Path B: Lumen Internet transit. Path C: Cogent Internet transit. Path D: the existing Equinix IX 10G connection for MLPE and other bilateral peers.

What this gives us

• Additional physical handoff capacity: If the HE circuit is genuinely new, its bandwidth does not consume the existing Equinix IX 10G port capacity.
• Potentially better separation: A physical problem on our Equinix IX port would not necessarily affect HE peering over the separate connection.
• Independent traffic measurement: We can monitor HE circuit utilization separately from IX and transit utilization.
• Additional cost and operations: We need another circuit, handoff, BGP session, monitoring, and failure/rollback design.

Separate circuits do not automatically guarantee independent failure domains. The HE circuit could still share a building, carrier transport, router, power source, or downstream 10G bottleneck with the existing connections. That must be checked.

And a dedicated HE peering circuit is not the same as buying HE IP transit. With bilateral peering, we may send HE traffic only for the routes HE advertises under the peering arrangement. If we purchase transit, the route scope, economics, and design are different.

Option 2: Peer With HE Over the Existing Equinix IX

In the second design, we do not add a separate HE circuit. Instead, we establish bilateral eBGP peering with HE over the Equinix IX connection, assuming HE is present in our IX metro and both sides agree to peer.

                         [ Enterprise INET Router ]
                                    |
              +---------------------+--------------------+
              |                     |                    |
        [ Lumen ISP ]         [ Cogent ISP ]      [ Equinix IX ]
         IP Transit             IP Transit         Single 10G
                                                       |
                                       +---------------+-----------+
                                       |                           |
                                 [ HE AS6939 ]             [ Other IX Peers ]
                                 Bilateral BGP               MLPE / Bilateral

Path A: The existing Equinix IX 10G connection carries both HE bilateral-peering traffic and traffic exchanged with other IX participants. Path B: Lumen Internet transit. Path C: Cogent Internet transit.

What this gives us

• Lower implementation cost: No new HE physical circuit is required if our existing IX handoff and HE’s available peering arrangements support the session.
• Potential transit offload: Some traffic currently using Lumen or Cogent may shift to the Equinix IX when our BGP policy prefers HE-learned routes or remote networks prefer our direct HE announcement.
• Shared 10G capacity: HE traffic and all other IX traffic compete for the same 10G physical handoff.
• Shared IX failure domain: An optic, interface, or IX access failure can affect both HE bilateral peering and existing Equinix peering traffic.

Equinix allows multiple bilateral peering sessions on the same physical port, and its MLPE documentation states that bilateral and multilateral peering share the same VLAN and IP. This is a normal Internet exchange design, not a special workaround.

HE also publishes a policy expectation to peer at all common exchange fabrics, rather than selectively establishing a session at only one shared site. If we operate in multiple metros, that requirement belongs in the design review and should be confirmed with HE.

Side-by-Side Comparison

What Other Providers Do Large Enterprises Evaluate?

Hurricane Electric is one option, not the only option. Enterprises with their own public address space and BGP-capable Internet edge commonly evaluate a mix of global IP transit providers, Internet exchanges, and selective private interconnects. Which providers are suitable depends on the actual locations, commercial terms, traffic patterns, and available routes. This is a candidate list, not a market-share ranking or a claim that every large enterprise uses these networks.

Other IP Transit and Dedicated Internet Providers

For an enterprise that already purchases transit from Lumen and Cogent, I would include the following providers in a request for information or architectural comparison. Their offerings are not identical: some provide wholesale-oriented BGP IP transit, while others offer enterprise Dedicated Internet Access (DIA). Confirm support for your own ASN, BGP routing, customer-owned prefixes, route feeds, and desired communities before treating a service as equivalent.

All of these are candidates to evaluate, not an endorsement or guarantee of carrier availability in a particular building. A provider’s published global backbone size doesn’t tell us whether its local handoff uses diverse fiber, whether the intended routes will actually prefer it, or whether the next downstream connection is already saturated.

Other Internet Exchange and Peering Options

We should also distinguish Internet exchange operators from IP transit carriers. Besides Equinix IX, exchanges such as DE-CIX, LINX, and AMS-IX support public peering models that may include route servers and bilateral BGP sessions. Availability varies by metro, and joining another exchange requires access to its peering fabric, directly or through a supported remote-peering arrangement.

• Another IXP (for example, DE-CIX): Could provide access to a different set of participating networks or infrastructure, but must be assessed for actual peers, cost, and physical independence.
• Bilateral peering on our existing Equinix IX 10G port: Can improve access to selected peers without another physical circuit, but shares the same 10G handoff and its failure domain.
• Private network interconnect (PNI): A direct interconnection with an eligible high-volume network or content provider may make sense when traffic measurements and peering agreements justify it.

How I’d Narrow the Provider Shortlist

I would start with traffic data by destination ASN, source ASN where observable, peak-hour utilization, and routing policy, then compare each candidate against the actual architectural objective:

• Need more general Internet capacity? Compare a separately provisioned IP transit circuit from HE, Arelion, NTT, GTT, Zayo, or another qualified provider against an upgrade to Lumen or Cogent.
• Need to offload specific high-volume networks? Check whether those networks will establish bilateral peering over the existing Equinix IX port, and whether the port has enough spare capacity.
• Need stronger resiliency? Validate independent entrances, fiber paths, carrier equipment, routers, power, and downstream forwarding capacity. A different provider name is not proof of end-to-end diversity.
• Need better regional reach or path quality? Compare actual BGP paths, latency, loss, route policies, and peer availability in each metro rather than selecting a carrier based on general reputation.

My preference: Build a short list of technically viable providers first, then let measured traffic, physical-path diversity, commercial terms, and operational readiness determine the winner. This is an architecture decision, not a race to accumulate the most carriers.

Remember: BGP Decides the Actual Traffic Paths

These diagrams show intended paths, not guarantees. Just because we advertise our public prefixes to HE does not mean every HE-connected customer will enter through that HE connection. Remote networks select routes according to their own BGP policies and alternatives.

On our side, local preference and other BGP attributes can influence which learned routes we prefer for outbound traffic. On the inbound side, route announcements, supported communities, and AS-path prepending can influence remote choices, but cannot dictate them unilaterally. Return paths may be asymmetric.

We also should not assume that opening a BGP session to HE somehow turns an ordinary Lumen or Cogent transit connection into a direct HE peering circuit. If the intent is to peer directly, use an agreed common IX or a separately provisioned interconnection. Otherwise, traffic reaching HE via Lumen or Cogent remains dependent on those transit paths.

What Does a 10G IX Port Mean for Capacity?

Consider a purely hypothetical example: our Equinix IX connection is already carrying 6 Gbps in one direction at a busy time. After adding HE bilateral peering over the same IX connection, another 3 Gbps in that same direction shifts from paid transit onto the IX.

Option 2 — HE over existing Equinix IX:
Existing IX traffic:     6 Gbps
Additional HE traffic:   3 Gbps
                         ------
Total on 10G IX port:    9 Gbps  (90% utilization)

Option 1 — Separate HE circuit:
Existing Equinix IX:     6 Gbps on 10G IX port
HE peering traffic:      3 Gbps on separate HE circuit

Option 2 might reduce transit utilization, but now the IX port operates much closer to its physical limit during that interval. Option 1 keeps that HE traffic off the existing IX handoff, assuming routing policy actually directs it to the new link.

But neither option automatically fixes downstream oversubscription. If all of these connections eventually feed the same congested aggregation interface, firewall, WAN link, or data-center path, adding another Internet circuit may simply move the bottleneck.

A Failure Scenario I Would Plan for

Imagine the Equinix IX 10G optical module starts developing errors. The interface remains operational, BGP sessions stay established, and some application traffic begins dropping packets. This is a real type of fault that can be difficult to diagnose if engineers monitor only whether BGP neighbors are up.

Under Option 2, HE bilateral traffic and the other IX participants share the impaired interface. Under Option 1, the separate HE connection may remain healthy and carry its eligible traffic. Neither configuration guarantees instantaneous recovery for affected flows; failure detection and BGP convergence still matter.

Monitoring should include physical interface errors, optical levels, utilization, BGP route changes, forwarding next hops, NetFlow/IPFIX, and end-to-end packet-loss tests. We need to know which interface packets actually use, not only which BGP neighbor supplied the route.

Don’t Implement It Until the Team Understands It

I would not enable either option merely because an additional path is available or someone believes it will provide "capacity relief." The existing Internet routing environment should first be stable, consistent, monitored, and well understood. Unexplained drops or routing anomalies deserve investigation before we add another variable.

The entire network engineering team should know what is changing, why we’re making the change, which prefixes will be exchanged, how routing priorities work, and what troubleshooting looks like afterward. Management should understand the objectives, risks, costs, and evidence that will define success. Not everyone needs to know every BGP command, but everyone responsible for operating the network needs a coherent picture of the design.

Before implementation, I would require:

• A measurable objective: Are we reducing paid transit utilization, gaining independent capacity, improving reachability, or increasing resiliency?
• A current baseline: Peak and average traffic per circuit, important source/destination ASNs, packet loss, errors, and BGP path selection.
• A topology and failure-domain review: Physical handoffs, shared transports, downstream bottlenecks, and circuit-level independence.
• Routing policy documentation: Authorized prefixes, inbound/outbound policy, local preference, maximum-prefix limits, filtering, and any peering-specific communities.
• Controlled implementation: Change window, verification steps, traffic-shift monitoring, and a rollback plan.
• A support runbook: Which commands and telemetry the team will use to identify a bad IX port, a failed HE session, or unexpected asymmetric routing.

My Recommendation

If the objective is primarily to offload measurable HE-eligible traffic from Lumen and Cogent, I would evaluate Option 2 first. It may allow us to use an IX connection we already pay for, without immediately introducing another circuit. But I would not proceed until we understand the current utilization and available headroom on that single 10G IX port.

If the objective is to add independent capacity or reduce dependence on the IX handoff, Option 1 deserves serious consideration. The value then comes from the genuinely separate transport, not merely from having another BGP neighbor. I would also compare the price and scope of HE peering with HE IP transit and other qualified transit providers if additional general Internet capacity is the goal.

Finally, I would avoid treating ordinary Lumen or Cogent IP transit as a substitute for a purpose-built direct HE peering connection. Those are different services and different failure domains.

References

• Equinix — Internet Exchange overview
• Equinix — Bilateral Peering
• Equinix — Multi-Lateral Peering Exchange
• Hurricane Electric — Peering Policy (AS6939)
• PeeringDB — Hurricane Electric
• Arelion — IP Transit (AS1299)
• NTT Global IP Network — IP Transit (AS2914)
• GTT — IP Transit
• Zayo — IP Transit Network Availability
• AT&T Business — Dedicated Internet
• Verizon Business — Dedicated Internet
• DE-CIX — Exchange Locations
• LINX — Peering and Route Servers
• AMS-IX — Route Servers