Data Center Interconnect: Understanding Layer 1, Layer 2, and Layer 3 Options

When designing connectivity between geographically separated data centers, network architects quickly encounter a confusing list of terms: dark fiber, DWDM, Metro Ethernet, EPL, EVPL, MPLS, L2VPN, VPLS, EVPN, VXLAN, and L3VPN. These terms are often compared as though they are mutually exclusive products. They are not.

Some describe physical transport. Others describe Ethernet services, VPN technologies, control planes, or managed IP routing. Understanding the difference matters before asking carriers such as Lumen and Cogent to quote a data center interconnect (DCI).

Let’s use a fictional enterprise operating four data centers: Miami (MIA), Dallas (DAL), Ashburn (ASH), and San Jose (SJC). It wants two carrier relationships for resilience and is evaluating two fundamentally different approaches: extending Layer 2 between the sites or routing at Layer 3.

1. First, separate what you buy from how it works

A carrier product is not always the same thing as the technology delivering it. For example, an Ethernet Private Line may be implemented over an optical system, MPLS pseudowires, or another carrier transport architecture. Similarly, MPLS can support both Layer 2 and Layer 3 VPN services.

2. Layer 1 choices: dark fiber and optical wavelengths

Dark fiber

The carrier leases unused fiber strands. The enterprise lights the fiber and is responsible for its optical equipment, routing, and switching. This provides substantial control but also requires optical engineering, operational expertise, and a practical fiber route. Long-distance dark fiber may be costly or unavailable between some facilities.

DWDM and dedicated wavelengths

A carrier supplies a dedicated optical transport channel, such as 100G or 400G, between endpoints. The carrier generally manages the optical line system while your enterprise determines whether Ethernet or routed IP runs over the delivered interface.

These are excellent transport candidates when large capacity and path diversity justify the economics. Neither dark fiber nor a wavelength dictates whether your enterprise will bridge or route traffic.

3. Layer 2 carrier options

Carrier Ethernet services are commonly described using three MEF service families: E-Line, E-LAN, and E-Tree. Their specific product variants matter.

What about VPLS and EVPN? These are possible mechanisms for delivering multipoint or point-to-point Ethernet connectivity. VPLS commonly relies on pseudowire and MAC-learning behavior. EVPN distributes Ethernet reachability using BGP and supports more sophisticated multihoming models. Neither term, by itself, guarantees a particular product or feature set from a carrier.

What about MPLS? A carrier may use MPLS internally to transport an EPL, EVPL, VPLS service, or IP VPN. Asking for “MPLS” is therefore incomplete; the useful question is which service and handoff behavior do we want?

4. Layer 3 choices: two very different models

Model A: Customer-routed Layer 3 over private transport

Your enterprise buys EPL circuits or wavelengths. Router interfaces use point-to-point addressing such as /31s, and you run BGP (or another suitable routing protocol) between your own devices. The provider transports packets, but does not need to learn your enterprise prefixes or decide intersite routes.

MIA router ==== Lumen EPL / wavelength ==== ASH router
      \                                           /
       +========= Cogent EPL / transport ========+

Enterprise routing: BGP + optional BFD on each eligible path
Provider service: transparent point-to-point transport

This model preserves enterprise control of routing policy, path preference, ECMP eligibility, route filtering, and convergence behavior. BFD can improve failure detection when supported and correctly tuned, but end-to-end failover time must be tested rather than assumed.

Model B: Carrier-managed Layer 3 VPN

With an MPLS L3VPN / private IP VPN, each customer edge (CE) router connects to a carrier provider edge (PE). Your network exchanges routes with the carrier, typically using BGP. The provider operates the private intersite routing and VPN forwarding infrastructure.

  MIA CE -----+                          +----- ASH CE
              |                          |
              +--- Carrier IP VPN / -----+
              |      MPLS L3VPN          |
  DAL CE -----+                          +----- SJC CE

Enterprise peers with the provider at each location.

That can simplify transport provisioning and provide a managed service boundary, but it gives the carrier responsibility for decisions and failure modes inside the WAN. Private IP VPN is also distinct from Internet access.

Other Layer 3 services

Dedicated Internet Access (DIA) and IP transit provide connectivity to the public Internet, not an equivalent private DCI service. SD-WAN and IPsec can connect the same four sites through an overlay, but still need underlay circuits such as Internet, private IP VPN, or Ethernet transport. They are additional architecture choices, not substitutes for specifying the underlying carrier service.

5. Four data centers: three real topology choices

Our fictional enterprise has four locations: MIA, DAL, ASH, and SJC. Before selecting a technology, it must choose the required connectivity pattern.

Option 1: Full mesh of point-to-point services

MIA -------- DAL
 | \        / |
 |  \      /  |
 |   \    /   |
 |    \  /    |
 |     \/     |
 |     /\     |
 |    /  \    |
 |   /    \   |
ASH -------- SJC

Six site pairs per carrier:
MIA-DAL, MIA-ASH, MIA-SJC, DAL-ASH, DAL-SJC, ASH-SJC

Four sites have six unique site pairs. Buying one point-to-point service on every pair from each of two carriers results in twelve carrier services. That can provide direct paths, but cost, provisioning effort, and routing-policy complexity grow quickly. The fact that circuits are purchased from two carriers does not by itself establish physical diversity.

Option 2: Hub-and-spoke or partial mesh

        MIA
       / | \
      /  |  \
    DAL ASH  SJC

Illustrative hub-and-spoke: three services per carrier,
six total when duplicated across Lumen and Cogent.

A partial mesh can reduce circuits while retaining redundancy, but spoke-to-spoke traffic may traverse a hub. That affects latency, resiliency, and bandwidth sizing. Dual hubs or selected direct links can improve the tradeoff.

Option 3: Multipoint Ethernet or carrier-managed IP VPN

        MIA              DAL
          \              /
           \            /
         [ Carrier multipoint ]
           /            \
          /              \
        ASH              SJC

L2 variant: E-LAN / VPLS / EVPN-delivered Ethernet
L3 variant: MPLS L3VPN / private routed IP WAN

All four sites connect to a provider service cloud. This avoids ordering a separate E-Line for every possible site pair, but the carrier’s internal topology, routing, SLA, and redundancy design still determine the actual path and failure behavior. To use two carriers, each provider typically supplies its own service domain; how the enterprise interconnects those domains must be engineered deliberately.

6. Side-by-side: which architecture should you evaluate?

7. Does a Layer 3 design automatically make the DCI more stable?

Not automatically. The optical routes, carrier facilities, and transport equipment may be the same whether you buy EPL or an L3VPN. A failure in the shared underlay can affect either service.

The stronger argument is about failure domains and operational simplicity. Imagine dozens of BGP adjacencies, each on a different VLAN, all crossing the same underlying carrier circuit. One transport event could reset all of those sessions. That is many BGP-down messages resulting from one physical incident, not necessarily many independent failures.

Replacing that design with a small number of purposeful routed adjacencies can reduce per-VLAN provisioning, simplify fault correlation, and make routing intent easier to understand. It does not prove that Layer 2 caused previous BGP resets or guarantee subsecond traffic recovery.

Design principle: Use Layer 2 across the DCI only where an application or operational requirement truly needs it. Otherwise, contain Layer 2 locally and route between data centers.

8. What I would ask Lumen and Cogent to quote

Rather than opening with “We need MPLS,” I would send an architectural request that compares several concrete service options:

1. Port-based 100G EPL for selected data-center pairs, with transparent Ethernet carriage and no per-customer-VLAN EVC provisioning.
2. Dedicated 100G wavelength services on the same requested site pairs, including optical termination and protection details.
3. Multipoint Ethernet (E-LAN, VPLS, or EVPN-backed Ethernet, if offered) for all four sites, including multihoming and MAC scale capabilities.
4. Managed MPLS L3VPN / private IP VPN connecting all four sites, including CE-PE BGP and traffic-engineering options.

For every option, require evidence of actual physical-path diversity, end-to-end MTU, 100G availability, latency, loss, SLA, fault propagation, BFD support where applicable, QoS/policing behavior, handoff model, and migration downtime. Ask whether Lumen and Cogent use any common last-mile, building entrance, colocation meet-me room, or underlying transport supplier. Carrier diversity is not automatically path diversity.

Commercial availability is site-specific. The fact that a carrier supports an architecture in its portfolio does not establish that it can deliver it at a particular address, capacity, or SLA.

9. My architectural recommendation

For a modern enterprise DCI whose primary purpose is moving IP traffic between MIA, DAL, ASH, and SJC, I would evaluate customer-managed Layer 3 routing over point-to-point EPL or wavelength transport first. Those options retain direct control of BGP path selection and allow deliberate failure-domain boundaries. The precise number and layout of links should be chosen based on latency, traffic matrices, bandwidth, and resilience requirements—not simply on a preference for full mesh.

I would evaluate managed MPLS L3VPN as a legitimate alternative where the business wants the carrier to operate intersite private routing. I would choose E-LAN, VPLS, or EVPN-based Layer 2 extension only if there is a documented requirement to extend Ethernet adjacency between the sites.

In short, the carrier supplies transport; the architecture determines where Layer 2 ends, where Layer 3 begins, and who is responsible for routing. That separation is the foundation of a scalable, supportable DCI design.

Further reading

• NVIDIA — DCI Reference Design Guide: Common Topologies
• MEF — Carrier Ethernet standards and service definitions
• RFC 4364 — BGP/MPLS IP Virtual Private Networks (VPNs)
• RFC 7432 — BGP MPLS-Based Ethernet VPN
• RFC 7348 — Virtual eXtensible Local Area Network (VXLAN)