A thought-provoking question surfaced on a social media platform recently, sparking a lively discussion:
“You have two L3 devices. You want to run a routing protocol like OSPF or BGP between them. You need multiple links for load sharing. Would you do:
1. L2 bundle with SVIs.
2. L3 bundle with routed ports.
3. Routed links with ECMP.”
Now, let’s dive into the discussion and share our insights. I’ll start by providing my answers below.
When it comes to connecting two Layer 3 (L3) devices and enabling routing protocols such as OSPF or BGP, especially with load sharing across multiple links, it’s essential to meticulously evaluate the options. Factors such as scalability, performance, ease of management, and redundancy should be taken into account. Let’s delve into three approaches and determine the most suitable one for this scenario.
Option 1: L2 Bundle with Switched Virtual Interfaces (SVIs)
This approach bundles multiple physical links between the devices at Layer 2, forming a single logical link. Switched Virtual Interfaces (SVIs) are configured on this bundle to provide Layer 3 connectivity.
Pros:
- Simplifies physical connectivity and VLAN management
- Single IP subnet across all links in the bundle
Cons:
- It relies on Spanning Tree Protocol (STP) to prevent loops, introducing potential delays and complexity
- L2 domains can be prone to broadcast storms, and larger failure domains
- Performance may be affected by the hashing algorithm’s ability to distribute traffic across bundle links
Use Case:
This is suitable when VLAN extension between switches is required and routing needs to be kept simple.
Option 2: L3 Bundle with Routed Ports
This setup involves creating a bundle at Layer 3, typically using protocols like LACP (Link Aggregation Control Protocol). The bundled links are then treated as a single-routed interface.
Pros:
- Efficient utilization of available bandwidth in the bundle
- Eliminates STP concerns as there are no L2 loops
Cons:
- Higher configuration complexity as both ends must support and be configured for L3 bundling
- Requires routing hardware that supports L3 bundling, which may not be available on all devices
Use Case:
Ideal for high-throughput scenarios where maximum bandwidth utilization and redundancy are critical and connected devices support L3 bundles.
Option 3: Routed Links with Equal-Cost Multi-Path (ECMP)
With ECMP, multiple parallel routed links are used for redundancy and load sharing. Each link is treated as a separate route, and routing protocols like OSPF or BGP can distribute traffic across these links based on routing metrics.
Pros:
- Provides robustness and fault isolation
- Scales easily by adding more links
- Widely supported by modern routing platforms
- Allows for granular control over routing policies and traffic engineering
Cons:
- Requires more IP subnetting work as each link will typically be in its subnet
- Monitoring and managing multiple individual links can be more complex
Use Case:
This solution is best for environments with crucial goals for robustness, scalability, and optimal use of available paths.
Recommendation:
Given the need for load sharing and assuming that robustness, scalability, and optimal use of available paths are critical, using Routed Links with ECMP (Option 3) is not just a good choice, it’s the best. It provides excellent fault isolation, scales easily by adding more links, and is supported by virtually all modern routing platforms. This option also allows for the most flexibility in traffic engineering, as each link can be controlled and routed independently. Additionally, it avoids the potential pitfalls of large L2 domains and the complexity of configuring L3 bundles, making it a versatile and reliable choice for connecting large networks and enabling routing protocols with load sharing across multiple links.