Underlay vs Overlay Networks

In IT networking, overlay networks and underlay networks refer to different layers or perspectives of network architecture. Here’s an explanation of the differences between overlay networks and underlay networks:

Underlay Networks:

Underlay networks refer to the physical or foundational network infrastructure that provides connectivity between network devices. It encompasses the physical layer and the underlying network protocols, such as Ethernet, IP, and routing protocols. The underlay network is responsible for the transport of data packets across the network.

Key characteristics of underlay networks include:

  • Physical connectivity: Underlay networks define the physical connections, cabling, and network devices (routers, switches) that form the network infrastructure.
  • Routing: Underlay networks use routing protocols, such as OSPF (Open Shortest Path First) or BGP (Border Gateway Protocol), to determine the best path for data packets to reach their destination.
  • Network scalability: Underlay networks focus on the scalability and efficiency of network transport, ensuring reliable and fast data transmission.

Overlay Networks:

Overlay networks, on the other hand, are virtual networks that are built on top of the underlying physical infrastructure (underlay network). Overlay networks provide logical or virtual network abstractions, decoupled from the physical infrastructure, to enable specific functionalities or services.

Key characteristics of overlay networks include:

  • Virtualization: Overlay networks abstract the underlying physical network by creating virtual networks or tunnels on top of it. These virtual networks are independent of the physical network topology and can be dynamically created or modified.
  • Service specific: Overlay networks are often used to enable specific services or functionalities such as virtual private networks (VPNs), software-defined networking (SDN), network segmentation, or multi-tenancy.
  • Overlay protocols: Overlay networks use encapsulation techniques and protocols, such as Virtual Extensible LAN (VXLAN), Generic Routing Encapsulation (GRE), or MPLS (Multiprotocol Label Switching), to create virtual connections between network endpoints.

Overlay networks provide flexibility, scalability, and isolation in deploying specific network services or architectures without requiring changes to the underlying physical infrastructure.

To summarize, underlay networks represent the physical infrastructure and protocols that provide basic connectivity and transport capabilities, while overlay networks provide virtual abstractions and specific services layered on top of the underlay network. Overlay networks enable network virtualization and the implementation of specialized functionalities or services without altering the physical network infrastructure.

Cisco ACI:

Cisco ACI (Application Centric Infrastructure) is an example of a software-defined networking (SDN) solution that incorporates both overlay and underlay networks to provide a comprehensive network architecture. ACI aims to simplify network management, improve agility, and enhance scalability in modern data center environments.

In Cisco ACI, the underlay network refers to the physical infrastructure that provides basic connectivity between networking devices, such as switches and routers. It typically consists of traditional networking protocols like Ethernet and IP. The underlay network is responsible for forwarding packets and establishing the foundation for communication within the data center.

On the other hand, the overlay network in Cisco ACI is built using a technology called VXLAN (Virtual Extensible LAN). VXLAN enables the creation of virtual network overlays on top of the physical underlay network. It allows the logical segmentation of network traffic and facilitates the deployment of network services without the need for manual configuration on individual networking devices.

To better understand how Cisco ACI utilizes overlay and underlay networks, let’s consider an example design:

  1. Underlay Network:
  • Physical switches: Cisco Nexus switches are deployed as the underlay network devices, providing high-speed connectivity between servers, storage systems, and other network devices.
  • Routing protocols: Underlay network uses standard routing protocols like OSPF (Open Shortest Path First) or BGP (Border Gateway Protocol) to establish connectivity and exchange routing information between switches.
  1. Overlay Network:
  • VXLAN encapsulation: Cisco ACI leverages VXLAN technology to create virtual network overlays. Each overlay network is associated with a unique VXLAN identifier (VNI), which provides segmentation and isolation.
  • Virtual Ports: ACI uses virtual ports, called “endpoints” or “endpoint groups” (EPGs), to define network policies and group related workloads together.
  • Virtual Switches: ACI employs distributed virtual switches, called “Application Virtual Switches” (AVS), which are integrated with hypervisors like VMware ESXi or Microsoft Hyper-V. These switches provide connectivity and policy enforcement at the virtual machine level.
  1. ACI Fabric:
  • Spine and Leaf Architecture: ACI utilizes a spine-and-leaf fabric architecture, where spine switches connect to multiple leaf switches. The leaf switches, in turn, provide connectivity to servers and other network devices.
  • Spine switches: These switches serve as the backbone of the fabric, providing high-bandwidth connectivity and handling traffic forwarding between leaf switches.
  • Leaf switches: Leaf switches connect to the endpoints (servers or virtual machines) and act as the access layer within the fabric. They provide connectivity, policy enforcement, and traffic forwarding within the ACI fabric.
  1. ACI Controller:
  • Application Policy Infrastructure Controller (APIC): ACI is managed by the APIC, which acts as a central controller for the entire fabric. It provides a single point of management and policy definition.
  • Policy-Based Management: APIC allows administrators to define and enforce network policies centrally. These policies can be based on application requirements, security rules, or quality of service (QoS) parameters. APIC then translates these policies into configuration instructions for the underlay and overlay networks.

In summary, Cisco ACI combines the underlay network for physical connectivity and the overlay network (based on VXLAN) for logical segmentation and policy enforcement. The underlay network provides the foundation for communication, while the overlay network offers virtualization, policy-driven management, and enhanced agility within the data center environment.