Underlay vs Overlay Networks



Underlay Network Overlay Network
The Underlay Network is the physical infrastructure on which the Overlay Network is built on top of. The Overlay Network is a virtual network built on top of an Underlying Network infrastructure / Network Layer (the underlay).
Related Protocols Ethernet Switching, VLAN, Routing, Etc. VXLAN, OTV, VPLS, EVPN
Scalability Less Scalable due to technology limitations They are designed to provide more scalability than the underlay network.  For example, VLAN (Underlay Network) provides 4096 VLANs while VXLAN (Overlay Network) provides up to 16 million identifiers.
Packet Control Hardware Orchestrated Software Orchestrated
Packet Delivery Responsible for delivery of packets Offloaded from delivery of packets
Packet Encapsulation and Overhead Packet delivery and reliability occur at layer 3 and layer 4 Need to encapsulate packets across the source and destination hence incurs additional overhead
Managing Multitenancy NAT or VRF-based segregation is required, which may face a challenge in big environments Ability to manage overlapping IP addresses between multiple tenants
Multipath Forwarding Less scalable options of multipath forwarding.  Using multiple paths can have associated overhead and complexity. Support for multipath forwarding within virtual networks
Deployment Time Less scalable and time-consuming activity to set up new services and functions. Ability to rapidly and incrementally deploy new functions through edge-centric innovations
Traffic Flow Transmits packets that traverse over network devices like switches and routers Transmits packets only along the virtual links between overlay nodes