Cisco ACI Multi-Pod vs Multi-Site

Fabric and Policy Domain Evolution:

ACI 1.0:
Single ACI Fabric
Single Leaf/Spine where all switches are in the same site
Managed by a single APIC cluster

ACI 1.1: Added support for stretched Fabric
Also a single Leaf/Spine topology
Leaf/Spine could be located in different sites connected over partial mesh configuration
Also managed by a single APIC Cluster

ACI 2.0: Support for Multi-iPod
Each pod is its own Leaf/Spin topology
All pods are connected over a layer 3 network
Also managed by a single APIC cluster making it a single change in the MGMT domain

Advantages of Multi-Pod over Stretched Fabric:
Higher scale
Fault Domain Isolation

ACI 3.0: Support for Multi-Site
Connects multiple ACI fabrics
Each managed by their own APIC Clusters over an IP network similar to Multi-Pod
Provides Layer 2 and Layer 3 extension across multiple sites
Allows to MGMT tenant policies across multiple sites using the ACI multi-site policy manager.

 

Data plane — Handles all the data traffic. The basic functionality of a Cisco NX-OS device is to forward packets from one interface to another. The packets that are not meant for the switch itself are called the transit packets. These packets are handled by the data plane.

Control plane — Handles all routing protocol control traffic. These protocols, such as the Border Gateway Protocol (BGP) and the Open Shortest Path First (OSPF) Protocol, send control packets between devices. These packets are destined to router addresses and are called control plane packets.

 

Multi-Pod:

aci1

  • Consists of multiple Pods
  • Each Pod is a Leaf/Spine Topology
  • Spine switches connect to an IP or Inter-Pod network
  • The Inter-Pod network is a Layer 3 network that provides connectivity between Pods and used to transport control plane and data plane traffic between the Pods
  • Multi-Pod fabric provides fault isolation for control plane protocols
  • A separate instance of IS-IS, COOP, and MP-BGP is running in each Pod
  • Managed by a single APIC Cluster
  • Single management and Policy Domain
  • Single Availability Zone
  • Uses MP-BGP – EVPN address family as a control plane protocol for advertising end point reachability across the entire Multi-Pod fabric
  • Tenant traffic is carried in VXLAN across the IP or Inter-Pod network
  • VXLAN also carries the end-to-end policy information and allows policy enforcement across the entire Multi-Pod fabric
  • Consists of multiple ACI fabrics
  • Each their own leaf/spine topology
  • Connected over an IP network
  • IP network provides connectivity between sites and used to transport control plane traffic, BGP-EVN, and data plane Traffic VXLAN between sites
  • Very similar to the Multi-Pod topology
  • One difference different to note between Multi-Pod and multisite – in ACI Multi-Pod fabric the IP network must support bidirectional PIM Multicast traffic which is used to transport BUM (Broadcast, Unknown Unicast, & Multicast) traffic between Pods. This is NOT a requirement in Multi-Site.  Multi-site uses head end replication on the Spines to transport BUM traffic between sites.
  • Each site is an independent ACI Fabric managed with independent APIC clusters which makes each site an Availability Zone

 

Multi-Site:

aci2

  • MP-BGP EVPN is used as the control plane protocol between sites for advertising end-point reachability
  • VXLAN is as the data plane protocol for carrying tenant traffic and policy information across sites
  • BGP-EVPN and VXLAN are used in both Multi-Pod and Multisite to allow Layer 3 and Layer 2 extension across sites
  • This simplifies the IP configuration by eliminated the need for other interconnect data center technologies such as Layer 3 VPN or OTV
  • Provides a single point for management to manage tenant polices that are extended across multiple sites
  • Pushes cross fabric configurations to multiple APIC clusters and provides scoping for all configuration changes by providing control over which tenant policies are extended and to which sites
  • Both multi-site and Multi-Pod provide L2 and L3 extension across multiple locations with consistent policy
  • ACI-Multi-Pod is commonly deployed as a single availability zone with operation simplicity as all polices are managed from Single APIC cluster for the entire Multi-Pod fabric
  • ACI-Multisite is commonly deployed as multiple availability zones with control over what policies are deployed and to where

A good comparison from Cisco:

Multi-Pod_vs_Multi-Site_Infographic