Cisco ACI Troubleshooting


#FABRIC MEMBERSHIP

show switch: Provides the information of the entire “switch”. Some times we refer to ACI as a “switch” because of the stateless and distributed architecture ACI has, with this command users can identify every single leaf and spine, and their respective information such as:

-Node IP
-Pod ID
-VTEP address
-In-Band IPv4
-In-Band IPv6
-OOB IPv4
-OOB IPv6
-Version of Code
-Flags
-Serial Number
-Name

show switch

apic1#show switch
ID Pod Address In-Band IPv4 In-Band IPv6 OOB IPv4 OOB IPv6 Version Flags Serial Number Name
---- ---- --------------- --------------- ------------------------- --------------- ------------------------- ------------------ ----- ----------------
------------------
101 1 10.9.200.129 10.0.236.177 :: 10.0.226.37 :: n9000-14.0(2c) asiv FDO221425XC S1
102 1 10.9.216.160 10.0.236.178 :: 10.0.226.38 :: n9000-14.0(2c) asiv FDO221425TL S2
201 1 10.9.216.162 10.0.236.171 :: 10.0.226.31 :: n9000-14.0(2c) aliv SAL1832Y6TL L1
202 1 10.9.240.58 10.0.236.172 :: 10.0.226.32 :: n9000-14.0(2c) aliv SAL1814PTBU L2
203 1 10.9.200.130 10.0.236.173 :: 10.0.226.33 :: n9000-14.0(2c) aliv FDO21280JKY L3
204 1 10.9.88.66 10.0.236.174 :: 10.0.226.34 :: n9000-14.0(2c) aliv FDO21400SS1 L4
205 1 10.9.240.57 10.0.236.175 :: 10.0.226.35 :: n9000-14.0(2c) aliv SAL1813PBJ8 L5
206 1 10.9.240.59 10.0.236.176 :: 10.0.226.36 :: n9000-14.0(2c) aliv SAL1813PBLQ L6
207 1 10.9.88.64 0.0.0.0 :: 0.0.0.0 :: n9000-14.0(2c) aliv FDO211218CD L7
208 1 10.9.200.128 0.0.0.0 :: 0.0.0.0 :: n9000-14.0(2c) aliv FDO21390QEL L8

Flags - a:Active | l/s:Leaf/Spine | v:Valid Certificate | i:In-Service

#LOCATING AN ENDPOINT

show endpoints ip 10.0.144.27

apic1#show endpoints ip 10.0.144.27
Legends:
(P):Primary VLAN
(S):Secondary VLAN


Dynamic Endpoints:
Tenant : aci_p04_tenant
Application : aci_p04_ap
AEPg : aci_p04_epg_web

End Point MAC IP Address Node Interface Encap Multicast Address
----------------- ---------------------------------------- ---------- ------------------------------ --------------- ---------------
00:50:56:02:01:05 10.0.144.27 207 208 aci_p04_intpolg_vpc vlan-551 not-applicable

Total Dynamic Endpoints: 1
Total Static Endpoints: 0

The next step is to identify which port(s) are part of this vPC, since the show endpoints ip have already provided the vPC Policy Group aci_p04_intpolg_vpc. Let’s identify the ports by doing the following command.

show vpc map aci_p04_intpolg_vpc

apic1#show vpc map aci_p04_intpolg_vpc

Legends:
N/D : Not Deployed


Virtual Port-Channel Name Domain Virtual IP Peer IP VPC Leaf Id, Name Fex Id PC Id Ports

-------------------------------- ------ ---------------- ---------------- ----- -------------------------------- ----- ------ ---------------
-----
aci_p04_intpolg_vpc 78 10.9.104.67/32 10.9.88.64/32 344 207,L7 po4 eth1/4 
aci_p04_intpolg_vpc 78 10.9.104.67/32 10.9.200.128/32 344 208,L8 po4 eth1/4 

As you can see the port members are eth1/4 in Leaf 207 and eth1/4 on Leaf 208.

Now you can look at the physical interface now that you ahve the Leaf # and Port:

fabric 207 show interface eth1/4

apic1# fabric 207 show interface eth1/4
----------------------------------------------------------------
Node 207 (L7)
----------------------------------------------------------------
Ethernet1/4 is up
admin state is up, Dedicated Interface
Belongs to po4
Hardware: 1000/10000/25000/auto Ethernet, address: 00a3.8ebf.fef3 (bia 00a3.8ebf.fef3)
MTU 9000 bytes, BW 10000000 Kbit, DLY 1 usec
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, medium is broadcast

#ACI – VMWARE INTEGRATION

show vmware domain name aci_p04_dc3_vds vm name POD04-WEB-SRV-02

apic1# show vmware domain name aci_p04_dc3_vds vm name POD04-WEB-SRV-02
VM Name       : POD04-WEB-SRV-02
DVS           : aci_p04_dc3_vds
vCenter       : 10.0.226.193
Host          : pod04-compute1.ecatsrtpdmz.cisco.com
Guest OS      : CentOS 7 (64-bit)
Configured OS : CentOS 7 (64-bit)
VM OID        : vm-7651
VM GUID       : 500c648e-ae86-9753-4147-45345076d111
Power State   : poweredOn
        
Virtual Nics:
        
Name         : Network adapter 2
Type         : Vmxnet3
MAC          : 00:50:56:02:01:05
IP           : 10.0.144.27
State        : up
Switch       : aci_p04_dc3_vds
Port Group   : aci_p04_tenant|aci_p04_ap|aci_p04_epg_web
Encap        : vlan-551
PrimaryEncap : --
Adjacency    : leafNone aci_p04_intpolg_vpc
        
Name         : Network adapter 1
Type         : Vmxnet3
MAC          : 00:50:56:02:00:05
IP           : 10.0.145.26
State        : up
Switch       : aci_p04_dc3_vds
Port Group   : aci_p04_tenant|aci_p04_ap_mgmt|aci_p04_epg_mgmt
Encap        : vlan-550
PrimaryEncap : --
Adjacency    : leafNone aci_p04_intpolg_vpc

#ROUTING COMMANDS

fabric 207 show ip route vrf overlay-1

Check the Underlay (overlay-1) routing table:

apic1# fabric 207 show ip route vrf overlay-1         
----------------------------------------------------------------
Node 207 (L7)
----------------------------------------------------------------
IP Route Table for VRF "overlay-1"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
in via output denotes VRF
        
3.0.0.102/32, ubest/mbest: 1/0
    *via 10.9.216.160, eth1/49.15, [115/2], 01w06d, isis-isis_infra, L1
10.9.0.0/27, ubest/mbest: 1/0, attached, direct
    *via 10.9.0.30, vlan13, [1/0], 02w06d, direct
10.9.0.1/32, ubest/mbest: 2/0
    *via 10.9.216.160, eth1/49.15, [115/12], 02w06d, isis-isis_infra, L1
    *via 10.9.200.129, eth1/50.16, [115/12], 02w05d, isis-isis_infra, L1

Check the routing table for aci_p04_tenant

fabric 207 show ip route vrf aci_p04_tenant:aci_p04_vrf

apic1# fabric 207 show ip route vrf aci_p04_tenant:aci_p04_vrf
----------------------------------------------------------------
    Node 207 (L7)
----------------------------------------------------------------
IP Route Table for VRF "aci_p05_tenant:aci_p05_vrf"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%' in via output denotes VRF 
        
0.0.0.0/0, ubest/mbest: 1/0
    *via 10.9.200.130%overlay-1, [200/1], 08:39:07, bgp-65503, internal, tag 65503
1.1.1.1/32, ubest/mbest: 1/0
    *via 10.9.200.130%overlay-1, [200/13], 08:39:07, bgp-65503, internal, tag 65503
10.0.0.5/32, ubest/mbest: 1/0
    *via 10.9.200.130%overlay-1, [1/0], 08:39:07, bgp-65503, internal, tag 65503
10.0.4.0/31, ubest/mbest: 1/0
    *via 10.9.200.130%overlay-1, [200/12], 08:39:07, bgp-65503, internal, tag 65503
10.0.5.0/31, ubest/mbest: 1/0
    *via 10.9.200.130%overlay-1, [200/0], 08:39:07, bgp-65503, internal, tag 65503

Then you can check OSPF as well:

fabric 203 show ip ospf neighbors vrf aci_p04_tenant:aci_p04_vrf

apic1#fabric 203 show ip ospf neighbors vrf aci_p04_tenant:aci_p04_vrf 
Total number of neighbors: 1
Neighbor ID     Pri State            Up Time  Address         Interface
10.0.4.1          1 FULL/DR          23:54:20 10.0.4.1        Eth1/4

Source:
https://aci-lab.ciscolive.com/lab/pod4/aci/aci-trouble

More Stories
Cisco UCS Best Practices