Home
Data Center
Cisco ACI Auto-Negotiation and Forward Error Correction

Cisco ACI Auto-Negotiation and Forward Error Correction

Kerry Cordero
Data Center
15 min read

New and Changed Information

The following table provides an overview of the significant changes up to the current release. The table does not provide an exhaustive list of all changes or of the new features up to this release.

Table 1. New Features and Changed Behavior in Cisco APIC

Cisco APIC Release

Feature

Description

5.1(3)

Additional auto-negotiation support

All of the switches that support auto-negotiation support additional speeds in the on-enforce mode. The Cisco N9K-C93180YC-EX and N9K-C9364C-GX switches now support the on-enforce mode.

Auto-negotiation and forwarding error correction is now supported on 4x10G and 4x25G breakout ports.

5.0(1)

Auto-negotiation

Auto-negotiation enables devices to automatically exchange link information about speed and duplex abilities.

AUTO-FEC

AUTO-FEC is supported for 10 to 100 Gbps ports.

KP-FEC

KP-FEC is supported for 4×100 and 400 Gbps ports.

Additional switches

Additional switches support FEC.

2.2(2e)

RS-FEC

RS-FEC is supported for 25 Gbps ports.

1.3(1g)

Forwarding Error Correction (FEC) was introduced.

—

Auto-Negotiation

About Auto-Negotiation

Auto-negotiation is an optional function of the IEEE 802.3u Fast Ethernet standard that enables devices to automatically exchange information over a link about speed and duplex abilities. ACI switches support auto-negotiation on these port types:

  • 10/25/40/100Gbps on copper passive cables

  • 10Gbps native copper ports

You can select from the following auto-negotiation modes:

  • off: Auto-negotiation is disabled.

  • on: Auto-negotiation is enabled on RJ45 ports. This is the default.

  • on-enforce: Auto-negotiation is enabled on copper passive cables. This mode is available beginning with Cisco ACI Release 5.0(1).

Auto-negotiation includes the auto-negotiation protocol exchange and link training. 10/40/100Gbps ports support fixed speed auto-negotiation. Ports with 25Gbps direct-attach-cable (DAC) copper cables support speed auto-negotiation, in which the switch advertises both 10Gbps and 25Gbps speeds. In this case, the negotiated speed is the fastest common speed advertised from both peers.

Ports with 25Gbps DAC copper cables support negotiation of forwarding error correction (FEC) as part of the auto-negotiation protocol exchange. Switches advertise their supported FEC modes, and both peers agree upon the best available mode.

Guidelines and Restrictions for Auto-Negotiation

  • When auto-negotiation is enabled, it should be enabled on both peers – the ACI switch and the link partner.

  • Auto-negotiation is supported on 4x10Gbps and 4x25Gbps passive (but not active) breakout cables, such as QSFP100G-4SFP25G-CU3M or QSFP40G-4SFP10G-CU3M.

  • In the 5.1(2) release and earlier, the on-enforce mode is not supported on N9K-C93180YC-EX, N9K-C93600CD-GX (ports 1-24), and N9K-C9364C-GX (ports 1-64).

  • Auto-negotiation is not supported with any of the QSFP-4x10G-AC transceivers on active copper cables.

Switch Support for Auto-Negotiation

Table 2. Support in release 5.1(2) and earlier
Switch AN mode OFF AN mode ON AN On-Enforce mode AN On-Enforce supported speed

N9K-C93180YC-EX

Off

Off

Not supported

N/A

N9K-C93180LC-EX

Off

On

Same as AN ON

100G and 40G passive copper cables

N9K-C93108TC-EX

Off

On

Same as AN ON

1G, 10G, and 100M

N9K-C9348GC-FXP

Off

Off

On

10G and 25G copper cables

N9K-C93108TC-FX

Off

On

Same as AN ON

1G, 10G, and 100M

N9K-C93180YC-FX

Off

Off

On

10G and 25G copper cables

N9K-C9336C-FX2

Off

Off

On

100G and 40G passive copper cables

N9K-C93240YC-FX2

Off

Off

On

10G and 25G copper cables

N9K-C93600CD-GX

Off

Off

Not supported on ports 1-24, supported on ports 25-34

100G, 40G, 4x10G, and 4x25G passive copper cables

N9K-C93360YC-FX2

Off

Off

On

10G and 25G copper cables

N9K-C93216TC-FX2

Off

On

Same as AN ON

1G, 10G, and 100M

N9K-C9364C-GX

Off

Off

Not supported

N/A
Table 3. Support in release 5.1(3) and later
Switch AN mode OFF AN mode ON AN On-Enforce mode AN On-Enforce supported speed

N9K-C93180YC-EX

Off

Off

On

10G and 25G

N9K-C93180LC-EX

Off

On

Same as AN ON

40G, 100G, 4x10G, and 4x25G passive copper cables

N9K-C93108TC-EX

Off

On

Same as AN ON

100M, 1G, 10G, 4x10G, and 4x25G

N9K-C9348GC-FXP

Off

Off

On

10G, 25G, 4x10G, and 4x25G copper cables

N9K-C93108TC-FX

Off

On

Same as AN ON

100M, 1G, 10G, 4x10G, and 4x25G

N9K-C93180YC-FX

Off

Off

On

10G, 25G, 4x10G, and 4x25G copper cables

N9K-C9336C-FX2

Off

Off

On

10G, 40G, 100G, 4x10G, and 4x25G passive copper cables

N9K-C93240YC-FX2

Off

Off

On

10G and 25G, 4x10G, and 4x25G copper cables

N9K-C93600CD-GX

Off

Off

On

10G, 40G, 100G, 4x10G, and 4x25G passive copper cables

N9K-C93360YC-FX2

Off

Off

On

10G, 25G, 4x10G, and 4x25G copper cables

N9K-C93216TC-FX2

Off

On

Same as AN ON

100M, 1G, 10G, 4x10G, and 4x25G

N9K-C93108TC-FX3P

Off

On

Supported on ports 1/49 to 52

10G, 40G, 100G, 4x10G, and 4x25G

N9K-C9364C-GX

Off

Off

On

10G, 40G, 100G, 4x10G, and 4x25G passive copper cables

Note

In the 3.2 and later releases, Auto-negotiation is always turned on for the GLC-TE optic on Cisco Application Centric Infrastructure (ACI), irrespective of the user configuration in the link level policy.

Forwarding Error Correction

About Forwarding Error Correction

Forwarding Error Correction (FEC) is a method of obtaining error control in data transmission over an unreliable or noisy channel in which the source (transmitter) encodes the data in a redundant way using Error Correcting Code, and the destination (receiver) recognizes it and corrects the errors without requiring a retransmission. The supported FEC modes are as follows:

  • AUTO-FEC: The switch uses the best FEC mode.

    In AUTO-FEC mode, the retimer ports of specific switches will be in the FC-FEC mode, as IEEE-RS-FEC is not supported in FEC advertising on these ports of those switches. As a result, you can use a cable of only up to 3 meters. A cable that is longer than 3 meters carries the risk of introducing CRC errors with FC-FEC. See the Ports of Switches That Do Not Support AUTO-FEC and IEEE-RS-FEC table for the affected switches and ports.

  • CL74-FC-FEC: Supports 25 Gbps speed.

  • CL91-RS-FEC: Supports 25 and 100 Gbps speeds.

  • CONS16-RS-FEC: Supports 25 Gbps speed.

  • IEEE-RS-FEC: Supports 25 Gbps speed.

    This mode is not supported on the retimer ports of specific switches when used with cables that are greater than 3 meters in length. See the Ports of Switches That Do Not Support AUTO-FEC and IEEE-RS-FEC table for the affected switches and ports.

  • KP-FEC: Supports 4×100 Gbps (KP1) or 400 Gbps (KP4).

  • Disable-FEC: Disables FEC.

  • Inherit: The switch uses FEC based on the port transceiver type. This is the default port configuration. See the Default FEC Support for Switches section for the inherited FEC mode optic.

The default is “Inherit.”

Table 4. Ports of Switches That Do Not Support AUTO-FEC and IEEE-RS-FEC

Switch

Ports

N9K-C9336C-FX2

1-6, 33-36

N9K-C93360YC-FX2

97-108

N9K-C93216TC-FX2

97-108

N9K-C93240YC-FX2

49-60

N9K-C93180YC-FX3S

1-54

N9K-C9364C-GX

1-64

N9K-C93600CD-GX

1-24

Note

FEC is configurable only on the front port and not on fabric ports.

Switch Support for FC-FEC

 

Switch

Details

N9K-93180YC-FX

Supported for 25Gbps ports

N9K-93180YC-EX

Supported for 25Gbps ports

N9K-C9348GC-FXP

Supported for 25Gbps ports

N9K-C93240YC-FX2

Supported for 25Gbps ports

N9K-C93360YC-FX2

Supported for 25Gbps ports

N9K-C93600CD-GX

Supported for 25Gbps ports

N9K-C9364C-GX

Supported for 25Gbps ports

Switch Support for CL91-RS-FEC

 

Switch

Details

1

N9K-C93180YC-FX

Supported for 100G speed

N9K-C93180YC-EX

Supported for 100G speed

N9K-C93108TC-FX

Supported for 100G speed

N9K-C93108TC-EX

Supported for 100G speed

N9K-C93180LC-EX

Supported for 100G speed

N9K-C9364C

Supported for 100G speed

N9K-C9348GC-FXP

Supported for 100G speed

N9K-C9336C-FX2

Supported for 100G speed

N9K-C93240YC-FX2

Supported for 100G speed

N9K-C9332C

Supported for 100G speed

N9K-C93360YC-FX2

Supported for 100G speed

N9K-C93216TC-FX2

Supported for 100G speed

N9K-C9316D-GX

Supported for 100G speed

N9K-C93600CD-GX

Supported for 100G speed

N9K-C9364C-GX

Supported for 100G speed

N9K-X9732C-EX (LC)

Supported for 100G speed

N9K-X9736C-FX (LC)

Supported for 100G speed
1 For QSFP-100G-LR4-S / QSFP-100G-LR4 transceivers, FEC is disabled by default.

Switch Support for CONS16-RS-FEC and IEEE-RS-FEC

 

Switch

Details

N9K-C93180YC-FX

Supported for 25G speed

N9K-C9348GC-FXP

Supported for 25G speed

N9K-C93240YC-FX2

Supported for 25G speed

N9K-C93360YC-FX2

Supported for 25G speed

N9K-C93600CD-GX

Supported for 25G speed, but only for IEEE-RS-FEC and only on ports 25 to 34

Default FEC Support for Switches

Table 5. EX Switches

Transceiver

Auto-FEC

Inherit

10G

Disable FEC

Disable FEC

25G-CU1,2,3M

FC-FEC (CL74)

FC-FEC (CL74)

25G-CU5M

IEEE-RS-FEC1

FC-FEC (CL74)

25G-optical

FC-FEC (CL74)2

FC-FEC (CL74)

40G

Disable FEC

Disable FEC

100G

RS-FEC (CL91)3

RS-FEC (CL91)3

1 On the Cisco N9K-C93180YC-EX switch and platforms with a retimer port, the FEC type is FC-FEC (CL74).

2 With the SFP-25G-SR and SFP-25-10G-LR transceivers on switches other than the N9K-C93180YC-EX switch, the FEC type is IEEE-RS-FEC.

3 With the QSFP-100G-LR, QSFP-100G-FR, QSFP-100G-DR, and QSFP-40/100G-SRBD transceivers, FEC is disabled.

Table 6. FX Switches and Later

Transceiver

Auto-FEC

Inherit

10G

Disable FEC

Disable FEC

25G-CU1,2,3M

FC-FEC (CL74)

FC-FEC (CL74)

25G-CU5M

IEEE-RS-FEC1

FC-FEC (CL74)

25G-optical

FC-FEC (CL74)2

FC-FEC (CL74)

40G

Disable FEC

Disable FEC

100G

RS-FEC (CL91)3

RS-FEC (CL91)3

1 On the Cisco N9K-C93180YC-EX switch and platforms with a retimer port, the FEC type is FC-FEC (CL74).

2 With the SFP-25G-SR and SFP-25-10G-LR transceivers on switches other than the N9K-C93180YC-EX switch, the FEC type is IEEE-RS-FEC.

3 With the QSFP-100G-LR, QSFP-100G-FR, QSFP-100G-DR, and QSFP-40/100G-SRBD transceivers, FEC is disabled.

Configuration

Configuring Auto-Negotiation Using the GUI

Procedure

Step 1

On the menu bar, click Fabric > Access Policies.
Step 2

In the Navigation pane, expand Policies > Interface
Step 3

Right click Link Level and click Create Link Level Policy.
Step 4

In the Create Link Level Policy dialog box, perform the following actions:

  1. In the Name field, enter a name for the policy.

  2. In the Auto Negotiation control, choose the desired auto-negotiation mode.

    The following modes are available:

    • off: Auto-negotiation is disabled.

    • on: (Default) Auto-negotiation is enabled on RJ45 ports.

    • on-enforce: Auto-negotiation is enabled on copper passive cables.

  3. Enter or choose the desired options for the remaining fields in the dialog box.

  4. Click Submit.

Configuring Auto-Negotiation Using the NX-OS Style CLI

This example shows how to configure auto-negotiation on a leaf interface.

Procedure

Step 1

Enter the configure mode.

Example:

apic1# configure
Step 2

Enter the switch mode.

Example:

apic1(config)# leaf 104
Step 3

Specify the interface and port.

Example:

apic1(config-leaf)# int eth 1/4
Step 4

Configure auto-negotiation on the interface.

The options are:

  • no negotiate auto to disable auto-negotiation

  • negotiate auto to enable auto-negotiation

  • negotiate on-enforce to enable auto-negotiation on copper passive cables

Note
The default auto-negotiation mode is negotiate auto.

Example:

apic1(config-leaf-if)# negotiate on-enforce
Step 5

Exit the interface mode.

Example:

apic1(config-leaf-if)# exit

Configuring FEC Using the GUI

Procedure

Step 1

On the menu bar, click Fabric > Access Policies.
Step 2

In the Navigation pane, expand Policies > Interface
Step 3

Right click Link Level and click Create Link Level Policy.
Step 4

In the Create Link Level Policy dialog box, perform the following actions:

  1. In the Name field, enter a name for the policy.

  2. In the Forwarding Error Correction field, choose the desired FEC value.

    The default is Inherit.

  3. Enter or choose the desired options for the remaining fields in the dialog box.

  4. Click Submit.

The Link Level Policy is created, and the desired FEC is configured with it.
Step 5

To verify the configuration, in the Navigation pane, click Policies > Link Level . The Work pane, lists all the policies.
Step 6

In the Work pane, choose the appropriate Link Level Policy, and verify in the Forwarding Error Correction column that the desired FEC is displayed.

Configuring FEC Using the NX-OS Style CLI

This example shows how to configure forward error correction (FEC) on a leaf switch interface.

Procedure

Step 1

Enter the configure mode.

Example:

apic1# configure
Step 2

Enter the switch mode.

Example:

apic1(config)# leaf 104
Step 3

Specify the interface and port.

Example:

apic1(config-leaf)# int eth 1/4
Step 4

Configure FEC.

The options are:

  • auto-fec

  • cl74-fc-fec

  • cl91-rs-fec

  • cons16-rs-fec

  • disable-fec

  • ieee-rs-fec

  • inherit

  • kp-fec

Note
The default forward-error-correction value is inherit.

Example:

apic1(config-leaf-if)# forward-error-correction cl91-rs-fec
Step 5

Exit the interface mode.

Example:

apic1(config-leaf-if)# exit

Configuring FEC Using the REST API

Before you begin

The interface selector is configured.

Procedure

Configure RS-FEC policy in a link level policy.

Example:


<!-- api/policymgr/mo/.xml -->
<polUni>
<infraInfra dn="uni/infra">
    <infraFuncP>
        <infraAccPortGrp name="25G-RS-FEC-PG">
            <infraRsHIfPol tnFabricHIfPolName="RS-FEC-POL"/>
        </infraAccPortGrp>
    </infraFuncP>
    <fabricHIfPol name="RS-FEC-POL" speed="inherit" fecMode="cl91-rs-fec"/>
</infraInfra>
</polUni>

The options for fecMode are:

  • auto-fec

  • cl74-fc-fec

  • cl91-rs-fec

  • cons16-rs-fec

  • disable-fec

  • ieee-rs-fec

  • inherit

  • kp-fec

 

Source:

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/kb/b_Cisco_ACI_and_Forward_Error_Correction.html

© 2024 cordero.me • All Rights Reserved