Two ISPs and an Equinix IX: what the route servers and collectors actually do

Two transit ISPs on an Internet edge make sense to most network engineers: redundancy, capacity, and some influence over how traffic enters and leaves the network. Then you open the BGP configuration and find more neighbors. There are sessions to Equinix route servers and a route collector, and your public prefixes appear in their advertisements too. Someone calls it the “Akamai IX.”

The obvious questions follow. Doesn’t Internet traffic come in through our ISPs? Are the Equinix peers just collecting information? If we’re prepending toward Cogent to prefer Lumen, why is traffic still arriving over the IX?

The answers come down to one fundamental distinction: BGP exchanges reachability information, but the forwarding table determines where packets actually go. Let’s walk through the design and then look at a troubleshooting mistake it can create.

The setup: two ISPs and one IX port

Assume our Internet edge has three external connections:

• Lumen: an Internet transit provider, carrying traffic to destinations beyond our direct peers.
• Cogent: a second Internet transit provider, used for redundancy and potentially additional capacity.
• Equinix Internet Exchange (IX): an Ethernet connection to a shared exchange fabric where participating autonomous systems can exchange traffic directly.

The IX isn’t Akamai’s network. Equinix operates the exchange, and Akamai may be one of many participants alongside other content networks, cloud providers, SaaS platforms, and ISPs. Networks that establish peering with us—either bilaterally or through the IX route servers—can exchange production packets over that fabric when their routing policies select those paths.

That means the IX port is a genuine third forwarding path, not a monitoring connection. It is not, however, a third full-Internet transit service. IX peering normally provides reachability to the prefixes advertised by participating peers, not a default route to the entire Internet.

Route servers and route collectors: control plane, not data plane

Seeing BGP neighbors on an IX doesn’t mean packets transit those neighbor devices. Two types of BGP service are often confused:

Think of an IX route server as a route-distribution service rather than a transit router. When it advertises an Akamai prefix to our router, the BGP NEXT_HOP normally remains Akamai’s peering address on the IX LAN. If our router installs that route, the packet is forwarded toward the Akamai router across the exchange fabric—not through the route server.

This behavior is an explicit part of RFC 7947, Internet Exchange BGP Route Server, which requires the route server to preserve the BGP NEXT_HOP attribute.

There is an important qualification: advertising a route to a route server does not guarantee every participant will receive or use it. Route-server export policy, prefix validation, participant-specific filters, and each participant’s own BGP decisions affect whether that path becomes usable. IX participants may also establish independent, bilateral BGP sessions instead of using the route servers.

In the diagram, the IX peer-to-peer links represent possible data-plane paths. The BGP sessions to route servers and collectors are control-plane relationships. The IX switches forward Ethernet frames between participating routers; the route-server hosts do not forward those production packets.

Outbound versus inbound: control and influence

Here’s a common design: we advertise a public /24 to Lumen and Cogent, and prepend our AS toward Cogent to encourage inbound traffic through Lumen. Using the documentation prefix 203.0.113.0/24 and a fictitious AS 64496, the conceptual AS paths look like this:

• Through Lumen: ... Lumen-AS 64496
• Through Cogent: ... Cogent-AS 64496 64496 64496

The ellipses represent upstream ASes and are not literal BGP configuration. The extra copies of our AS make Cogent’s path longer. When a remote network considers both routes and gets as far as AS-path length in its decision process, it may prefer Lumen.

But AS-path prepending is an influence, not an instruction. A remote network can select Cogent for its own reasons: higher local preference, commercial routing policy, another available path, or an upstream decision we don’t control. Even the same prefix can arrive through different paths from different networks.

Now add the IX. We also announce 203.0.113.0/24 to the Equinix route servers. An eligible IX participant might learn a direct path to our AS. For an Akamai network that learns and accepts this route, its choices might resemble:

• Direct peering: 64496, with our IX-facing router as the BGP next hop.
• Transit: one or more paths received from its upstream networks.

If Akamai’s routing policy prefers that IX peering route, traffic originating from the relevant Akamai network can enter our edge directly over the exchange. Our prepend toward Cogent doesn’t compel Akamai to send those packets through Lumen. It only changes the AS path that Cogent advertises onward, subject to further policy.

It’s tempting to say the IX route wins because it has the shortest AS path. That skips the more important point: BGP policy comes first. Many networks assign different local preferences to customer, peer, and transit routes. Those preferences depend on the network’s own policy; they are not universal, and a shorter AS path is not guaranteed to win.

Similarly, Cogent may prefer the prefix it learns directly from us because we’re its customer, even when we prepend our AS. Lumen, Cogent, Akamai, and other IX participants all independently choose their own forwarding paths.

Outbound routing is more directly under our administrative control. We can use import policy and local preference to rank the routes our router learns from transit providers and IX peers. But even outbound decisions depend on which routes are actually received, their validity, next-hop reachability, and other policy. If multiple paths have the same local preference, additional BGP decision criteria apply; it isn’t simply “shortest AS path always wins.”

What traffic actually uses the IX?

Two traffic patterns are especially relevant to an enterprise with public Internet access.

1. Content and return traffic for employees

An employee downloads a software update, streams training content, or opens a SaaS application. A CDN such as Akamai might serve some of that content. Our outbound request follows whatever route our edge selects for the destination, while the response follows the content provider’s independently selected route back to our public source or NAT address.

If that content is delivered from an Akamai network that learns and prefers our prefix via the IX, the response may return over the exchange even if the request left through a transit ISP. Internet routing can be asymmetric, and that alone is not a fault. It also doesn’t mean every Akamai-served request will traverse the IX; actual paths depend on the specific delivery server and routing policies.

2. Origin connections for public applications

Suppose Akamai provides a CDN, WAF, or other reverse-proxy service in front of one of our public applications. Users connect to an Akamai edge, and Akamai may establish a separate connection to our origin when it needs to retrieve content or proxy a request.

That origin connection may enter our network through the IX if the source network selects our peering route and the applicable product architecture permits it. It can also use transit or other connectivity. We should verify actual source IPs, route selection, and application flow rather than assume every Akamai-to-origin connection uses the exchange.

Those two patterns are different. One involves traffic returning to our users; the other involves traffic from a reverse proxy toward our servers. They should not be conflated when troubleshooting application flows.

From a cost and engineering perspective, peering can reduce paid transit utilization, add capacity for destinations reachable through the IX, and shorten or stabilize some paths. Many public-peering relationships are settlement-free on a per-peer basis, but the IX port, cross-connect, and other connectivity services still have costs. Performance improvements are possible, not automatic.

If we want to restrict an IX path, options may include route-server export controls or supported BGP communities, prefix-specific policies, or bilateral peering arrangements. Any change needs to be evaluated for its effects on reachability, redundancy, and traffic distribution. Consult Equinix’s Internet Exchange documentation for the capabilities of the relevant service and metro.

The ping test that does not prove IX packet loss

Now imagine a ticket that says: “Packet loss on the Equinix route-server path.” The evidence is a continuous ping to a route server’s BGP peering address, sourced from an unrelated remote office across the public Internet.

That test is not valid evidence of loss on our IX port. There are three reasons:

1. It may not traverse our IX connection at all. The remote source follows its own routing table. Without verifying the forward and return paths, we cannot attribute any loss to our edge, our cross-connect, or the exchange fabric.
2. ICMP responses from infrastructure may be rate-limited. Routers and servers often treat packets addressed to their control planes differently from transit traffic. Some rate-limit ICMP echo responses. But it is incorrect to say all ICMP is deprioritized or that ordinary transit ICMP always takes a slow path. ICMP traversing a router can be hardware-forwarded under the applicable forwarding and QoS policies.
3. An IX peering-LAN IP may not be globally reachable. The address can be publicly numbered without the peering subnet being advertised into the global routing table. A remote ping failure may simply reflect intended reachability restrictions, not packet loss.

Even a real problem communicating with a route server must be interpreted correctly. The route server does not carry production packets, but it can still influence production reachability. Session loss, route withdrawals, incorrect filtering, or unexpected route announcements may change which prefixes our edge or remote peers can reach over the IX. A BGP session can remain established while a particular prefix disappears or changes paths.

How to troubleshoot the IX properly

Start by defining the symptom. Is a BGP session unstable? Is a particular prefix missing? Are production applications suffering loss or latency? Or is the only observation that a route server doesn’t reply consistently to ICMP?

1. Validate Layer 1 and Layer 2. Check optical levels where supported, port status, CRC/input errors, discards, utilization, and any evidence of congestion or microbursts. Coordinate with Equinix on the handoff, cross-connect, and IX fabric if the evidence points there.
2. Confirm IX peering reachability. From our edge router, inspect the route to the route-server IP and verify the expected directly connected IX VLAN or interface. A source-specific ping to the peering address can help test local reachability, but ICMP loss alone is not proof of forwarding loss.
3. Inspect both route-server BGP sessions. Check established state, resets, uptime, prefix counts, and changes over time. A stable session only confirms that the BGP relationship remains up; it doesn’t prove all expected routes are present or usable.
4. Check routes, not just neighbors. Verify that our intended public prefixes are advertised, that expected peer routes are learned and accepted, and that the selected BGP next hops resolve through the IX-facing interface.
5. Validate actual forwarding. Examine the routing and CEF/FIB entries for a specific affected destination. Correlate interface counters, flow records, or end-to-end application measurements with the reported problem. A good control plane does not guarantee a healthy data plane.
6. Compare peers and both route servers cautiously. Test multiple reachable peer endpoints, but recognize that each may rate-limit or ignore ICMP differently. A response pattern across several devices can guide the investigation; it does not by itself isolate a defective fabric.
7. Escalate with evidence. Supply the metro, port, VLAN, affected prefixes, time window, interface counters, BGP changes, and reproducible application symptoms. Ask Equinix to verify port/fabric statistics and route-server policy or reachability relevant to the case.

Example: Cisco IOS XE verification

These are illustrative read-only checks. Replace placeholders with actual addresses and interface identifiers; available syntax can vary by IOS XE release and platform.

show bgp ipv4 unicast summary
show bgp ipv4 unicast neighbors <RS-IP>
show bgp ipv4 unicast neighbors <RS-IP> advertised-routes
show bgp ipv4 unicast neighbors <RS-IP> routes
show bgp ipv4 unicast <PREFIX>
show ip route <RS-IP>
show ip route <DESTINATION-IP>
show ip cef <DESTINATION-IP> detail
show interfaces <IX-INTERFACE>

For the public prefix we originate, verify the configured outbound policy and whether it is being advertised to the route server. For a peer prefix we learn, inspect the BGP path and its NEXT_HOP, confirm next-hop resolution, and verify the installed forwarding entry. The routes neighbor view shows paths received from that neighbor and accepted into BGP, subject to platform behavior and policy; it is not a substitute for verifying the installed best route and forwarding entry. A detailed received-routes view may require inbound soft reconfiguration on some platforms; don’t enable it solely for an ad hoc check without considering resource impact.

For packet-loss complaints, also compare what the application experiences. Router CPU replies to ICMP, BGP session health, physical error counters, and user traffic are different measurements. None should be treated as a proxy for the others without corroborating evidence.

The short version

• An Internet Exchange is a real forwarding path. The IX switch fabric carries peer-to-peer packets; route servers and collectors do not.
• Route servers distribute eligible routes. They simplify multilateral peering, but filters and each participant’s routing policies determine what is learned and used.
• AS-path prepending influences inbound traffic; it doesn’t dictate it. A peer may prefer a direct IX path even when we prepend on a transit provider.
• Traffic patterns matter. CDN responses to employees and reverse-proxy connections to origins are different flows, and neither is guaranteed to use a particular IX path.
• ICMP loss to an IX route-server address is not proof of IX forwarding loss. Test the relevant path, confirm installed routes, examine actual counters and application performance, and establish whether the reported loss affects production traffic.
• Stable BGP is necessary, not sufficient. A session can stay up while a route or a forwarding path is unhealthy.

The engineering lesson is simple: understand which devices exchange routing information, which devices actually forward packets, and which routing policies select the path. When those roles are clear, both network design and troubleshooting become much less mysterious.

References

• RFC 7947 — Internet Exchange BGP Route Server (route-server behavior, policy, AS_PATH, and NEXT_HOP transparency).
• Equinix Internet Exchange documentation (multilateral peering and route-server services).
• Equinix Internet Exchange product overview (bilateral and multilateral peering models).
• Cisco IOS BGP command reference (BGP neighbor verification commands).