Arista @ NFD40: Modern Fabrics for AI

Arista @ NFD40: Modern Fabrics for AI

AI networking, much like everything else in the space, is developing very quickly and has a new acronym per day. Between UEC, ESUN, NCCL and a bunch of other protocols/technologies, it's easy to lose track of all the new innovations. What kind of network are we even building with these things? For what workloads? At what scale?

Arista's session at NFD40 started by answering those questions with a most helpful framework that really helped me understand the Networking for AI "State of the Art" a little better. They showed four fabric types with four distinct sets of requirements.

A Four Fabric Taxonomy

Arista broke down AI networking into four fabric types for us, each with distinct requirements:

  • Front End - This is the traditional data center network fabric we all know and love. Standard Ethernet switching for inference workloads, user-facing traffic, storage. Nothing exotic here, but it's still a real fabric that needs appropriate design and care.
  • Scale Out - These are the big Ethernet fabrics connecting racks of XPUs within a data center with numbers in the hundreds of thousands of said XPUs. This is where most of the industry's attention has been focused: Multiple, massive 800G, and now 1.6Tbps, links and the tuning of these networks for maximum performance.
  • Scale Across - A new comer in the emerging multi-data-center story which is driven by geo-distributed DCs due to power grid constraints. We get longer distances (10km, 100km, even 1000km), deep buffering, ZR/ZR+ optics, MACsec encryption. Meta is doing this publicly and others are working toward it as well.
  • Scale Up - This is the intra-rack interconnect. XPU-to-XPU communication within a single node or rack, where bandwidth requirements are an order of magnitude higher than scale-out (3.6 Tbps vs 400 Gbps per GPU). This is where tech like ESUN lives.

What made this framework valuable wasn't that any of this was particularly new. It was the act of naming them, defining their boundaries and letting practitioners think clearly about which problem they're actually solving. Arista laid out a pretty nice map.

4 AI fabrics, easy peasy!

ESUN and the Scale Up Opportunity

ESUN (Ethernet Scale Up Networking) was one of the more technically interesting things Arista showed on the protocol side. ESUN takes Ethernet, strips it down to the absolute minimum header (14 bytes Ethernet + 4 bytes routing, compared to 30+ bytes for IPv4/6), adds credit-based flow control and link-layer retries and makes it fast enough for intra-rack XPU communication.

The result is 8-9x more bandwidth per XPU compared to scale out connections. Single-hop architecture eliminates multi-hop flow control headaches. It's a purpose-built protocol for a specific, high-value problem: making Ethernet competitive for the communication patterns that have historically required proprietary interconnects.

The stripped down header is clever. When you're pushing 3.6 Tbps per XPU, every byte and watt of overhead matters. Less header means more payload and more efficient power usage, which means more useful work per cycle.

ESUN is still early, but the direction is open, standards-based alternatives for every layer of the AI networking stack. That's good for the industry regardless of who gets it out the door first.

Shedding a light on ESUN

Why Extra-Dense Pluggable Optics (XPO)?

Arista covered three optics tracks (XPO, LPO, CPO) but it was clear, XPO is where they're planting their flag.

XPO (Extra Dense Pluggable Optic) delivers 4x density over OSFP in a liquid cooled only form factor. It has operating temperatures 20-25°C lower than air cooled alternatives, a claimed 5-8x reliability improvement and support for up to 400W per module. It's also backed by an open MSA with 100+ members covering roughly 90% of the module market which is a lot of support out of the gate.

Arista's position is that 800G is likely the end of the road for air cooled optics. At 1.6T, the thermal math just doesn't work without liquid moving forward. By committing to liquid cooling now with XPO, they're getting ahead of a transition that the rest of the industry will eventually have to make. The side benefits compound: lower fan speeds, no air preheating of adjacent components, simplified thermal management across the entire switch.

The power story ties it together. An 800G optic runs around 30W and at 1.6T, you're looking at roughly 45W. Multiply that across a 100K XPU cluster with hundreds of thousands of optics and the aggregate power budget is enormous. LPO (Linear Pluggable Optics) helps here too, cutting roughly 60% of power by eliminating the retiming/DSP function. LPO works well at 800G and is expected to carry through to 1.6T. Between XPO's density and LPO's power reduction, Arista is attacking the two biggest constraints at AI cluster scale: physical space and watts.

CPO (Co-Packaged Optics) got a measured mention. Arista acknowledged the power savings are equivalent to LPO but noted the current DR reach limitation and supply chain immaturity. Their stance: they want an open CPO ecosystem, not single-vendor integration and they're not rushing deployment until the supply chain can support 100M+ scale. That's being realistic about when it's ready.

What came through most clearly was that XPO is in direct response to customer constraints. Networking targets under 10% of total DC power, but at AI cluster scale, even that 10% is a massive number. Every watt you save on optics is a watt you can spend on compute. Arista's open-standards approach (contributing OSFP, XPO, LPO and VISTA to open MSAs) means they're pushing the industry toward a common solution.

That's Dense!

AI Job-Centric Operations

Arista's CloudVision and AVA platform are shifting focus from traditional network telemetry (link state, interface counters, BGP sessions) to the health of the AI training job itself.

Arista claims 44% improved job completion time through better network observability and optimization. Networks that can see the relationship between fabric congestion and XPU utilization can pay for themselves by keeping expensive compute resources productive.

What makes this more than marketing is the data ingestion model. AVA pulls from EOS streaming telemetry (every switch state change in real-time), but also from third-party sources: AI job orchestrators, compute and storage systems via Prometheus and OpenMetrics. You don't have to be a single-vendor shop to get value from the correlation engine.

The natural language query capability (Ask AVA) is easy to demo and hard to evaluate without production deployment but the underlying concept is sound: if your management platform understands both "spine link X is congested" and "training job Y is running slow" The correlation between those two facts is where the operational value lives.

This guy seems stressed about networks!

Wrapping Up

Arista's NFD40 presentation was refreshingly straightforward and to the point and I felt it was a well organized tour of how modern AI networking works in production, where the hard problems are and what the roadmap looks like.

Sometimes the most valuable thing a vendor can do at a Tech Field Day presentation is to help you better understand the state of play for their space. Arista did just that and I feel was one of the more dense and concise presentations at all of NFD40. You can tell these guys have seen their way around an AI network or two.

DISCLAIMER: I was fortunate to participate in Network Field Day 40 as a delegate by Gestalt IT who put me up in San Jose California, bought me food, paid for snacks and left me some cool swag from the participants. I did not receive any compensation to attend this event and I am under no obligation whatsoever to write any content related. The contents of these blog posts represent my personal opinions about the products and solutions presented during NFD.

More NFD40 Content From My Esteemed Delegate Peers!

Tony Mattke

Nokia at NFD40: Networking in the AI Era
I’ve been building networks for nearly thirty years. I understand leaf-spine fabrics, BGP design, VRF isolation, ECMP, and congestion management. I’ve designed data centers for financial institutions …
Lightyear at NFD40: Can Software Fix the Telecom Lifecycle?
Anyone who has managed carrier circuits for a living knows the pain. Quoting takes forever. Installs drag on with zero visibility. Renewals sneak up because everything lives in a spreadsheet that …
One Throat to Choke, or Any Silicon You Want: Cisco at NFD40
Cisco showed up to NFD40 selling two different pitches in the same session, and nobody on stage reconciled them. Richard Licon, Faraz Taifehesmatian, and Paymon Mogharabi walked through Silicon One …

Ben Story

Try Before You Buy: Nokia Is Taking Validated Designs Seriously (And That’s Rarer Than You Think) -
Vendors publish “validated designs” and let them rot. Nokia is doing something different — and the four-year support commitment is the part worth paying attention to.
AIOps Fatigue Is Real, And It’s Your Vendor’s Fault, Not Yours -
Mike Hoffman has been doing this longer than most of us. His first network troubleshooting tool was an oscilloscope on thick coax, shooting down the cable and picking up a 50 ohm resistor looking for the little bleeps from vampire taps. His next tool was the first-generation Network General LAN Doctor. A briefcase with an […]
Mind the Gap: How MSPs Can Use Lightyear to Own the Telecom Lifecycle -
I’m sitting in the audience at Network Field Day watching Dennis Thankachan and Ryan Schrack from Lightyear demo what they call a “telecom operating system,” and my brain is doing that thing it does where it stops listening to the presenter and starts thinking about my customers. Lightyear is pitching to enterprises, Fortune 500s, big-name […]

Peter Welcher

NFD40: Cisco Networking for AI
Networking Field Day 40 (#NFD40) had some key content for those designing AI datacenters. On the hardware front, Arista, Cisco, and Nokia all presented.
NFD40: Arista Networking for AI
Networking Field Day 40 (#NFD40) had some key content for those designing AI datacenters. On the hardware front, Arista, Cisco, and Nokia all presented.
NFD40: Nokia Networking for AI
Networking Field Day 40 (#NFD40) had some key content for those designing AI datacenters. On the hardware front, Arista, Cisco, and Nokia all presented.