EDM: An Ultra-Low Latency Ethernet Fabric for Memory Disaggregation

Paper: EDM: An Ultra-Low Latency Ethernet Fabric for Memory Disaggregation
Reviewer: The Guardian

---

Summary

This paper proposes EDM, a radical redesign of the Ethernet fabric for memory disaggregation that aims to achieve ultra-low latency. The core proposal involves two aggressive architectural changes: 1) moving the entire network protocol stack for remote memory access from above the MAC layer into the Physical (PHY) layer, and 2) implementing a centralized, priority-based PIM scheduler within the switch's PHY layer to create virtual circuits and eliminate queuing. The authors support their claims with a small-scale 25GbE FPGA hardware prototype, which demonstrates a ~300ns unloaded latency, and larger-scale C-language simulations to evaluate performance under load. While the stated performance goals are ambitious, the work rests on a series of optimistic assumptions and its evaluation lacks the scale and rigor to substantiate its claims of practical viability.

Strengths

1. Problem Motivation: The paper does an excellent job of identifying and articulating the fundamental latency and bandwidth overheads imposed by the standard Ethernet stack (MAC layer constraints, IFG, L2 switching latency) for small, latency-sensitive memory messages (Section 2.4, page 4). This background provides a clear and compelling motivation for exploring unconventional solutions.
2. Scheduler Design: The design of the priority-based PIM scheduler in hardware is detailed and appears well-considered (Section 3.1.2, page 7). The use of constant-time data structures for priority queue operations and the 3-cycle implementation of a PIM iteration demonstrate a clear path toward a high-performance ASIC implementation.
3. Holistic Approach: The authors present an end-to-end design, considering the necessary modifications at the host NIC, the switch, and the protocol semantics. This is a commendable effort compared to papers that focus only on one piece of the puzzle.

Weaknesses

My primary concerns with this work revolve around its fundamental feasibility, the limited scope of its evaluation, and the dismissal of significant real-world complexities.

1. Fundamental Violation of Standardization: The core premise of implementing a custom protocol within the PHY layer (Section 3.2, page 8) is a critical flaw from a practical standpoint. The PHY layer is a complex and highly standardized domain for a reason. The paper completely fails to address how EDM interacts with essential PHY-level functions like Forward Error Correction (FEC), Auto-Negotiation, or training sequences. These are not trivial features; they are essential for reliable communication over modern high-speed links. By creating custom /M* block types, the authors effectively propose a proprietary, non-standard physical layer that would be incompatible with the entire existing ecosystem of transceivers, optics, and diagnostic tools. The claim in Section 3.3 (page 10) that EDM simply "creates a parallel pipeline" is a gross oversimplification that ignores the physical and logical realities of the SerDes interface.
2. Insufficient and Unrepresentative Hardware Evaluation: The hardware results, while showing impressive latency, are derived from a toy-scale testbed consisting of just two host nodes and one 2-port switch (Section 4.2, page 11). Extrapolating performance from this minimal setup to a 512-port switch, as is implied by the ASIC synthesis discussion (Section 4.1), is a significant logical leap. Such a small system cannot exhibit the complex contention patterns, scheduler hotspots, or clock-domain crossing challenges that emerge at scale. The ~300ns latency figure is an ideal, best-case number that has not been validated under any meaningful stress.
3. Understated Overheads and Unaddressed Trade-offs:
   • Write Latency Penalty: The design requires an explicit notification for write requests (WREQ), incurring an RTT/2 latency penalty before data can be sent (Section 3.1.1, page 6). The authors dismiss this as "nominal" and a "small price to pay" (Section 3.1.4, page 8). This is not a credible assessment. In a rack- or cluster-scale network, the propagation delay alone can be hundreds of nanoseconds to several microseconds, making this "small price" potentially larger than the entire unloaded latency EDM claims to achieve. This fundamentally biases the fabric's performance towards read-heavy workloads, a point that is not adequately analyzed.
   • Penalty on Standard Traffic: The intra-frame preemption mechanism (Section 3.2.3, page 10) imposes a buffering delay on all non-memory traffic on the receive path, equal to the transmission delay of a maximum-sized frame. For a 9KB jumbo frame on a 100Gbps link, this is a ~720ns latency tax paid by all standard IP and storage traffic, even when there are no memory messages to preempt. This is a significant performance regression for co-existing traffic that the paper fails to evaluate or even acknowledge.
4. Grossly Simplified Handling of System-Level Challenges: The section on "Practical Concerns" (Section 3.3, page 10) hand-waves away monumental engineering challenges. The proposed solution for fault tolerance—state machine replication for the in-switch scheduler state—is presented as a straightforward extension. In reality, implementing state replication for a nanosecond-scale, line-rate scheduler without compromising performance is an enormous, unsolved research problem that would introduce significant latency and complexity. Dismissing this with a single paragraph is a serious omission.
5. Questionable Simulation Baselines: The simulation results that show CXL performing up to 8x worse than EDM (Figure 8b, page 14) are highly suspect. The paper attributes this to credit-based flow control and head-of-line blocking (Section 4.3.1, page 13). While this is a known potential issue, an 8x performance degradation suggests that the CXL model used in the simulation may be uncharitable or configured to exhibit a worst-case pathology that is not representative of typical CXL switch implementations or workloads. Without a detailed validation of the CXL simulation model, these comparative results lack credibility.

Questions to Address In Rebuttal

1. Please provide a detailed account of how EDM's PHY-layer modifications would co-exist with standard and essential PHY functions like FEC (e.g., Reed-Solomon RS-FEC), link training, and auto-negotiation. How can a standard transceiver correctly interpret a link that interleaves standard 66-bit blocks with EDM's custom /M* blocks?
2. Can the authors justify the claim that the performance observed in a 2-node, 1-switch FPGA testbed is representative of a large-scale deployment? What scheduler or system-level bottlenecks do you anticipate when scaling from 2 ports to 128 or 512 ports, and why are these not captured in your evaluation?
3. Provide a quantitative analysis of the RTT/2 latency overhead for write operations. At what physical distance (e.g., 10m, 50m, 100m optical links) does this overhead cease to be "nominal" and instead dominates the total transaction latency?
4. Please provide a detailed specification of the CXL model used in your C-language simulator. What are the specific buffer sizes, credit exchange parameters, and traffic patterns that lead to the 8x worse message completion time shown in Figure 8b? Please provide evidence that your model accurately reflects the behavior of state-of-the-art CXL fabrics.
5. Regarding fault tolerance, the paper proposes state machine replication. Can you elaborate on how you would implement this for a scheduler that must make decisions every few nanoseconds without introducing significant latency that would nullify EDM's primary benefit? Please acknowledge the state-of-the-art in this area and how your proposal relates to it.

Reviewer Persona: The Synthesizer (Contextual Analyst)

---

Summary

This paper presents EDM, a novel network fabric designed to achieve ultra-low latency for memory disaggregation over Ethernet. The authors identify the standard Ethernet protocol stack, particularly the MAC layer, as a fundamental latency and overhead bottleneck for small, latency-sensitive memory messages.

The core contribution is a radical architectural shift: bypassing the MAC layer entirely and implementing a specialized network protocol for remote memory access directly within the Ethernet Physical Layer (PHY). This is complemented by a second key idea: a fast, centralized, in-network scheduler, also implemented in the switch's PHY. This scheduler creates dynamic, nanosecond-scale virtual circuits between compute and memory nodes, eliminating L2 processing and queuing delays.

Through an FPGA-based prototype and larger-scale simulations, the authors demonstrate that EDM can achieve a remote memory access latency of ~300 ns, which is an order of magnitude better than existing Ethernet-based RDMA solutions and is competitive with emerging PCIe-based fabrics like CXL, while retaining the scalability and cost benefits of Ethernet.

---

Strengths

The true strength of this paper lies in its ambition and the elegance of its core idea. It challenges a long-held assumption about network layering and, in doing so, opens a compelling new design point for datacenter fabrics.

1. A Bold and Foundational Contribution: The central idea of moving the memory fabric into the PHY is not an incremental optimization; it is a fundamental re-architecting of the network stack for a specific, high-value workload. By identifying the MAC layer's frame-based semantics (minimum frame size, inter-frame gap, lack of preemption) as the root cause of latency for small messages (Section 2.4, page 4), the authors make a convincing case for their radical approach. This is the kind of high-level conceptual thinking that can inspire a new line of research.

2. Excellent Contextualization and Positioning: The authors do a superb job of placing their work within the current landscape of memory disaggregation. The comparison is not just against other Ethernet protocols but squarely against CXL, the leading alternative fabric. The paper effectively frames EDM as a way to achieve CXL-like latency without abandoning the cost, distance, and bandwidth-scaling advantages of the Ethernet ecosystem (Section 2.2, page 3). This demonstrates a keen awareness of the broader industry and academic trends.

3. A Holistic and Well-Considered System Design: The work goes far beyond a single clever trick. The design is comprehensive, encompassing the host NIC, the switch, and the scheduling protocol that ties them together. The in-PHY scheduler, inspired by Parallel Iterative Matching (PIM), is thoughtfully designed to be implemented in high-speed hardware (Section 3.1.2, page 7). Furthermore, the design for intra-frame preemption (Section 3.2.3, page 10) is a crucial and elegant feature that acknowledges the reality of converged networks where memory traffic must coexist with traditional IP traffic. This demonstrates deep systems thinking.

4. Connecting Disparate Concepts: This work synthesizes ideas from several domains. It takes the classic networking concept of virtual circuits, implements it using modern high-speed scheduling algorithms (PIM), and places it in a novel part of the network stack (the PHY), a layer previously explored more for timing or covert channels. By applying PHY-level engineering to a mainstream datacenter problem, the paper connects what were once niche research areas to a problem of significant practical importance.

---

Weaknesses

While the core idea is powerful, the paper would be strengthened by a more direct engagement with the practical and systemic challenges its adoption would entail. My concerns are less about the validity of the idea and more about its path to real-world impact.

1. The "Ecosystem" Barrier to Adoption: The most significant challenge for EDM is that it requires a non-standard PHY in both the NIC and the switch. This creates a chicken-and-egg problem for adoption. Unlike software-based solutions or even P4-based programmable switches that work within the existing Ethernet framework, EDM requires new hardware from the ground up. This is a formidable barrier and represents the biggest threat to the work's practical impact.

2. The Limits of Centralized Scheduling: The proposed design is centered on a single, centralized scheduler within one switch, targeting rack- or cluster-scale deployments. This is a reasonable starting point, but the broader vision for datacenter-scale memory disaggregation will inevitably involve multi-switch topologies. The paper does not discuss how the EDM model would extend beyond a single switch domain. Would traffic revert to standard RoCEv2 when crossing switch boundaries, negating the latency benefits? A discussion of the architectural implications for larger-scale networks is a missing piece of the puzzle.

3. Understated Engineering Complexity in ASICs: While the authors have successfully prototyped their design on FPGAs using an open-source PHY, translating this to a commercial, multi-terabit switch ASIC is a non-trivial leap. The PHY/SerDes in modern ASICs is a highly complex piece of mixed-signal hardware, often licensed as hardened IP. The paper could better acknowledge the engineering challenges of modifying this layer, especially concerning signal integrity, clocking domains, and integration with existing FEC (Forward Error Correction) logic, which also operates at the PHY level.

---

Questions to Address In Rebuttal

1. Path to Impact: Given the requirement for custom hardware on both the host and switch side, what do you envision as a plausible adoption path for EDM? Could a version of this be implemented using emerging technologies like programmable SmartNICs and programmable switches, even if at a higher latency, as a bridge to full ASIC integration?

2. Scaling Beyond a Single Switch: Could you elaborate on how you see the EDM architecture evolving for multi-rack or datacenter-scale deployments? How would an EDM-enabled rack interconnect with another, and what would the end-to-end latency properties of such a connection be? Does the centralized scheduling model fundamentally limit EDM to a single-switch domain?

3. ASIC Implementation Feasibility: Based on your work, what are the most critical interactions between EDM's logic and the core functionality of a modern high-speed Ethernet SerDes (e.g., equalization, FEC)? Do you believe that EDM's logic can be cleanly separated from the analog and signal-processing components of the PHY, making it a feasible addition to future switch/NIC ASICs?

Reviewer: The Innovator (Novelty Specialist)

Summary

The paper presents EDM, an ultra-low latency Ethernet fabric designed for memory disaggregation. The core proposal is to circumvent the traditional network stack by implementing a complete protocol for remote memory access, including a centralized in-network scheduler, entirely within the Physical (PHY) layer of the Ethernet NIC and switch. By operating at the granularity of 66-bit PHY blocks, the authors claim to eliminate the fundamental overheads of the MAC layer (minimum frame size, inter-frame gap, lack of preemption) and the processing latency of the switch's L2 forwarding pipeline. The authors demonstrate a ~300ns end-to-end fabric latency on an FPGA prototype, an order of magnitude lower than standard Ethernet-based protocols.

Strengths

The primary strength of this paper lies in the novelty of its architectural design point. While individual components of the proposed system have conceptual precedents in prior work, their synthesis into a cohesive, high-performance system operating exclusively at the PHY layer is, to my knowledge, new. The specific novel contributions are:

1. Protocol Relocation: The radical proposition of moving the entire network protocol stack for a specific traffic class (memory access) into the PHY is the paper's most significant novel claim. This is a fundamental departure from decades of layered network architecture.
2. Scheduler Integration: Placing a centralized, hardware-accelerated scheduler inside the switch PHY (Section 3.1, page 5) is a clever and novel mechanism. This placement is the key enabler that allows EDM to create virtual circuits and bypass the entire L2 packet forwarding pipeline, which is a major source of latency in conventional switches.
3. PHY-level Preemption: The mechanism for intra-frame preemption at the 66-bit block level (Section 3.2.3, page 10) is a novel implementation. It provides a much finer granularity of control than existing MAC-level preemption standards (e.g., IEEE 802.3br), which is critical for protecting latency-sensitive memory traffic.

The performance benefits demonstrated are not marginal; a reduction in fabric latency by an order of magnitude is substantial enough to justify the consideration of such a non-standard and complex architecture.

Weaknesses

While the overall architecture is novel, the paper could do a better job of positioning its contributions against the closest conceptual prior art. The novelty is in the synthesis and location, not necessarily in the foundational concepts themselves.

1. Repurposing PHY Constructs: The idea of using idle or otherwise unused portions of the PHY layer for data transmission is not entirely new. Prior work on PHY-level covert channels, such as Lee et al. [37] ("PHY Covert Channels: Can you see the Idles?"), established the principle of repurposing idle characters for out-of-band communication. The authors cite this work but should more explicitly frame their contribution as elevating this concept from a low-bandwidth "covert channel" to a first-class, high-bandwidth protocol, which is a significant delta but builds on the same foundational insight.
2. Scheduling Algorithm: The core scheduling algorithm is a priority-based version of the classic Parallel Iterative Matching (PIM) from Anderson et al. [6]. The novelty here is not the algorithm but its highly optimized, constant-time hardware pipeline implementation (Section 3.1.2, page 7) and its unique placement. The paper should be more precise in claiming novelty for the implementation and integration, rather than the scheduling algorithm itself.
3. Centralized Scheduling: The concept of a centralized scheduler for datacenter networks was explored in depth by works like Fastpass [51]. Fastpass, however, used a separate server, which introduced different bottlenecks. EDM's novelty is in decentralizing the scheduler logic to the switch hardware itself, specifically the PHY. The paper should more clearly articulate this distinction: it is an in-network centralized scheduler, not a server-based one.

Questions to Address In Rebuttal

The authors should use the rebuttal to sharpen the articulation of their novel contributions.

1. Novelty vs. PHY Covert Channels: How do the authors differentiate their work from the conceptual precedent set by papers like [37], which proposed repurposing PHY-level constructs for data transfer? Is the primary novelty the scale (a full protocol vs. a covert channel) or a more fundamental architectural difference? Please clarify the key inventive step that allows for this scaling.
2. Novelty of the Scheduler: The paper's scheduler is an optimized hardware implementation of the well-known PIM algorithm. Could the authors confirm if there are any novel algorithmic contributions to the scheduler itself, or if the novelty lies exclusively in its efficient hardware pipeline and its unique placement within the PHY?
3. Comparison to Preemption Standards: The IEEE 802.3br standard defines frame preemption at the MAC layer to support time-sensitive networking. While the authors' PHY-level approach is different and finer-grained, could they comment on the trade-offs and explain why the existing standard is insufficient for their goals? This would help solidify the necessity of their novel approach.
4. Justifying Complexity: The proposed architecture requires deep, non-standard modifications to the Ethernet PHY in both NICs and switches, a significant engineering and ecosystem cost. Given that CXL offers comparable unloaded latency, and the primary benefit of EDM appears under heavy network load (Figure 8, page 14), what specific class of applications justifies this massive trade-off of standardization for performance? Is the primary driver truly memory disaggregation, or is it a more general fabric for HPC/ML workloads where congestion is the dominant problem?