Delegato: Locality-Aware Atomic Memory Operations on Chiplets

---

Review Form

Reviewer: The Guardian (Adversarial Skeptic)

Summary

The authors propose "Delegato," a mechanism to improve the performance of Atomic Memory Operations (AMOs) in chiplet-based architectures. The proposal consists of two new "far AMO" types (delegated and migrating) to supplement existing near and centralized AMOs. These new types allow the directory to choose a more optimal execution location for an AMO. To guide this choice, the paper introduces a tracing mechanism to convey reuse information from private L2 caches to the directory, which feeds a simple predictor. The authors claim that Delegato improves performance by 1.07x over a centralized AMO baseline and 1.13x over the state-of-the-art predictor, DynAMO.

While the problem of AMO performance on NUMA systems is well-established, this paper's solution rests on a simplistic prediction heuristic, an evaluation that appears tuned to flatter the proposal, and an underestimation of the practical implementation complexity. The claims of superiority are not convincingly supported when the details are scrutinized.

Strengths

1. The paper correctly identifies a relevant and challenging problem: the high latency of AMOs in chiplet systems due to expensive cross-chiplet communication.
2. The exploration of alternative execution locations for far AMOs beyond a single, centralized point is a logical direction for investigation.

Weaknesses

1. Simplistic and Potentially Inaccurate Tracing Mechanism: The core of the predictor's intelligence relies on Delegato, which uses a single reuse_bit to signal usage from the private cache to the directory (Section 5.2, page 6). This is an exceptionally coarse heuristic. It only captures reuse that occurs between two consecutive delegate transactions. It cannot capture more complex temporal patterns, distinguish between frequent and infrequent reuse, or handle cases where a line is used for non-AMO purposes between AMOs. The entire premise of making accurate predictions rests on this fragile, low-information signal, which is a fundamental weakness.

2. Flawed and Self-Serving Baseline Comparison: The paper claims a 1.13x speedup over the "state-of-the-art" predictor, DynAMO [102], which is the authors' own prior work. This comparison is problematic. DynAMO is an L1-based predictor that decides if an AMO should be sent far, whereas Delegato is a directory-level predictor that decides where a far AMO should execute. These are not mutually exclusive. A rigorous comparison would have been to augment the baseline DynAMO to enable it to issue the newly proposed delegated/migrating AMOs, thereby isolating the performance contribution of the prediction logic itself. As presented, the comparison conflates the benefits of the new AMO types with the benefits of the predictor. Furthermore, the authors admit in Section 6.4 (page 10) that combining Delegato with an L1 predictor could fix performance degradation, which concedes that the presented comparison is incomplete.

3. Unconvincing and Potentially Biased Evaluation:

   • The choice of a 50 ns cross-chiplet latency (Table 3, page 7) is extremely high for a modern interconnect and heavily penalizes any data movement, thereby creating an environment where a mechanism like Delegato is destined to show benefit. The sensitivity study in Section 6.6 (page 10) only considers a 2 ns latency, an opposite extreme. The lack of analysis for more moderate and realistic latencies (e.g., 10-20 ns) casts doubt on the robustness of the conclusions.
   • The geomean results mask significant performance regressions. In Figure 9 (page 9), Delegato is shown to be slower than the baseline near AMOs for the FMM, BAR, and LFQ benchmarks. A proposed optimization that results in a slowdown for multiple applications cannot be considered a clear success. The paper fails to adequately diagnose or discuss the cause of these regressions.
   • The CAS Counter microbenchmark (Figure 8, page 8) is a "best-case" scenario of pure, high-contention updates to a single address. While illustrative, its performance is not representative of real applications and overstates the benefits of the Pinned Owner policy.

4. Understated Implementation Complexity and Lack of Rigor:

   • The paper proposes adding ALUs to L2 caches (Section 4.1, page 5) and new, complex transaction types (SnpAMO) to the coherence protocol. The verification and validation effort for such changes is monumental, yet it is not discussed. This is a critical omission for a hardware proposal.
   • The suggestion to reuse the DataPull field in the AMBA CHI protocol for the reuse_bit (Section 6.8, page 11) is an ad-hoc solution that demonstrates a lack of implementation rigor. Such a modification would likely be non-compliant with the standard and create conflicts with features that legitimately use that field, like Stashing. A serious proposal must detail a compliant extension to the protocol.

Questions to Address In Rebuttal

1. Please provide a quantitative analysis to justify that a single reuse_bit is a sufficient and accurate signal for predicting AMO behavior. How does this compare to more established reuse prediction mechanisms (e.g., counters, RRIP)?

2. Can the authors justify the fairness of the comparison against DynAMO? Why was the baseline DynAMO not modified to take advantage of the new delegated and migrating primitives? Such a study would provide a true apples-to-apples comparison of the predictor's efficacy.

3. The performance results are highly sensitive to the 50 ns interconnect latency. Please provide a sensitivity analysis across a more realistic range of latencies (e.g., 10 ns, 20 ns, 30 ns) to demonstrate the robustness of Delegato's performance claims. Furthermore, please provide a detailed analysis of the performance degradation observed in FMM, BAR, and LFQ. What is the specific mechanism in Delegato that causes this harm?

4. Please elaborate on the verification and validation challenges for the proposed protocol changes. Furthermore, is your proposed reuse of the DataPull field compliant with the AMBA 5 CHI specification? If not, what would a compliant implementation entail?

5. The predictor's state machine (Figure 6b, page 7) migrates a cache line after only two consecutive requests from the same core. What is the rate of "ping-ponging" induced by this aggressive policy, where the line is migrated away from an active owner only to be immediately requested back? Please quantify the performance impact of these incorrect migrations.

---

Review Form:

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper addresses the growing performance challenge of Atomic Memory Operations (AMOs) in modern chiplet-based architectures. The authors identify the limitations of the traditional binary choice between 'near' (local execution, causing costly data movement) and 'far' (centralized execution, causing serialization) AMOs. The core contribution is the introduction of two new types of far AMOs—'delegated' and 'migrating'—which allow for remote execution without a single point of centralization, by sending the operation directly to the cache line's current owner or migrating ownership to the requester on demand.

They complement this expanded architectural capability with 'Delegato,' a hardware tracing and prediction mechanism. Delegato enables the directory to dynamically select the optimal AMO type (from the newly expanded set of centralized, delegated, or migrating) based on observed data access patterns. This fundamentally expands the design space for handling atomic operations in NUMA systems, moving beyond simple prediction to intelligent operational dispatch.

Strengths

1. Novel and Significant Conceptual Contribution: The paper's primary strength lies in its conceptual reframing of remote AMO execution. Instead of treating "far" AMOs as a monolithic, centralized action, the authors decompose the problem and propose a more flexible, decentralized approach. Proposing new coherence primitives (delegated, migrating) is a significant architectural contribution that moves beyond the state-of-the-art, which has largely focused on building better predictors for the two existing modalities (e.g., DynAMO [102]). This work correctly identifies that the palette of available operations was itself a limitation.

2. Excellent Problem Motivation and Contextualization: The motivation is exceptionally well-grounded in a pressing industry trend: the shift to chiplet architectures. The analysis in Section 1 and Figure 1 (p. 2) clearly demonstrates that existing solutions are insufficient as systems scale and NUMA factors become more pronounced. By showing cases where centralized AMOs actually regress in performance on a dual-chiplet system, the authors create a compelling narrative for why a new approach is necessary. This work fits perfectly at the intersection of cache coherence research and next-generation system design.

3. A Pragmatic Hardware-Software Bridge: This work can be seen as a hardware realization of principles previously explored in software. The concept of "delegation" echoes software-based techniques where data is partitioned and operations are routed to the thread that "owns" the data to avoid costly synchronization (as noted in Section 2.2, p. 3). Delegato provides a transparent, hardware-managed mechanism to achieve a similar outcome without burdening the programmer, which is a powerful and highly desirable direction for architectural innovation.

4. Thorough and Convincing Evaluation: The evaluation is comprehensive. The authors not only propose the new primitives but also explore the design space of static policies (Section 4.2, p. 5) before introducing their dynamic predictor. The benchmark suite is well-chosen, including classic parallel kernels, graph analytics, and, importantly, modern lock-free data structures, which are notoriously sensitive to AMO performance. The comparison against both a baseline and a state-of-the-art predictor (DynAMO) provides strong evidence of the proposal's efficacy.

Weaknesses

1. Inherent Implementation Complexity: The primary weakness, inherent to the proposal's strength, is its complexity. Introducing new coherence transactions (SnpAMO), requiring ALUs at the L2 cache level for delegation (as discussed in Section 6.8, p. 11), and adding two new predictor tables represents a non-trivial increase in design and verification effort for a CPU core and its uncore components. While the authors perform a reasonable area analysis, the cost of validating such a fundamental change to the coherence protocol is significant and could be a barrier to adoption.

2. Potential for Negative Interactions with Other Optimizations: The paper evaluates Delegato in a relatively clean environment. However, its interaction with other advanced coherence and prefetching mechanisms is not explored. For instance, how would Delegato's decisions (which implicitly influence cache line placement) interact with a sophisticated, learning-based data prefetcher that is also trying to manage where data should reside? There is a risk of the two mechanisms working at cross-purposes, leading to performance oscillations or degradation that is not captured in this study.

3. Simplicity of the Predictor: While the proposed Delegato predictor is effective and simple (which is a strength), its state machine (Figure 6b, p. 7) is based on relatively simple heuristics (e.g., two consecutive requests from the same core triggers a migration). This leaves open the question of whether more sophisticated prediction techniques (e.g., incorporating stream/stride information, or machine learning-based predictors) could yield further gains, or if the problem space is such that these simple heuristics capture the majority of the available benefit.

Questions to Address In Rebuttal

1. On Complexity and Verification: Could the authors comment on the anticipated verification challenges of introducing new snoop and request types like SnpAMO into a complex, industry-standard coherence protocol like AMBA CHI? Is there a simplified path to implementation, perhaps by overloading existing message fields or transaction types, that could reduce this burden while still capturing some of the benefits?

2. On Software vs. Hardware Delegation: The paper rightly positions itself as a hardware alternative to software delegation (Section 2.2). Could you elaborate on the scenarios where you believe a hardware-only approach like Delegato provides a decisive advantage over programmer-driven directives (e.g., __builtin_prefetch style hints for AMOs) or compiler analysis that could achieve a similar effect by pinning data to a specific NUMA node and routing AMOs there?

3. On the Tracing Mechanism's Richness: The Delegato tracing mechanism relies on piggybacking a single 'reuse_bit' on SnpResp messages. This is elegantly lightweight. Have the authors considered the potential benefits of a richer feedback channel? For example, would providing a reuse counter or other metadata from the private cache to the directory allow for more nuanced predictions (e.g., distinguishing between a line that was used once vs. ten times), and what would the associated network and storage overheads be?

---

Review Form

Reviewer: The Innovator (Novelty Specialist)

Summary

This paper addresses the performance challenges of Atomic Memory Operations (AMOs) in chiplet-based architectures, which suffer from high inter-chiplet communication latencies. The authors propose two new types of "far" AMOs: "delegated" and "migrating". Delegated AMOs forward the operation to the current owner of the cache line for local execution. Migrating AMOs transfer ownership to the requester, similar to a near AMO, but are initiated by the directory.

The central novel claim appears to be a tracing mechanism called "Delegato," which leverages the delegated AMO message path to piggyback a "reuse bit" from the owner back to the directory. This bit informs a directory-side predictor about the owner's local use of the line, allowing for more intelligent decisions about whether to keep the line with the current owner (delegate), transfer it to the requester (migrate), or handle it centrally.

While the packaging and evaluation are comprehensive, the novelty of the core primitives is questionable. The "migrating" AMO is functionally a relabeling of existing ownership transfer mechanisms under a new policy. The "delegated" AMO is a specific implementation of request forwarding, a known concept. The most significant novel contribution is the "Delegato" tracing mechanism, which provides an elegant, low-cost method for conveying reuse information back to the directory. The work's overall contribution is an evolutionary, not revolutionary, step in coherence design.

Strengths

1. The "Delegato" Tracing Mechanism: The core innovation of this paper lies in the design of Delegato (Section 5.2, page 6). The problem of an "information outage" at the directory is well-established in coherence prediction literature. The proposed solution—using the SnpResp message of a delegated AMO to carry a single bit of reuse information—is a clever and low-overhead mechanism. It elegantly couples the proposed delegated AMO primitive with the predictor's need for feedback, creating a "heartbeat" to confirm the utility of the line's current placement. This specific mechanism for feedback appears to be new.

2. Formalization of an Owner-Executed AMO: While the concept of forwarding requests is not new, the paper does a good job of formalizing the "Delegated AMO" within a modern, complex coherence protocol (AMBA CHI). Defining the specific transaction flows (SnpAMO) and state transitions (Figure 4, page 4) required to implement this is a valuable, albeit implementation-focused, contribution.

Weaknesses

1. Overstated Novelty of AMO Primitives: The paper presents "delegated" and "migrating" AMOs as two "new types" of far AMOs. This claim requires significant qualification.

   • Migrating AMOs: As described, a migrating AMO is simply a directory policy that chooses to resolve a far AMO request by initiating an ownership transfer to the requester. The underlying mechanism—transferring a cache line to a new owner—is fundamental to all modern coherence protocols. Calling this a "new type" of AMO is misleading; it is a new policy for when to trigger an existing action. The state-of-the-art predictor, DynAMO [102], already makes a similar decision (near vs. far), albeit at the L1 cache. This is a shift in where the decision is made, not the introduction of a new primitive.
   • Delegated AMOs: The concept of forwarding a request to the node holding the most up-to-date data (the owner) is a well-known pattern in distributed systems and cache coherence. For example, the idea of data forwarding (e.g., Koufaty and Torrellas, 1998, [63]) and producer-initiated communication (e.g., Goodman et al., 1989, [40]) involves sending data directly between caches. The delta here is forwarding the operation instead of just the data. While this specific formalization as an AMO primitive is notable, the conceptual foundation is not entirely new, and the paper should position its contribution more precisely against this backdrop.

2. Complexity vs. Benefit Trade-off: The proposed solution introduces non-trivial complexity. Specifically, it requires placing ALUs in the L2 caches to handle delegated AMOs (mentioned in Section 4.2, page 5) and adds new message types to the coherence protocol. The evaluation shows that Delegato achieves a 1.07x speedup over centralized AMOs and a 1.13x speedup over the state-of-the-art DynAMO predictor (Figure 9, page 9). While positive, these gains are modest. An innovator must ask if adding hardware ALUs to another level of the cache hierarchy and extending the coherence protocol is justified for an average 7-13% performance improvement. The case for this trade-off is not overwhelmingly strong.

Questions to Address In Rebuttal

1. Regarding "Migrating AMOs," please clarify why this should be considered a novel primitive rather than a new directory-level policy. Functionally, how does the resulting ownership transfer differ from the one initiated by a conventional ReadUnique request in a near AMO transaction, other than the point of initiation?

2. Regarding "Delegated AMOs," can the authors please elaborate on the delta between this proposal and prior art in request forwarding within coherence protocols? Please situate this contribution in the context of owner-driven resolution and remote invocation concepts from the broader literature.

3. The proposed design necessitates ALUs at the L2 cache, adding area and complexity. Given the reported average speedups of 1.07x–1.13x, please provide a stronger argument for why this architectural modification is more compelling than a less invasive, purely policy-based optimization at the directory that does not require new hardware at the L2.