ComPASS: A Compatible PIM Protocol Architecture and Scheduling Solution for Processor-PIM Collaboration

---

Review Form

Reviewer: The Guardian (Adversarial Skeptic)

Summary

The authors propose ComPASS, a system aimed at improving the integration of Processing-In-Memory (PIM) devices into general-purpose systems. The solution consists of three main components: 1) a new memory command, PIM-ACT, intended to provide a compatible interface for initiating PIM operations across different architectures; 2) a PIM request generator within the memory controller to offload command generation from the host CPU; and 3) two scheduling policies, Static (ST-BLC) and Adaptive (AT-BLC) Throughput Balancers, to manage concurrent PIM and conventional memory requests. The evaluation, conducted via simulation, claims that the proposed system achieves high PIM performance (up to 10.75× GEMV speedup over non-PIM) while successfully co-existing with memory-intensive CPU workloads.

Strengths

1. The paper addresses a significant and timely problem in computer architecture: the practical integration of PIM hardware. The challenges of protocol incompatibility and resource scheduling are indeed major barriers to adoption.
2. The proposed solution is comprehensive, considering the protocol layer (PIM-ACT), hardware support (request generator), and system-level scheduling (ST-BLC/AT-BLC).
3. The evaluation of the scheduling policies, particularly the demonstration in Figure 8 that AT-BLC can meet PIM Quality-of-Service (QoS) targets where other policies fail, presents a compelling case for the adaptive approach.

Weaknesses

My analysis reveals several significant weaknesses that undermine the paper's core claims of compatibility, practicality, and rigor.

1. The Claim of a "Compatible" Protocol is Overstated and Misleading. The central premise of a unified PIM-ACT command is fundamentally weakened by the necessity of the "Architecture-Aware Optimization" (AAO) mechanism described in Section 4.5 (Page 5). The paper claims PIM-ACT allows "different PIM devices to communicate with the host using the same PIM-ACT interface, ensuring compatibility." However, AAO requires the memory controller to load device-specific timing parameters, bank activation granularities, and command semantics from the SPD. This means the controller is not interacting with a unified interface; it is interacting with a configurable one that requires explicit, a priori knowledge of the specific PIM device it is controlling. This is a crucial distinction. The complexity is not removed; it is merely shifted to SPD tables and the MC's interpretation logic. This approach seems far from the "drop-in" compatibility implied.

2. The Foundation of the Adaptive Scheduler (AT-BLC) Relies on an Unjustified Assumption. The paper's strongest result, the AT-BLC scheduler, is critically dependent on a "target completion time (T)" which "must be initialized before the PIM operation begins" (Section 5.4, Page 8). The paper provides no details on how this target T is determined. Is it provided by a compiler? A runtime profiler? An oracle? The feasibility and robustness of AT-BLC are entirely contingent on the accuracy of this prediction. The authors fail to analyze the sensitivity of their mechanism to mispredictions in T. A scheduler that only works with perfect future knowledge is of limited practical value. This omission represents a major logical flaw in the evaluation of the paper's primary scheduling contribution.

3. Key Overheads are Ignored or Dismissed Without Evidence.

   • Host CPU Offload: The paper claims the PIM request generator alleviates the burden on host processor cores (Section 4.2, Page 4), but this crucial benefit is never quantified. There is no measurement of CPU cycles saved or utilization reduced. Without this data, the contribution of the request generator is unsubstantiated.
   • Memory Management: The authors propose using HugeTLB for contiguous memory allocation (Section 4.6, Page 6). They acknowledge that memory compaction may be required if huge pages are fragmented but dismiss the overhead by claiming it is "amortized" for LLM inference. This is an unsupported assertion. For systems under high memory pressure or with different workload characteristics, compaction can induce significant, non-trivial latency spikes. The lack of any quantitative analysis on this front is a serious methodological weakness.

4. Performance Claims Lack Critical Nuance. The PIM-only evaluation in Section 7.1 (Page 9, Figure 7) shows that for GDDR6-AiM, PIM-ACT results in a performance regression for GEMV, even with AAO enabled (0.59% slower). For LPDDR-AiM, the regression is even larger (2.84% slower). To conclude from this data that the protocol "maintains performance comparable to... device-specific protocols" is an oversimplification. Furthermore, the system performance gain attributed to AAO is shown to be only 1.2-1.5% (Section 7.3, Page 12, Figure 10b). This marginal improvement calls into question whether the added complexity of the AAO mechanism is justified.

Questions to Address In Rebuttal

The authors must address the following points directly:

1. On Compatibility: Please reconcile the claim of a "compatible" and "unified" protocol with the explicit requirement for the memory controller to parse and implement device-specific behaviors via the AAO mechanism. Is "configurable" not a more accurate description? If so, how does this meaningfully reduce integration complexity compared to existing device-specific controller modifications?

2. On the AT-BLC Scheduler: The functionality of AT-BLC hinges entirely on the pre-supplied target time T.

   • a) What specific mechanism is proposed to calculate T in a real system?
   • b) Provide a sensitivity analysis showing how AT-BLC's performance (both PIM QoS and CPU throughput) degrades when T is mispredicted by ±10%, ±25%, and ±50%.

3. On Overheads:

   • a) Provide quantitative data demonstrating the reduction in host CPU utilization as a direct result of the PIM request generator. Compare a system with the generator to one without, where the host CPU issues all micro-requests.
   • b) What is the measured performance impact (e.g., tail latency for CPU requests, delay in PIM task initiation) of a memory compaction event triggered to create a huge page for a PIM workload?

4. On Performance Regressions: Please justify the claim that PIM-ACT offers "comparable" performance to native protocols when your own data (Figure 7) shows clear performance regressions for GEMV on AiM-based architectures. At what threshold does a performance loss become significant enough to disqualify a "compatible" claim?

---

Review Form

Reviewer: The Synthesizer (Contextual Analyst)

Summary

The paper presents ComPASS, a comprehensive solution aimed at solving the critical system-level integration challenges of Processing-In-Memory (PIM). The core contribution is a two-pronged approach that addresses both the hardware interface and the performance management aspects of PIM in general-purpose systems. The first prong is a compatible PIM protocol (PIM-ACT) and a memory controller extension (PIM request generator) designed to create a unified, flexible hardware interface that can support diverse PIM architectures (like HBM-PIM and GDDR6-AiM) while adhering to existing DRAM standards. The second prong is a set of scheduling policies, culminating in an adaptive throughput balancer (AT-BLC), to intelligently manage memory bus contention between PIM operations and conventional host CPU memory requests, ensuring that PIM meets its performance targets (QoS) without starving CPU applications.

Essentially, this work is not about inventing a new PIM device but rather about designing the standardized "plumbing" and "traffic control" necessary to make the burgeoning ecosystem of PIM devices a practical and integrated part of modern computer systems.

Strengths

1. Addresses a Critical and Timely Problem: The field has demonstrated the potential of PIM with several real-world devices. The primary bottleneck to widespread adoption is no longer the feasibility of in-memory computation itself, but the challenge of system integration. This paper correctly identifies the core obstacles—protocol incompatibility and resource contention—and proposes a holistic solution. It shifts the conversation from "Can we build PIM?" to "How do we use PIM effectively in a heterogeneous system?" This is exactly the direction the field needs to move in.

2. Pragmatic and Elegant Protocol Design: The PIM-ACT command is a clever and practical solution to the limited command space problem. By leveraging existing RFU commands or unused bits in commands like NOP (Section 4.1, Figure 3, page 4), it avoids the need for a radical departure from JEDEC standards. The inclusion of an optype field is the key to its power, creating a flexible abstraction layer. This allows device manufacturers to innovate on their specific PIM architectures while communicating with the host through a single, standardized command. The concept of Architecture-Aware Optimization (AAO) (Section 4.5, page 5), where device-specific timing or bank-grouping information is loaded from SPD, is an excellent mechanism for supporting this diversity without sacrificing compatibility.

3. Holistic System-Level Perspective: The authors recognize that a protocol alone is insufficient. The tight coupling of the PIM-ACT protocol with the PIM request generator in the memory controller and the AT-BLC scheduler demonstrates a strong system-level understanding. The request generator offloads the host CPU from the tedious task of issuing micro-operations, connecting it to a long line of work on "macro instructions" (e.g., TRiM [48], AESPA [27]). The AT-BLC scheduler directly addresses the inevitable performance interference, transforming PIM from a disruptive accelerator into a well-behaved citizen in the memory subsystem.

4. Strong Connection to Real-World Architectures: The work is well-grounded by demonstrating how ComPASS can be applied to existing commercial PIM architectures like Samsung's HBM-PIM and SK Hynix's GDDR6-AiM (Section 4.7, Table 1, page 7). This case study is crucial as it proves the proposal is not merely a theoretical exercise but a viable path toward unifying disparate industry efforts under a common framework.

Weaknesses

1. Hardware Complexity and Overhead are Underexplored: The paper proposes adding a non-trivial "PIM request generator" and additional scheduling logic to the memory controller. While conceptually sound, the potential cost in terms of die area, power consumption, and design complexity is not quantified. For a solution targeting practical adoption, understanding this overhead is crucial. A simple analysis of the storage requirements (request/data buffers) and decoding logic would strengthen the proposal significantly.

2. The Software-Hardware Interface is Abstract: The AT-BLC scheduler relies on the host providing the target completion time (T) and total number of micro-requests (W) for a PIM operation. The paper briefly mentions this is handled by the OS and PIM libraries (Section 4.4, page 5), but this interface is a critical and complex part of the system. How are these values accurately estimated by the runtime? What is the mechanism for passing them to the memory controller? What happens when these predictions are inaccurate? The robustness of the adaptive scheduler depends heavily on the quality of these inputs, and this link feels underdeveloped.

3. Limited Applicability of the Scheduling Model: The evaluation focuses on large, monolithic GEMV operations typical of LLM inference. In this context, a pre-calculated W and T is plausible. However, the future of PIM may include more diverse workloads, such as graph analytics or database queries, which feature more irregular, data-dependent memory access patterns and potentially many smaller, concurrent PIM tasks. It is unclear how well the AT-BLC model would adapt to scenarios where PIM execution is less predictable and cannot be easily characterized by a single W and T pair.

Questions to Address In Rebuttal

1. Could the authors provide an estimate, even if high-level, of the hardware overhead (e.g., buffer sizes in KB, gate count estimate for logic) introduced by the PIM request generator and the additional scheduling queues in the memory controller?

2. Could you please elaborate on the software stack's responsibility for the AT-BLC? Specifically, how does the runtime or driver determine the W and T parameters for a given PIM task, and what is the proposed hardware mechanism for communicating this information to the memory controller?

3. The proposed adaptive scheduler (AT-BLC) appears well-suited for predictable, throughput-oriented PIM workloads like GEMV. Could you discuss how the ComPASS framework, and particularly the AT-BLC, might be extended or adapted to support more dynamic and latency-sensitive PIM workloads with less predictable execution times?

4. The Architecture-Aware Optimization (AAO) concept is very compelling as a way to future-proof the protocol. Do you envision this information being communicated solely through a static mechanism like SPD, or could there be a more dynamic, runtime registration process for new PIM device capabilities?

---

Review Form

Reviewer: The Innovator (Novelty Specialist)

Summary

The authors propose ComPASS, a solution aimed at improving the compatibility and efficiency of integrating Processing-In-Memory (PIM) devices into general-purpose systems. The work identifies two primary challenges: the lack of a standardized PIM protocol and the difficulty of scheduling PIM and conventional memory requests concurrently. The proposed solution consists of three core components:

1. PIM-ACT: A new, unified memory command that leverages unused command space (RFU or NOP bits) in existing DRAM standards to trigger multi-bank PIM operations. An "optype" field within the command allows it to be adapted to different PIM architectures.
2. PIM Request Generator: A hardware unit within the memory controller that offloads the host CPU by receiving high-level "macro requests" and decomposing them into a sequence of low-level "micro requests" for PIM execution.
3. Static and Adaptive Schedulers (ST-BLC and AT-BLC): Scheduling policies that manage the interleaving of PIM and non-PIM requests. While the static balancer uses a fixed threshold, the adaptive balancer dynamically adjusts scheduling priorities based on whether the PIM workload is meeting its Quality-of-Service (QoS) targets.

The central claim is that this combination of a compatible protocol and a QoS-aware scheduler provides a novel and effective solution for processor-PIM collaboration.

Strengths

The primary strength of this work lies in the synthesis of its components to address a practical and timely problem. While the individual concepts have precedents, their integration into a cohesive system architecture designed for compatibility is commendable.

The most novel element is the Adaptive Throughput Balancer (AT-BLC). The concept of a feedback loop where PIM execution progress (Wcur/Tcur vs. W/T') directly influences the memory scheduler's behavior (N) appears to be a new contribution in the context of PIM scheduling. This elevates the work beyond a simple static priority scheme.

The architectural decision to place the PIM request generator inside the memory controller (Section 4.2, Page 4), rather than within the PIM device itself, is a well-reasoned choice. It correctly identifies the limitation of prior work (e.g., TRiM [48], Darwin [32]) where an external generator obscures bank state from the MC, thereby preventing efficient request interleaving. This specific architectural delta is a key enabler for the proposed scheduling policies.

Weaknesses

While the paper presents a well-engineered system, its core ideas, when deconstructed, are largely incremental advancements or applications of known principles from other domains. My primary concern is the degree of fundamental novelty.

1. The PIM-ACT Command: The idea of using reserved/unused command bits to introduce new functionality is a standard industry practice, not a novel academic concept. The true contribution here is the proposed protocol that uses the command, not the mechanism of the command itself. Furthermore, both HBM-PIM and GDDR6-AiM already have mechanisms to trigger multi-bank operations. ComPASS's proposal is a unifying abstraction layer, which is a valuable engineering contribution but a limited conceptual leap. The authors themselves demonstrate in Table 1 (Page 7) that PIM-ACT primarily serves as a wrapper for existing PIM device functionality.

2. The PIM Request Generator: The concept of a hardware unit that translates macro-instructions into micro-operations is not new. This is functionally analogous to DMA controllers, command queue (CQ) mechanisms in NVMe, or instruction decoders in other types of accelerators. As noted in the paper's own related work section (Section 8, Page 12), works like TRiM [48] and Darwin [32] have already proposed instruction generators for PIM. The novelty of ComPASS is limited to the placement of this generator within the MC to enable better scheduling, which, while important, is an architectural refinement rather than a new paradigm.

3. The Static Throughput Balancer (ST-BLC): This is a classic threshold-based or round-robin arbitration policy. The paper acknowledges a recent work [15] that is "conceptually similar to our ST-BLC" (Section 8, Page 12). This suggests that the static scheduling approach is not novel. The paper's main contribution in scheduling is therefore entirely dependent on the adaptive nature of AT-BLC.

In summary, the novelty of this paper is not in the invention of new foundational concepts, but in the clever integration and adaptation of existing ones to the specific problem of PIM-CPU co-execution. The contribution is more of an elegant system design than a fundamental breakthrough.

Questions to Address In Rebuttal

The authors should use the rebuttal to more sharply define the novelty of their contributions against the closest prior art.

1. Regarding the Static Throughput Balancer (ST-BLC), the paper concedes it is "conceptually similar" to the virtual channel-based scheduler in [15]. Can the authors elaborate on the novel delta between ST-BLC and this prior work? If the novelty is minimal, the paper's contribution should be more narrowly framed around the adaptive policy (AT-BLC).

2. The novelty of the PIM Request Generator rests on its placement in the MC. While this enables scheduling, prior works like TRiM [48] also used macro instructions. Beyond location, is there any novelty in the macro-request interface itself (Figure 4c, Page 5) or the decoding logic that allows it to be more general-purpose than prior PIM-specific generators?

3. The PIM-ACT protocol's claim to compatibility is evaluated on two similar GEMV-focused accelerators (HBM-PIM and GDDR6-AiM). How would this protocol generalize to a fundamentally different PIM architecture, such as the general-purpose RISC-based cores in UPMEM-PIM [10]? Would the proposed 6-bit optype and the macro-request format be sufficient to express the diverse instruction set of such a device without becoming a bottleneck or requiring an impractically large optype mapping? Please justify the claimed generality of the protocol beyond the domain of neural network accelerators.