HEAT: NPU-NDP HEterogeneous Architecture for Transformer-Empowered Graph Neural Networks

---

Review Form:

Reviewer: The Guardian (Adverserial Skeptic)

Summary

The authors propose HEAT, a heterogeneous NPU-NDP architecture designed to accelerate inference for Transformer-empowered Graph Neural Networks (TF-GNNs). The architecture assigns the compute-intensive Transformer frontend to a Neural Processing Unit (NPU) and the memory-intensive Graph Neural Network (GNN) backend to a DIMM-based Near-Data Processing (NDP) unit. The paper introduces three primary optimizations: (1) a topology-aware encoding scheme that quantizes vertex features based on their degree; (2) a two-phase subgraph scheduling strategy to reduce DRAM bank conflicts and redundant memory accesses; and (3) a decoupled dataflow to enable parallel execution of the NPU and NDP. The authors claim significant speedup and energy efficiency gains over a high-performance GPU and state-of-the-art specialized accelerators.

Strengths

1. Problem Formulation: The paper correctly identifies a relevant and challenging problem. The performance bottleneck created by the sequential, two-phase nature of TF-GNNs—combining a compute-bound Transformer with a memory-bound GNN—is well-established. The roofline analysis in Figure 3(b) (page 4) provides clear motivation for a heterogeneous approach.

2. Architectural Concept: The high-level decision to map the Transformer to a compute-centric NPU and the GNN to a memory-centric NDP is logical and well-aligned with the computational characteristics of each model component.

Weaknesses

The paper's claims rest on a foundation of overly simplistic assumptions, questionable experimental design, and insufficiently justified heuristics. The reported performance gains appear to be a product of these weaknesses rather than a demonstrated architectural superiority.

1. Under-justified Topology-Aware Encoding: The central premise of the topology-aware encoding (Section 4, page 5) hinges on the simplistic assumption that vertex degree is a sufficient proxy for importance. The authors provide only a coarse masking experiment (Figure 4, page 4) as justification. This neglects more nuanced centrality metrics (e.g., PageRank, betweenness centrality) that often provide a more accurate measure of a node's influence in a graph. The choice of degree feels arbitrary and is not rigorously defended against alternatives, making the entire optimization seem superficial. Furthermore, the selection of the hyperparameter α (Section 6.4.2, page 11) is presented as a simple trade-off, but no sensitivity analysis is provided to show how this choice interacts with different graph structures or model architectures.

2. Questionable Optimality of Scheduling Heuristics: The proposed subgraph scheduling strategy (Section 5.3, page 7) is based on a series of greedy heuristics. While the intra-SG scheduling algorithm aims to balance bank accesses, its optimality is not discussed. The paper fails to quantify how close the resulting schedule is to an optimal or even a lower-bound solution for bank conflict minimization. Similarly, the inter-SG scheduling relies on a heuristic that "subgraphs sampled from neighboring starting points tend to have more overlapped vertices." The generalizability of this observation across diverse and complex graph structures is asserted, not proven. The entire scheduling component lacks theoretical grounding.

3. Unfair and Potentially Misleading Baselines: The primary weakness lies in the evaluation methodology (Section 6.1, page 9).

   • The "FACT+MEGA" baseline appears to be a strawman. The authors have constructed a baseline by serially chaining two independent, state-of-the-art accelerators. This naive composition is inherently limited by the sequential data dependency that HEAT is specifically designed to break. It does not represent a genuinely co-designed heterogeneous competitor and is therefore an unfair point of comparison. HEAT's outperformance of this baseline is expected and fails to demonstrate superiority over a competently designed alternative.
   • The performance comparison against the A100 GPU lacks critical details. It is unclear what level of software optimization (e.g., CUDA graph fusion, optimized libraries) was applied to the GPU baseline. A poorly optimized GPU baseline can artificially inflate the speedup of a custom accelerator.

4. Insufficient Modeling of System-Level Overheads: The evaluation relies on simulation (ONNXim, DRAMsim3), but the paper provides insufficient detail on the modeling of the interconnect between the host, NPU, and NDP. The "NPU Memory Bus" in Figure 7 (page 6) is a black box. The practical overheads of communication, synchronization, and data coherency in a real system with such a decoupled dataflow are non-trivial and could significantly erode the claimed performance gains. The paper seems to assume an idealized communication fabric.

5. Neglect of Dynamic Graph Scenarios: The paper dismisses the cost of its offline scheduling algorithms by comparing it to the one-time cost of model training (Table 5, page 9). This is a critical omission for inference workloads. In many real-world applications (e.g., social networks, recommendation systems), graphs are dynamic and evolve over time. The proposed offline approach would require frequent and costly re-computation, rendering it impractical. The paper completely fails to address the applicability of its methods to dynamic graphs.

Questions to Address In Rebuttal

1. Please provide a more rigorous justification for using vertex degree as the sole metric for importance in your topology-aware encoding. Have you evaluated other centrality metrics, and if so, how do they compare in the accuracy-performance trade-off?
2. Can the authors provide an analysis of the optimality of the greedy subgraph scheduling algorithm? For instance, how does it compare to a theoretical upper bound or a more complex scheduling method like simulated annealing for a small, representative graph?
3. Please defend the fairness of the FACT+MEGA baseline. Why is a simple sequential combination of two distinct accelerators a valid point of comparison for your co-designed architecture, rather than a hypothetical but more fairly integrated heterogeneous baseline?
4. Provide more details on the communication model assumed between the NPU and NDP. What bandwidth and latency are assumed for the "NPU Memory Bus," and how do these assumptions impact the overall performance, especially in the decoupled dataflow? Please conduct a sensitivity analysis on these parameters.
5. How would the proposed offline scheduling strategies adapt to dynamic graphs where the topology changes over time? What is the overhead of re-computing the schedule, and at what rate of graph change does this overhead nullify the benefits of the scheduling?

---

Review Form:

Reviewer: The Synthesizer (Contextual Analyst)

Summary

The authors present HEAT, a heterogeneous hardware architecture designed to accelerate Transformer-empowered Graph Neural Networks (TF-GNNs). The paper identifies a key challenge in this emerging model class: the front-end Transformer is compute-bound, while the back-end GNN is memory-bound, and existing accelerators typically focus on only one of these domains. HEAT addresses this by co-designing a Neural Processing Unit (NPU) for the Transformer and a Near-Data Processing (NDP) unit for the GNN.

The central contribution is not merely the hardware itself, but a set of co-design principles that exploit the coupling between the Transformer front-end and the GNN back-end. These include: (1) a topology-aware encoding scheme that uses vertex importance (degree) from the graph to apply variable precision quantization in the Transformer, reducing its computational load; (2) a subgraph scheduling strategy for the GNN that bundles and reorders subgraphs to mitigate DRAM bank conflicts and improve data locality; and (3) a decoupled dataflow that breaks the strict sequential dependency between the two stages, enabling parallel execution on the NPU and NDP. The paper demonstrates significant speedup and energy efficiency gains over a high-performance GPU and state-of-the-art specialized accelerators.

Strengths

1. Novel Cross-Domain Co-Optimization: The key intellectual contribution of this paper is the idea of using information from one domain (graph topology) to optimize computation in another (Transformer encoding). The topology-aware encoding scheme (Section 4, page 5) is an elegant insight. It recognizes that not all nodes in a graph are equally important and that this importance, relevant to the GNN, can be back-propagated to guide the precision of the feature extraction in the Transformer. This is a powerful co-design principle that moves beyond optimizing individual components in isolation.

2. A Holistic, System-Level Perspective: The authors have successfully avoided the pitfall of local optimization. Instead of just bolting a Transformer accelerator onto a GNN accelerator, they have re-architected the entire execution flow. The decoupled dataflow (Section 5.4, page 8), enabled by their heterogeneous hardware split, is a prime example of this. It directly addresses the pipeline bubble that would otherwise form due to the inherent dependency, showing a deep understanding of the full system's performance limiters.

3. Addresses a Timely and Forward-Looking Problem: This work is situated at the confluence of several important trends: the rise of large language models (Transformers), their integration into structured data analysis (GNNs), and the persistent need for specialized hardware. TF-GNNs represent a class of complex, multi-stage AI models that are becoming more prevalent. This paper provides not just a point solution, but a potential architectural template for accelerating future composite AI systems.

4. Strong Supporting Optimizations: The subgraph scheduling strategy (Section 5.3, page 7) is a well-reasoned and effective technique for tackling the notorious memory access challenges of GNNs. The distinction between intra-SG scheduling (for bank parallelism) and inter-SG scheduling (for temporal locality via a Hot Buffer) is particularly clever and demonstrates a nuanced approach to memory system optimization.

Weaknesses

My critiques are less about fundamental flaws and more about the boundaries and future implications of the proposed ideas.

1. Limited Exploration of "Importance" Heuristics: The entire topology-aware encoding scheme hinges on using vertex degree as a proxy for importance. While the authors validate this heuristic empirically (Figure 4, page 4), vertex degree is a relatively simple centrality measure. In more complex graphs, other metrics like PageRank, betweenness centrality, or learned attention scores might be better indicators of importance. The work would be strengthened by a discussion on the sensitivity of the architecture to this choice and its generalizability beyond degree-based importance.

2. Static Nature of the Optimizations: The proposed optimizations, particularly the topology-aware encoding and subgraph scheduling, appear to be performed offline based on a static graph structure. This is suitable for many inference scenarios, but the paper does not address how HEAT would perform in dynamic settings, such as social networks with evolving connections or real-time recommendation systems where graph structures change frequently. This static assumption limits the applicability to a subset of real-world problems.

3. Potential Programmability and Abstraction Challenges: The co-designed nature of HEAT, while powerful, implies a tight coupling between the algorithm and the hardware. It is unclear how a developer would program such a system for a new or slightly different TF-GNN model. The paper would benefit from a brief discussion on the software stack or compiler support required to map new models onto this heterogeneous architecture and manage its unique dataflow.

Questions to Address In Rebuttal

1. Could the authors comment on the sensitivity of their topology-aware encoding scheme to the choice of vertex importance metric? Have they considered or run preliminary experiments with other metrics (e.g., PageRank) and how might that impact the trade-off between computation reduction and accuracy?

2. The decoupled dataflow is a key enabler of parallelism. Could the authors provide more insight into the overhead and management of the "Encode Table" (Section 5.4, page 8)? Specifically, how does the system handle synchronization between the NPU and NDP to ensure vertex features are ready when needed without introducing significant control overhead?

3. How would the proposed subgraph scheduling strategy adapt to a streaming or dynamic graph scenario where the graph topology changes over time? Would the intra-SG and inter-SG schedules need to be recomputed frequently, and what would be the associated overhead?

---

Review Form

Reviewer: The Innovator (Novelty Specialist)

Summary

This paper proposes HEAT, a heterogeneous NPU-NDP architecture designed to accelerate Transformer-empowered Graph Neural Networks (TF-GNNs). The authors identify the performance challenges arising from the compute-bound Transformer front-end and the memory-bound GNN back-end. HEAT's contributions are presented as a three-pronged optimization strategy: (1) A topology-aware encoding scheme that uses GNN vertex importance (proxied by node degree) to apply variable-precision quantization to the front-end Transformer. (2) A two-phase subgraph scheduling strategy for the GNN back-end that bundles subgraphs to mitigate DRAM bank conflicts and reorders execution to improve data locality for "hot" vertices. (3) A decoupled dataflow that enables pipelined execution by having the Transformer on the NPU generate only the vertex features currently required by the GNN on the NDP.

My analysis concludes that the work contains one core, genuinely novel idea—the co-design linking graph topology to Transformer quantization. The other two contributions, while well-engineered and effective, are applications of well-established principles from the systems and architecture communities to the specific TF-GNN workload. The novelty in those areas lies in the specific heuristics and implementation, rather than in the concepts themselves.

Strengths

1. Novel Cross-Domain Optimization: The most significant and novel contribution is the topology-aware encoding algorithm (Section 4, page 5). The idea of using a structural property from the back-end graph computation (node degree) to guide the precision of the front-end text-encoding Transformer is a genuinely new co-design principle. Prior work on Transformer quantization is typically data-driven (e.g., ANT), while GNN quantization focuses on features within the GNN domain (e.g., MEGA). Linking the two is a clever insight specific to the TF-GNN model class and represents a clear conceptual advance.

2. Holistic System Co-design: The authors present a complete system that addresses bottlenecks across the entire TF-GNN pipeline. Rather than focusing on just the Transformer or just the GNN in isolation, the architecture and its accompanying algorithms consider the interplay between the two, which is commendable.

3. Well-Defined Problem and Solution: The paper does an excellent job of motivating the problem with clear profiling data (Figure 3, page 4) and systematically proposing solutions for each identified challenge. The connection between an observation (e.g., irregular memory access) and the proposed solution (e.g., subgraph scheduling) is logical and well-articulated.

Weaknesses

My critique focuses on the degree of conceptual novelty of the latter two contributions, which appear to be expert applications of existing ideas rather than the creation of new ones.

1. Incremental Novelty in Subgraph Scheduling: The subgraph scheduling strategy (Section 5.3, page 7), while effective, is conceptually an extension of known techniques. The problem of optimizing GNN memory access by reordering computations to improve locality and mitigate bank conflicts is well-trodden ground in GNN accelerator literature (e.g., I-GCN [12], GraNDe [59]). The authors' contribution is a specific two-phase greedy heuristic: bundling subgraphs based on complementary Bank_MAX/Bank_MIN access patterns is a new algorithm, but the underlying principle of co-scheduling tasks to balance resource utilization is a classic systems problem. Similarly, reordering based on neighboring starting points to exploit "hot vertices" is a direct application of the principle of temporal locality. The novelty here is algorithmic and heuristic, not conceptual.

2. Decoupled Dataflow as Applied Pipelining: The proposed decoupled dataflow (Section 5.4, page 8) is a direct application of producer-consumer pipelining to the TF-GNN workload. The "naive dataflow" (Figure 11b) is a classic bulk-synchronous process. The "decoupled dataflow" (Figure 11c) introduces fine-grained pipelining by enabling the consumer (NDP) to start as soon as the producer (NPU) provides the first piece of necessary data. This is a fundamental optimization in computer systems. The use of an "Encode Table" to avoid re-computation is a form of memoization, another standard technique. While this is excellent system engineering and crucial for performance on their heterogeneous architecture, it does not represent a fundamentally new concept in dataflow or execution models.

3. Complexity vs. Benefit Justification: The system introduces significant offline and online complexity (topology analysis, two-phase scheduling, inter-module control for the decoupled dataflow). The performance gains are substantial. However, much of this gain comes from applying known systems principles. The primary novel idea (topology-aware quantization) provides a 1.5x speedup in the ablation study (Figure 17, page 11, base to v1). The subsequent, more conventional optimizations (scheduling and pipelining) provide the remaining boost to the final performance. This suggests that while the system is effective, its performance is built upon a single core novel idea augmented by strong, but standard, engineering.

Questions to Address In Rebuttal

1. On Subgraph Scheduling: Could the authors please clarify the conceptual novelty of the intra-SG scheduling algorithm beyond being a new greedy heuristic for the well-known problem of DRAM bank-conflict reduction in irregular applications? What is the core insight that distinguishes it from prior work on data rearrangement or task scheduling for GNNs or other graph algorithms?

2. On the Generality of the Core Novelty: The paper's primary novel claim hinges on using node degree as a proxy for vertex importance to guide Transformer quantization. This is compelling for the social network and citation network graphs evaluated. How does this principle generalize to graph types where high-degree nodes are not necessarily the most informative (e.g., hubs in protein-protein interaction networks, or in knowledge graphs where relation types are more important than connectivity)? Is there a risk of this optimization being a domain-specific heuristic rather than a general principle for TF-GNNs?

3. On System-Level Conflicts: The decoupled dataflow necessitates fine-grained, on-demand feature generation from the Transformer on the NPU. This contrasts with conventional high-performance Transformer inference, which relies heavily on batching large numbers of inputs to maximize PE utilization. Does this on-demand execution model introduce significant throughput degradation or efficiency loss on the NPU, and if so, how does HEAT mitigate this? Is there a trade-off between latency reduction from pipelining and the NPU's overall throughput?