Dissecting and Modeling the Architecture of Modern GPU Cores

---

Review Form

Reviewer: The Guardian (Adverserial Skeptic)

Summary

The authors attempt to reverse engineer the core microarchitecture of modern NVIDIA GPUs (Turing, Ampere, and Blackwell) through a series of microbenchmarks. Based on their inferences, they propose a new core model for the Accel-sim framework. The paper claims this new model significantly improves simulation accuracy, reducing the Mean Absolute Percentage Error (MAPE) from over 34% to approximately 13.5% for an Ampere GPU when compared to the baseline Accel-sim. The primary contributions center on elucidating the function of compiler-managed control bits for dependency tracking, a new issue scheduler policy, and refined models for the register file and memory pipeline.

Strengths

1. The ambition of the work is commendable. Tackling the opaque nature of modern commercial GPU architectures is a difficult and labor-intensive task.
2. The paper provides a wealth of specific, quantitative data in its tables (e.g., Table 2, Memory instruction latencies) which, if accurate, could serve as a useful reference.
3. The authors have clearly invested significant effort in creating and running numerous microbenchmarks to generate the timing data that forms the basis of their hypotheses.

Weaknesses

My primary concerns with this manuscript are the opacity of the methodology, the logical leaps made from limited evidence, and the potential for an unfair baseline comparison, which may inflate the significance of the reported results.

1. Methodological Rigor and Reproducibility: The reverse engineering methodology described in Section 3 is presented anecdotally. The authors provide two illustrative examples (Listing 1, scheduler policy) but fail to describe the systematic process and full scope of their investigation. How many microbenchmark variants were run to derive each conclusion? What was the process for ruling out alternative hypotheses? Without a rigorous account of the methodology, the findings appear to be a collection of observations from hand-picked scenarios rather than the result of a comprehensive, scientific dissection. The work is therefore difficult to validate or trust.

2. Overstatement of Inferred "Discoveries": The paper presents its hypotheses as definitive facts. For instance, the proposed "Compiler Guided Greedy Then Youngest (CGGTY)" issue policy (Section 5.1.2) is a strong claim based on insufficient evidence. Figure 4 shows only three specific scenarios with homogeneous, independent instructions. This is hardly a comprehensive evaluation required to definitively characterize a complex scheduler. How does the policy behave under heavy memory contention, with long-latency instructions, or around synchronization primitives? The evidence supports CGGTY as a plausible hypothesis for a limited case, not as a confirmed mechanism. This pattern of over-claiming permeates the paper (e.g., register file organization, front-end policy).

3. Questionable Baseline for Comparison: The validation in Section 7 hinges on a comparison against "the Accel-sim simulator." It is well-known that the public Accel-sim model is largely based on the much older Tesla/Fermi architecture. Comparing a new model tailored for Ampere against a decade-old architectural model and then claiming a >20% reduction in MAPE is not a fair or insightful comparison. It proves that Ampere is different from Fermi, which is already known. The authors have not demonstrated that their model is superior to a state-of-the-art academic model properly configured for a more recent architecture. This appears to be a "strawman" comparison.

4. Unsupported Claims Regarding the Blackwell Architecture: The claim to have modeled the very recent Blackwell architecture with high accuracy (17.41% MAPE, Table 4) is not adequately substantiated. In Section 6, the authors mention only supporting new SASS instructions and extending the L2 hashing function. These are minor adjustments and it is highly improbable that they capture the full microarchitectural evolution from Ampere to Blackwell. This claim feels premature and potentially misleading.

5. Contamination of Validation Results: A critical detail is buried at the end of Section 6: for some kernels, where SASS code was unavailable, a "hybrid mode" using "traditional scoreboards" was employed. This is a major confounding variable. The paper’s central thesis is about the accuracy of a new model based on compiler-set control bits. However, the final accuracy numbers are an amalgam of this new model and an entirely different, traditional dependency model. The authors do not state for which of the 128 benchmarks this hybrid mode was used, making it impossible to assess the true accuracy of their primary contribution.

Questions to Address In Rebuttal

The authors must provide clear and concise answers to the following questions to alleviate the concerns raised.

1. Regarding your methodology (Section 3), can you provide a quantitative summary of the reverse engineering effort? For example, for the issue scheduler policy alone, how many distinct microbenchmark scenarios (beyond the three in Figure 4) were constructed and tested to validate the CGGTY policy against other plausible alternatives (e.g., LRR, GTO)?

2. Regarding the baseline (Section 7.2), please specify the exact version and configuration of Accel-sim used as the baseline. Justify why this configuration, which primarily models an older architecture, is considered a fair and relevant baseline for comparison against modern Ampere/Turing hardware.

3. Regarding the "hybrid mode" for dependency tracking mentioned in Section 6, please provide a list of the benchmarks from your validation suite (Table 3) that required this mode. Furthermore, present a separate MAPE analysis for the subset of benchmarks that ran purely on your proposed control-bit model versus the subset that used the hybrid scoreboard model.

4. Regarding your Blackwell model, please provide a comprehensive list of all microarchitectural changes implemented beyond the Ampere model. How do you justify that this limited set of changes is sufficient to claim an accurate model of the Blackwell core architecture, rather than simply an "Ampere-plus" model that coincidentally performs well on your chosen benchmarks?

---

Review Form:

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper presents a comprehensive reverse-engineering study of modern NVIDIA GPU core microarchitectures, from Turing to the latest Blackwell generation. The authors' primary contribution is bridging the significant and growing gap between the aging architectural models used in academic simulators (like Accel-sim) and the reality of contemporary commercial hardware. Through meticulous micro-benchmarking using hand-written SASS code, the work uncovers crucial, previously undocumented design details. These include the compiler-driven mechanism for managing data dependencies (replacing traditional hardware scoreboards), a novel "Compiler Guided Greedy Then Youngest" (CGGTY) issue policy, and detailed models of the multi-banked register file, its associated cache, and the memory pipeline. The authors integrate these findings into a new, publicly available simulator model, demonstrating a dramatic improvement in accuracy—reducing the Mean Absolute Percentage Error (MAPE) from over 34% to 13.45% on an Ampere GPU compared to the baseline Accel-sim. This work serves as both a significant contribution to the scientific understanding of modern GPU design and a vital update to the community's research infrastructure.

Strengths

1. High-Impact Contribution to Research Infrastructure: The most significant strength of this work is its direct and potent address of the "relevance crisis" in academic GPU simulation. For over a decade, much of the community's research has been predicated on simulators modeling architectures from the Tesla era (circa 2006). This paper performs the herculean task of updating our collective understanding and providing a tangible, validated tool that will elevate the quality and relevance of future research in areas like scheduling, memory systems, and compiler optimizations for GPUs. The validation across three major architectural generations (Turing, Ampere, Blackwell) underscores its immediate and likely future utility.

2. Unveiling a Fundamental Architectural Paradigm Shift: The paper's detailed exposition of the software-hardware co-design for dependency management (Section 4, page 3 and Section 7.5, page 13) is a profound insight. The move away from complex, power-hungry hardware scoreboards towards compiler-managed Stall counters and Dependence counters represents a major philosophical shift in GPU design. This finding contextualizes modern GPUs within a broader architectural trend towards compiler-led complexity management, reminiscent of VLIW principles. This discovery alone is a major contribution to computer architecture literature.

3. Methodological Depth and Rigor: The authors' methodology of using carefully crafted, low-level SASS microbenchmarks to probe the hardware's behavior is commendable. This is a non-trivial undertaking that requires deep expertise. The detailed examples provided (e.g., Listing 1 for register file conflicts, the analysis in Figure 4 for the scheduler policy) lend significant credibility to their inferred models. This empirical, bottom-up approach is exactly what is needed to demystify these otherwise black-box systems.

4. Holistic and Coherent Model: Unlike prior works that often focused on reverse-engineering a single component (e.g., a cache, a specific unit), this paper presents a coherent model of the entire core pipeline. It successfully connects the dots between the front-end fetch/decode, the issue logic, the register file, and the memory subsystems, showing how they interact. The discovery of the CGGTY issue policy and its interaction with the compiler-set Yield bit is a perfect example of this holistic view.

Weaknesses

As a contextual analyst, I view these less as flaws and more as inherent limitations or opportunities for deeper discussion.

1. Inherent Ambiguity of Reverse Engineering: The work constructs a highly plausible and well-validated model, but it is ultimately an inferred model. The authors are commendably transparent about this (e.g., acknowledging in Section 5.1, page 6, that they "could not find a model that perfectly fits all the experiments"). The paper could benefit from a brief, consolidated discussion on the limitations of this approach and the confidence bounds on their conclusions. While the MAPE reduction is impressive, it's important to frame the resulting model as a powerful and accurate approximation, not necessarily ground truth.

2. Limited Exploration of the "Why": The paper excels at detailing the "what" (the mechanisms) and the "how" (their operation). However, it offers little speculation on the "why"—the architectural trade-offs that likely motivated these design choices. For example, why did NVIDIA pivot so heavily to compiler-managed dependencies? Was the primary driver area savings, power reduction, clock speed improvements, or simpler hardware verification? Adding a short discussion section to hypothesize on these design rationales would elevate the paper from a descriptive masterpiece to a more complete architectural analysis.

3. NVIDIA-Centric Focus: The paper's deep dive is exclusively on NVIDIA architectures. This is a practical and understandable choice given NVIDIA's market position and the sheer scope of the work. However, it implicitly positions the NVIDIA way as the modern GPU way. While a brief comparison to AMD's waitcnt is included (Section 5, page 5), the work would be even more valuable to the broader community if it could contextualize its findings by more explicitly contrasting them with the known design philosophies of other major GPU vendors.

Questions to Address In Rebuttal

1. The core of your work rests on a complex, inferred model of the GPU pipeline. Beyond the aggregate MAPE scores, were there any specific micro-architectural behaviors or corner-case instruction sequences that your final model still struggled to accurately predict? Discussing these outliers could provide valuable clues for future reverse-engineering efforts.

2. Could you elaborate on the likely architectural motivations behind NVIDIA's shift to the compiler-managed dependency system? What are the primary trade-offs (e.g., in terms of area, power, performance, and compiler complexity) compared to the traditional hardware scoreboard approach that your own results in Section 7.5 (page 13) so clearly favor?

3. This work represents a monumental effort to re-synchronize academic tools with a fast-moving industry target. Looking forward, how sustainable is this manual, intensive reverse-engineering process? Does your experience suggest any potential avenues for semi-automating the discovery of such microarchitectural properties for future GPU generations, to ensure the community's research tools do not fall so far behind again?

---

Review Form

Reviewer: The Innovator (Novelty Specialist)

Summary

The authors present a work of microarchitectural reverse engineering and modeling for modern NVIDIA GPU cores (Turing, Ampere, and Blackwell). The central claim is that existing academic simulators are based on antiquated architectural assumptions (dating back to the Tesla architecture) and are therefore inaccurate. This paper seeks to remedy this by discovering and modeling key features of modern designs. The core novel claims center on the detailed mechanics of a compiler-hardware co-design for managing data dependencies, a specific issue scheduler policy ("Compiler Guided Greedy Then Youngest"), the internal structure of the register file (RF) and its cache, and the absence of an operand collector stage. These findings are integrated into the Accel-sim framework, and the resulting model is shown to be significantly more accurate than the baseline.

Strengths

The primary strength of this paper is the novelty of its empirical findings. While many papers propose new architectural ideas, this work provides a rare and valuable service by reverse-engineering and documenting a complex, proprietary, state-of-the-art commercial architecture. The novelty is not in inventing new mechanisms, but in revealing existing ones for the first time in the public domain.

1. Novel Semantics of Dependence Management: The most significant novel contribution is the detailed elucidation of the software-based dependence management system (Section 4, pages 3-4). The existence of compiler-inserted "hints" is not new; this was observed in the Kepler architecture and noted by Jia et al. [46, 47] for Volta/Turing. However, prior work has not provided a functional, mechanistic model. This paper's detailed breakdown of the Stall counter for fixed-latency hazards and the system of six Dependence counters (SBx registers) with producer-increment/consumer-decrement semantics for variable-latency hazards is a genuinely new contribution to public knowledge. The explanation of the DEPBAR.LE instruction and the Yield bit provides a complete, plausible model that has not been described elsewhere.

2. Refutation of the Operand Collector Assumption: The paper makes a strong claim that modern NVIDIA GPUs do not use an operand collector unit (Section 5.3, page 8). This is a direct and novel refutation of a core assumption in the widely-used GPGPU-Sim and Accel-sim models. The authors' reasoning—that variable latency from an operand collector would break the compiler's ability to calculate static stall cycles—is logical and compelling. Disproving a long-held assumption in a dominant model is a significant form of novelty.

3. Specific Characterization of the Issue Scheduler: While greedy scheduling policies are well-known, the specific "Compiler Guided Greedy Then Youngest" (CGGTY) policy (Section 5.1.2, page 6) is a novel finding. This moves beyond the canonical "Greedy Then Oldest" (GTO) policy and demonstrates a tight coupling between the scheduler's fallback mechanism (picking the youngest warp) and the compiler's explicit instructions to yield the pipeline (Yield bit). The detailed experimental timeline in Figure 4 provides strong evidence for this specific, previously undocumented policy.

4. Timeliness of Blackwell Modeling: The claim of being the first work to provide an accurate, validated model for the NVIDIA Blackwell architecture (Section 1, page 1 and Section 7.2, page 12) is a strong point of novelty. Given the recent release of this architecture, this contribution is at the cutting edge of academic GPU modeling.

Weaknesses

The weaknesses of the paper, from a novelty perspective, lie in areas where the contributions are more confirmatory or incremental rather than fundamentally new concepts.

1. Incremental Novelty of the RF Cache: The concept of a compiler-managed register file cache for GPUs is not a new idea. The paper's model is explicitly and correctly identified as being "similar to the work of Gebhart et al. [34]" (Section 5.3.1, page 8). While the reverse-engineered parameters—such as one entry per bank and specific software management via a "reuse bit"—are new empirical details for NVIDIA's implementation, the core architectural concept has been established in prior art for over a decade. The delta here is one of implementation-specifics, not of fundamental mechanism.

2. Assumed Prefetcher Design: The front-end model relies on a simple stream buffer for instruction prefetching (Section 5.2, page 7). This is a classic mechanism, and the authors state that they "suspect it is a simple scheme" and "assume" its size is 8 entries. While this assumption is validated by the model's overall accuracy, the contribution lacks the rigor of the reverse engineering applied elsewhere. As a contribution, proposing a standard, well-known prefetcher is not novel.

3. Known Trade-off Analysis: The analysis in Section 7.5 (page 13) comparing the discovered software-based dependence management to a traditional hardware scoreboard is an evaluation of a known design trade-off. The conclusion that a software-managed approach has lower area overhead is expected. The value is in quantifying this trade-off with realistic parameters, but this does not represent a new conceptual insight into computer architecture.

Questions to Address In Rebuttal

1. On the Fetch Policy's Novelty: The front-end fetch policy is assumed to mirror the issue scheduler's greedy logic (Section 5.2, page 7). This is presented as a plausible assumption rather than a direct finding. What experiments were performed to rule out other well-known fetch policies (e.g., round-robin, ICOUNT)? How much of the model's accuracy hinges on this specific assumption, which appears less rigorously proven than other claims in the paper?

2. Clarifying the Delta vs. Gebhart et al.: The paper acknowledges the similarity of its RF cache model to that of Gebhart et al. [34]. Beyond the parametric differences and the lack of a two-level scheduler, could the authors more sharply define the novel architectural principle, if any, that their findings reveal? Is NVIDIA's implementation merely a modern instantiation of the 2011 concept, or is there a more fundamental conceptual difference that this review has missed?

3. On the Generality and Longevity of Findings: The detailed semantics of the control bits and dependence counters are a key contribution. These were reverse-engineered across three recent architectures. Based on this, do the authors have evidence to suggest that this specific mechanism is a stable, long-term feature of NVIDIA's design philosophy, or is it an artifact of a particular architectural era? How much risk is there that the next generation invalidates this detailed model, thereby limiting the durable novelty of these specific findings?