GCC: A 3DGS Inference Architecture with Gaussian-Wise and Cross-Stage Conditional Processing

---

Review Form

Reviewer: The Guardian (Adversarial Skeptic)

Summary

The authors present GCC, a hardware accelerator architecture for 3D Gaussian Splatting (3DGS) inference. The central thesis is that the conventional "preprocess-then-render" tile-wise dataflow is fundamentally inefficient. To address this, they propose a "Gaussian-wise" dataflow, which processes one Gaussian completely before moving to the next, coupled with "cross-stage conditional processing" to skip work dynamically. While the motivation is sound, the submission suffers from several methodological and analytical weaknesses that call its central claims of superior efficiency into question. The proposed dataflow appears to trade one set of problems (redundant loads) for another, more severe set (serialization, complex control flow, and scalability issues), and the evaluation fails to adequately quantify these trade-offs.

Strengths

1. Problem Identification: The authors correctly identify and quantify two well-known inefficiencies in the standard 3DGS rendering pipeline: the significant fraction of preprocessed Gaussians that are ultimately discarded (Figure 2a) and the repeated loading of the same Gaussian's data for different tiles (Figure 2b). The motivation for a new dataflow is, therefore, well-grounded.

2. Co-design Principle: The work attempts a full-stack co-design, proposing a novel dataflow and a hardware architecture tailored to it. This holistic approach is commendable in principle.

Weaknesses

My primary concerns with this work are threefold: 1) the proposed Gaussian-wise blending introduces a critical performance pathology that is not analyzed, 2) the cost of the proposed boundary identification method is not justified against simpler alternatives, and 3) the evaluation framework contains significant omissions that weaken the conclusions.

1. Unanalyzed Performance Bottleneck in Blending Stage: The fundamental premise of a Gaussian-wise pipeline is that each Gaussian is rendered to completion across all pixels it covers. This necessitates random-access writes to an on-chip Image Buffer. The paper acknowledges in Section 4.5 (page 9) that strict back-to-front blending order must be maintained at the block level. The authors state: "If a later Gaussian in the sorted sequence attempts to access a block whose previous Gaussian has not yet completed processing, the pipeline stalls..." This is a critical admission. In any reasonably complex scene, multiple Gaussians will overlap within the same pixel block. This creates a high probability of data hazards, leading to frequent pipeline stalls. The paper provides zero quantitative analysis of the frequency or duration of these stalls. Without this data, the claimed performance improvements are unsubstantiated, as this serialization hazard could easily negate any gains from reduced data loading.

2. Insufficient Justification for Alpha-based Boundary Identification: The authors propose a runtime, breadth-first search (BFS) algorithm (Algorithm 1, page 6) to identify the exact pixel footprint of each Gaussian. While Table 1 (page 6) shows this method processes fewer pixels than AABB or OBB methods, this comparison is misleading. It compares the number of pixels rendered, not the total work done. The proposed BFS-style algorithm introduces significant control flow overhead (queue management, neighbor checking, visited map updates) that must be executed for every single rendered Gaussian. The paper fails to provide a cycle-level or runtime cost comparison between their complex identification algorithm and simply iterating through all pixels within a conservative OBB. It is entirely possible that the simpler data path of the OBB method, despite performing more alpha calculations, is faster overall due to the absence of this complex control logic. The claim of efficiency is therefore unsupported.

3. Flawed and Incomplete Evaluation:

   • The "Compatibility Mode" (Section 4.6, page 9) is presented as a feature, but it is an admission of a fundamental scaling limitation. The Gaussian-wise approach requires the entire image's transmittance and color buffers to be on-chip, which is infeasible for high resolutions. The proposed solution is to tile the image into sub-views, which partially re-introduces the very tile-wise processing paradigm the authors claim is inefficient. Figure 6 (page 7) shows that as the sub-view size decreases, the number of redundant Gaussian invocations increases. The evaluation does not analyze the performance impact of this mode on a high-resolution target (e.g., 1080p or 4K), which would require significant tiling and likely erode the claimed benefits over GSCore.
   • The GPU comparison in Section 6 (page 12) is weak. The authors state that their dataflow performs poorly on GPUs due to data races requiring atomic operations. This is not a justification for custom hardware; it is strong evidence that the proposed dataflow is inherently hostile to massively parallel architectures. An effective dataflow should be scalable, not fundamentally serial in its memory access pattern. This result undermines the generality and soundness of the proposed dataflow itself.

4. Redundant Processing Stages: The dataflow in Figure 3 (page 5) includes a global "Gaussian Grouping by Depth" (Stage I) followed by an "intra-group depth sorting" in Stage III. If a global sort order is established in Stage I, the necessity of a second, intra-group sort is unclear and seems redundant. This adds complexity and cost without clear justification.

Questions to Address In Rebuttal

1. Provide a quantitative analysis of the pipeline stalls mentioned in Section 4.5. For each benchmark, please report the total number of stall cycles as a percentage of total execution cycles. How does this scale with scene complexity (i.e., the number of overlapping Gaussians)?

2. Provide a direct, cycle-level comparison of the proposed Alpha-based Boundary Identification algorithm versus a simpler OBB-based approach. The comparison should not be based on the number of pixels processed (as in Table 1), but on the total execution time (or cycles) for the entire identification-plus-blending stage for a single Gaussian.

3. The "Compatibility Mode" is critical for practical deployment. Provide a performance and energy breakdown for rendering a standard 1920x1080 frame, which would require your architecture to process 120 sub-views of 128x128. How do the total number of Gaussian loads and overall performance in this mode compare to the GSCore baseline at the same resolution?

4. Please justify why the poor performance of your dataflow on a GPU (Section 6) should be interpreted as a need for custom hardware, rather than as an inherent parallelism flaw in the Gaussian-wise processing model itself.

5. Clarify the necessity of the second sorting stage (intra-group sorting in Stage III). Given the global depth grouping in Stage I, what specific ordering problem does this second sort solve, and what is its computational cost?

---

Review Form

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper introduces GCC, a novel hardware accelerator architecture for 3D Gaussian Splatting (3DGS) inference. The core contribution is a fundamental rethinking of the 3DGS dataflow, moving away from the conventional, GPU-inspired "preprocess-then-render" and tile-wise paradigm. Instead, the authors propose a "Gaussian-wise" rendering pipeline, augmented by "cross-stage conditional processing." In this new dataflow, Gaussians are sorted globally by depth once per frame and then processed sequentially. The entire pipeline—from 3D-to-2D projection to final color blending—is executed for a single Gaussian before moving to the next. This structure allows the system to dynamically skip all processing for Gaussians that are occluded or otherwise do not contribute to the final image, addressing major inefficiencies in the standard approach. The paper also introduces a novel alpha-based boundary identification method to more accurately define a Gaussian's region of influence, further reducing computational waste. The proposed architecture is evaluated against the state-of-the-art GSCore accelerator, demonstrating a significant 5.24× speedup and 3.35× energy efficiency improvement.

Strengths

1. Fundamental Dataflow Innovation: The paper's primary strength is its departure from merely optimizing the existing 3DGS pipeline. The authors correctly identify that the standard tile-wise dataflow, while a natural fit for traditional GPU rasterization of triangles, introduces systemic redundancies for 3DGS primitives. The proposed Gaussian-wise dataflow is a paradigm shift that directly attacks the root causes of inefficiency: redundant preprocessing of unused Gaussians and repeated memory access for Gaussians spanning multiple tiles. The illustration in Figure 1 (page 3) provides an exceptionally clear and compelling argument for this new approach.

2. Excellent Contextualization and Problem Motivation: The work is situated perfectly within the current landscape of neural rendering acceleration. It correctly identifies 3DGS as a key algorithm for real-time rendering on edge devices and GSCore [19] as the relevant state-of-the-art hardware baseline. The analysis in Section 2.2 and the motivating data in Figure 2 (page 4) crisply define the problem and quantify the opportunity for improvement, making the authors' proposed solution feel both necessary and impactful.

3. Elegant Algorithm-Architecture Co-design: GCC is a strong example of co-design. The Gaussian-wise dataflow enables the possibility of cross-stage conditional processing. This, in turn, motivates the need for a more efficient way to determine a Gaussian's screen-space footprint. The proposed "alpha-based boundary identification" (Section 3, page 6) is a clever algorithmic solution that replaces coarse bounding boxes (AABB/OBB) with a more accurate, dynamically computed region. This synergy between the high-level dataflow and the low-level compute kernels is a hallmark of excellent architecture research.

4. Significant and Well-Supported Results: The performance and efficiency gains over GSCore are substantial and position GCC as a new state-of-the-art. The authors not only present strong headline numbers but also provide insightful breakdown analyses (Figure 11, page 10) and sensitivity studies (Section 5.4, page 11) that attribute these gains to their specific innovations. The comparison with GPU implementations in Section 6 further strengthens the paper by demonstrating why custom hardware is necessary to fully exploit the potential of this new dataflow.

Weaknesses

While the core idea is powerful, the paper could benefit from a deeper exploration of its own architectural trade-offs and limitations.

1. Scalability of the Global Sorting Stage: The entire Gaussian-wise dataflow hinges on the ability to perform an initial global sort of all Gaussians by depth (Stage I, page 5). While this is amortized over a frame, the complexity of this sort grows with scene size. The paper does not analyze the potential bottlenecks (memory bandwidth, compute latency) of this stage for truly massive scenes containing tens or hundreds of millions of Gaussians. It is a critical preprocessing step that could become a new limiter at scale.

2. Pragmatism vs. Purity in Compatibility Mode: The "Compatibility Mode" (Section 4.6, page 9) is a necessary and practical solution for handling large-resolution images on memory-constrained devices. However, it essentially reintroduces a form of tiling, which the paper's core premise argues against. While the authors show the overhead is minimal at a 128x128 tile size, this feels like a compromise of the central design philosophy. A more detailed analysis of the performance trade-offs as this sub-view size changes would be valuable to understand the robustness of the architecture.

3. Complexity of Runtime Boundary Identification: The alpha-based boundary identification algorithm (Algorithm 1, page 6) is more accurate but also appears more complex than a simple geometric bounding box test. It involves an iterative, breadth-first search. The paper does not provide a detailed analysis of the hardware cost or cycle latency of this runtime search process compared to the simpler methods it replaces. It's possible that for some distributions of Gaussians, the overhead of this search could erode some of the gains from reduced blending computations.

Questions to Address In Rebuttal

1. Could the authors elaborate on the scalability of the initial global depth grouping stage (Stage I)? What are the projected memory traffic and latency characteristics for this stage on scenes that are an order of magnitude larger than those in the benchmarks (e.g., 50M+ Gaussians)? At what point does this upfront sorting cost begin to dominate the per-frame latency?

2. Regarding the Compatibility Mode, could you provide data showing how performance and memory access overhead scale as the sub-view size is reduced below 128x128 (e.g., to 64x64 or 32x32)? Understanding this curve would help clarify the robustness of the architecture for even more severely resource-constrained systems.

3. The alpha-based boundary identification is an elegant technique. Could you provide a brief analysis of the average number of iterations or pixels evaluated by Algorithm 1 per Gaussian? How does the cycle cost of this dynamic search in the Alpha Unit compare to the cost of a simpler OBB intersection test followed by processing more redundant pixels, as might be done in a baseline system?

---

Review Form

Reviewer: The Innovator (Novelty Specialist)

Summary

The authors present GCC, a hardware architecture and dataflow for 3D Gaussian Splatting (3DGS) inference. The central claim of novelty rests on a fundamental restructuring of the standard 3DGS processing pipeline. Instead of the conventional two-stage "preprocess-then-render" dataflow with tile-wise rendering, GCC introduces a "Gaussian-wise" rendering approach coupled with "cross-stage conditional processing." This new dataflow processes Gaussians sequentially in depth order, completing all operations for one Gaussian before starting the next. This structure enables the early termination of both preprocessing and rendering for Gaussians that are determined to be occluded or visually insignificant, thereby addressing the redundancy issues of repeated data loading and extraneous preprocessing inherent in prior art. A supporting algorithmic contribution is an alpha-based boundary identification method to dynamically compute a tight processing region for each Gaussian.

Strengths

The primary strength of this work is its core conceptual novelty in redefining the 3DGS accelerator dataflow. My analysis confirms that this contribution is significant and well-motivated.

1. Novel Dataflow Inversion: The most significant novel contribution is the shift from a tile-wise to a Gaussian-wise rendering paradigm within the context of 3DGS acceleration. Prior dedicated hardware, notably the baseline GSCore [19], adheres to the GPU-style tile-based approach. By processing one Gaussian completely at a time (as detailed in Section 3, page 4 and Figure 1), the authors eliminate the "Challenge 2" they identify: the repeated loading of a single Gaussian's parameters for every tile it overlaps. This is a clean and fundamental departure from the established methodology in this specific domain.

2. Cross-Stage Conditional Processing: This concept is inextricably linked to the Gaussian-wise dataflow and represents a crucial element of the work's novelty. Standard pipelines decouple preprocessing from rendering, leading to wasted computation on Gaussians that are later discarded during alpha blending ("Challenge 1," page 4). GCC's proposed interleaving of these stages allows rendering-dependent information (i.e., per-pixel accumulated transmittance) to gate the execution of the entire processing chain for subsequent Gaussians. This is a genuinely new mechanism for 3DGS accelerators that directly prunes the pipeline at a much earlier stage than previously possible, saving both data movement and computation.

3. Principled, Dynamic Boundary Identification: The proposed alpha-based boundary identification method (Section 3, page 6, "Alpha-based Gaussian Boundary Identification") is a strong supporting contribution. While prior work uses static approximations like the 3σ rule (AABBs) or slightly better OBBs, this work derives the exact elliptical support boundary as a function of the Gaussian's opacity ω (Equation 8, page 5). The runtime breadth-first traversal (Algorithm 1) to identify the minimal pixel set is a specific and novel implementation of this principle, moving beyond coarse bounding boxes to a fine-grained, accurate evaluation region. This is essential for making the Gaussian-wise splatting approach computationally efficient.

Weaknesses

While the application of the core ideas to 3DGS acceleration is novel, the underlying concepts have precedents in the broader field of computer graphics. The manuscript would be stronger if it more clearly situated its contributions relative to this foundational prior art.

1. Conceptual Precedent of Primitive-Wise Rendering: The concept of "Gaussian-wise" rendering is functionally analogous to classic immediate-mode or primitive-order rasterization, where each primitive (e.g., a triangle) is fully processed and rasterized before moving to the next. Modern tile-based rendering was developed specifically to overcome the memory bandwidth issues of this classic approach. The authors have effectively re-introduced a primitive-wise model for 3DGS, arguing its benefits in this new context. While its application here is novel and the performance justification is strong, the paper should acknowledge this conceptual lineage to more precisely define its delta over foundational graphics architectures.

2. Scalability of Global Sorting: The proposed dataflow is predicated on a global, front-to-back ordering of all Gaussians. This is accomplished in "Stage I: Gaussian Grouping by Depth" (page 5). This initial global sorting step could represent a new scalability bottleneck, especially for scenes containing tens of millions or billions of Gaussians. The paper describes a hierarchical grouping scheme (Section 4.2, page 7) but does not provide a rigorous analysis of this stage's performance characteristics or its asymptotic complexity as scene size increases. The novelty of the rendering pipeline could be undermined if this new preprocessing step does not scale effectively.

3. Complexity of Irregular Memory Access: A key reason for the prevalence of tile-based rendering is its highly coherent access to on-chip tile memory. The proposed Gaussian-wise approach, by contrast, splats elliptical footprints onto a global image buffer, creating a scattered and potentially irregular memory write pattern. While the authors mention processing in pixel blocks (8x8 PEs) to manage this (Section 4.4, page 8), the paper lacks a detailed analysis of the memory subsystem's design for handling this irregularity. The trade-off between reducing repeated DRAM loads (a clear win) and introducing less coherent on-chip memory accesses (a potential loss) is not fully explored.

Questions to Address In Rebuttal

1. Could the authors please elaborate on the novelty of "Gaussian-wise" rendering in the context of historical, primitive-order rasterization pipelines? Please clarify how your approach differs from, or is specifically adapted for, the unique properties of 3DGS compared to traditional triangle rasterization.

2. The enabling step for your dataflow is the global depth sorting of Gaussians. Can you provide an analysis of the performance and scalability of this sorting stage? How does its runtime cost scale with the number of Gaussians in a scene, and at what point, if any, might it become the dominant performance bottleneck?

3. The transition to Gaussian-wise rendering changes the memory access pattern on the Image Buffer from the coherent accesses of a tile-based system to more scattered writes. Could you provide more detail on how your memory hierarchy (e.g., buffer banking, queuing) is designed to mitigate the potential performance penalties associated with this irregularity, and quantify any remaining overhead?