Titan-I: An Open-Source, High Performance RISC-V Vector Core

---

Review Form

Reviewer: The Guardian (Adversarial Skeptic)

Summary

The authors present Titan-I (T1), a parameterizable, out-of-order (OoO), lane-based RISC-V vector core generator. The paper introduces several microarchitectural techniques aimed at improving scalability and performance, including a floor-planning solver, a dedicated permutation unit, and a shadow cache for mask registers. The evaluation section presents performance comparisons against high-end GPUs (NVIDIA GA102/GB202) and contemporary CPU vector architectures (HiSilicon KP920, SpacemiT X60).

While the ambition of the project is noted, the paper’s central claims of superior performance are predicated on a narrow and highly favorable set of benchmarks. Several key architectural decisions appear to trade correctness and generality for performance in specific scenarios, and the quantitative analysis of the design's own scaling properties is questionable. The comparisons to other architectures, particularly GPUs, are fundamentally flawed, raising serious doubts about the validity of the conclusions drawn.

Strengths

1. Open-Source Contribution: The commitment to providing an open-source RTL generator is a significant contribution that enables community verification and research.
2. Area Efficiency: The reported area efficiency against the HiSilicon KP920 core (Section 6.2.1, page 12) is impressive, assuming the area measurement methodology for the baseline is sound. Achieving comparable performance in 19% of the area is a noteworthy engineering result.
3. Focus on Scalability: The paper correctly identifies critical bottlenecks in scaling vector architectures (permutation, masking, scheduling) and makes a concerted effort to address them, even if the proposed solutions have weaknesses.

Weaknesses

1. Fundamentally Flawed GPU Comparison: The central claim of outperforming NVIDIA GPUs is based on an inappropriate comparison. The authors benchmark T1 against a single Streaming Multiprocessor (SM) on two integer-only cryptographic workloads (NTT, MMM) (Section 6.1, page 11). These workloads are known to be pathologically ill-suited for the SIMT execution model of GPUs and are perfectly suited for a wide-vector architecture. This is a clear case of cherry-picking benchmarks to maximize the perceived advantage. The comparison completely ignores workloads where GPUs excel, such as dense floating-point linear algebra or highly divergent codes. The claim of "outperforming" a GPU is meaningless without a balanced and representative benchmark suite.

2. Questionable Area Scaling Analysis: The area scaling results presented in Figure 4 (page 6) are highly suspect. Specifically, Figure 4c suggests that increasing the LaneScale (i.e., making individual lane datapaths wider) leads to a decrease in the total area of T1. This is counter-intuitive and physically implausible. While the authors attribute this to sharing control logic, such a dramatic reduction in total area (nearly 40% when moving from LaneScale=1 to 4) is an extraordinary claim that requires a much more detailed explanation and justification than is provided. It suggests a potential flaw in the area model or an omission of critical information.

3. Unsafe Architectural Shortcuts for Exception Handling: The mechanism for handling long-latency indexed memory operations relies on a "chicken bit" in a CSR to suppress access-fault exceptions (Section 4.2.1, page 7 and Section 4.2.6, page 9). This is not a robust architectural solution; it is a hardware hack that offloads the burden of ensuring memory safety entirely to software. For a core intended for high-performance, general-purpose workloads, this is a critical design flaw. It renders the core unsuitable for environments where precise exceptions are required for memory management, debugging, or security.

4. Misleading Comparison to In-Order Cores: The performance comparison against the SpacemiT X60 (Section 6.2.3, page 12) results in a claimed 8.05x speedup. However, the X60 is a simpler, in-order core. It is neither surprising nor particularly insightful that a complex OoO architecture significantly outperforms an in-order one. This comparison serves more to inflate T1's performance numbers than to provide a meaningful benchmark against a peer competitor.

5. Insufficient Detail on Novel Contributions: The "coarse-grained floor-planning solver" (Section 4.1.1, page 6) is presented as a key innovation. However, the paper provides no details on the heuristic algorithm itself. Without this information, it is impossible to assess its novelty or effectiveness beyond the single, potentially cherry-picked example in Figure 5. It is unclear how this differs from standard P&R scripting. Similarly, the "shadow mask" cache (Section 4.1.2, page 6) is described, but the mechanism for ensuring correctness and handling coherence with pending writes to v0 is glossed over, despite this being a potential serialization point.

Questions to Address In Rebuttal

1. On GPU Benchmarking: Can the authors justify their claim of GPU superiority by providing performance comparisons on workloads where GPUs are traditionally strong, such as FP32/FP16 SGEMM, stencil computations, or graph analytics? If not, the claims regarding GPU performance should be significantly moderated to reflect the narrow, integer-only context.

2. On Area Scaling: Please provide a detailed breakdown and justification for the claim in Figure 4c that total core area decreases as LaneScale increases. What specific components are shrinking, and why does this effect overwhelm the expected area increase from wider datapaths and crossbars within the lane?

3. On Exception Handling: Please defend the use of a "chicken bit" for indexed memory operations. How does this design support robust, general-purpose software that relies on precise exceptions for virtual memory paging, memory protection, or runtime error handling?

4. On the Floorplan Solver: Please provide sufficient detail on the heuristic algorithm used in your floorplan solver to allow the reader to understand its novelty and distinguish it from a trivial script that drives a standard placement tool. What are the constraints and the objective function it optimizes?

5. On the X60 Comparison: Please clarify if the SpacemiT X60 core used for comparison is an in-order design. If so, please justify why this is presented as a meaningful comparison for an OoO architecture, rather than an expected outcome.

---

Review Form

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper presents Titan-I, an open-source, highly parameterized generator for an out-of-order (OoO) RISC-V vector (RVV) processor. The work's core contribution is a suite of microarchitectural innovations designed to holistically address the long-standing challenge of scaling both data-level parallelism (DLP), via wide vector datapaths and long vector lengths (VLEN), and instruction-level parallelism (ILP), via fine-grained, OoO execution.

The authors identify that traditional superscalar OoO techniques do not scale well to the massive state of wide vector registers, while traditional vector machines often sacrifice ILP. Titan-I tackles this gap with several key techniques: a coarse-grained floor-planning solver to manage routing complexity in wide designs, a datapath-wide permutation unit to efficiently handle shuffle-heavy workloads, a shadow cache for mask registers to reduce broadcast traffic, and a novel "issue-as-commit" mechanism to decouple the scalar and vector pipelines. Central to its ILP capabilities is a fine-grained chaining microarchitecture that operates at the sub-register (lane) level.

The authors validate their architecture with strong empirical results, including two ASIC tape-outs. Their evaluations show Titan-I outperforming high-end NVIDIA GPUs on specific cryptographic kernels and demonstrating competitive performance-per-area against a high-performance ARM SVE core (HiSilicon KP920) in HPC workloads. The work is presented not as a single point design, but as a flexible generator, positioning it as a significant contribution to the open-source hardware ecosystem.

Strengths

1. Addresses a Fundamental Architectural Trade-off: The paper targets the difficult, yet crucial, intersection of CPU and GPU design philosophies. The quest to unify high single-thread performance (ILP) with massive data throughput (DLP) is a central theme in modern computer architecture. Titan-I offers a compelling and well-reasoned "vector-first" approach to this problem, standing as a modern successor to the philosophy of classic vector supercomputers like the Cray-1.

2. Holistic, System-Level Design: The strength of this work lies not in a single trick, but in the co-design of multiple solutions to solve the wider problem of scalability. The authors correctly identify that simply widening a datapath creates cascading problems in routing, control logic, and data movement. Their solutions—the floorplanner for physical layout (Section 4.1.1, page 6), the dedicated permutation unit for data shuffling (Section 4.1.3, page 7), and the mask register cache (Section 4.1.2, page 6) for predication—demonstrate a deep understanding of both the logical and physical barriers to scaling.

3. Credible and Impressive Empirical Validation: The authors provide a robust evaluation against relevant and powerful commercial counterparts. Comparing against NVIDIA GPUs for crypto and a flagship ARM server CPU for HPC is ambitious and gives the results significant weight. The fact that the project has yielded two physical tape-outs (Section 5.3, page 10) lends enormous credibility to the claimed performance and area results, moving it beyond a purely academic simulation study.

4. Contribution as an Open-Source Generator: Perhaps the most significant aspect of this work is that its deliverable is a highly parameterized generator. This elevates its potential impact substantially. Instead of a single, static design, the authors provide a framework that can be adapted to different application domains and PPA (Power, Performance, Area) targets, from edge devices to data centers. This is a powerful enabler for the RISC-V ecosystem and the broader hardware community.

Weaknesses

While the microarchitectural contributions are excellent, the paper could be strengthened by providing more context on its place within a complete system, particularly regarding software and memory.

1. The Software Abstraction Challenge: The paper rightly celebrates its hardware's performance, but this performance is unlocked via hand-tuned assembly or a custom MLIR-based toolchain (Section 5.1, page 10). A significant advantage of competitors like NVIDIA is the maturity and accessibility of the CUDA ecosystem. For Titan-I to have broader impact, the path from high-level code (e.g., C++, Python) to high-performance vectorized execution needs to be more thoroughly explored. The current presentation leaves the impression that achieving these results requires expert-level programming effort, which could limit its adoption.

2. The Scalar Core as a Potential Bottleneck: The architecture is presented as a vector co-processor that relies on a scalar core for control flow, address generation, and non-vector instructions. The "issue-as-commit" policy (Section 4.2.1, page 7) is a clever way to decouple the pipelines, but the performance of many real-world vector applications (e.g., sparse matrix operations) is often limited by the efficiency of the scalar code that prepares the data for the vector unit. The paper provides little detail on the assumed capabilities of the scalar core or the potential for it to become an "Amdahl's Law" bottleneck.

3. System-Level Memory Integration: The paper demonstrates impressive memory latency tolerance (Section 6.2.2, page 12) and discusses a Memory Management Unit (MMU) as future work (Section 7, page 13). However, in a real system, the interaction with virtual memory—handling page faults, TLB misses, and maintaining coherence with other cores in an SoC—is a first-order design constraint, not just a feature to be added later. A deeper discussion of how the massive, long-latency memory accesses would be managed within a virtual memory system would contextualize the design's practicality for general-purpose computing.

Questions to Address In Rebuttal

1. Could the authors elaborate on the maturity and usability of their MLIR-based software toolchain? For the HPC benchmarks, how much of the performance was achieved through fully automatic auto-vectorization versus manual intervention or the use of compiler intrinsics?

2. The "issue-as-commit" mechanism effectively decouples the scalar and vector units. However, what are the key performance considerations for the scalar core that pairs with Titan-I? Are there specific workload characteristics (e.g., complex address generation, frequent loop-carried dependencies) where the scalar front-end is likely to become the primary performance limiter?

3. The paper's discussion on memory latency tolerance is a highlight. Could the authors comment on how the design's philosophy of handling long-latency operations would be extended to manage system-level complexities like page faults? Would a page fault on one element of a VLEN=16384 operation require stalling the entire vector instruction for its duration, and what would be the performance implications?

4. Section 7 mentions the possibility of adopting a Multiple Streaming Processor (MSP) approach, akin to the Cray X-MP, to improve TLP. Given the generator's flexibility, have the authors explored configurations that might partition a wide Titan-I core into several narrower, independent vector contexts? This seems like a natural architectural evolution to bridge the gap between this powerful single-thread vector model and the many-thread GPU model.

---

Review Form

Reviewer: The Innovator (Novelty Specialist)

Summary

The authors present Titan-I (T1), a generator for an open-source, out-of-order (OoO), lane-based RISC-V Vector (RVV) core. The central thesis is that a combination of novel microarchitectural techniques can overcome the traditional scaling challenges of vector processors, enabling simultaneous scaling of both Data-Level Parallelism (DLP) and Instruction-Level Parallelism (ILP). The proposed contributions are a collection of solutions targeting specific bottlenecks: a coarse-grained floor-planning solver and a dedicated permutation unit for DLP scaling, and a fine-grained chaining mechanism, a scalar-vector decoupling policy ("issue-as-commit"), and specialized memory scheduling for ILP. The authors provide an open-source RTL generator and validate their design with two ASIC implementations and extensive performance comparisons.

My review focuses exclusively on the novelty of the proposed ideas relative to the vast body of prior work in vector and parallel architectures.

Strengths

The primary strength of this work lies not in a single, revolutionary concept, but in the clever synthesis and specific implementation of multiple techniques, some of which represent a genuine delta over prior art.

1. Fine-Grained, Configurable Chaining: The concept of chaining dates back to the Cray-1. However, its application in modern, highly-laned architectures presents new challenges. The closest academic prior art cited, Berkeley's Saturn [51], implements chaining at a coarse "DLEN-granular" level. T1's proposal for configurable, fine-grained chaining that can operate down to the element (ELEN) level (Section 4.2.2, page 8) is a significant and novel advancement. It directly addresses the need for maximizing pipeline utilization in the presence of wide, partitioned datapaths, a key limitation of prior academic designs like Ara [9, 35], which lacks chaining entirely.

2. Physical Design-Aware Microarchitecture: The introduction of a "coarse-grained floor-planning solver" (Section 4.1.1, page 6) is a noteworthy contribution. While floorplanning is a standard part of physical design, explicitly incorporating a heuristic solver into the architectural design flow of a generator to minimize cross-lane routing latency is novel. It acknowledges that at the scale the authors are targeting, physical realities can no longer be an afterthought for the microarchitect. This is a commendable step towards a true hardware-software-physical co-design methodology.

3. Specific Bottleneck Alleviation: The "shadow-cache for mask registers (v0)" (Section 4.1.2, page 6) is an elegant solution to a well-known and painful bottleneck in lane-based RVV designs. While caching is a fundamental computer science concept, the creation of a specialized, dedicated cache within the permutation unit to solve the v0 broadcast problem is a specific, novel, and practical microarchitectural innovation.

Weaknesses

While the paper contains novel elements, several of the core ideas are clever engineering applications of well-established architectural concepts. The paper would be stronger if it more precisely delineated its contributions from this foundational work instead of presenting them as entirely new pillars.

1. Derivative ILP Concepts: Several techniques presented to enhance ILP are modern implementations of classic ideas.

   • "Issue-as-commit" (Section 4.2.1, page 7): This is a form of scalar-vector decoupling. The idea that a scalar core can run ahead of a long-latency vector unit as long as there are no dependencies is conceptually similar to Decoupled Access/Execute (DAE) architectures and the function of scoreboards in early machines like the CDC 6600. The contribution here is a specific, low-overhead scoreboard implementation for RVV, not a fundamentally new execution model. The novelty is in the implementation, not the concept.
   • "Memory Interleaving" and "Memory Delay Slot" (Sections 4.2.5 and 4.2.6, page 9): Overlapping load and store operations and scheduling independent instructions to hide memory latency are canonical compiler and architecture techniques. The paper presents a robust implementation for RVV (e.g., the Conflict Region Table), but the underlying principles are not new.

2. Conflation of Contributions: The paper aggregates a large number of techniques. This makes it difficult to assess the novelty and benefit of each individual contribution. The impressive final results are a product of the entire system, but the value of, for instance, the floor-planning solver is not isolated from the value of the fine-grained chaining. An ablation study would be necessary to truly weigh the merit of each proposed idea against its complexity. The performance gains are substantial, but it's unclear if they come from one or two key breakthroughs or the aggregation of many marginal improvements.

Questions to Address In Rebuttal

1. Clarification on Chaining Novelty: The claim of "fine-grained chaining" is compelling. Could the authors please elaborate on the delta between their linked-list scoreboard approach (Section 4.2.4, page 8) and other forms of fine-grained dependency tracking in prior SIMD or vector architectures, academic or industrial? Is the novelty in the data structure, the configurability, or both?

2. Comparison to Classic Decoupling: Please contrast the "issue-as-commit" mechanism with classic scoreboard-based designs and more formal Decoupled Access/Execute architectures. What is the precise, novel contribution beyond applying a known decoupling principle to the specific scalar-vector interface of RVV?

3. Quantifying Individual Contributions: The paper presents a complex system with multiple interacting optimizations. To better assess the significance of each novel idea, can the authors provide any data (even from simulation) that isolates the performance benefit of the key contributions? For example, what is the performance of T1 with fine-grained chaining disabled (i.e., DLEN-granular like Saturn)? What is the performance impact of using a naive, grid-like floorplan instead of the solver's output?

4. Generality of the Floor-Planner: The floor-planning solver is an interesting design-time contribution. Is the solver's heuristic specifically tuned for the permutation patterns of RVV, or does it represent a more generalizable approach for optimizing communication in tiled accelerator architectures?

Recommendation: Accept with minor revisions.

The paper presents a powerful and well-engineered vector core. While some of its ILP-enhancing concepts are derived from established principles, the work contains several genuinely novel and significant contributions, particularly in its approach to fine-grained chaining and the integration of physical design constraints into the microarchitecture generator. The rebuttal should focus on more precisely situating their work in historical context and, if possible, providing data to deconvolve the benefits of their many contributions.