TAIDL: Tensor Accelerator ISA Definition Language with Auto-generation of Scalable Test Oracles

Review Form

Reviewer: The Guardian (Adversarial Skeptic)

---

Summary

The authors present TAIDL, a domain-specific language for defining the instruction set architectures (ISAs) of tensor accelerators. The central idea is to leverage a high-level tensor intermediate representation, XLA-HLO, to describe the operational semantics of instructions. From a TAIDL specification, the system can automatically generate a Python-based "test oracle" which, under the hood, compiles the operations into an XLA computational graph for execution on multi-core CPUs or GPUs. The authors claim this approach improves productivity and yields test oracles that are orders of magnitude faster and more scalable than existing instruction-level functional simulators like Gemmini Spike and Intel SDE.

However, the fundamental premise of using a high-level compiler IR to define a low-level ISA is questionable. The work appears to conflate a high-level functional model with a precise, bit-accurate ISA specification. Furthermore, the empirical evaluation, while showing dramatic speedups, rests on an inequitable comparison between a JIT-compiled, parallelized tensor program and traditional serial instruction interpreters, making the performance claims misleading.

Strengths

1. Problem Motivation: The paper correctly identifies a critical gap in the academic hardware accelerator community: the lack of robust, well-defined hardware-software interfaces and accessible correctness-testing tools (Section 1.1, Table 1). The motivation is clear and compelling.
2. Productivity Goal: The goal of automating the generation of functional simulators ("test oracles") is a laudable one. Reducing the significant, repetitive engineering effort required to build such tools for each new accelerator design is a valuable research direction.
3. Open Source: The authors have made their implementation publicly available, which is a positive contribution to the community, enabling reproducibility and further investigation.

Weaknesses

1. Fundamental Mischaracterization of an "ISA Definition Language": The core weakness of this paper is its premise. An ISA is a low-level contract specifying precise, bit-level behavior. TAIDL, by using XLA-HLO as its computational primitive, is not defining an ISA in the rigorous sense that languages like Sail [52] do. It is defining a high-level functional behavior. This abstraction is problematic:

   • It cannot naturally express novel data types, non-standard floating-point formats, or specific rounding/saturation behaviors not already supported by XLA. The paper's proposed workaround—using custom_call to an external C function (Section 5.5)—is an admission of the language's limitation and defeats the purpose of a self-contained specification.
   • Subtle side effects or interactions between instructions that are not representable as a clean dataflow graph of tensor operations would be difficult, if not impossible, to model. TAIDL appears best suited for accelerators whose ISAs map cleanly to an existing compiler IR, which questions its utility for defining genuinely novel architectures.

2. Misleading Performance Evaluation: The scalability analysis in Section 7 is fundamentally flawed due to an apples-to-oranges comparison.

   • The TAIDL-TO "oracle" is not an instruction-level simulator. It is a JIT compiler that transforms a sequence of high-level API calls into an optimized XLA graph, which is then executed by a highly-optimized, parallel backend (e.g., on an A100 GPU).
   • In contrast, Gemmini Spike and Intel SDE are true instruction-level simulators/emulators that process one instruction at a time, often in a single-threaded manner.
   • The "orders of magnitude" speedup reported in Figures 19 and 20 is therefore not a measure of a better simulation technique, but rather a demonstration that running a compiled, parallelized tensor program is faster than interpreting a sequence of instructions serially. This outcome is expected and does not validate the claims about TAIDL's superiority as a simulation methodology. A fair comparison would be against a hand-optimized C++ functional model of the accelerator, compiled with aggressive optimizations.

3. The "Oracle" is Not a Golden Reference: A test oracle, by definition, should serve as an independent, trusted source of truth. The TAIDL-TO artifact is generated and executed via the complex XLA compiler toolchain. This introduces a significant risk of common-mode failures. A bug or semantic interpretation within the XLA compiler could manifest in both the code being tested (e.g., a compiler stack targeting the accelerator) and the TAIDL-generated oracle, thereby masking the bug entirely. This approach lacks the semantic independence required for a trustworthy verification tool.

4. Overstated Expressivity: The examples provided (AMX, TPUv1, Gemmini) are for accelerators with relatively well-structured, data-parallel semantics that map nicely to XLA-HLO operators. The paper does not provide convincing evidence that TAIDL could handle ISAs with more irregular control flow, complex state management, or fine-grained bit-manipulation instructions that are common in hardware but do not have a clean high-level tensor abstraction. The inclusion of IF and REPEAT blocks (Section 4.5) operating on control registers is a minimal step and does not address complex, data-dependent control flow at the instruction level.

Questions to Address In Rebuttal

1. Please defend the characterization of TAIDL as an ISA Definition Language rather than a High-Level Functional Modeling Framework. How would TAIDL precisely specify the semantics of an instruction that implements a novel 8-bit floating-point format with a non-standard rounding mode, without resorting to an external custom_call?
2. How do you justify the performance comparison in Section 7? Acknowledge that you are comparing a JIT-compiled, parallel XLA graph against serial instruction interpreters. What insights do these results provide beyond the trivial conclusion that compiled, parallel code runs faster than interpreted, serial code?
3. The term "test oracle" implies a golden reference. Given that your oracle is dependent on the large and complex XLA compiler stack, how do you mitigate the risk of common-mode failures where bugs in the underlying XLA implementation could mask bugs in the software being tested?
4. The transform algorithm in Figure 16 appears to perform constant propagation on control registers (state) and unroll loops. What happens if an instruction's behavior depends on a value in a tensor buffer (not a control register)? Does TAIDL support this, and if so, how does the transformation to a static XLA graph handle such data-dependent control flow?

---

Review Form

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper introduces TAIDL, a domain-specific language for defining the instruction set architectures (ISAs) of tensor accelerators. The authors identify a critical and widening gap in the hardware-software ecosystem: while numerous novel accelerators are proposed, they almost universally lack the software tooling (specifically, well-defined ISA semantics and fast functional simulators) necessary for the software community to build compilers and applications for them.

The core contribution is twofold: 1) The TAIDL language itself, which uniquely leverages a high-level tensor IR, XLA-HLO, as its semantic foundation for describing instruction behavior. 2) A novel methodology to automatically generate fast, scalable "test oracles" (functional simulators) from TAIDL definitions. This is achieved by transforming a sequence of TAIDL instructions into a single XLA-HLO computation graph, which can then be compiled by mature tensor compilers (like XLA) to run efficiently on multi-core CPUs or GPUs.

The authors demonstrate TAIDL's expressivity by modeling diverse, real-world accelerators (Google TPU, Intel AMX, Gemmini). Crucially, they show that the auto-generated test oracles are orders of magnitude faster than established, hand-crafted simulators like Gemmini Spike and Intel SDE, making them practical for large-scale software testing and even end-to-end model simulation.

Strengths

The primary strength of this work lies in its elegant and highly effective synthesis of ideas from different domains to solve a pressing, real-world problem.

1. Problem Significance and Framing: The paper correctly identifies a major bottleneck in hardware innovation. The "disconnect between the software and hardware research" (Section 1.1, Page 1) is a well-known but undertreated problem. The authors' analysis in Table 1 (Page 2) clearly motivates the need for a solution that provides both programmability (via a clear ISA) and testability (via fast oracles). This work is not a solution in search of a problem; it is a direct and thoughtful response to a critical community need.

2. The Core Insight: Semantics-as-IR: The most profound contribution is the decision to use a high-level, functional tensor IR (XLA-HLO) to define instruction semantics. Traditional ISA specification languages (e.g., Sail, as mentioned in Section 9) operate at a scalar or bit-vector level. By elevating the semantic definition to the level of tensor operations, the authors unlock the ability to leverage the entire, highly-optimized ML compiler ecosystem for simulation. This is a paradigm shift from writing simulators to compiling them, and it is the key insight that enables the impressive performance results.

3. Synergistic and Practical Toolchain: The proposed workflow (visualized well in Figure 14, Page 9) is exceptionally clever. Instead of building a new simulation engine from scratch, the authors' transformation algorithm effectively retargets the simulation task to the XLA compiler. This allows the generated oracles to automatically benefit from decades of compiler research in optimization (fusion, layout changes) and parallelization (multi-threading, GPU offload). This synergy makes the approach both powerful and practical.

4. Compelling Empirical Validation: The performance evaluation in Section 7 is not just incremental; it demonstrates a transformative improvement. The speedups of 1200x to 5600x over Gemmini Spike (Figure 19, Page 11) and significant gains over the industrial-grade Intel SDE (Figure 20, Page 12) are dramatic. These results elevate the tool from a theoretical concept to something genuinely usable for interactive development cycles and testing large, complex kernels, as shown in the I-BERT case study (Section 8.2).

Weaknesses

The weaknesses of the paper are less about flaws in the existing work and more about the inherent limitations of the chosen approach and unaddressed future challenges.

1. The XLA Abstraction Leash: The paper's greatest strength is also its most significant potential limitation. By tying TAIDL's semantics to XLA-HLO, the language is fundamentally constrained by what can be cleanly expressed in XLA-HLO. While the authors discuss forward-compatibility for custom datatypes (Section 5.5), it is less clear how TAIDL would handle accelerator features that are philosophically misaligned with the XLA model. Examples could include complex, low-level memory dependencies, explicit cache management instructions, or novel synchronization primitives that don't map to standard tensor operations. The expressivity might break down for ISAs that are not "HLO-like."

2. Scope of a "Test Oracle": The work focuses exclusively on the functional simulation of the data path. A complete software stack also needs to interact with the accelerator's control plane (e.g., command submission, interrupt handling, synchronization with the host). While TAIDL models some state with "control registers," it does not seem equipped to describe the dynamic, asynchronous interactions between the accelerator and a host system. This limits its utility for developing drivers or runtimes, which are also critical parts of the software ecosystem.

3. The Authoring Effort is Opaque: The paper excellently demonstrates the benefits for the user of a test oracle (the software programmer). However, it does not quantify the effort for the creator of the TAIDL specification (the hardware architect). Translating the intricate microarchitectural behavior of a new accelerator into a sequence of pure XLA-HLO operators may be a significant conceptual and engineering challenge in itself. The paper would be stronger if it acknowledged and discussed this potential shift in the "tooling burden" from writing a C++ simulator to writing a complex TAIDL-HLO specification.

Questions to Address In Rebuttal

1. Could the authors elaborate on the "escape hatches" for accelerator features that are difficult to model in XLA-HLO? For instance, how would TAIDL model an instruction that performs a scatter operation with data-dependent addressing, or a hardware feature that exposes explicit control over a software-managed cache? Is the expectation that these would be modeled via custom_call (Section 5.5), and what are the performance implications of that for the generated oracle?

2. The current focus is on functional correctness of kernel execution. What is the team's vision for extending the TAIDL framework to model the broader system context? Specifically, how might you model the host-accelerator interface, command queues, and memory coherence, which are essential for testing the correctness of the runtime and driver software, not just the compiled kernels?

3. While the benefits of having a TAIDL specification are clear, what is the anticipated learning curve and effort for a hardware architect to write a specification for a novel, complex accelerator? Is there a risk that precisely defining complex hardware behavior using only the constrained vocabulary of HLO operators is as difficult as writing a traditional simulator?

---

Review Form

Reviewer: The Innovator (Novelty Specialist)

Summary

The paper introduces TAIDL, a domain-specific language for defining the Instruction Set Architectures (ISAs) of tensor accelerators. The authors' central novel claim is not the language itself, but the methodology for automatically generating fast and scalable test oracles (functional simulators) from TAIDL definitions. This is achieved by a unique approach: expressing instruction semantics using a high-level tensor Intermediate Representation (XLA-HLO) and then leveraging a production-grade tensor compiler (XLA) to generate highly optimized, parallel simulator code that can execute on multi-core CPUs and GPUs. The authors demonstrate this by modeling ISAs like Intel AMX and Gemmini, and show that their generated oracles significantly outperform existing, hand-crafted instruction-level simulators.

Strengths

The primary strength of this paper lies in the novel synthesis of two established fields: ISA specification and tensor compilation. The core innovative idea can be summarized as "semantic piggybacking" on a mature compiler ecosystem. To my knowledge, this is a new approach for generating functional simulators.

1. Novel Compilation Methodology: Prior work on ISA specification languages, such as Sail [8, 52] or ILA [56], primarily focuses on formal correctness and typically generates C code or theorem prover inputs for emulation and verification. These generated artifacts are often serial and not designed for high-performance simulation of large workloads. This paper's core insight—to define semantics in an IR that is the input to an optimizing compiler rather than a low-level language that is the output of a simple transpiler—is the key novelty. This choice directly enables the generation of highly scalable oracles, a claim well-supported by the performance results in Section 7 (pages 11-12).

2. Pragmatic Abstraction Choice: The decision to use XLA-HLO as the semantic foundation is a clever and domain-appropriate one. Tensor accelerators are, by definition, designed to execute operations that map well to a tensor IR. By staying at this high level of abstraction, the authors avoid the notoriously difficult problem of compiling low-level, bit-precise semantic definitions into efficient, parallel code. They effectively offload this complexity to the XLA compiler team at Google, which is a significant and pragmatic engineering choice that enables their results.

3. Significant Delta Over Prior Art: The "delta" between this work and the closest prior art (e.g., generating C emulators from Sail) is the performance and scalability of the resulting artifact. The orders-of-magnitude speedup shown in Figures 19 and 20 is not a marginal improvement; it represents a qualitative shift in what is practical for pre-silicon software testing, enabling full end-to-end model simulation (Section 8.2, page 13) where it was previously infeasible.

Weaknesses

My critique is focused on the boundaries and generalizability of the claimed novelty. While the core idea is new and effective, its scope may be narrower than implied.

1. Limited Generality of the Novel Approach: The central novelty is critically dependent on the semantics of the target ISA being easily expressible in a tensor IR. This works exceptionally well for tensor accelerators whose instructions are coarse-grained operations like matrix multiplies or convolutions. However, this approach would likely fail or be exceedingly cumbersome for general-purpose ISAs or even accelerators with fine-grained, scalar control logic or complex bit-level manipulation instructions (e.g., cryptography or networking accelerators). The paper acknowledges backward compatibility with scalar and bit-vector representations (Section 5.4, page 8), but using a tensor compiler to simulate complex bit-fiddling seems highly inefficient and misaligned. The novelty, therefore, seems confined to a specific, albeit important, class of architectures.

2. The DSL Contribution is Secondary: The paper presents TAIDL as a new language. However, based on the examples provided (e.g., Figure 2b, page 3), the language itself appears to be largely syntactic sugar—a thin, user-friendly wrapper—around XLA-HLO operations. The fundamental innovation lies in the compilation pipeline, not in the language constructs of TAIDL itself. The paper does not sufficiently argue for the novelty of the language design independent of its compilation target. An alternative approach of a Python library for programmatically constructing XLA-HLO graphs could have achieved similar results, questioning the necessity of a new DSL.

Questions to Address In Rebuttal

1. The reliance on XLA-HLO seems to tightly couple the approach to accelerators whose semantics are naturally expressed as coarse-grained tensor operations. How would the authors' framework handle instructions with complex, non-tensor semantics, such as intricate bit-level manipulations (e.g., a Galois Field multiply instruction) or stateful control flow logic not easily captured by XLA's select or while operators? Is the claimed novelty fundamentally restricted to the domain of DNN accelerators?

2. The TAIDL language appears to be a direct mapping to XLA-HLO constructs. What is the fundamental novel contribution of the language design itself, separate from the novel compilation methodology? Could the same result have been achieved by providing a Python library that directly constructs XLA-HLO graphs, and what is the distinct advantage of introducing a new DSL that justifies its novelty?

3. The framework's novelty and performance are tied to the capabilities of the XLA ecosystem. What happens if future accelerators introduce computational paradigms (e.g., sparsity patterns, dynamic data structures) that are not well-supported or efficiently compiled by XLA-HLO? Does this dependency represent a long-term limitation to the approach's novelty and applicability?