LEGOSim: A Unified Parallel Simulation Framework for Multi-chiplet Heterogeneous Integration

Review Form:

Reviewer: The Guardian (Adversarial Skeptic)

Summary

This paper introduces LEGOSim, a parallel simulation framework for heterogeneous multi-chiplet systems. The authors identify two key challenges: the difficulty of modularly integrating diverse simulators and the performance overhead of existing synchronization schemes. To address this, they propose a framework built on three core ideas: 1) a Unified Integration Interface (UII) for integrating existing simulators ("simlets") with supposedly "minimal modifications"; 2) an "on-demand" synchronization protocol managed by a central Global Manager (GM) to reduce overhead; and 3) a three-stage decoupled simulation methodology to handle inter-chiplet network latencies.

While the paper addresses a relevant and challenging problem, its central claims rest on a methodologically flawed validation approach, unsubstantiated assertions about performance, and an overstatement of the framework's modularity. The reliance on an optimistic rollback mechanism for correctness—whose overhead is never quantified—casts significant doubt on the reported performance benefits.

Strengths

1. Problem Relevance: The work tackles a critical and timely problem in computer architecture. The need for a flexible and fast simulation framework for multi-chiplet systems is undeniable.
2. Conceptual Approach to Synchronization: The idea of "on-demand" synchronization, which avoids unnecessary global barriers, is a sound principle for reducing simulation overhead compared to per-cycle or fixed time-quantum methods.
3. Inclusion of Artifact: The authors provide access to the source code via GitHub and Zenodo, which is commendable and allows for verification of their implementation.

Weaknesses

1. Fundamental Methodological Flaw in Core Simulation Loop: The paper's three-stage simulation process (Section 3.2, page 5) is critically flawed. In Stage 1, simulation proceeds with zero-load latency estimates. In Stage 3, it re-runs with accurate latencies from a separate NoI simulation. The authors acknowledge that this can lead to timing violations and reordering of memory accesses. Their proposed solution is an "optimistic [27] execution approach" using "checkpointing and rollback to resolve conflicts" (Section 3.2, page 6). This is a massive red flag. The authors then claim that "such violations are rare." This is an extraordinary claim that requires extraordinary evidence, yet no data is provided anywhere in the paper to substantiate this. The frequency of rollbacks, the overhead of checkpointing, and the performance penalty of re-simulation are entirely ignored. Without this data, the performance results presented are meaningless, as they may not account for the potentially crippling cost of ensuring correctness.

2. Weak and Indirect Accuracy Validation: The validation in Section 5.2 (page 9) is unconvincing. The authors compare LEGOSim's results for the SIMBA and CiM-based architectures against performance numbers reported in the original papers [69, 14]. This is not a rigorous validation. It is an indirect comparison susceptible to countless confounding variables, including minor differences in architectural configuration, workload inputs, and internal simulator assumptions. A proper validation requires a direct, head-to-head comparison against a "golden reference" simulator (e.g., a sequential, cycle-accurate gem5 model) running the exact same configuration and workload. The comparison in Figure 7 is a step in this direction, but it lacks the necessary details on the gem5 configuration to be verifiable.

3. "Minimal Modification" Claim is Overstated: The UII, presented in Section 4 (page 7-8), is described as enabling integration with "minimal modifications." The evidence provided contradicts this. Integrating Sniper required inserting Sleep() calls to manage its non-cycle-driven model. Integrating Scale-Sim involved writing to and reading from files. Integrating GPGPU-Sim required wrapping calls with cudaMemcpy(). These are significant, non-trivial, and highly simulator-specific engineering tasks. This is not a unified interface but a set of bespoke integration strategies. The claim of "minimal modification" is misleading.

4. Unaddressed Central Bottleneck: The paper's own scalability analysis in Section 5.4 (Figure 12, page 10) demonstrates that the centralized Global Manager (GM) becomes a performance bottleneck under high inter-chiplet traffic volumes. The authors acknowledge this and propose a "distributed management scheme" as a solution. However, this solution is neither designed, implemented, nor evaluated. It is pure hand-waving. A framework that claims to be scalable must provide a scalable solution to its core coordination mechanism, not just mention one as future work.

Questions to Address In Rebuttal

1. Provide quantitative data on the optimistic execution mechanism. For the benchmarks presented, what is the precise frequency of timing violations that trigger a rollback? What is the performance overhead (in cycles or wall-clock time) of the checkpointing mechanism and the cost of re-executing from a checkpoint?

2. Justify the choice of validating against published numbers instead of a direct, controlled comparison against a golden-reference simulator. For the gem5 comparison in Figure 7, please provide the exact configuration scripts and command lines used for both LEGOSim and gem5 to ensure the comparison is on an identical architectural model.

3. Please provide a concrete example of integrating a new, third-party, event-driven simulator. Detail the specific lines of code and internal simulator logic that must be modified to conform to the UII, to give a more realistic picture of the integration effort beyond the "minimal" claim.

4. How would the proposed "distributed management scheme" maintain global causal consistency? A distributed system of managers introduces its own complex synchronization and communication overhead. Please provide a preliminary design and analysis showing how this scheme would not simply shift the bottleneck from computation to inter-manager communication.

---

Review Form

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper introduces LEGOSim, a parallel simulation framework designed to address the increasingly critical challenge of modeling heterogeneous multi-chiplet systems. The work identifies two primary shortcomings in the current simulation landscape: the lack of modular flexibility in monolithic simulators (e.g., gem5) and the prohibitive synchronization overhead of existing parallel simulation techniques.

The core contribution of LEGOSim is a pragmatic meta-framework that integrates existing, specialized simulators (termed "simlets") as independent processes. This is enabled by two key innovations:

1. A Unified Integration Interface (UII), which provides a standardized API for communication and control, aiming to minimize the modifications required to plug in existing simulators like Sniper, GPGPU-Sim, etc.
2. An on-demand synchronization protocol built upon a three-stage decoupled simulation methodology. This intelligently triggers synchronization only when inter-chiplet communication occurs, drastically reducing overhead compared to per-cycle or fixed time-quantum approaches while preserving accuracy.

The authors validate LEGOSim's fidelity against published results for real architectures and demonstrate its utility through several case studies exploring the design space of interconnect topologies, memory protocols, and on-chip buffer sizes. The work is positioned as an open-source tool to facilitate community-wide research in the burgeoning field of chiplet-based design.

Strengths

The primary strength of this work is its timeliness and direct relevance to a critical, emergent problem in computer architecture. As the industry pivots from monolithic SoCs to heterogeneous, chiplet-based integration (e.g., AMD's Zen, Intel's Ponte Vecchio), the need for fast, accurate, and flexible pre-silicon evaluation tools has become paramount. This paper is not an academic exercise; it is building a necessary piece of infrastructure for the next generation of hardware design.

The conceptual approach is both elegant and pragmatic. Rather than building another monolithic simulator from scratch, LEGOSim cleverly leverages the vast ecosystem of existing, highly-detailed simulators. This "standing on the shoulders of giants" approach is powerful. The core idea of treating simulators as composable "LEGO bricks" is not new in spirit (SST and SimBricks come to mind), but the specific implementation here is compelling.

The on-demand synchronization mechanism (detailed in Section 3.2, p. 5) is the technical heart of the paper and represents a significant contribution. It provides a sophisticated middle ground in the classic speed-vs-accuracy trade-off. By tying synchronization to actual communication events, it avoids the brute-force overhead of per-cycle locks while mitigating the accuracy loss of coarse-grained time quanta. The three-stage simulation flow (Figure 4, p. 5)—estimating latency, simulating the interconnect in isolation, and then re-simulating with accurate latencies—is a well-reasoned methodology borrowed from optimistic simulation principles and applied effectively here.

Finally, the thoroughness of the evaluation and the inclusion of diverse case studies (Section 6, p. 10-13) are major strengths. The authors don't just present a framework; they demonstrate its concrete value in solving real-world design space exploration problems, from analyzing HBM3 vs. DDR5 (Section 6.4) to selecting an interconnect topology (Section 6.3). This showcases the tool's utility and significantly bolsters the paper's impact. The decision to open-source the framework is commendable and will be a great service to the research community.

Weaknesses

While the work is strong, its positioning within the broader context of parallel discrete event simulation (PDES) and existing modular frameworks could be sharpened.

1. Positioning Relative to Existing Modular Frameworks: The paper mentions SST and SimBricks but dismisses them somewhat cursorily (Section 2.1, p. 2). A more detailed, principled comparison would be beneficial. For example, SST is also a parallel framework designed for integrating diverse simulation components. What are the fundamental architectural differences in LEGOSim's UII and Global Manager (GM) that enable it to have lower overhead or require fewer code modifications? A deeper discussion of the trade-offs (e.g., SST's distributed scheduler vs. LEGOSim's centralized GM) would help readers better situate this work.

2. Scalability of the Global Manager: The current architecture relies on a centralized Global Manager (GM) to coordinate all inter-simlet communication and synchronization. This introduces a potential single-point-of-failure and a scalability bottleneck. The authors acknowledge this and suggest a distributed alternative in their scalability analysis (Section 5.4, p. 10), which is a good first step. However, the potential limitations of the current, foundational architecture should be discussed more upfront. The framework's performance is fundamentally tied to the efficiency of this centralized component.

3. Overhead of Optimistic Execution: The paper notes that timing violations discovered in Stage 3 of the simulation are handled via "checkpointing and rollback" (Section 3.2, p. 6), a classic technique in optimistic PDES. The authors assert that such violations are "rare." This claim is critical to the framework's overall performance, as rollbacks can be extremely costly. However, this assertion is not backed by data. The frequency of rollbacks is highly dependent on the communication patterns and timing characteristics of the workload. Some quantitative evidence on how often these rollbacks occur in their experiments would significantly strengthen the claims of efficiency.

Questions to Address In Rebuttal

1. The Global Manager (GM) is a central coordinator, which raises concerns about scalability. While you demonstrate a distributed GM can improve performance in Section 5.4, could you elaborate on the performance limits of the baseline single-GM architecture? Specifically, at what number of simlets or inter-chiplet communication rate does the GM itself become the primary performance bottleneck?

2. Your on-demand synchronization relies on a three-stage process, with the potential for timing violations in Stage 3 to be corrected by checkpointing and rollbacks. You state in Section 3.2 that these violations are "rare." Could you provide quantitative data from your experiments on the frequency of these rollback events and their associated performance overhead? How does this frequency change with workloads that have more irregular or bursty communication patterns?

3. Could you provide a more detailed, qualitative comparison of your Unified Integration Interface (UII) to the component interface of a framework like SST? What specific design choices in the UII make the process of integrating an existing simulator like gem5 or Sniper fundamentally simpler or require less code modification than doing so within the SST framework?

Review Form

Reviewer: The Innovator (Novelty Specialist)

Summary

This paper introduces LEGOSim, a parallel simulation framework for heterogeneous multi-chiplet systems. The authors identify two key challenges with existing simulators: a lack of modular integration flexibility and inefficient synchronization mechanisms (per-cycle and time-quantum). The proposed solution consists of two primary components: a Unified Integration Interface (UII) to facilitate the modular integration of diverse simulators ("simlets"), and an "on-demand" (OD) synchronization protocol, managed by a central Global Manager (GM), to reduce simulation overhead.

The core novel claim appears to be the OD synchronization protocol, which is realized through a three-stage decoupled simulation flow. While the framework is well-engineered and addresses a timely problem, its foundational concepts are not entirely new. The idea of synchronizing only when communication occurs is a cornerstone of Parallel Discrete Event Simulation (PDES). The contribution of this paper lies not in inventing this concept, but in its specific application and methodological refinement for the multi-chiplet domain.

Strengths

1. A Novel Simulation Methodology: The most significant novel contribution is the three-stage decoupled simulation workflow described in Section 3.2 and Figure 4 (page 5). This flow—(1) initial simulation with zero-load latency to generate traffic traces, (2) offline NoI simulation on those traces to get accurate latencies, and (3) a final, corrected simulation run—is a clever and practical methodology. It effectively decouples the timing of inter-chiplet communication from the execution of the chiplet simulators themselves, which is the key enabler for the proposed on-demand synchronization. This workflow represents a tangible advancement over prior art that attempts to resolve latencies online.

2. Well-Considered Abstraction for Integration (UII): While component-based simulation frameworks with defined interfaces are not new (e.g., SST [63]), the UII presented in Section 4 (page 7-8) is thoughtfully designed for the specific challenges of integrating highly disparate simulators. The authors' consideration for handling cycle-accurate (gem5), non-cycle-driven (Sniper), and abstracted DSA simulators within a single API demonstrates a high degree of engineering novelty and provides a valuable blueprint for future work.

Weaknesses

1. Overstated Novelty of the Core Synchronization Concept: The paper presents "on-demand synchronization" as a new idea that stands in contrast to per-cycle and time-quantum methods. However, this is a well-established concept in the PDES community, often referred to as event-driven synchronization. The work of Fujimoto [27] and others on conservative and optimistic synchronization protocols laid this foundation decades ago. Frameworks like SST [63] are also built on an event-driven core. The authors fail to position their work within this broader context, giving the impression that the idea of synchronizing only on interactions is novel, when in fact it is the application methodology (the three-stage flow) that is new. The contribution is one of methodology, not of a fundamental synchronization principle.

2. Misleading "Formal Analysis for Validation": Section 3.3 (page 6) is titled "Formal Analysis for Validation." This is a mischaracterization. The section presents a standard queuing theory model (G/G/1) for estimating NoI latency. This model is used to parameterize the simulation, not to validate the correctness of the LEGOSim framework or its synchronization protocol. A formal validation would involve proofs of correctness, such as demonstrating the absence of causality errors or deadlocks in the synchronization mechanism. The current section does not provide this.

3. Unexamined Scalability of the Central Global Manager: The entire OD synchronization scheme hinges on a centralized Global Manager (GM) that arbitrates all inter-simlet requests. This design creates an obvious potential for a serial bottleneck, limiting the scalability of the parallel simulation. The authors briefly acknowledge this in Section 5.4 (page 10), suggesting a distributed management scheme as a remedy, but this is presented as an afterthought. A core contribution of the paper is simulation performance, yet the performance limitations of its central component are not analyzed. The problem of creating a correct, deadlock-free distributed time management system is a major research challenge in PDES and cannot be waved away as a simple extension.

Questions to Address In Rebuttal

1. Please clarify the novelty of your on-demand synchronization scheme with respect to established conservative Parallel Discrete Event Simulation (PDES) protocols. How does your Global Manager-based approach fundamentally differ from the time-management kernels in existing modular frameworks like SST [63]? Is the primary novelty the three-stage simulation flow, and if so, should the paper's contributions be re-framed to emphasize this methodological aspect rather than the general concept of on-demand synchronization?

2. Could you justify the title of Section 3.3, "Formal Analysis for Validation"? As the section describes a latency model rather than a formal proof of the simulator's correctness, please explain what is being formally validated.

3. The reliance on a single, centralized Global Manager (GM) appears to be a critical scalability bottleneck. Beyond the brief mention in Section 5.4, have you analyzed the performance impact of this centralized design as the number of simlets and the frequency of their communication increases? What are the specific challenges (e.g., ensuring global event ordering, deadlock avoidance) in implementing the proposed distributed management scheme, and how would that affect the framework's complexity and correctness?