ClosureX:Compiler Support for Correct Persistent Fuzzing

Paper Title: CLOSUREX: Compiler Support for Correct Persistent Fuzzing
Reviewer: The Guardian (Adversarial Skeptic)

Summary

The authors present CLOSUREX, a compiler-based instrumentation framework designed to enable high-performance, semantically correct persistent fuzzing. The core idea is to eliminate the process management overhead (e.g., fork) inherent in traditional, correct fuzzing approaches. CLOSUREX achieves this by instrumenting the target program at the LLVM IR level to track and reset program state between test case executions within a single, long-running process. The state restoration targets global variables, the program stack, heap allocations, and file descriptors. The evaluation, conducted on ten benchmarks, claims that CLOSUREX achieves a 3.5x speedup in executions per second over AFL++'s forkserver mode, finds bugs 1.9x faster, and maintains semantic correctness equivalent to that of a fresh process execution.

Strengths

1. Problem Significance: The paper addresses a well-understood and significant bottleneck in fuzzing: the performance cost of process creation and initialization. The motivation to bridge the performance gap between incorrect persistent fuzzing and correct fork-based fuzzing is strong.

2. Sound Core Concept: The approach of using compile-time instrumentation to manage state rollback is a logical and powerful technique. It correctly identifies the primary sources of state pollution in many C-based programs.

3. Impressive Performance Results: A 3.5x average increase in test case throughput (Table 5, p. 9) over the state-of-the-art forkserver is a substantial performance gain. If correct, this is a significant engineering achievement.

4. Tangible Bug-Finding Impact: The discovery of 15 0-day bugs, including 4 CVEs (Abstract, p. 1), provides strong evidence that the tool is practically effective, at least on the selected targets.

Weaknesses

My primary concerns with this submission revolve around the strength and generalizability of its correctness claims, the limitations of its state restoration model, and the robustness of its bug-finding evaluation.

1. Overstated and Unsubstantiated Correctness Claims: The central premise of the paper is "correctness." However, the validation of this claim in Section 6.5 (p. 10) is methodologically weak and does not support the strong assertions made.

   • The authors' method for verifying equivalence relies on running queued test cases and comparing dataflow (heap state) and control-flow (path coverage) against a fresh process execution.
   • Critically, the authors state they "exclude test cases that induce a non-deterministic execution path on the target." This is a fatal flaw in a proof of general correctness. Fuzzing inherently explores complex, often-unstable program behaviors, including those involving uninitialized data, certain PRNGs, or environmental interactions, which can be non-deterministic. By excluding these cases, the evaluation demonstrates correctness only for the subset of well-behaved, deterministic executions, fundamentally undermining the claim of general semantic equivalence. The claim of "maintaining semantic correctness" (Abstract, p. 1) is therefore not fully supported.

2. Incomplete State Restoration Model: The paper claims to provide "whole-application persistent fuzzing" (Section 4, p. 5), but the described state restoration is partial. CLOSUREX handles globals, stack, malloc/free, and fopen/fclose. This neglects numerous other critical sources of process state that can cause cross-test-case contamination:

   • Memory Maps: State created via mmap is not discussed. A target that maps a file, modifies it in-memory, and does not munmap it will pollute subsequent test cases.
   • IPC and Networking: Sockets, shared memory segments, pipes, and other forms of inter-process communication are not handled. A server application under test would almost certainly enter an invalid state.
   • Static State in Libraries: The function replacement technique for malloc and fopen will fail if a statically-linked, pre-compiled library makes direct syscalls or uses its own internal state/allocators that are not visible at the LLVM IR level of the main application. The authors acknowledge this as future work in Section 7.4 (p. 11), but it is a fundamental limitation of the current approach and its claims. The "correctness" is conditional on a very specific and limited programming model.

3. Weak Bug-Finding Metric: The claim that CLOSUREX "finds bugs more consistently and 1.9x faster than AFL++" (Abstract, p. 1) is based on the "time-to-first-bug" metric (Table 7, p. 10). This metric is notoriously noisy and can be misleading. A superior evaluation would compare the total number of unique crashes or unique code paths discovered by each fuzzer over the entire 24-hour campaign. It is possible that while CLOSUREX finds the first bug faster due to raw speed, AFL++'s different execution pacing might explore a different, ultimately more productive part of the state space over the long term. Without this data, the claim of superior bug-finding effectiveness is weak.

4. Ambiguous Comparison Baseline: The paper rightly positions AFL++'s forkserver as the primary correct baseline. However, it dismisses AFL++'s own persistent mode as incorrect without providing a performance comparison. While the mode is indeed fragile, quantifying the performance gap between it and CLOSUREX would clearly demonstrate how much of the "unsafe speed" has been recovered "safely." Similarly, kernel-based snapshotting is dismissed on portability grounds (Table 2, p. 4), but its performance on a supported configuration is not compared. Is CLOSUREX faster than all correct approaches on their optimal platforms? The paper does not provide the evidence to support such a strong claim.

Questions to Address In Rebuttal

1. Given that your correctness evaluation explicitly excludes non-deterministic test cases, how can you justify the general claim that CLOSUREX "maintains semantic correctness"? Please re-frame the contribution to accurately reflect that correctness has only been demonstrated for deterministic program paths.

2. Please provide a more exhaustive list of stateful behaviors your system does not currently handle (e.g., mmap, sockets, direct syscalls from linked libraries, dlopen). How would the presence of any of these in a target program break CLOSUREX's correctness guarantees?

3. To substantiate the claim of superior bug-finding, please provide data comparing the total number of unique crashes (or a similar metric like unique edges/paths) discovered by CLOSUREX and AFL++ over the full 24-hour fuzzing campaigns, rather than just the time to the first crash. A time-series plot would be most effective.

4. Can you clarify the practical limitations of your approach regarding the fuzzing of complex, real-world applications that rely heavily on pre-compiled third-party libraries (e.g., OpenSSL, zlib)? If all source code must be available and instrumented, this should be stated as a primary constraint of the system.

Reviewer: The Synthesizer (Contextual Analyst)

Summary

This paper presents CLOSUREX, a compiler-based system designed to solve a fundamental trade-off in fuzzing: the choice between the high performance of persistent fuzzing and the semantic correctness of fresh-process execution. The authors correctly identify that while persistent mode offers the highest throughput by reusing a single process, it suffers from state contamination across test cases, leading to incorrectness and missed bugs. Conversely, approaches like fork-server are correct but incur significant process management overhead.

The core contribution of CLOSUREX is a novel point on this "state restoration continuum." By using a series of LLVM passes, CLOSUREX instruments a target program to become "self-resetting." It injects a persistent fuzzing loop and automatically adds code to track and roll back key sources of program state—specifically global variables, heap allocations, and file descriptors—between each test case. This approach effectively simulates a fresh process for each input while eliminating the overhead of process creation and destruction. The evaluation demonstrates that CLOSUREX achieves a ~3.5x performance increase over AFL++'s standard fork-server mode and finds bugs 1.9x faster, all while maintaining the semantic correctness of fresh-process execution.

Strengths

The true strength of this work lies in its elegant and highly practical approach to a long-standing, important problem in the fuzzing community.

1. Excellent Problem Formulation and Positioning: The authors do a superb job of contextualizing their work. The continuum from fresh-process (correct, slow) to persistent (fast, incorrect) is a perfect framing. Table 1, in particular, is a masterful piece of communication, immediately showing the gap in the design space that CLOSUREX aims to fill: a portable, correct, high-performance solution that works for whole applications.

2. A Pragmatic and Portable Solution: By choosing a compiler-based approach (LLVM), the authors sidestep the major pitfalls of competing high-performance solutions. Unlike kernel-based snapshotting (e.g., [7, 34]), CLOSUREX is OS-agnostic and does not rely on fragile, version-specific kernel interfaces. This makes the solution far more deployable and maintainable. It represents a user-space, compile-time alternative to both kernel-level and binary-level snapshotting techniques [29], occupying a sweet spot of performance and accessibility.

3. Addresses the "Annoying Last Mile" of Persistent Fuzzing: While experts have long known how to write manual reset functions for persistent mode, this process is tedious, error-prone, and requires deep target-specific knowledge. CLOSUREX automates this harness generation, democratizing high-performance, correct fuzzing. This is a significant engineering contribution that lowers the barrier to entry for effective fuzzing campaigns.

4. Strong Empirical Validation: The performance gains are substantial and well-documented across a diverse set of standard fuzzing benchmarks. The most critical part of the evaluation, the correctness check in Section 6.5, is well-designed. By verifying both dataflow and control-flow equivalence against a ground-truth fresh-process execution, the authors provide strong evidence for their central claim of maintaining semantic correctness.

Weaknesses

The weaknesses of the paper are primarily related to the boundaries and limitations of the proposed technique, which could be explored more deeply.

1. Scope of State Restoration: The current implementation handles the most common sources of state (globals, heap, file descriptors). However, real-world applications often involve more complex state. The discussion in Section 7.4 acknowledges this, but the paper would be stronger if it more formally defined the classes of state it cannot handle. For example, state hidden in linked, non-instrumented libraries, state managed by custom allocators, interactions with hardware, or persistent changes to the filesystem are all outside the current scope. Similarly, the proposed method for handling threads in Section 7.3 seems optimistic; managing thread-local storage and ensuring clean thread teardown is notoriously difficult.

2. The Challenge of Initialization Overhead: The paper's primary performance comparison is against the fork-server. A key advantage of the fork-server is that it snapshots the process after program initialization. CLOSUREX, by looping around main, re-executes this initialization code for every single test case. For programs with a heavy, one-time setup cost, this could significantly erode the performance benefits. The authors acknowledge this as future work (Section 7.1), but it remains a notable limitation of the current approach when compared to the fork-server model.

3. Lack of Comparison to Expert-Crafted Harnesses: The performance evaluation is missing a key baseline: a manually-written, expert-crafted persistent mode harness for one of the simpler targets (like zlib). While the value of CLOSUREX is its automation, understanding the performance overhead of its generic state-tracking mechanisms compared to a bespoke, minimal reset function would provide valuable context. Is there a performance price for this automation?

Questions to Address In Rebuttal

1. Regarding initialization overhead: Could you provide an estimate or measurement for one of your benchmarks (e.g., bsdtar) of the proportion of execution time spent in one-time initialization code that CLOSUREX re-executes but a fork-server would not? This would help clarify the types of targets where CLOSUREX offers the greatest benefit.

2. Regarding complex state: How would the CLOSUREX model handle C++ programs, specifically with respect to static object constructors/destructors and exceptions? Does the setjmp/longjmp mechanism for replacing exit() correctly unwind C++ objects, or could it lead to resource leaks?

3. The proposed solution for resetting heap state by hooking malloc/free is elegant. However, many complex C/C++ applications use custom memory allocators (e.g., arena or slab allocators) for performance. How would a user adapt CLOSUREX to support a target with a custom allocator that manages a large, contiguous block of memory internally? Would this require manual annotation or a new set of passes?

Reviewer: The Innovator (Novelty Specialist)

Summary

This paper presents CLOSUREX, a system for achieving high-performance, correct persistent fuzzing. The authors' core claim is a method for transforming standard C programs into "naturally restartable" ones using a series of compile-time LLVM passes. This instrumentation automatically injects code to track and reset key sources of program state—specifically global variables, the heap (via malloc/free hooks), the stack (via setjmp/longjmp), and file descriptors (via fopen/fclose hooks)—between fuzzing iterations within a single process. This avoids the overhead of process creation (fork/exec) while maintaining the semantic correctness lost in traditional persistent fuzzing modes.

Strengths

The primary strength of this paper lies in its specific implementation strategy. The application of compiler-level instrumentation to automate state-reset for persistent fuzzing is an elegant engineering approach. By operating at the LLVM IR level, the authors have a principled way to intercept state-modifying library calls and manage memory sections. This source-based approach is a clean alternative to OS-level primitives or binary-level rewriting and offers the potential for fine-grained, precise state management.

Weaknesses

The central weakness of this paper, from a novelty perspective, is that the core conceptual idea—in-process state-saving and restoration to enable fast, correct fuzzing—is not new. The work is best characterized as a novel implementation of a pre-existing concept.

1. Overlap with Prior Art: The work is conceptually very similar to WinFuzz (Stone et al., NDSS '21) [11] and its follow-on work (Stone et al., USENIX Security '23) [29]. WinFuzz also implements in-process snapshotting and restoration to bypass OS-level overhead. It achieves this by rewriting the binary to save and restore memory regions (heap, stack, globals) and other process state. While the mechanism differs (LLVM instrumentation vs. binary rewriting), the fundamental idea of creating a self-resetting process for fuzzing is the same. The authors acknowledge this work in Section 8.2 (Page 12) but claim their approach is superior due to being "fine-grain" and avoiding "runtime code injection overhead." However, this claim is presented qualitatively and is not substantiated with a direct comparison, making the "delta" over prior art seem incremental—an alternative engineering choice rather than a new paradigm.

2. Limited Scope of State Restoration: The novelty is further constrained by the specific types of state handled. The presented solution hooks standard libc functions (malloc, fopen, exit). This approach is well-understood but does not address more complex, yet common, scenarios. For instance, programs employing custom memory allocators, memory-mapped files (mmap), direct syscalls for I/O, or state stored in shared memory would not be correctly reset by CLOSUREX out-of-the-box. The authors acknowledge this in Section 7.4 (Page 11), but this limitation implies that the "automatic" solution is only automatic for a well-behaved subset of programs. The novelty is thus in the specific implementation of these hooks, not in a generalizable state-reset framework.

3. Well-Known Techniques: The techniques used for state restoration are, individually, not novel. Using wrappers around malloc/free to track allocations is a standard technique used in memory debuggers for decades. The use of setjmp/longjmp to hijack control flow from exit() is a classic C programming idiom for implementing exception-like behavior. The novelty lies only in the composition of these specific techniques for the fuzzing use case.

Questions to Address In Rebuttal

1. Clarify the Delta vs. WinFuzz [29]: The authors claim that binary-level snapshotting (as in WinFuzz) is subject to "imprecision with its state checkpoints." Please provide a concrete example of a state-related bug or inconsistency that would be missed by a binary-level approach like WinFuzz but correctly handled by CLOSUREX's compiler-level instrumentation. What, precisely, is this "imprecision"?

2. Generalizability of the Technique: How much manual, target-specific effort would be required to adapt CLOSUREX to a target that uses a custom slab allocator instead of malloc? If the technique relies on developers manually identifying and hooking all sources of latent state, how does this represent a significant leap over the manual reset handlers already used in tools like libFuzzer?

3. Robustness of longjmp: The use of longjmp to unwind from a call to exit() is a C-style mechanism. Have the authors considered its correctness in C++ programs, where this would bypass the execution of destructors for stack-allocated objects, potentially leading to resource leaks or incorrect state for the next fuzzing iteration? This could undermine the core claim of "semantic correctness."