CyKor, Korea University Hacking Club

Home About Contact Sponsors

[Paper Review] SyzGen: Automated Generation of Syscall Specification of Closed-Source macOS Drivers

Written by Youngin Won

This post reviews SyzGen, a system that generates Syzkaller-compatible specifications for closed-source macOS kernel drivers. It uses the Catalina 10.15.4 environment evaluated in the paper as a reference and discusses what would need to be revalidated for Sonoma.

In addition to reviewing the paper, I tested the components that were feasible to evaluate in a Catalina VM on my MacBook. This was not an end-to-end reproduction of SyzGen or a live Sonoma port.

SyzGen combines driver-binary analysis, application traces, dynamic symbolic execution, and coverage feedback. Its templates describe argument layouts, constraints, and explicit value dependencies; useful call orderings are discovered during fuzzing and retained in the corpus. Here, “syscall” follows the paper’s broad terminology for IOKit driver-interface calls.

1. Background

Syzkaller Specifications and Programs

Syzkaller is a coverage-guided kernel fuzzer. Instead of treating each request as an arbitrary byte array, it uses a declarative description language, commonly called Syzlang, to describe callable operations and their arguments. A specification can express constants, integer ranges, arrays, pointers, structures, buffer lengths, and resource types.

A specification is not itself a test case. Syzkaller instantiates the descriptions into programs containing one or more calls, mutates those programs, and retains inputs that produce useful coverage. SyzGen generates descriptions for driver-specific operations built on top of the generic IOKit calling interface and supplies them to its modified Syzkaller pipeline.

Explicit and Implicit Dependencies

Driver interfaces are often stateful, so individual calls cannot always be fuzzed independently.

  • Explicit or value dependency: one call produces a value that a later call must reuse, such as a connection identifier, object ID, or handle. Syzkaller resources encode this producer-consumer relationship.
  • Implicit or ordering dependency: calls must occur in a valid order even when no value is directly reused. For example, an object may need to be created and configured before it can be started or destroyed.

A simplified sequence is:

OPEN
-> CREATE = object_id
-> CONFIGURE(object_id)
-> USE(object_id)
-> DESTROY(object_id)

SyzGen infers explicit dependencies from application traces and generalizes them to related interfaces. Useful call orderings are explored during coverage-guided fuzzing and retained in the corpus.

Snapshot-Based Dynamic Symbolic Execution

SyzGen does not symbolically execute the entire macOS kernel. It first drives a concrete test case to the target driver interface, pauses execution there, and captures the relevant kernel-memory state. It then uses angr to symbolize the interface inputs and explore paths from that concrete context. Observed reads, comparisons, pointer dereferences, and copy operations help recover field boundaries, types, nested objects, and constraints.

This design makes binary analysis more tractable, but the recovered specification covers explored paths rather than every possible behavior. Hardware interaction, concurrency, unsupported kernel functions, and path explosion can still limit recovery.

Why Driver Specifications Matter

Random bytes rarely reach deep driver logic. A useful fuzzer must know how to open an interface, construct inputs that pass early checks, reuse identifiers returned by earlier calls, and invoke operations in a valid order.

SyzGen focuses on two kinds of information:

  • Argument structure and constraints: constants, ranges, arrays, pointers, nested objects, and supported field relationships.
  • Cross-call dependencies: explicit value dependencies and implicit ordering dependencies.

Manual recovery of this information from closed-source drivers is slow and error-prone. SyzGen automates part of that work, but traces are still needed to seed explicit-dependence inference. Without traces, it degrades to interface recovery only.

2. Target IOKit Path

SyzGen targets IOKit UserClients implemented in kernel drivers. DriverKit drivers run in user space and are outside the paper’s scope.

Service discovery
IOServiceMatching(className)
    -> matching dictionary
IOServiceGetMatchingService(...)
    -> io_service_t

Connection setup
IOServiceOpen(service, owningTask, type, &connection)
    -> IOService::newUserClient(..., type, ...)
    -> IOUserClient + io_connect_t connection handle

Request dispatch
IOConnectCallMethod(connection, selector, scalar/structure input, ...)
    -> IOUserClient::externalMethod(...)
    -> driver-specific dispatch and handler

IOServiceMatching() only creates the matching dictionary; the lookup step returns the service object used by IOServiceOpen(). Every IOConnectCallMethod() call carries a top-level selector, but the command identifier used by a particular driver may instead be stored in inputValues or inputStruct.

3. SyzGen Workflow

The paper combines four components in an iterative feedback loop.

syzgen-system-architecture.png

Figure 4 from the SyzGen paper. Application traces seed explicit-dependence analysis and the initial corpus, while driver binaries supply service, command-identifier, and interface information. Generated test cases support iterative refinement, and coverage feedback guides corpus growth.

  1. Syscall logger and analyzer. The logger records observed driver calls and the analyzer finds values produced by one interface and consumed by another. These traces seed explicit dependencies and an initial corpus, but cover only application-exercised behavior.
  2. Service and command identifier determination. Candidate service class names come from binary symbols. SyzGen symbolically explores the UserClient type path and searches for input values that lead to different driver functions, without assuming a selector-indexed dispatch table.
  3. Interface recovery. At a reached interface, SyzGen uses a concrete kernel-memory state for angr-based dynamic symbolic execution. It approximately recovers supported field boundaries, basic types, constraints, and nested userspace objects on explored paths. Known dependencies are generalized to interfaces without traces only when SyzGen matches the same internal dependence operation or a compatible validation signature.
  4. Fuzzer with kernel coverage. Explicit value dependencies are encoded as Syzkaller resources, while useful multi-call sequences are retained in the corpus. In the evaluated Intel/x86 prototype, INT3 breakpoints collected basic-block coverage through a custom in-kernel module.

The generated syscall specifications are consumed by Syzkaller, but the evaluated pipeline was not an entirely unmodified upstream build: the prototype added roughly 1,000 lines of Go integration code to Syzkaller and used a separate in-kernel coverage module.

4. Evaluation Reported in the Paper

Catalina Evaluation Platform

Item Paper setup
Guest OS macOS Catalina 10.15.4 (x86_64)
Host hardware Three Intel-based Macs
Hypervisor VMware Fusion 11.5.7
Campaign duration 24 hours per campaign, repeated three times per target; AppleAPFSUserClient was fuzzed for 72 hours because throughput was about five test cases per minute

The paper evaluated macOS Catalina 10.15.4; later Catalina releases were not part of the evaluation.

Metric Paper result and scope
Candidate discovery 254 drivers → 56 valid service names → 60 identified UserClients in the Catalina/VM environment
Selected fuzzing targets 25 of the 60 UserClients, selected by code size and input complexity; 504 valid command identifiers
Coverage proxy 48% mean per-target relative block-coverage increase over SyzGen-Base; this is not a specification-accuracy measurement or a pooled-total increase
Trace availability 36 of 338 interfaces across the nine UserClients for which logs were collected
Dependence generalization 33 dependency usages with SyzGen-IMF versus 271 with full SyzGen across five applicable UserClients: 238 additional usages
Coverage effect of generalization 16.5% mean per-target relative increase over SyzGen-IMF across those five UserClients
Bug finding 34 unique bugs after manual triage and stack-trace deduplication. SyzGen-Base found 29; five were found only by full SyzGen. Two had received CVE IDs by publication time.

SyzGen-Base already included service and command identification plus simple dispatch-table constraints. SyzGen-IMF retained logged dependencies and interface recovery but disabled signature-based dependence generalization. The 48% result therefore measures coverage improvement over a nontrivial baseline, not ground-truth specification correctness.

Bluetooth Case Studies

  • CVE-2020-9929: an arbitrary read using a userspace-supplied address. Reaching it required a correctly reconstructed nested input and valid connection_0 and connection_1 resource values obtained through the proper call sequence.
  • CVE-2020-9928: classified by the paper as an out-of-bounds (OOB) read-and-write vulnerability. A request can sleep while waiting for firmware, allowing later requests to modify associated global state; the authors state that the resulting race can be exploited for privilege escalation.

5. Significance and Limitations

Significance

  • SyzGen automates the generation of syscall specifications for closed-source macOS drivers and facilitates interface-aware fuzzing.
  • It does not assume that a command identifier must be passed through a particular parameter or used as an index into a function dispatch table. It handles function tables, switch cases, if-else statements, and identifiers embedded in inputValues or inputStruct.
  • SyzGen extracts explicit dependencies from a small number of execution traces and uses internal dependence operations and signature matching to identify related dependencies in interfaces without traces.
  • It connects interface recovery, specification generation, a modified Syzkaller-based fuzzer, and kernel coverage collection into an end-to-end workflow.

Limitations

  • The target driver must be loaded, and its user client must be reachable from userspace so that its interfaces can be invoked.
  • If no execution trace is available, SyzGen cannot infer or extrapolate explicit dependencies and degrades to a mode in which only interface recovery is performed.
  • Generated specifications were not always exact: Syzkaller could not express some cross-field relationships, and SyzGen did not recognize string-based dependency variables.
  • Retrofitting sanitizers such as Kernel Address Sanitizer (KASAN) into closed-source kernel binaries remains a challenge.
  • The paper evaluated SyzGen only on macOS 10.15.4 running in VMware Fusion 11.5.7. Compatibility and reproducibility on later macOS versions were not evaluated.

Later Work

Research on macOS driver fuzzing later expanded to Apple Silicon. KextFuzz, presented at USENIX Security 2023, fuzzes closed-source kernel extensions by instrumenting their binaries to collect coverage and analyzing user-space wrappers to infer interface information. In 2024, CrossFire was presented at ACM CCS with a different focus: memory shared between the CPU and XPUs such as GPUs. It uses an m1n1-based hypervisor to observe accesses to these regions and mutate the shared memory during fuzzing.

6. What Would Be Needed for a Sonoma Port

Porting SyzGen to Sonoma would require more than recovering a selector map. Generated inputs would need to run against a live UserClient, and kernel coverage would have to feed back into Syzkaller.

For this post, my Sonoma work was limited to static analysis of driver binaries extracted from the macOS Sonoma 14.8.9 (23J631) update payload. I did not test a live Sonoma UserClient, selector-to-handler reachability, or coverage-guided fuzzing. This work is preparation for a port, not a completed partial port.

7. Tests in a Catalina VM

The local checks used a macOS Catalina 10.15.7 (19H15) x86_64 guest under VMware Fusion 13.6.4. This differs from the paper’s macOS 10.15.4 and VMware Fusion 11.5.7 setup.

Service Discovery and Connection

A user-level probe found five services conforming to IOAudioEngine: two AppleHDAEngineInput instances and three AppleHDAEngineOutput instances. I selected one output engine using a fixed rule, and both IOServiceOpen(..., type=0) and IOServiceClose() succeeded.

A successful IOServiceOpen() confirms only that a connection was created; it does not demonstrate that a selector-specific handler was reached.

Using a second type-0 connection, I submitted one IOConnectCallMethod request with selector 2, which the IOAudioFamily source maps to GetConnectionID. The request returned 0xe00002c1 (kIOReturnNotPrivileged).

Catalina-selector2-live-result-2026-09-06.svg

Catalina VM test on my MacBook. The type-0 connection opened and closed successfully, while selector 2 returned kIOReturnNotPrivileged.

Manual Interface Mapping and Syzlang Smoke Test

Inspection of Apple’s IOAudioFamily 300.2 source identified six selectors: registerBuffer64, unregisterBuffer64, getConnectionID, clientStart, clientStop, and getNearestStartTime. I used this map to write a minimal Syzlang specification by hand and verified that it compiled in the local toolchain.

This checks only the manually written specification and build pipeline; it does not reproduce SyzGen’s snapshot-based dynamic symbolic execution.

The Entitlement Check

Apple’s IOAudioFamily 300.2 source checks the entitlement represented by kDriverHelper_DriverHostEntitlement in IOAudioEngineUserClient::externalMethod() before selector dispatch. The SyzGen setup repository identifies the corresponding private entitlement as com.apple.private.audio.driver-host. The VM’s kIOReturnNotPrivileged result is consistent with failure at this pre-dispatch check, although the runtime test did not identify the exact return site.

The public SyzGen setup repository contains private entitlements and signing helpers. Its VM instructions explicitly disable SIP, install a development kernel, and configure boot arguments including amfi_get_out_of_my_way=0x1. The repository does not isolate which of these changes is specifically required for the audio entitlement.

Entitlement Experiment

To see whether the embedded private entitlement changed the probe’s behavior, I compared two ad hoc-signed copies. During this comparison, I did not change SIP, AMFI, boot arguments, or kernel settings. In both runs of the private-entitlement variant, the shell reported status 137 and the program terminated before printing anything. The empty-entitlement control completed one open, selector-2 request, and close sequence; its selector-2 request returned kIOReturnNotPrivileged.

The two variants behaved differently, but this comparison does not isolate the cause. Both produced the same amfid message, signature not valid: -67050, while only the empty-entitlement control completed. Because the private-entitlement variant produced no call record, the experiment provides no evidence that a selector-specific handler was reached.

Catalina-entitlement-bracket-2026-09-06.svg

Comparison in the same Catalina VM: the private-entitlement copy ended before producing output, while the empty-entitlement control completed one selector-2 call and returned kIOReturnNotPrivileged.

8. Conclusion

In my Catalina VM, I enumerated five IOAudioEngine services, opened a type-0 UserClient, and converted a six-selector map from the IOAudioFamily 300.2 source into a small Syzlang specification. A selector-2 request returned kIOReturnNotPrivileged, consistent with the source’s pre-dispatch entitlement check; I did not confirm the exact runtime return site or entry into the source-mapped handler.

SyzGen’s dynamic recovery, live Sonoma execution, and coverage-guided fuzzing were outside the scope of this experiment.

Sources

Hacking macOS Kernel Fuzzing SyzGen Review CyKor