Captain Bible's 25,829 commands become a clean-room AI testbed

Peter Kelly’s Captain Bible reverse-engineering repository is a meticulous archaeology project: a reproducible FreeDOS/QEMU environment, a recovered 24-byte archive directory format, and a clean-room Rust engine for a 1994 Christian-themed DOS adventure game. The headline numbers are striking. The project maps 145 opcodes in the game’s scene bytecode, names 140 functions and 134 distinct BIN handlers, and audits all 25,829 shipped commands against a decoder. That is not a weekend hobby. It is a complete, byte-for-byte behavioral specification of a legacy software artifact, and it is exactly the kind of ground truth that AI evaluation pipelines currently lack.

The AI angle is not the game. It is the corpus. Kelly has produced something rare: a deterministic, fully specified software system with a verified executable, a clean-room reimplementation, and a documented mapping between the two. Every opcode has a semantic name. Every handler has a confidence score. Every command in the shipped game data has been checked against the decoder, with the per-opcode results in analysis/opcode-audit.tsv. For anyone building benchmarks that test whether a model can read disassembly, infer data formats, or generate compatible code, this is a goldmine. Most reverse-engineering datasets are unstructured binary dumps with no ground truth. This one has a spec book, a working engine, and a test suite.

What the project actually recovered

The repository’s documentation is organized as two mdBook volumes: a reverse-engineering record and a clean-room engine specification. The distinction matters. The research notes document what the DOS executable does, with evidence and memory dumps. The spec defines the game mechanics as a portable contract, deliberately independent of the original program’s internals. That separation is the legal and methodological backbone of clean-room reimplementation, and it is also a template for how AI systems could be trained on proprietary software without copying it.

The technical recovery is deep. The main resource archive, DD1.DAT, holds 369 members with a recovered 24-byte directory format and custom LZW-family compression. The 62 BIN resources contain scene programs, and the recovered decoder knows the operand layout and dispatch effect of all 145 opcodes. The project even corrected a two-word layout error in opcode 0x69, which removed 11 phantom commands from the linear corpus. That level of correction, verified against QEMU memory dumps, is the difference between a guess and a fact.

The audio work is similarly thorough. All 41 ABT members are compressed 9,000 Hz unsigned eight-bit mono sound effects. The 32 XMI members are one-sequence IFF/XMIDI music resources. The sound-driver chapter maps all 34 game-side int 66h sites and the DIGPAK and MIDPAK service contracts. The game’s Miles AIL/MIDPAK OPL timbre library is inspectable with a dedicated tool. Every layer of the stack, from the EXEPACK-compressed 16-bit MZ executable to the VGA palette bytes, has been documented.

Why this matters for AI

The AI evaluation field has a known weakness: benchmarks are either too synthetic or too opaque. Synthetic tasks like “reverse this toy VM” test narrow skills. Real-world tasks like “analyze this malware sample” lack verified ground truth. Captain Bible sits in the middle. It is a real, shipped, 1990s DOS game with real complexity, but Kelly has turned it into a fully labeled dataset. Every opcode is named. Every command is audited. Every save-state byte is mapped.

That enables several concrete AI use cases. First, bytecode-level code understanding. A model could be shown a BIN resource and asked to reconstruct the scene semantics, then scored against the recovered opcode catalog. Second, format inference. The DD1.DAT extractor validates directory, payload magic, compressed stream, expanded size, and exact input consumption. A model that can reproduce that extractor from the raw archive alone demonstrates genuine reverse-engineering skill. Third, cross-language porting. The Rust engine consumes the same original CB/ data directory and includes a standard-library-only core plus terminal and SDL3 frontends. A model that can generate a compatible engine from the spec, without seeing the DOS internals, would demonstrate clean-room reasoning.

The deterministic RNG alignment is a bonus. The launcher supports --rng-seed 1 for RNG-aligned, action-for-action DOS/Rust comparisons against the same game data and save state. That means a model’s output can be checked not just for structural correctness but for behavioral equivalence. Run the DOS original and the Rust engine with the same seed, compare the resulting save states, and you have a rigorous notion of “did the port preserve the game’s behavior.”

The clean-room precedent for AI training

The project’s clean-room structure has a policy angle too. The spec is written without relying on the DOS program’s internals, which is the standard legal path for interoperable reimplementation. As AI models are increasingly trained on copyrighted software, the clean-room methodology offers a defensible template: train on a behavioral specification, not on the original code. The Captain Bible project demonstrates that such specs can be complete enough to reproduce a working system. The Rust engine is not a toy. It validates the complete shipped resource set and runs the full game.

There are limits. The game is small by modern standards. Its scene bytecode is far simpler than, say, a JavaScript engine’s IR. The 25,829 commands are large for a 1990s game but trivial next to the instruction streams of modern software. The project’s value is not scale. It is precision. Every claim in the repository is backed by a tool, a memory dump, or a test. For AI evaluation, precision is the scarce resource.

What to watch

The repository is a template. If the AI community wants deterministic, verified reverse-engineering benchmarks, this is the shape they should take: a real artifact, a documented recovery process, an independent reimplementation, and a test harness that checks behavioral equivalence. The tools are all here, from audit_bin_opcodes.py to render_fullscreen_gallery.py. The next step is for someone to package this corpus into a benchmark suite and measure how well current models handle it.

The most interesting test would be simple: give a model the raw DD1.DAT archive and the QEMU memory dumps, and ask it to produce the opcode catalog from scratch. Then compare against the recovered 145-opcode ground truth. That single task would measure format inference, disassembly skill, and cross-referencing ability in one shot. No current benchmark does that. The Captain Bible project makes it possible.

Kelly’s repository sits quietly on GitHub, a DOS game from 1994 turned into a complete behavioral specification. The AI field has spent years building benchmarks that are either synthetic or unverifiable. This project is neither. It is a real artifact with exhaustive ground truth, and it is waiting for someone to build the test.