// engineering retrospective

How and why the Eshyra rules pack is built

Eshyra ships an openly licensed, machine-readable D&D 5e SRD 5.1 rules pack: 1812 structured records with per-record provenance, regenerated deterministically from the licensed source document. This is the story of why that artifact exists, how the tool that produces it stopped being a simple “importer” and became a source-grounded rules compiler, and what we learned about the boundary between what a language model should decide and what deterministic software should own.

It is written for people who might be interested in: AI game systems, deterministic rules engines, structured game data, compiler and data-pipeline design, provenance and reproducibility, the practical seam between LLM reasoning and ordinary software and the general process of working through a problem that turns out to be bigger than expected. You do not need to know anything about Eshyra to read it. If you just want the download, it is on the home page. Everything below is the engineering behind it.

Why an AI Dungeon Master needs authoritative rules data

Eshyra is a text-first, persistent AI Dungeon Master built for long-running fantasy campaigns, the kind that unfold over months, remember what happened forty sessions ago, and expect the world to stay consistent while you play. A large language model is remarkably good at most of what that job asks for:

• understanding what a player actually meant
• improvising a scene, an NPC, a consequence
• narration and tone
• living with ambiguity
• contextual judgment
• reasoning inside the fiction

What a model is not the right authority for is the part of a tabletop game that has to come out the same way every single time:

• arithmetic
• whether an action is legal by the rules
• resource accounting
• derived values
• the action economy
• reset and recharge behavior
• persistent state transitions from one turn to the next

None of this is hostility toward language models (it would be pretty awkward to be building an AI DM if we were anti-LLM). It is a division of responsibility. Asking a model to re-derive a character’s current hit points, remaining spell slots, and encumbrance from a prose transcript on every turn is asking it to be a database and an arithmetic-logic unit as well as a storyteller, and to do it identically forever. That is exactly the class of work where you want boring, delightfully uneventful, deterministic code.

That still leaves a question the architecture cannot avoid: where do the rules come from? We wanted the mechanics grounded in openly licensed primary source material: not a model’s training-data memory, not opaque generated text, and not some unexplained third-party database whose chain of custody we could not inspect. The natural candidate is Wizards of the Coast’s System Reference Document 5.1, released under a Creative Commons Attribution license. It is authoritative, it is redistributable, and it can be pinned and hashed. So the first job was to turn that document into data.

The original bet: a machine-readable reference pack

The initial architecture was a reasonable hypothesis, and it is worth stating without hindsight’s smug, condescending glare. The plan was:

• build an accurate, complete, machine-readable reference pack
• give the DM model reliable access to authoritative rules prose and a set of useful structured fields
• automate the operations that are clearly deterministic
• let the DM model adjudicate much of the rest of the game directly from the prose

In other words: get a faithful, source-complete reference corpus into the machine, automate the obvious math, and lean on the model’s judgment for everything situational. That is a defensible first design. It scopes the ingestion work, it avoids prematurely building a rules engine, and it plays to the model’s strengths. The open question was whether authoritative prose plus selective deterministic tooling would actually hold up to actual game play. That was not something we could answer from a design doc, so we built it and went to find out!

The deterministic importer

The reference pack is produced by a pipeline, not typed by hand. The original shape of that pipeline was:

• a licensed source corpus, vendored in the repository
• a pinned source identity and SHA-256 hash
• deterministic extraction from that source
• generated records with stable keys and typed record kinds
• per-record provenance naming the source location
• reproducible generation: same input, same output
• schema validation, plus tests and audits over the result

The single most important rule sits underneath all of that, and it is worth being explicit about because it is easy to get wrong when the tool building your data is itself a language model:

A model can help build, inspect, and review the machinery. It is not the authoritative source of the rules.

A coding agent is genuinely useful here. It can write parser code, propose how a source field maps onto our schema, spot anomalies in the output, audit records for suspicious values, and suggest test cases. What it must never do is fill in canonical mechanics from its own training-data memory. If the model “knows” that a particular cantrip deals 1d10 fire damage, that knowledge is not admissible as pack content, even when it happens to be correct, because it is not reproducible from the vendored source and cannot be audited against it. The authoritative bit of every record has to trace back to the licensed document.

This is the boundary the project’s architecture decision records call the model-assistance boundary: the licensed primary corpus is authoritative; the model is a development tool, not a data source. Everything the pipeline emits is reproducible by re-running it over the pinned source, and reviewable in isolation from the data it produces. A change to the records has to show the change to the code or the curated input that produced them.

The first freeze, and what playtesting taught us

The project reached an initial quality bar, froze the pack, and started using it in real play. The freeze was not ceremonial: the artifact was source-complete, every record carried provenance, an independent audit accounted for the source document region-by-region, and regeneration was byte-for-byte reproducible. As a reference corpus, it was solid. Better than solid, it’s exactly what we said we needed! Surely our DM will now be a wise and knowledgeable Rules Lawyer: Eshyra Esq. Attorney at Law!

Then we actually played it. Oof. Like our favorite ace attorney, every beautifully narrated scene the DM proposed was met with a vigorous Objection! from the auditor LLM* author’s asideIf you are thinking that it is the lawyers who object, not the (LLM-as-)judge, then you are my kind of people. But also, look over there, a shiny object!, which kept refusing to sign off because it could not prove what the canon mutation tools needed proven. And character creation, without a deterministic curated experience underneath it, was a hot mess.

What matters is what the auditor kept tripping over. So let me start with the failures, before I pretend I understood them.

What actually broke

Character creation

The early character-creation runs did not produce subtly wrong characters. They produced nonsense. The system would cheerfully offer class and ancestry combinations the source does not support, hand back ability-score arrays no legal method could generate, and, my personal favorite, fail a level-one hit-point check by announcing that the maximum allowed value was -4. A first-level character with a negative hit-point ceiling is a strong hint that something upstream has stopped meaning anything.

The shape of the mistake was consistent. We were asking the model to generate a whole interdependent character in a single pass, then validate and repair it after the fact. But character creation is not a single object you emit and then inspect. It is a dependency graph, where earlier choices constrain which later ones are even legal and some unlock decisions that did not exist a moment before. Generate the whole thing at once and you have already discarded the ordering that made any of it valid.

Combat

Combat failed more quietly, which made it easier to overlook. Attacks and damage were routed through deterministic tools, so at least the dice were honest. But many of those rolls never reached the player as a coherent chain of cause and effect: you would feel a result without seeing the roll that produced it. Underneath that presentation problem sat a sharper one. The system could look up a creature and read its rules, and that lookup told the runtime nothing about how any given mechanic was supposed to execute. Knowing what a monster’s ability says is a long way from knowing when it triggers, what it changes, and who applies the change. The model could read the rules; what it could not do, and should never have been asked to do, was execute them reliably, turn after turn, as though it were a rules engine.

Were these just bugs?

For a while I assumed so, and that we were working through an unlucky pile of unrelated defects. Fix the character validator. Fix the roll presentation. Add the missing state. Tighten the auditor. Some of that was genuinely true, and those fixes stuck. But every one of them kept uncovering the same question underneath: did the rules pack actually contain the information the deterministic parts of the system needed, or only an accurate description of it?

The three examples that finally made it click did not come from that first playtest. They came out of the follow-up debugging, audits, and design work. Each one starts from a rule the frozen pack described perfectly well and ends where a deterministic consumer still could not do its job.

The choice was in the prose, not in a structure you could enforce

A lot of character creation is choosing from constrained menus: Warlock Pact Boons, Eldritch Invocations, Fighting Styles, Metamagic, the starting-equipment options. The pack carried the prose for every one faithfully, and a person or a model could read it. But a deterministic character builder does not want the paragraph. It needs to know whether the choice is required, how many selections are allowed, the closed set of legal options, a stable identifier for each, and the prerequisites that gate them. None of that survives as English. The clearest symptom was an Eldritch Invocation whose prerequisite the builder had boiled down to the string Pact of. Prerequisite satisfied: Pact of. Pact of what was, apparently, not load-bearing. The source was right there, complete and correct; the representation had simply never captured the part of the rule the consumer relied on.

A condition reference is weak until you say what it means

The second example is subtler, and more important. A mention of a condition tells the runtime almost nothing on its own. Does the rule impose the condition, remove it, require it as a precondition, grant immunity to it, or end when it ends? Five different behaviors wearing the same word.

The Mimic’s False Appearance is where this caught us. An early structured projection captured that the creature is indistinguishable while motionless, and quietly dropped the qualifier that it is indistinguishable while motionless and in object form. The result was typed, machine-readable, and noticeably cleaner than the prose it came from. It was also wrong, which is the uncomfortable lesson: adding structure does not make a rule more correct. The structure has to preserve the rule’s applicability, triggers, and consequences, or all you have built is a tidier way to be mistaken.

A stateful rule runs straight into the execution boundary

The third example looks trivial and is not. Take a magic item with 7 charges that regains 1d6+1 charges at dawn. One paragraph of source prose. Follow what a system actually has to do with it and you cross three separate concerns.

Some of it is pack semantics, fixed by the source and true of every copy of the item: the maximum charge count, what an activation costs, the recharge formula, and the trigger that resets it. Some of it is live state, true only of one particular item in one particular campaign at one moment: this wand has 3 charges left. And some of it is deterministic behavior that has to come out the same way every time: reject an activation that would cost more than the charges remaining, subtract the charges when the use is legal, notice dawn, roll 1d6+1, add the result without exceeding the maximum, and persist the new value for next time.

Put the three side by side and the common cause is hard to miss. The pack never needed more data. The frozen pack was genuinely strong at the question it had been built to answer:

What does the source say?

Play forced a second question the reference corpus had never been built to answer:

What does the system need to know, and who is responsible for making it happen?

That is a question about semantic structure and ownership, not volume, and it does not make the compiler own the game. It still produces immutable, source-grounded semantics and names the execution each rule requires; holding live state and running the deterministic behavior stay in separate layers. The playtest only changed how much the pack itself had to say for those other layers to function.

The pack needed to become a semantic substrate for the model/engine execution boundary, not merely an authoritative reference corpus.

Two product stages, not six moving targets

The goal changed once, not six times. There were two stages: first a complete, accurate, source-grounded reference pack, with deterministic structure wherever that was clearly useful; then a richer source-grounded semantic substrate capable of supporting a substantially more deterministic engine. The second stage does not repeal the first. A source-complete reference pack is still necessary.

What the second stage added was a vocabulary for treating correctness as several independent dimensions rather than one vague notion of “done.” An artifact can be strong on one and weak on another:

• reproducibility: re-running the compiler over the pinned source reproduces the pack exactly.
• source completeness: every source rules element is accounted for, not quietly dropped.
• structural fidelity: stat blocks, tables, and rows are reconstructed faithfully, source typos and all.
• semantic modeling: the pack carries the typed mechanics the execution boundary needs.
• runtime ownership: each rule’s behavior is explicitly owned, not left unclassified.
• end-to-end playable validation: the rule actually works in a played turn over the real engine.

These are evaluation lenses, not six historical rewrites. A reference-complete pack with no semantic layer is a real thing; so is a semantically rich record whose engine hook does not exist yet. Keeping them distinct is what lets us say precisely which one is failing.

From importer to rules compiler and executable curator

This is the conceptual center of the story. A lot of paths, commands, and issue IDs still say importer, and we deliberately did not rename them: churn is not architecture. But what those names point at is better described now as a source-grounded rules compiler with executable curation, and its value is not that it is a magic general-purpose PDF parser. It is not one, and pretending otherwise would be the wrong lesson to take from it. The SRD mixes shapes that no single technique handles well:

• places where a genuine structural grammar repeats: stat blocks, spell metadata, table rows.
• document-specific extraction problems: multi-column text, wrapped table cells, continuation pages.
• irregular semantic content: mechanics expressed in prose, exceptional tables, meaning that generic parsing cannot honestly infer.

So the compiler uses different techniques for different shapes of source, and picks the cheapest one that is still trustworthy. The ordering matters:

1. Structural parsers: where real grammar exists

Where the source repeats a genuine structural grammar, parse it: stat blocks, spell metadata, table and equipment rows, class progression, consistent headings. Enough regularity that a parser is cheaper and more trustworthy than curating each record by hand.

2. Shared semantic grammars: for recurring mechanics

Some meaning recurs even when the typography does not: recurring forms of damage, healing, saving throws, usage limits, recharge language, modifiers, common effect structures. For those, a reusable semantic grammar earns its keep. Crucially, these grammars fail closed: prose that falls outside the reviewed shape contributes no semantics at all, rather than being guessed at into a plausible-looking fabrication.

3. Declarative executable curation: for irregular families

Then there are the irregular semantic families, the ones where a human already has to read and classify each clause individually before any extraction logic could be written. The magic items are the archetype: a couple hundred items with bespoke economies, state machines, curses, and containment semantics that share no parseable grammar.

For those, the honest move is to represent the reviewed decisions explicitly, as source-grounded deterministic compiler inputs: a registry of clauses, each with a stable id, bound to its representation and the engine hooks it needs. This is what we mean by executable curation. It is usually clearer than building elaborate natural-language heuristics that merely re-encode, in fragile regular expressions, decisions a human already made during review.

4. Exceptional procedural handling: for genuine one-offs

Finally, narrow one-off code is acceptable for true exceptions: a single item with a bespoke state shape, one table whose columns are irrecoverably interleaved. What it must not become is the silent default. If you find yourself writing per-record if (key === …) branches across dozens of records, you actually wanted a shared grammar or a curated specification.

The point of the whole hierarchy is one sentence:

The goal is trustworthy, source-grounded semantics, not maximizing how much English the software appears to infer automatically.

One distinction here is easy to misread, so state it flatly. Executable curation is not hand-editing the generated output. A curated clause registry is a compiler input: source-grounded, schema-checked, and it fails loudly if the source drifts underneath it. The generated records.json stays a generated output that nobody hand-edits. We will come back to why preserving that line is worth the machinery.

Three layers: pack semantics, live state, engine services

The charge example already fell apart into three concerns, and that was not a quirk of magic items. The same three-layer separation kept surfacing as the semantics grew, and the compiler owns exactly one of the three.

Pack semantics: immutable, source-derived

What a rule or item can do: effects, formulas, prerequisites, usage and recharge contracts, engine-hook requirements. Compiled from the source, never carrying play-state. This is the compiler’s layer.

Live state: mutable, per-campaign

What is currently true in one campaign: current hit points, charges, attunement, current form, stored spells, ownership. It lives in a per-campaign store, keyed per instance, and pack records never carry it.

Engine services: shared behavior

The shared runtime machinery: dice, action economy, concentration, spell slots, reset and recharge, rest processing, character-build resolution, derived math. A magic item subscribes to the shared dawn/rest reset; it does not carry its own private recharge loop.

Rules records name what mechanics require and which hooks they need; the engine executes the shared deterministic behavior. The compiler is emphatically not the game engine.

The hybrid execution model

The model interprets intent and makes contextual rulings; the engine owns the arithmetic and state transitions that must repeat. The subtle part, and the part it took playtesting to see, is that a single rule does not always belong wholly to one or the other. Ownership is often clause-level. Cover is the canonical example:

• the model reads the fiction (positioning, terrain, line of sight) and decides whether a creature qualifies for half cover or three-quarters cover.
• the deterministic rules surface owns applying the corresponding +2 or +5 bonus to Armor Class and Dexterity saving throws once that determination is made.

One rule, two owners, a clean seam. That is the general shape of most “situational” rules: the contextual determination is a model judgment, and the numeric modifiers, dice transformations, state changes, and resource effects that follow are deterministic.

It cuts both ways: not every rule needs its own rigid subsystem. Some are correctly left as terminal model rulings, when the open question is genuinely contextual, the deterministic primitives for randomness, arithmetic, and state already exist, and that ownership was consciously reviewed rather than accidentally omitted. The requirement is not an engine for everything. It is that ownership is explicit: every deterministic rule is code-enforced, a known engine gap, a deliberately deferred decision, or a reviewed model ruling. By classification, never by accident.

How the project tries to know when it is wrong

A pipeline like this is only as trustworthy as its ability to catch itself failing, so over time the verification became at least as important as the extraction. The mechanisms fall into a few families:

• source identity and provenance: hashing and pinning the source, with per-record provenance back to it.
• source inventory and coverage: structure inventories and explicit coverage accounting, so owned source content must prove it was emitted.
• schema, reference, and semantic validation: schema and reference-integrity checks, plus semantic projections that fail closed rather than guess.
• runtime ownership and readiness: explicit execution-boundary classification, fail-closed dispositions for anything not yet modeled, and runtime readiness checks.

The guiding instinct is this:

When you discover a failure class, the best response is usually to turn it into a broader invariant that makes the whole class visible, not to patch the one record.

There is a real tension, though: not every finding should spawn a new registry or parallel proof system. Audit machinery has a cost and can drift from what it claims to measure, so before adding another checker the project asks whether an existing invariant can be strengthened instead. The aim is strong, converging, fail-closed evidence, not a claim of proven perfection. We are trying to make being wrong loud, not to pretend it is impossible.

The skeptical question: why not just hand-write records.json?

This is the objection I have the hardest time dismissing: if so much of the hard part is human review anyway, why not skip the parsers and just curate a database by hand?

Honestly: in some areas, hand-authoring would have been faster, and I won’t pretend every parser paid for itself against simply writing the data down. For irregular, low-repetition material, careful curation is a legitimate technique, which is exactly why executable curation is a first-class part of the design rather than a grudging exception to it.

The reason the source-bound compiler model still earns its place is everything that a bag of hand-typed JSON quietly gives up:

• a chain of custody from source to artifact
• reproducible regeneration
• reviewable transformations rather than opaque values
• bulk schema evolution and corpus-wide migrations
• independent auditing against the source
• preservation of past corrections
• a visible distinction between repeated grammar and reviewed exceptions
• detection when the source itself drifts

So the useful question was never the binary “parser or hand-edited JSON.” It is sharper than that:

Which source structures should be parsed, which recurring semantics deserve shared grammars, and which irregular families should be represented as explicit, source-grounded executable curation?

Answering that per family, rather than committing to one technique for the whole document, is most of what “compiler with executable curation” means in practice.

What transfers to future systems such as Pathfinder

So here is the uncomfortable question: if the D&D PDF heuristics are largely disposable, and they are, what did we build that outlasts this one document? Not the heuristics. What transfers is the architecture and methodology. A future system (Pathfinder is the obvious next test) would reuse the contracts, not the document tricks:

• establish source and licensing authority first, and pin authoritative versions
• inventory the source’s structures early, failing closed on anything unaccounted for
• choose a format-appropriate extraction boundary rather than imitating PDF-layout reconstruction; an HTML source should not pretend to be a PDF just because the D&D compiler is one
• choose deliberately, per family, among parsing, shared grammars, and curation
• preserve provenance and enforce verification and explicit runtime ownership

Reuse contracts, not document heuristics.

We also do not yet know which abstractions are genuinely cross-system. Generalizing from a sample size of one is how you end up with a “framework” shaped exactly like D&D, so the second system is the real test: make the shared extension points explicit and let implementation pressure, not speculation, reveal what is truly shared. Pathfinder support is not shipped; it is a deferred, planned direction.

What you can download today

The downloadable rules pack is a structured, machine-friendly representation of the D&D 5e SRD 5.1: 1812 records across 16 record kinds, in a single JSON array, alongside a manifest with license and source provenance. It is roughly 586.6 KB zipped. Every record carries a kind, a stable key, a name, a data payload, and a provenance block naming the SRD location it came from. It is built from the same committed pack data the project uses, on 2026-07-10, and both the source document and the ZIP publish their SHA-256 hashes so you can verify exactly what you have.

Plausible uses:

• rules lookup
• search and indexing
• tabletop software experiments
• AI assistants and bots
• structured data analysis
• rule-reference tooling
• other source-attributed software

A precise word on status, because overclaiming here would be exactly the wrong note to end on. The pack is source-complete, provenance-tracked, and reproducibly generated. It was frozen once as an audited reference artifact, and it is currently under a stricter complete/accurate/playable re-audit while the semantic substrate described above is closed out. Its source and pack identities stay pinned throughout. So: this is a genuinely useful, openly licensed, verifiable structured representation of the SRD, not a claim that every gameplay mechanic is already perfectly modeled or executable. We would rather you find it solid than find it oversold.

It is distributed under the Creative Commons Attribution 4.0 International License. If you redistribute the records or derivatives, preserve the attribution notice that ships with the bundle. The pack is reference data derived from CC BY 4.0 source material; it is not original game content and not an endorsement by Wizards of the Coast.