plai_testr 0.2.0 → 0.2.1

data/docs/ingesting-a-game-with-an-agent.md

@@ -0,0 +1,138 @@
+# Ingesting a game with an agent
+
+*Ingesting* a game means turning your rulebook into a *harness*: code that knows your game's rules well enough to deal the cards, list each player's legal moves, apply the one they pick, and tell when the game is over. Language models are unreliable at tracking a game's state over a long game, so the harness does the bookkeeping and the model only chooses.
+
+You can have an AI coding assistant (an *agent*, such as [Claude Code](https://claude.com/claude-code)) write the harness for you. This guide covers what to prepare, how to ask, and how to check the result. It assumes you've finished [Getting started](getting-started.md). To write the harness yourself instead, see [Ingesting a game manually](ingesting-a-game-manually.md). The two mix well: have the agent write the first version, then read and change it yourself.
+
+The examples below use Blackjack, whose [rules](https://bicyclecards.com/how-to-play/blackjack) are public domain.
+
+## Step 1: Create the game's folder
+
+From your app folder:
+
+```sh
+bundle exec plai_testr generate game blackjack
+```
+
+Game names are lowercase, with `_` instead of spaces. This creates `games/blackjack/` with a starting file for everything the harness needs:
+
+```
+games/blackjack/
+  data/rules.md            The rules, as the model players will read them
+  data/components/         Card lists and other components
+  data/rules.json          The game's name, version, player count, and numbers the code uses
+  data/personas.yml        Optional: strategies and play styles for the model players
+  data/bots.yml            Optional: settings for the bots
+  lib/                     The harness code (the agent writes this)
+  spec/                    Automated checks (the agent writes these too)
+```
+
+You fill in the first two; the agent does the rest.
+
+## Step 2: Write the rules
+
+Put your rules in `games/blackjack/data/rules.md`, in [Markdown](https://www.markdownguide.org/basic-syntax/). There's no required format: write it the way you'd explain the game to a new player. The model players read this file every turn, so:
+
+- Describe the procedure: setup, what happens on a turn, how the game ends, how scoring works.
+- Long examples and flavor text can go; they cost the models reading time.
+- Leave out house rules and variants you aren't testing.
+
+Don't worry about gaps. The agent is told to find them and ask you.
+
+## Step 3: List the components
+
+Put every card, tile or token list in `games/blackjack/data/components/`, one file per kind of component. The file name becomes the component's name (`cards.csv` is "cards").
+
+A file is a list with one row per component and a column for each thing about it. A spreadsheet saved as CSV works:
+
+```csv
+name,suit,rank
+A♣,clubs,1
+2♣,clubs,2
+```
+
+YAML works too, and suits cards with longer text:
+
+```yaml
+- name: Ambush
+  cost: 2
+  text: Play when an opponent attacks. Cancel the attack.
+  copies: 3
+```
+
+A `copies` column makes that many identical copies. `games/hearts/data/components/cards.csv` is a complete standard deck.
+
+Already have your components in another form (a spreadsheet, an export from a card tool, a PDF)? Point the agent at it instead; it will convert it and check the counts with you.
+
+## Step 4 (optional): Sketch some personas
+
+A *persona* is a play style: a strategy the model players follow, and settings that make the bots play differently. Botruns then compare how each style does.
+
+Personas build on a *basic strategy* that every player follows. For Blackjack that might be "never hit on 21" and "always hit under 11". Each persona then adds its own approach: a `by_the_book` player who hits under 16 and stands on 16 or more, a `21_obsessed` player who always tries for exactly 21. It's a fun way to test design hypotheses, and to catch rules regressions later.
+
+You don't need to write `personas.yml` yourself. Describing the styles in plain words to the agent is enough.
+
+## Step 5: Ask the agent
+
+The app comes with instructions for agents. Its `CLAUDE.md` pulls in PlAI Testr's [agent guide](AGENTS.md), which explains how the harness works, the order to build it in, and to ask you before guessing at a rule. Claude Code reads `CLAUDE.md` automatically when started in the app folder. With another agent, tell it to read `AGENTS.md` and `.plai_testr/gem/docs/AGENTS.md` first.
+
+Start the agent in the app folder (for example, run `claude`) and give it a prompt like this:
+
+```
+I'm building a PlAI Testr harness for my game, blackjack. I ran `plai_testr generate game blackjack`, wrote the rules into @./games/blackjack/data/rules.md, and put the components in @./games/blackjack/data/components/.
+
+Build out the rest of the harness so I can run botruns and model games:
+
+  1. Read the rules and components closely. Before writing any code, list
+     every gap, ambiguity or rule you'd have to guess at, and ask me about
+     them. Record my answers in a Rulings section in rules.md.
+  2. Fill in data/rules.json, then build the state, runner and bot following
+     the order in the PlAI Testr agent guide, with specs for the rules as you go.
+  3. Get `bundle exec plai_testr blackjack botrun --games 100` finishing
+     with no crashes and sensible endings, and `bundle exec rspec
+     games/blackjack/spec` passing.
+  4. Write the player views, debrief facts and debrief questions, and show
+     me one turn prompt and one debrief prompt to review.
+
+Don't change the rules to make the harness easier. Don't run any model games
+(play, preflight, resume, debrief) — stop and tell me when it's ready.
+```
+
+Add your personas to the prompt if you have them in mind ("Add three personas: …").
+
+Expect rules questions. Answering them is often useful in itself: it has turned up ambiguities in my own rules I hadn't noticed. The answers go in a *Rulings* section at the end of `rules.md`, so the model players see them too. A small game like Blackjack typically takes a few minutes; a large one can take much longer.
+
+## Step 6: Check the result
+
+When the agent says it's ready:
+
+1. **Run the checks.** `bundle exec rspec games/blackjack/spec` should pass. These include the standard checks every game gets: games reach an ending, saved games resume exactly, and no player can see another's hand.
+2. **Run a botrun** and read the report:
+
+   ```sh
+   bundle exec plai_testr blackjack botrun --games 100
+   ```
+
+   Do the results make sense? A cautious persona that busts more often than the gambler, or games that all end the same unusual way, point to a bug. Describe what looks wrong to the agent.
+3. **Read the prompts** the agent showed you. Is everything a player needs to decide there? Is anything there that the player shouldn't know, such as the dealer's face-down card? Does any option tell the model what to pick? Options should state facts ("Hit (your total: 14)"), never advice ("Hit (recommended)").
+4. **Run a model game** when you're happy:
+
+   ```sh
+   bundle exec plai_testr blackjack preflight
+   bundle exec plai_testr blackjack play
+   ```
+
+See [Running playtests](running-playtests.md) for the options and for reading reports.
+
+## Asking for more
+
+The agent guide stays with your app, so you can come back to the agent at any time. Some requests that work well:
+
+- *Write bot rules for blackjack so the gambler persona plays differently, run a 1000-game botrun, and summarize the report.*
+- *Read the newest report in games/blackjack/reports and tell me what it says about balance. Separate what might be the bot's habits from what might be the game.*
+- *The model players in blackjack keep making the same bad choice. Show me the prompt they see and suggest how to make the menu clearer, without telling them what to pick.*
+- *I've ruled that a tie goes to the dealer. Update the engine, add a spec, and record the ruling in data/rules.md.*
+- *The debrief answers mention things that didn't happen. Add debrief facts for what the questions ask about.*
+- *I've changed the rules in data/rules.md to version 0.4. Update the harness to match and tell me what changed.*
+
+The agent guide tells the agent to ask before running model games, which can take a long time. Specs and botruns are fast, so it runs those freely.

data/docs/running-playtests.md

@@ -0,0 +1,193 @@
+# Running playtests
+
+This guide covers running PlAI Testr from the command line: every command and its options, settings, where results go, and how to work through a game efficiently. It assumes your app is set up ([Getting started](getting-started.md)) and you're comfortable with command-line options and a little shell scripting.
+
+Run every command from the app folder, with `bundle exec`.
+
+## Commands
+
+```
+plai_testr <game> play      [--seed N] [--model M] [--think] [--mode llm|hybrid|bot] [--players N] [--personas A,B]
+                            [--referee M] [--debrief-model M] [--debrief-think] [--skip-preflight] [--quiet | --verbose]
+plai_testr <game> botrun    [--games N | --seeds 1,2,3] [--players N] [--personas all|A,B] [--quiet | --verbose]
+plai_testr <game> resume    <session_id> [--skip-preflight] [--quiet | --verbose]
+plai_testr <game> debrief   <session_id> [--model M] [--think] [--skip-preflight] [--verbose]
+plai_testr <game> preflight [--seed N] [--model M] [--think] [--referee M] [--players N] [--personas A,B]
+plai_testr <game> compare   [--since ID]
+plai_testr <game> list
+plai_testr new <dir>
+plai_testr generate game <name>
+plai_testr games
+plai_testr help [<command>]
+plai_testr --version
+```
+
+Every command takes `--help` to list its options (`plai_testr hearts botrun --help`, or `plai_testr help botrun`). A command refuses options it doesn't take.
+
+```
+$ bundle exec plai_testr hearts botrun --help
+Usage: plai_testr hearts botrun [options]
+
+Play many games with bot rules only (no model), then report
+
+Options:
+        --games N                    Number of games, seeds 1..N (default 100)
+        --seeds LIST                 Comma-separated seeds, instead of --games
+        --players N                  Number of players, within the game's player_count (default: the game's default)
+        --personas LIST              One persona per player, comma-separated; * for random, "all" to cycle every arrangement (default: random)
+        --quiet                      Show only a progress line and the result
+        --verbose                    Show every game
+```
+
+### botrun
+
+Plays many games with bots only, no model, and writes one report:
+
+```sh
+bundle exec plai_testr hearts botrun --games 1000
+bundle exec plai_testr hearts botrun --seeds 4,17,52
+bundle exec plai_testr hearts botrun --games 500 --personas all
+```
+
+`--games N` plays seeds 1 to N. Nothing is saved but the report, `games/<game>/reports/botrun-<time>.md`:
+
+- how games ended, with the share of each ending
+- wins by persona and by seat, and persona matchups, each with a 95% range
+- game length, and how often each bot rule was used
+- warnings, crashes, and seeds worth replaying
+
+A *95% range* is how far the true win rate could plausibly be from this sample. Two ranges that overlap are not a real difference; play more games to narrow them.
+
+Bot results describe how the bots play, not how people play. Confirm a surprising result with a few model games before changing your rules.
+
+### play
+
+Plays one game with model players:
+
+```sh
+bundle exec plai_testr hearts play
+bundle exec plai_testr hearts play --seed 12 --model qwen3:30b --think
+```
+
+Before playing, `play` runs preflight: it checks Ollama, downloads missing models, warms them up, checks the first prompt fits the model's context, and plays 5 bot games to catch bugs. `--skip-preflight` skips it.
+
+| Option | Effect |
+|---|---|
+| `--seed N` | The seed for shuffles and dice. Without it, a random seed from 1 to 100 is picked and printed. |
+| `--model M` | The Ollama model for the players (default `qwen3:8b`). |
+| `--think` | The model reasons before each choice. Stronger play, several times slower. |
+| `--mode hybrid` | The bot plays moves its rules mark *obvious*; the model plays the rest. Faster. |
+| `--mode bot` | No model at all; one bot game saved as a session. |
+| `--players N` | Number of players, within the game's player count. |
+| `--personas A,B` | One persona per player, in seat order; `*` for a random one (`--personas moon_shooter,*,*,*`). Default: random. |
+| `--referee M` | A second model for yes/no rulings, for games that ask for them. |
+| `--debrief-model M`, `--debrief-think` | A different model, or thinking, for the end-of-game debrief only. |
+
+When the game ends, each player answers the game's debrief questions, and the report goes in `games/<game>/reports/<session_id>.md`:
+
+- the result, model reliability (retries, invalid answers)
+- how often the model agreed with the bot's obvious moves
+- speed and wall time
+- the debrief answers, and every decision with the model's reasoning
+
+### resume
+
+Ctrl+C (or `kill`, which sends SIGTERM) stops a game after the current move and saves it. Continue it with the session id it prints:
+
+```sh
+bundle exec plai_testr hearts resume 20261024-143012-a1b2
+```
+
+A resumed game plays exactly as it would have without the interruption. If the game's data, code or the PlAI Testr version changed since the game started, `resume` refuses, because it could no longer replay exactly; start a new game with `play`. Ctrl+C twice quits at once, losing only the move in progress.
+
+### debrief
+
+Asks a finished game's players the debrief questions again and rebuilds its report:
+
+```sh
+bundle exec plai_testr hearts debrief 20261024-143012-a1b2 --model qwen3:30b --think
+```
+
+By default it uses the game's player model, without thinking. Use it to try a bigger model, thinking, or changed debrief questions on the same game without replaying it. Earlier answers are kept in the saved game.
+
+### preflight, compare, list
+
+- `preflight` runs `play`'s checks without playing.
+- `compare` writes a table of every finished model game, grouped by model and host: games, speed, reliability, endings and wins by persona. `--since 20261020` limits it to sessions from that date on (session ids start with the date and time).
+- `list` shows every saved session with its status.
+
+## Settings
+
+`config/plai_testr.rb` holds settings for the whole app. All are commented out until you set them:
+
+```ruby
+PlaiTestr.configure do |config|
+  config.max_decisions  = 100_000       # stops any game that runs this long
+  config.player_model   = 'qwen3:8b'
+  config.ollama_host    = 'localhost'
+  config.ollama_port    = 11434
+  config.model_settings = { context_size: 8192 }
+end
+```
+
+Environment variables set the same defaults, handy for one-off runs: `PLAI_TESTR_PLAYER_MODEL`, `PLAI_TESTR_REFEREE_MODEL`, `PLAI_TESTR_OLLAMA_HOST`, `PLAI_TESTR_OLLAMA_PORT`.
+
+```sh
+PLAI_TESTR_OLLAMA_HOST=studio.local bundle exec plai_testr hearts play
+```
+
+To use Ollama on another computer, set `ollama_host` to its name or address, and start Ollama there with `OLLAMA_HOST=0.0.0.0` so it accepts connections from the network.
+
+### Model settings
+
+| Setting | Default | Used for |
+|---|---|---|
+| `context_size` | 8192 | How much text the model reads per turn (without `--think`); the referee |
+| `long_context_size` | 16384 | Games with `--think`; the debrief |
+| `menu_budget` | 512 | The longest answer allowed for a menu choice |
+| `think_budget` | 8192 | The longest answer allowed for a thinking turn |
+| `debrief_budget` | 1024 | The longest answer allowed for the debrief |
+| `referee_budget` | 2048 | The longest answer allowed for a ruling |
+| `temperature` | 0.6 | How varied the players' choices are |
+
+Sizes and budgets are in tokens (roughly ¾ of a word each). Values outside sensible limits are adjusted with a warning, and the values used are listed in each report. A game can set its own in its registration.
+
+## Output files
+
+Each game folder holds:
+
+- `reports/<session_id>.md`: a model game's report
+- `reports/botrun-<time>.md`, `reports/comparison-<time>.md`: botrun and compare reports
+- `sessions/<session_id>/state.json`: the saved game, replaced after every decision
+- `sessions/<session_id>/transcript.jsonl`: one JSON object per decision, with the model's reasoning. After a crash the last line may be cut off.
+- `sessions/<session_id>/think.log`: every model reply, with Ollama's timings
+
+Both folders are ignored by git. Delete old sessions freely; reports don't depend on them, except that `resume` and `debrief` need theirs.
+
+## Working effectively
+
+A sequence that works well, from fast and cheap to slow and detailed:
+
+1. **Botrun first.** `botrun --games 1000 --personas all` gives balance numbers in seconds to minutes, and finds crashes and games that never end. Rerun it after every rules change.
+2. **Replay what looks odd.** The report lists seeds worth a look. `play --seed N --mode bot` replays one as a saved session, with every decision in its report.
+3. **Hybrid model games.** `play --mode hybrid` checks a surprising bot result with model players, at a fraction of the time of a full model game.
+4. **Full model games**, a few seeds at a time, for reasoning and debrief feedback. Use the same seeds when comparing models or rules versions, so the deals match.
+5. **Compare.** `compare` puts models and hosts side by side.
+6. **Re-debrief** instead of replaying, when you only want better answers about a game.
+
+Larger models and `--think` play better but take far longer: try a small model without thinking first.
+
+### Scripting
+
+To play several seeds in a row:
+
+```sh
+for seed in 1 2 3; do
+  bundle exec plai_testr hearts play --seed "$seed" --quiet || break
+done
+```
+
+- `--quiet` prints only a progress line and the result; `--verbose` adds the raw model output (`play`) or every game (`botrun`).
+- An interrupted `play` exits with status 130, so `|| break` stops the loop on Ctrl+C.
+- Errors print one line. `PLAI_TESTR_DEBUG=1` shows the full backtrace.
+- A botrun prints its report when it finishes, so it can be piped: `bundle exec plai_testr hearts botrun --games 1000 | less`.

data/docs/technical/README.md

@@ -0,0 +1,15 @@
+# Technical reference
+
+How PlAI Testr works, for anyone writing or reviewing harness code. [Ingesting a game manually](../ingesting-a-game-manually.md) gives the order to build a game in; these pages cover each part in depth.
+
+- [The runner](runner.md): the decision loop, every hook, moves and actions, helpers, randomness and limits, checking a prompt
+- [State](state.md): game and player state, hidden information, checkpoints and resume
+- [Rules and components](rules-and-components.md): `rules.json`, `rules.md`, and component files
+- [Personas and bots](personas-and-bots.md): `personas.yml`, `bots.yml`, `bot_choice`, and the bot's part in model games
+- [Debrief](debrief.md): how the debrief prompt is built, and writing good facts and questions
+- [Reports](reports.md): adding sections to session and botrun reports
+- [Specs](specs.md): the standard checks every game runs, and writing rules specs
+- [Nested decisions with FrameQueue](frame-queue.md): attacks and responses, triggered effects, decisions inside decisions
+- [Yes/no rulings with the referee](referee.md): rulings on free-text conditions by a second model
+
+The source is the final word: hooks and their defaults are in [`lib/plai_testr/session/runner.rb`](../../lib/plai_testr/session/runner.rb), and the example games in [`templates/examples/`](../../templates/examples) use most of what's described here.

data/docs/technical/debrief.md

@@ -0,0 +1,55 @@
+# Debrief
+
+After a model game, each player answers the game's `debrief_questions`, and the answers go in the report. Small local models can't reliably recall a long game from its log, and they invent what they can't recall, so the gem gives them a summary of the facts first.
+
+## The prompt
+
+The gem builds the debrief prompt in a fixed order:
+
+1. **`debrief_summary(player)`**, built by the gem (don't override it):
+   - **Outcome**: winner, end reason, final scores
+   - **Game facts**: `debrief_facts(player)`, shown as YAML
+   - **Your decisions**: how many the model made, how often it chose the bot's obvious move, fallbacks
+   - **Your plan as it changed**: the player's own `plan` each time it changed, with the decision and step
+2. **`debrief_view_for(player)`**: the final position. Default: the result, final scores, `view_text_for` and the player's visible log.
+
+The model is told to treat the summary as correct, to cite the round of any event it mentions, and to say when something isn't in the facts rather than guess.
+
+The debrief uses the player model without thinking, unless `play` is given `--debrief-model` or `--debrief-think`.
+
+## `debrief_facts`
+
+Returns a Hash with plain-language keys:
+
+```ruby
+def debrief_facts(_player)
+  {
+    'how_it_ended' => "Player 3 reached 100 points after 9 hands",
+    'players' => players.to_h do |p|
+      [p, { 'final_score' => pstate(p).score, 'hearts_taken' => data.dig('tally', p, 'hearts'), 'moons_shot' => data.dig('tally', p, 'moons') }]
+    end
+  }
+end
+```
+
+- **Count what the questions ask about.** Per player: final score and points by source, counts of key actions, races or majorities and who won them; and who ended the game, when and how.
+- **Facts, not conclusions.** "Player 3 had 4 Candy Corn, the most" is a fact; "Candy Corn was the strongest card" answers the question for the model.
+- **Compute only from saved state** (`data`, player state). `debrief <session_id>` rebuilds the facts from the checkpoint, possibly after the code changed, so instance variables are gone. Keep the counters the facts need in `data` as the game is played; `tally(player, key)` is made for this.
+
+Without `debrief_facts`, the facts are `botrun_stats`. `check!`, run by every game command, warns when a game defines neither.
+
+## `debrief_questions`
+
+Ask about the player's own experience, pointed at something the facts or its own plans can answer:
+
+- *Which of your decisions mattered most, and why? Name the round.*
+- *Which of your plays scored less than you expected? What happened instead?*
+- *Which rule was unclear or confusing when you had to use it? Name the situation.*
+
+Avoid open questions with nothing behind them ("what felt pointless?"): models fill them with invented events. `debrief_questions(player)` takes the player, so questions can differ by persona.
+
+## Checking and re-asking
+
+- Preflight builds every player's debrief prompt after its lint games, and the [standard checks](specs.md) build them too, so a broken hook shows up before a model game.
+- `debrief <session_id> [--model M] [--think]` re-asks a finished game's players and rebuilds the report, keeping earlier answers under `debrief_history` in the checkpoint. Use it to try better facts or questions, or a bigger model, on the same game without replaying it.
+- Debrief answers are model opinions. Check a claim against the facts or the turn log before acting on it.

data/docs/technical/personas-and-bots.md

@@ -0,0 +1,67 @@
+# Personas and bots
+
+A **persona** is a play style. For model players it is strategy text in the system prompt; for bots, a set of parameters the bot rules read. Each player gets one persona per game, so a botrun or a set of model games can compare styles. A **bot** is the game's scripted decision-maker, `bot_choice`.
+
+Both are optional: a game without personas plays everyone the same way, and a game without a bot can't botrun (and model games fall back to random moves when a model never answers validly).
+
+## `data/personas.yml`
+
+```yaml
+basic_strategy: &basics
+  pass_dangerous: Pass away the Q♠, high spades (A♠, K♠) and high hearts unless you have many low spades to protect them.
+  duck: Play just under the winning card to avoid taking tricks that carry points.
+
+personas:
+  safe_player:
+    <<: *basics
+    goal: Take as few points as you can in every hand. Never win a trick you could lose.
+  moon_shooter:
+    <<: *basics
+    goal: Watch for a hand strong enough to take every heart and the Q♠. Go for it when you can; otherwise play safe.
+    pass_dangerous: Keep high hearts and the Q♠ when your hand is strong enough to take every trick.
+```
+
+- `basic_strategy` is advice every player gets. `<<: *basics` merges it into each persona.
+- Each key under `personas` is a persona's name, used with `--personas`. Every field under it is shown to the model. Reusing a `basic_strategy` key overrides that item; a new key adds one.
+- The gem reads only `basic_strategy` and `personas`. Other top-level keys are yours, for a custom `persona_for`.
+- The default `persona_for(player)` builds the system prompt's strategy text from this file; `display_name` shows the persona's name, title-cased, in the console and reports.
+
+Personas are assigned per game: `--personas a,b` in seat order (`*` for random), otherwise at random from the seed. `botrun --personas all` cycles every arrangement of personas across seats, and needs at least as many personas as seats.
+
+Personas shape debrief answers as well as play: a player told to favor a card will praise it. Compare debriefs across personas before concluding anything.
+
+## `data/bots.yml`
+
+```yaml
+bot_strategy: &bot_defaults
+  pass_dangerous: true
+  win_clean_tricks: true
+
+bots:
+  safe_player:
+    <<: *bot_defaults
+    win_clean_tricks: false
+```
+
+`bot_strategy` holds the defaults; `bots` holds per-persona overrides, keyed by the persona names in `personas.yml` (an unknown name is an error). The bot reads them with `bot_strategy_for(player)`. The model never sees this file.
+
+## `bot_choice`
+
+`bot_choice(player, moves, pick: nil)` lives in `lib/<name>/bot.rb`, a module included in the runner. It gets the checked menu and returns `nil` (no rule applies) or:
+
+```ruby
+{ 'choice' => 2, 'rule' => 'duck', 'obvious' => false }        # choice is 1-based
+{ 'choices' => [1, 4, 5], 'rule' => 'pass_dangerous', 'obvious' => false }   # pick turns
+```
+
+- **`rule`** names the idea behind the move (`duck`, `dump_queen`, `shed_deadwood`). Botrun reports count how often each rule was used; `none` means the bot had no rule and took option 1.
+- **`obvious`** marks a move that is never worse than any alternative. `--mode hybrid` plays obvious moves without asking the model, and every model game's report scores the model against them (*Bot agreement*). Mark a move obvious only when it really is; a wrong mark skews both.
+- Find options by pattern-matching their actions, and return a 1-based choice: `i = moves.index { |m| m[:action] in { play: { value: ^uid } } }` then `{ 'choice' => i + 1, ... }`.
+
+Keep the bot simple and readable. It is a baseline for balance numbers and a crash test, not a strong player. Bot results describe the bot's strategy; confirm a surprising one with a few model games.
+
+### Where the bot is used
+
+- **Botruns and `--mode bot`**: the bot makes every decision.
+- **`--mode hybrid`**: the bot makes its obvious decisions; the model makes the rest.
+- **Model games**: the turn log records the bot's choice next to the model's, for *Bot agreement*. When the model never answers validly, the bot's choice is played.

data/docs/technical/reports.md

@@ -0,0 +1,57 @@
+# Reports
+
+Every model game and botrun ends in a markdown report. The gem's sections cover what every game has; a game adds its own for what matters in that game.
+
+## Session reports
+
+Each model game's report (`reports/<session_id>.md`) has a header (session, rules version, players and personas, model and settings, seed, decisions, wall time), then the result, model reliability, bot agreement, speed, warnings, the debrief, and the turn log.
+
+`report_sections` adds sections after the gem's own, as an array of markdown strings:
+
+```ruby
+def report_sections
+  rows = players.map { |p| "| #{p} | #{pstate(p).score} | #{data.dig('tally', p, 'hearts')} |" }
+  ["## Hearts\n\n| Player | Score | Hearts taken |\n|---|---|---|\n#{rows.join("\n")}"]
+end
+```
+
+Tables are aligned automatically. Return `false` from `report_debrief?` to leave out the gem's debrief section and render the answers yourself in `report_sections` (from `data['debrief']`).
+
+Reports are rebuilt from saved state (by `resume` and `debrief`), so compute sections from `data`, not instance variables.
+
+## Botrun reports
+
+A botrun keeps a small result for each game: seed, winner, personas, end reason, decisions, bot rules used, warnings. The report summarizes them: endings, wins by persona and seat, matchups, game length, bot rules, and seeds worth replaying.
+
+To add game-specific numbers:
+
+1. Return them from `botrun_stats` in the runner, as JSON-safe values. It is called once at the end of each game.
+
+   ```ruby
+   def botrun_stats = { 'hands' => data['hand'], 'taken' => totals }
+   ```
+
+2. Subclass `PlaiTestr::Reporter::BotrunReport` and return extra sections from `extra_sections`. Each game's numbers are in `@played` (games that didn't crash), under `'stats'`. `table(title, headers, rows, note)`, `pct` and `percentile` are available.
+
+   ```ruby
+   module Hearts
+     module Reporter
+       class BotrunReport < PlaiTestr::Reporter::BotrunReport
+         private
+
+         def extra_sections
+           counts = @played.filter_map { |r| r.dig('stats', 'hands') }.sort
+           [table('## Hands per game', %w[Mean Max], ["| #{counts.sum.fdiv(counts.length).round(1)} | #{counts.max} |"])]
+         end
+       end
+     end
+   end
+   ```
+
+3. Register it: `PlaiTestr.register('hearts', ..., botrun_report_class: Hearts::Reporter::BotrunReport)`.
+
+`botrun_stats` doubles as the debrief facts when a game has no `debrief_facts`.
+
+## Model comparison
+
+`compare` reads every finished session and writes one row per model and host. To add columns, subclass `PlaiTestr::Reporter::ModelComparison`, override `extra_columns` (a list of `[header, ->(sessions) { value }]` pairs; each session is a hash with `:state`, its saved game state, plus `:turns` and `:seed`) and `notes` (footnotes), and register it with `comparison_class:`.

data/docs/technical/rules-and-components.md

@@ -0,0 +1,61 @@
+# Rules and components
+
+A game's `data/` folder describes the game: what the model players read, the numbers the engine uses, and the components.
+
+## `data/rules.md`
+
+The rules text the model players read every turn (the default `rules_text_for`). Write it for a player, not a programmer.
+
+- End it with a **Rulings** section: rules questions settled with the designer, dated, so models and future readers see them.
+- Add a **Simplifications** section for any rule the harness deliberately simplifies (see `games/gin_rummy`).
+- Keep open questions and engine choices the designer hasn't confirmed out of `rules.md`, in `games/<name>/NOTES.md`, until they're settled.
+- If the rules are long, override `rules_text_for` to send only the sections the current step needs, calling `super` for the file.
+
+## `data/rules.json`
+
+```json
+{
+  "game": {
+    "name": "Hearts",
+    "version": "1.0.0",
+    "description": "A 4-player trick-taking card game. ...",
+    "player_count": { "min": 4, "max": 4, "default": 4 }
+  },
+  "points_to_end": 100
+}
+```
+
+- `game.name` and `game.version` are required. The version appears in every report, so bump it when the rules change.
+- `game.description` is what the model players read first.
+- `game.player_count` sets the allowed `--players` range and the default. The [standard checks](specs.md) play every count in the range.
+- Anything else is the game's: points to win, hand sizes, limits. The runner reads them from `@rules` (`@rules['points_to_end']`), so tuning a number needs no code change.
+
+The file is validated when the game loads, and errors name the field.
+
+## Components
+
+Card lists, tiles and other components go in `data/components/`, one file per component, named after the file:
+
+```ruby
+PlaiTestr::Deck.build(@rules, component: 'cards')   # data/components/cards.csv (or .yml, .json)
+PlaiTestr::Deck.build(@rules)                       # the first component
+```
+
+`Deck.build` returns a list of hashes with string keys, one per copy, each with its own `uid` (1, 2, 3, … in file order). Shuffle it with `@rng`: `Deck.build(@rules).shuffle(random: @rng)`.
+
+A component file is a list of records:
+
+- **CSV**: a header row, then one row per entry. Numbers become numbers; empty cells are left out.
+- **YAML or JSON**: a list of hashes.
+
+A `copies` field makes that many copies. A malformed file fails to load with the file and entry named. Both example games read `data/components/cards.csv`.
+
+A component can also be inline under `components` in `rules.json` (`{ "name": "deck", "type": "card_deck", "values": [...] }`), but files are easier to edit and review.
+
+### Converting a designer's files
+
+PlAI Testr reads only this format. Converting a designer's own files (spreadsheets, YAML with metadata, exports from card tools) is the game's job:
+
+- Keep the fields the engine or the players need; leave out print layout, art notes and other metadata.
+- If the source keeps changing, write a small rerunnable script in the game folder (`games/<name>/import.rb`) that rebuilds the file, and a spec that fails when the copy is behind.
+- Check the counts (cards per type, total) after converting.