ftl-bench 0.1.0__tar.gz

ftl_bench-0.1.0/.gitignore

@@ -0,0 +1,31 @@
+# Python
+__pycache__/
+*.py[cod]
+.venv/
+venv/
+*.egg-info/
+.pytest_cache/
+.coverage
+htmlcov/
+
+# Env / secrets
+.env
+*.local
+
+# Runtime artifacts
+runs/
+logs/
+*.log
+trajectories/
+state.json
+action.json
+
+# OS
+.DS_Store
+
+# Build
+build/
+dist/
+# vendored into the package at build time by harness/hatch_build.py (not a source of truth)
+src/ftl_bench/_bundled/
+harness/src/ftl_bench/_bundled/

ftl_bench-0.1.0/PKG-INFO

@@ -0,0 +1,66 @@
+Metadata-Version: 2.4
+Name: ftl-bench
+Version: 0.1.0
+Summary: A turn-based, intent-level environment + scenario suite for evaluating LLM agents on FTL: Faster Than Light
+Project-URL: Homepage, https://github.com/ogabrielluiz/ftl_bench
+Project-URL: Repository, https://github.com/ogabrielluiz/ftl_bench
+Author: ogabrielluiz
+License-Expression: MIT
+Keywords: agents,benchmark,evaluation,ftl,game-playing,llm
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: MacOS
+Classifier: Operating System :: Microsoft :: Windows
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.11
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == 'dev'
+Provides-Extra: screenshot
+Requires-Dist: pyobjc-framework-quartz>=10.0; (sys_platform == 'darwin') and extra == 'screenshot'
+Description-Content-Type: text/markdown
+
+# ftl-bench
+
+An agent-evaluation benchmark that has an LLM agent play **FTL: Faster Than Light** through a
+turn-based, intent-level interface built on the FTL-Hyperspace Lua API. The agent reads a
+decision-complete JSON observation and replies with one command; the harness scores how far it
+gets on a suite of reproducible, seed-pinned scenarios.
+
+This package ships the Python harness, the scenario suite, the agents, and the `ftlbench`
+command line. Driving the real game additionally needs FTL installed (via Steam) plus the bench
+Hyperspace mod.
+
+## Install
+
+```bash
+pip install ftl-bench
+```
+
+## Use
+
+```bash
+ftlbench run --agent scripted              # run the scenario suite with the scripted baseline
+ftlbench run --agent random --tier public  # the legal-move floor on the public tier
+ftlbench run --agent llm --backend anthropic --model claude-sonnet-4-6   # a model plays the suite
+ftlbench play obs                          # print the live observation the agent sees
+ftlbench install-mod --url <release-asset> # install the prebuilt bench Hyperspace mod into FTL
+ftlbench version
+```
+
+`ftlbench run --help` and `ftlbench play` show the full options. Results and a reproducibility
+manifest are written under `runs/benchmark/`.
+
+## Platforms
+
+The harness runs on native Windows, WSL, or macOS and launches FTL for you (via Steam on
+Windows). It reads/writes the FTL user folder, resolved per OS or overridden with `FTL_SAVE_DIR`.
+
+## More
+
+Full design, architecture, and the in-game bridge live in the project repository:
+<https://github.com/ogabrielluiz/ftl_bench>.

ftl_bench-0.1.0/README.md

@@ -0,0 +1,40 @@
+# ftl-bench
+
+An agent-evaluation benchmark that has an LLM agent play **FTL: Faster Than Light** through a
+turn-based, intent-level interface built on the FTL-Hyperspace Lua API. The agent reads a
+decision-complete JSON observation and replies with one command; the harness scores how far it
+gets on a suite of reproducible, seed-pinned scenarios.
+
+This package ships the Python harness, the scenario suite, the agents, and the `ftlbench`
+command line. Driving the real game additionally needs FTL installed (via Steam) plus the bench
+Hyperspace mod.
+
+## Install
+
+```bash
+pip install ftl-bench
+```
+
+## Use
+
+```bash
+ftlbench run --agent scripted              # run the scenario suite with the scripted baseline
+ftlbench run --agent random --tier public  # the legal-move floor on the public tier
+ftlbench run --agent llm --backend anthropic --model claude-sonnet-4-6   # a model plays the suite
+ftlbench play obs                          # print the live observation the agent sees
+ftlbench install-mod --url <release-asset> # install the prebuilt bench Hyperspace mod into FTL
+ftlbench version
+```
+
+`ftlbench run --help` and `ftlbench play` show the full options. Results and a reproducibility
+manifest are written under `runs/benchmark/`.
+
+## Platforms
+
+The harness runs on native Windows, WSL, or macOS and launches FTL for you (via Steam on
+Windows). It reads/writes the FTL user folder, resolved per OS or overridden with `FTL_SAVE_DIR`.
+
+## More
+
+Full design, architecture, and the in-game bridge live in the project repository:
+<https://github.com/ogabrielluiz/ftl_bench>.

ftl_bench-0.1.0/hatch_build.py

@@ -0,0 +1,54 @@
+"""Hatchling build hook: vendor the repo's sibling data dirs INTO the package tree so that the
+wheel AND the sdist are both self-contained.
+
+The runnable benchmark needs files that live in repo top-level dirs (adapter/, scenarios/,
+prompts/, mod/, scripts/) and the README, all OUTSIDE this package root (harness/). A wheel-only
+`force-include "../adapter"` can reach them when building from the repo, but the sdist cannot
+(paths above the project root are not shipped, and the sdist->wheel rebuild then fails trying to
+reach a `../adapter` sibling that no longer exists). So instead we copy them under
+`src/ftl_bench/_bundled/` at build time and ship that as package data (declared via `artifacts`),
+which both targets include cleanly without reaching above the project root.
+
+Idempotent: when building from an unpacked sdist the siblings are absent but `_bundled/` was
+already shipped, so we keep it as-is.
+"""
+from __future__ import annotations
+
+import shutil
+from pathlib import Path
+
+from hatchling.builders.hooks.plugin.interface import BuildHookInterface
+
+# repo top-level entries to vendor under src/ftl_bench/_bundled/ (preserving names so each
+# module's own `REPO = __file__.parent.parent` path logic keeps resolving them).
+_VENDOR = ["adapter", "scenarios", "prompts", "mod", "scripts", "README.md", "LICENSE"]
+_IGNORE = shutil.ignore_patterns(
+    "__pycache__", "*.pyc", "runs", "dist", "build", ".venv", "node_modules", "*.log"
+)
+
+
+class CustomBuildHook(BuildHookInterface):
+    PLUGIN_NAME = "custom"
+
+    def initialize(self, version, build_data):
+        root = Path(self.root)                  # project root = harness/
+        repo = root.parent                      # repo root: the siblings live here
+        bundled = root / "src" / "ftl_bench" / "_bundled"
+
+        if (repo / "adapter").is_dir():
+            # building from the repo working tree: refresh a clean vendored copy
+            if bundled.exists():
+                shutil.rmtree(bundled)
+            bundled.mkdir(parents=True)
+            for name in _VENDOR:
+                src, dst = repo / name, bundled / name
+                if src.is_dir():
+                    shutil.copytree(src, dst, ignore=_IGNORE)
+                elif src.is_file():
+                    shutil.copy2(src, dst)
+        elif not (bundled / "adapter" / "run_benchmark.py").exists():
+            # not in the repo and nothing was pre-vendored: cannot produce a runnable package
+            raise RuntimeError(
+                "ftl_bench build: neither the repo data dirs nor a pre-vendored "
+                "src/ftl_bench/_bundled/ are present — build from the repo or a complete sdist."
+            )

ftl_bench-0.1.0/pyproject.toml

@@ -0,0 +1,60 @@
+[project]
+name = "ftl-bench"
+version = "0.1.0"
+description = "A turn-based, intent-level environment + scenario suite for evaluating LLM agents on FTL: Faster Than Light"
+readme = "README.md"
+license = "MIT"
+requires-python = ">=3.11"
+authors = [{ name = "ogabrielluiz" }]
+keywords = ["benchmark", "agents", "llm", "evaluation", "ftl", "game-playing"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Science/Research",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Operating System :: Microsoft :: Windows",
+    "Operating System :: MacOS",
+    "Operating System :: POSIX :: Linux",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+]
+dependencies = []
+
+[project.urls]
+Homepage = "https://github.com/ogabrielluiz/ftl_bench"
+Repository = "https://github.com/ogabrielluiz/ftl_bench"
+
+[project.scripts]
+ftlbench = "ftl_bench.cli:main"
+
+[project.optional-dependencies]
+dev = ["pytest>=8.0"]
+# Occlusion-proof FTL window capture for the `screenshot` action (macOS). Without it the
+# capture falls back to an AppleScript-bounds region grab (may catch an overlapping window).
+screenshot = ["pyobjc-framework-Quartz>=10.0; sys_platform=='darwin'"]
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+# A build hook vendors the repo's sibling data dirs (adapter/, scenarios/, prompts/, mod/,
+# scripts/, README) into src/ftl_bench/_bundled/ so BOTH the wheel and the sdist are
+# self-contained. See harness/hatch_build.py for why force-include alone cannot do this.
+[tool.hatch.build.hooks.custom]
+path = "hatch_build.py"
+
+# Include the build-vendored data even though it is .gitignored.
+[tool.hatch.build]
+artifacts = ["src/ftl_bench/_bundled/**"]
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/ftl_bench"]
+exclude = ["**/__pycache__", "**/*.pyc", "**/runs/**"]
+
+[tool.hatch.build.targets.sdist]
+include = ["/src", "/tests", "/hatch_build.py", "/README.md", "/pyproject.toml"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+pythonpath = ["src"]

ftl_bench-0.1.0/src/ftl_bench/__init__.py

@@ -0,0 +1,68 @@
+from ftl_bench.observation import (
+    Observation,
+    ObservationClient,
+    ObservationValidationError,
+)
+from ftl_bench.scenario import Scenario, SubObjective, load_suite
+from ftl_bench.scoring import (
+    achieved_metrics,
+    score_instance,
+    score_observation,
+    score_trajectory,
+)
+from ftl_bench.session import (
+    AgentSession,
+    choose_event,
+    fire_weapon,
+    jump,
+    move_crew,
+    set_system_power,
+    start_game,
+    store_buy,
+    store_sell,
+    upgrade_system,
+    cloak,
+    set_doors,
+    mind_control,
+    battery,
+    fire_beam,
+    hack_system,
+    deploy_drone,
+    recall_drones,
+    teleport_crew,
+)
+from ftl_bench.trajectory import TrajectoryRecorder, load_trajectory
+
+__all__ = [
+    "Observation",
+    "ObservationClient",
+    "ObservationValidationError",
+    "AgentSession",
+    "set_system_power",
+    "move_crew",
+    "jump",
+    "choose_event",
+    "fire_weapon",
+    "start_game",
+    "store_buy",
+    "store_sell",
+    "upgrade_system",
+    "cloak",
+    "set_doors",
+    "mind_control",
+    "battery",
+    "fire_beam",
+    "hack_system",
+    "deploy_drone",
+    "recall_drones",
+    "teleport_crew",
+    "TrajectoryRecorder",
+    "load_trajectory",
+    "score_observation",
+    "score_trajectory",
+    "Scenario",
+    "SubObjective",
+    "load_suite",
+    "score_instance",
+    "achieved_metrics",
+]

ftl_bench-0.1.0/src/ftl_bench/_bundled/LICENSE

@@ -0,0 +1,33 @@
+MIT License
+
+Copyright (c) 2026 ftl_bench Contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+---
+
+License notes (informational; not part of the MIT grant above)
+
+This license covers the ftl_bench source code in this repository only. It does
+not grant rights to, and explicitly excludes:
+
+- FTL: Faster Than Light, a commercial game by Subset Games, including any of
+  its assets or data. You must own and install the game separately.
+- FTL-Hyperspace, the third-party modding framework this project builds on,
+  which is distributed under its own separate license and terms.

ftl_bench-0.1.0/src/ftl_bench/_bundled/README.md

@@ -0,0 +1,102 @@
+# ftl_bench
+
+An agent-evaluation benchmark that lets LLM coding agents **play FTL: Faster Than Light** through a clean, intent-level interface built on the [FTL-Hyperspace](https://github.com/FTL-Hyperspace/FTL-Hyperspace) Lua API.
+
+FTL is a real-time-with-pause roguelike: resource management, risk under uncertainty, combat micro, and long-horizon planning across a branching map. That makes it a rich substrate for measuring agent decision-making. `ftl_bench` wraps it into a reproducible, turn-based environment with structured observations, an intent-level action space, seed-pinned runs, and full trajectory logging.
+
+## The benchmark: a scenario suite scored by goal achievement
+
+`ftl_bench` evaluates agents on a suite of reproducible **scenario instances**, not on raw play.
+
+- **Instance** = a fully-specified, seeded scenario `(seed, ship, difficulty, goal)`. The seed pins the map + events; the goal is a set of weighted sub-objectives.
+- **The agent decides everything in-game** (fight, flee, target, power, repair, navigate). The harness scores **only goal achievement** — no decision policy is baked into the env or scoring.
+- **Goal-conditioned partial credit**: each instance earns `r ∈ [0,1]` = the weighted intersection of achieved vs. requested sub-objectives, × a legitimacy gate that collapses metric-gaming (e.g. jumping in place). `Score = 100·r`.
+- **Headline metric: FTL score** = the mean of FTL's own native run score (scrap, kills, sectors, flagship, times difficulty) over the suite (± seed SE), alongside a strict **Solve / Win Rate** and an **efficiency** axis (jumps/turns per instance).
+- **Anti-memorization split**: a `public` tier to tune against and a held-out `semi_private` tier that is the leaderboard number.
+- **Baseline ladder**: a `random`-legal floor and a `scripted` heuristic floor, so a high agent score is interpretable.
+
+**Scenario types:** `survive_n_jumps` (make N jumps alive), `reach_sector` (advance to sector K), `reach_sector_healthy` (reach K with hull + crew intact — a multi-attribute goal), `full_run` (milestone progress toward beating the flagship — the unsaturated ceiling). Higher-signal micro-encounters (`win_this_combat`, `escape_a_crisis`, `event_risk_choice`) and the flagship/store tiers are next.
+
+**Run it** (the harness runs on native Windows, WSL, or macOS, and drives FTL for you):
+```bash
+# One-time setup, per platform:
+#   Windows: install FTL via Steam + the bench Hyperspace mod (scripts/setup_pc.sh). The runner
+#            launches and restarts FTL through Steam itself, so no env vars are needed.
+#   macOS:   defaults write com.example.FTL NSAppSleepDisabled -bool YES   # keep it ticking unfocused
+#            scripts/restart_ftl.sh none                                    # launch FTL to the menu
+
+cd harness && uv run python ../adapter/run_benchmark.py --agent scripted   # scripted floor
+cd harness && uv run python ../adapter/run_benchmark.py --agent random      # random floor
+cd harness && uv run python ../adapter/run_benchmark.py --agent scripted --tier semi_private  # held-out leaderboard number
+# A real frontier model plays the suite (the LLM track), two backends:
+cd harness && uv run python ../adapter/run_benchmark.py --agent llm --backend anthropic --model claude-sonnet-4-6  # needs ANTHROPIC_API_KEY
+cd harness && uv run python ../adapter/run_benchmark.py --agent llm --backend claude-cli --model claude-opus-4-8   # no key: local `claude -p`
+```
+The **LLM track** (`adapter/llm_agent.py`) drives the model over the same intent-level surface the baselines use: each turn it gets the decision-complete observation + the scenario goal + a short action history and replies with one command, dispatched through the shared `apply_command()` in `play_cli.py`. It decides everything — no scripted policy. `--backend anthropic` is the canonical, portable track (Anthropic Messages API); `--backend claude-cli` shells out to a local `claude -p` so you can run it with no API key. The agent's rules/instructions are a **version-controlled operating manual** at `prompts/ftl_agent_<v>.md` (select with `--prompt-version`); the version is recorded in each run's manifest and agent label, so a manual change is a distinct, comparable agent — not a silent drift.
+Output: per-instance `ftl_score` + breakdown, then the aggregate `FTL score ± SE | Solve N/M` with per-type/tier breakdown. Each instance's trajectory + a reproducibility manifest (seed, ship, schema, runner/agent version) is saved under `runs/benchmark/`.
+
+**Native baseline (scripted heuristic floor, 12-instance v1 suite, native Windows + Steam, no WSL).** The headline metric is FTL's own native run score (mean over the suite, ± seed SE):
+
+| Agent | FTL score | Solve | survive_n_jumps | reach_sector | reach_sector_healthy | full_run |
+|---|---|---|---|---|---|---|
+| **scripted** (heuristic floor) | **143.75 ± 13.05** | 3/12 | 133.3 | 124.0 | 195.0 | 157.5 |
+
+Median 11 jumps per instance; public tier 142.9, held-out `semi_private` tier 145.0. The full suite runs end to end on native Windows with no crashes. A native `random` floor and a frontier-LLM row (`--agent llm`, above, scored identically over the same observe/act surface) are the next rows to fill. Earlier macOS/Rosetta numbers used a goal-conditioned 0-100 score (scripted 70.2, random 5.2) and are not comparable to FTL's native score here.
+
+## Why Hyperspace
+
+Hyperspace is an open-source C++ "exe mod" that exposes FTL's engine to **Lua via SWIG bindings**. It already lets scripts *read* full game state (`Hyperspace.ships.player`/`.enemy`, crew, systems, weapons, map) and *drive* much of the simulation (move crew, allocate power, teleport, toggle cloak). It also supports **seeded runs** with the seed readable from Lua, the basis for reproducibility. Where capabilities aren't yet bound — the harness **transport** (the Lua sandbox disables `io`/sockets), JSON serialization, and a few UI-driven actions (weapon room-targeting, event-choice confirm, jump trigger, store) — we extend Hyperspace itself with new SWIG bindings rather than resorting to brittle screen/input automation. The source-grounded map of what's exposed vs. what we build is in [`docs/deepdive/hyperspace-lua-surface.md`](docs/deepdive/hyperspace-lua-surface.md).
+
+## Architecture (four layers)
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│  Coding agent (LLM)                                          │
+│   observes JSON, returns intent-level actions                │
+└───────────────▲──────────────────────────┬──────────────────┘
+                │ tools (MCP / func-calling)│
+┌───────────────┴──────────────────────────▼──────────────────┐
+│  adapter/   — exposes env as agent tools                     │
+├──────────────────────────────────────────────────────────────┤
+│  harness/   — gym-like env server (reset/observe/step),      │
+│               episode + seed + scoring + trajectory logging  │
+└───────────────▲──────────────────────────┬──────────────────┘
+                │ transport (file / socket) │
+┌───────────────┴──────────────────────────▼──────────────────┐
+│  mod/ftl_bench_bridge  — Hyperspace Lua mod inside FTL:      │
+│   • per-frame hook gates the sim (event-driven pause)        │
+│   • serializes Observation JSON                              │
+│   • applies Action commands via the Lua API                 │
+│  (+ extended Hyperspace C++/SWIG bindings for action gaps)  │
+└──────────────────────────────────────────────────────────────┘
+```
+
+| Dir | Purpose |
+|-----|---------|
+| `mod/ftl_bench_bridge/` | In-game Hyperspace Lua mod: state serialization, action application, sim gating |
+| `harness/` | External environment server (Python): `reset()/observe()/step()`, episodes, seeds, scoring, logging |
+| `adapter/` | Exposes the env to a coding agent as MCP / function-calling tools |
+| `scenarios/` | Benchmark scenario definitions + pinned seeds (full runs and cheap micro-encounters) |
+| `docs/deepdive/` | Source-grounded analysis of the Hyperspace Lua surface |
+
+## Core idea: making a real-time game turn-based
+
+The harness keeps the game **paused by default** and unpauses in controlled increments. The default **event-driven gating** mode runs the sim until the next significant decision point (enemy weapon about to fire, system damaged, projectile incoming, event/store/jump screen) then re-pauses and requests an action — mirroring how a skilled human micro-pauses. A simpler **fixed-tick** mode is available for cheaper runs.
+
+## Platform notes
+
+On **native Windows**, FTL must be launched through Steam (`steam.exe -applaunch 212680`), which the runner does for you; a direct executable launch skips the Hyperspace injection and the bridge never loads. Windows Defender can briefly lock the observation/action files, so the harness retries those file operations. On **macOS**, keep FTL from being App-Napped so it keeps ticking when unfocused: `defaults write com.example.FTL NSAppSleepDisabled -bool YES`.
+
+## Documentation
+
+A documentation site covering the scoring model, the action set and observation schema, the per-platform install guides, and the architecture is in [`site/`](site/) (Astro Starlight). The Hyperspace Lua state/action surface is mapped in [`docs/deepdive/hyperspace-lua-surface.md`](docs/deepdive/hyperspace-lua-surface.md).
+
+## Known gaps
+
+- **Beam weapons** are not yet in the action set; they need two-point, room-to-room targeting.
+- The higher-signal micro-encounter scenarios and the flagship tier are planned but not yet in the suite.
+
+## Related
+
+- [FTL-Hyperspace](https://github.com/FTL-Hyperspace/FTL-Hyperspace) — the modding API this is built on
+- [FTLAV](https://github.com/Niels-NTG/FTLAV) — savefile parser (basis for the state fallback)

ftl_bench-0.1.0/src/ftl_bench/_bundled/adapter/README.md

@@ -0,0 +1,67 @@
+# adapter
+
+Exposes the `ftl_bench` environment to an LLM agent as **MCP tools**, so a
+tool-capable model can play FTL directly.
+
+`ftl_mcp_server.py` wraps `AgentSession` and serves these tools (each returns a
+compact, agent-readable summary of the resulting state):
+
+| Tool | What it does |
+|------|--------------|
+| `observe()` | Current state: context (menu/in_space/combat/event), your ship (hull, reactor, systems+power, crew, weapons+charge), enemy (hull, shields, rooms), jump beacons, event choices |
+| `reset(mode)` | Start a run — `'new'` (fresh seeded) or `'continue'` (resume the save) |
+| `do_jump(beacon_index)` | FTL-jump to a connected beacon |
+| `pick_choice(choice_index)` | Choose an event option |
+| `power_system(system_id, level)` | Set a system's power (0=shields 1=engines 3=weapons …) |
+| `send_crew(crew_id, room_id)` | Move a crew member to one of your rooms |
+| `shoot(weapon_slot, enemy_room_id)` | Aim+fire a weapon at an enemy room (auto-fires as it charges) |
+| `advance(frames)` | Let game time pass (charge weapons, finish a jump/combat) then re-pause |
+| `run_strategy(code)` | **Code mode**: run agent-authored Python against the env (loops, whole combats) in one call — fewer round-trips than per-action tool calls |
+
+## Prerequisites
+
+The game must be running with the bridge live and not napping in the background:
+
+```bash
+defaults write com.example.FTL NSAppSleepDisabled -bool YES   # one-time: keep ticking unfocused
+scripts/restart_ftl.sh continue                                # launch + start a run
+```
+
+## Run the server
+
+```bash
+cd harness && uv run --with "mcp[cli]" python ../adapter/ftl_mcp_server.py
+```
+
+To register with Claude Code (`.mcp.json` / `claude mcp add`), point the command at
+that line. The server drives a single live FTL instance.
+
+## Baseline agent
+
+`baseline_agent.py` is a scripted heuristic agent (no LLM) that plays a few jumps —
+powers shields/weapons, resolves events, and fights — useful as a smoke test and a
+scoring baseline:
+
+```bash
+cd harness && uv run python ../adapter/baseline_agent.py --jumps 5
+```
+
+## Eval harness
+
+`eval.py` runs N seeded episodes (fresh `reset_episode` between each — no FTL restart),
+records a trajectory per episode, and aggregates scores (survival rate, mean kills/hull/…):
+
+```bash
+cd harness && uv run python ../adapter/eval.py --seeds 1,2,3 --jumps 6
+```
+
+## Two ways an agent plays
+
+- **Per-tool MCP** — the model calls `observe`/`do_jump`/`shoot`/… step-by-step (good for
+  introspection). No built-in Claude Code "code mode" toggle exists for MCP; tools are normal calls.
+- **Code mode** — the model writes Python against this env and runs it. Either via `run_strategy`
+  (an MCP tool that execs agent code with `session` + action builders in scope) or, in a
+  code-execution agent (Claude Code), by writing+running a script that imports `ftl_bench` directly.
+  Recommended for FTL: one script can drive a whole combat without flooding context with
+  intermediate observations.
+