hippocamp 0.2.11 → 0.2.13

package/AGENTS.md

@@ -29,6 +29,13 @@ This repository is a local-first MCP package for Git-backed agent memory. Keep c
 - `scripts/eval-lagoon.cjs`: read-only retrieval check against real Lagoon cases in `<Lagoon root>/evals/retrieval-cases.json`.
 - `scripts/eval-qa.cjs`: LongMemEval-S answer-accuracy check (reader and grader models) for Hippocamp snippets against the labeled answer sessions.
 - `scripts/eval-retrieval.cjs`: retrieval-only LongMemEval-S check for `searchMemory` against a BM25 reference.
+- `scripts/eval-continuity.cjs`: work continuity benchmark runner and CLI: real headless Claude Code or Codex through successive sessions per arm (`cc-none`, `cc-native`, `cc-hippocamp`, `codex-native`, `codex-hippocamp`) in isolated sandboxes, with a frozen run manifest and `--report`.
+- `scripts/continuity-adapters.cjs`: one adapter per benchmark arm, each installed the way a developer has it (Hippocamp through its real `install-claude` or `install-codex` installer).
+- `scripts/continuity-judge.cjs`: blind task judge, miss attribution, and the memory-quality judge.
+- `scripts/continuity-report.cjs`: outcome, efficiency and memory-quality tables, pooled over runs, with a probe-level bootstrap.
+- `scripts/continuity-dataset.cjs`: loader and validator for the continuity dataset, including the retrieval-hint check on prompts.
+- `scripts/eval-locomo.cjs`: LoCoMo answer-accuracy check with the MemoryTax protocol; its `complete` model client is shared with the continuity judge.
+- `bench/continuity/`: synthetic continuity dataset: two seed repos, session tickets, planted facts and the run timeline.
 - `scripts/install-common.cjs`: shared installer helpers: the MCP launch command and Lagoon repo preparation.
 - `scripts/install-claude.cjs`: one-step installer for Claude Code MCP setup and user-level `CLAUDE.md` guidance.
 - `scripts/install-codex.cjs`: one-step installer for the Hippocamp skill and MCP server in Codex.
@@ -48,7 +55,7 @@ This repository is a local-first MCP package for Git-backed agent memory. Keep c
 - Global memory lives under `HIPPOCAMP_GLOBAL_ROOT/`.
 - Project memory lives under `HIPPOCAMP_GLOBAL_ROOT/projects/<slug>/`.
 - The current project slug is inferred from `HIPPOCAMP_PROJECT_ROOT` or the current working directory. An existing folder with nothing but `.git` has no project; project scope then resolves to the global root.
-- Event logs are append-only daily Markdown files under `events/YYYY-MM-DD.md`, with sibling `events/YYYY-MM-DD.index.json` keyword indexes.
+- Event logs are append-only daily Markdown files under `events/YYYY-MM-DD.md`, with sibling `events/YYYY-MM-DD.index.json` keyword indexes. The one exception is the installer's keyword-link format upgrade of older events.
 - Curated files such as `current_state.md` and `open_threads.md` use explicit file writes.
 - MCP `append_event` requires a title and 1-8 keywords; MCP `write_memory_file` rejects paths under `events/`; an explicit `projectRoot` must be a directory inside a Git repository.
 - `append_event` auto-stamps `Agent:` / `Session:` provenance when resolvable (installer env, MCP client name, or process session id). No manual user config.
@@ -79,10 +86,11 @@ When changing code, use the smallest verification that proves the change:
 - Run `npm run dream -- --help` when changing Dream CLI arguments.
 - Run `npm run eval:retrieval -- --limit 100` when changing search ranking or snippets; report Recall@5 against the BM25 row and snippet evidence. Also run `npm run eval:lagoon` when a Lagoon is available.
 - Run `npm run eval:qa -- --limit 3 --dry-run` when changing `scripts/eval-qa.cjs`; a full run calls paid models, so run it only when asked.
+- Run `npm test` and `npm run eval:continuity -- --dry-run` when changing the continuity benchmark or its dataset; a real run spawns paid Claude Code sessions, so run it only when asked.
 
 ## Notes For Future Agents
 
 - The cloud/API version is intentionally deferred.
 - Do not reintroduce Next.js, GitHub API token auth, or GitHub App auth for the local MVP.
 - Keep Dream minimal: scheduled updates and coherent compaction, two curated files plus bounded indexed evidence, one PR per project.
-- The local install story should stay: `npx hippocamp@latest install`, a local Lagoon repo created if missing, normal Git auth. Installers never create remotes or push.
+- The local install story should stay: `npx hippocamp@latest install`, a local Lagoon repo created if missing, normal Git auth. Installers never create remotes or push; their only event rewrite is one local keyword-link format upgrade of older events, the exception to the append-only event invariant.

package/README.md

@@ -52,6 +52,8 @@ Reproducible with the scripts in this repo.
 | Text returned per search | 5.9k chars | Whole sessions: 65.5k |
 | Answer accuracy, Claude Sonnet 5, top 10 results | 77% | Correct sessions given directly: 90% |
 | Unrelated questions that return nothing (our own test set) | 10 / 10 | |
+| Work continuity, 21 real Claude Code sessions, checks passed | 100% (20 / 20) | Claude Code native memory: 96% (25 / 26) |
+| Work continuity, same 21 sessions in Codex, checks passed | 86% (24 / 28) | Codex native memory: 86% (24 / 28) |
 
 Full tables, method, and limits: [docs/results.md](docs/results.md).
 

package/docs/development.md

@@ -10,6 +10,7 @@ npm run mcp:help
 npm run mcp:smoke
 npm run dream
 npm run dream:railway
+npm run eval:continuity
 npm run eval:lagoon
 npm run eval:qa
 npm run eval:retrieval
@@ -21,6 +22,26 @@ npm run upgrade:claude
 npm run upgrade:grok
 ```
 
+## Work continuity benchmark
+
+`npm run eval:continuity` measures what Hippocamp is built for: a coding agent that keeps working on a project across independent sessions and acts on earlier decisions, constraints and state without being told to look them up.
+
+- The dataset in `bench/continuity/` is synthetic: two small engineering projects (`relay`, a TypeScript webhook relay, and `ingest`, a Python CSV pipeline) on one 21-session timeline. Work sessions are ordinary tickets that plant facts; probe sessions are ordinary tickets with hidden checklists. No prompt hints at retrieval.
+- Arms, each set up the way a developer has it: `cc-none` (auto-memory off), `cc-native` (Claude Code defaults) and `cc-hippocamp` (Hippocamp through its real installer, auto-memory off). Codex arms run `codex exec` (`gpt-6.1-sol`) on the ChatGPT sign-in copied from `~/.codex/auth.json`: `codex-native` (Codex `memories` on, with `min_rollout_idle_hours = 0` because sessions run back to back) and `codex-hippocamp` (Hippocamp through `install-codex`). Codex reports no dollar cost, so its cost column stays at $0.
+- Each session is a fresh headless `claude -p --model claude-opus-5-5` process. The repo is committed after each session with the agent's own summary as the message. If the agent ends with an offer or a question, the simulated user answers once with "yes, go ahead". A retried session restores both the repo and the memory store.
+- Sandboxes live in `HIPPOCAMP_BENCH_DIR` (default `~/.cache/hippocamp-bench`), each with its own `HOME` and `CLAUDE_CONFIG_DIR`, so your real `~/.claude` and `~/.lagoon` are never read or written, and runs survive reboots.
+- Judging: a blind judge grades each check from the diff and a tool log with memory tools and paths anonymized. A second call explains each miss (never stored, stale memory, wrong scope, not recalled, ignored, asked instead of acting). At the end, a third grades every stored memory entry for coverage, precision, staleness, noise and duplicates. Prompts and replies are kept next to each transcript, and `validation.csv` lists every verdict for hand-checking.
+- Each session records tokens, time, memory tool calls and memory bytes read or auto-loaded.
+
+```bash
+npm run eval:continuity -- --dry-run                                         # free: validates and prints the cost range
+npm run eval:continuity -- --systems cc-native,cc-hippocamp --limit 3 --run smoke
+npm run eval:continuity -- --run final-1                                     # full run
+npm run eval:continuity -- --report final-1,final-2                          # pooled report, spawns nothing
+```
+
+Sessions and the Claude judge run through Claude Code, so they use your Claude subscription when `CLAUDE_CODE_OAUTH_TOKEN` is set (create it with `claude setup-token` and put it in `.env`). Without it they bill `ANTHROPIC_API_KEY`. `--dry-run` prints the API-price range, and `--budget` caps each call. Results go to `.context/continuity/<run>/`. A run is frozen by `manifest.json` (dataset, harness, model and Claude Code version): reruns with the same `--run` skip finished sessions, new sessions appended to the timeline extend it, and any frozen change is refused unless `--force-manifest` records it. The report shows n next to every percentage, defines memory-required items as those `cc-none` fails in every run, and gives bootstrap intervals over probes.
+
 ## Releases
 
 Pull requests and pushes to `main` run syntax and MCP smoke checks in GitHub Actions.

package/docs/how-it-works.md

@@ -78,7 +78,7 @@ Event writes update a sibling `events/YYYY-MM-DD.index.json` file. The Markdown
 
 - Keep memory concise.
 - Prefer curated summaries over raw event history.
-- Give events a short `Keywords:` section; keywords weigh the most in search ranking. `append_event` writes each keyword as a `[[keyword]]` wiki link, so a graph view such as Obsidian's groups events by topic. Older events with plain keywords or a `Cues:` section still search the same way.
+- Give events a short `Keywords:` section; keywords weigh the most in search ranking. `append_event` writes each keyword as a `[[keyword]]` wiki link, so a graph view such as Obsidian's groups events by topic. Install and upgrade convert clear keyword bullets in older events, including legacy `Cues:` sections, to this format and commit the change in the Lagoon; the next memory sync pushes it. A file is rewritten only if search reads exactly the same keywords from it afterwards.
 - Do not duplicate GitHub-owned facts such as commits, PRs, issues, reviews, or CI results.
 - Store artifact references plus the missing rationale, preference, assumption, or follow-up context.
 - Use project scope for project-specific state.

package/docs/results.md

@@ -39,6 +39,56 @@ The same reader reaches 90% with the correct sessions, so the remaining gap is r
 | Right event in top 5, 43 answerable questions | 42 / 43 |
 | No results for 10 questions no memory is about | 10 / 10 |
 
+## Work continuity
+
+Does memory carry real engineering work from one coding session to the next? `npm run eval:continuity` runs real headless Claude Code (`claude-opus-5-5`, Claude Code 2.1.318 to 2.1.319) through 21 sessions over two small synthetic projects (a TypeScript webhook relay and a Python ingest job). Early sessions plant facts the way work does: a rejected library, a pinned Node version, a blocked vendor, a parked branch, a user preference. Later sessions give tickets that only go well if the agent remembers them. The prompts never say "remember" or point at memory.
+
+In short: the same Hippocamp server, plugged unchanged into Claude Code or Codex, keeps up with each one's own memory (100% vs 96% in Claude Code, 86% vs 86% in Codex), with fewer than 5 points of difference and one run per arm. In Codex, whose built-in memory stored almost nothing, Hippocamp's memory covered 35% of the planted facts against 4%.
+
+Each arm is installed the way a developer has it, in its own sandbox home: Claude Code defaults with native auto-memory, or Hippocamp through its real installer (`install-claude`) with auto-memory off. A blind judge (Opus, arm names and memory paths scrubbed) grades 28 checks against the diff and the tool calls. A check is n/a when the session wrote no code it applies to.
+
+Run `full-1`, one run per arm:
+
+| | Claude Code native memory | Hippocamp in Claude Code |
+| --- | ---: | ---: |
+| Checks passed | 96% (25 / 26) | 100% (20 / 20) |
+| Checks marked as needing memory | 92% (11 / 12) | 100% (10 / 10) |
+| Tokens per session, median | 179k | 224k |
+| Time per session, median | 62 s | 73 s |
+| Agent cost per run, API price | $8.84 | $9.19 |
+| Memory entries at the end | 8 | 15 |
+| Planted facts covered by memory | 22% | 30% |
+| Memory entries that are useful (precision) | 88% | 80% |
+
+What it shows: Hippocamp, plugged into Claude Code as a plain MCP server, does as well as Claude Code's own memory, at about the same cost (+4% dollars, +25% tokens per session). Native memory's one miss was a parked `fix/late-dedupe` branch it no longer knew about, so it rewrote the fix from scratch. Hippocamp's 8 n/a checks come mostly from one session where it correctly held a blocked vendor back and wrote no code; native also held the vendor back, but with gated code the checks could grade.
+
+What it does not show: that Hippocamp is better. The difference is one check, from one run per arm (bootstrap 95% interval over probes: +4 points, 0 to +16). There is no no-memory run, so the "needing memory" label comes from the dataset, not from a measured baseline. An earlier development run (27 / 27 vs 24 / 27) is not used: its cases were added after a tie and its harness was unfair to native memory.
+
+The diffs, judge prompts, grades with evidence and memory snapshots of this run are in [results/continuity-full-1](results/continuity-full-1/) (start with `report.md` and `validation.csv`). Full transcripts are left out for size.
+
+### Same benchmark in Codex
+
+Run `codex-1` repeats the 21 sessions in Codex (`codex exec`, codex-cli 0.159.2, `gpt-6.1-sol`, ChatGPT sign-in), same checks and same blind Opus judge. Codex native memory is its `memories` feature, turned on (it is off by default) with `min_rollout_idle_hours = 0`, because the benchmark runs sessions back to back instead of days apart. Hippocamp is installed with `install-codex`; nothing else changes.
+
+| | Codex native memory | Hippocamp in Codex |
+| --- | ---: | ---: |
+| Checks passed | 86% (24 / 28) | 86% (24 / 28) |
+| Checks marked as needing memory | 67% (8 / 12) | 75% (9 / 12) |
+| Tokens per session, median | 166k | 238k |
+| Time per session, median | 63 s | 82 s |
+| Memory calls per session | 1.0 | 4.9 |
+| Memory entries at the end | 4 | 17 |
+| Planted facts covered by memory | 4% | 35% |
+| Memory entries that are useful (precision) | 25% | 76% |
+
+What it shows: the same Hippocamp server, with no Codex-specific code, works in Codex as it does in Claude Code. Codex called `wake_up`, `search_memory` and `append_event` on its own, and the memory it built covers about as many planted facts as in Claude Code (35% vs 30%). The two Codex arms tie on checks (bootstrap: 0 points, -14 to +14).
+
+Codex's own memory stored almost nothing in this run: its background summaries never produced a memory from a past session, and the one real entry is a changelog preference the agent wrote as a note. The other three entries are Codex's empty templates, which the judge counts as noise. Codex still passed most checks without it, from the repository itself (code, commit messages, notes it wrote in the repo); that is why the score does not drop more. Hippocamp's misses include one stale next step and one fact it never saved (a blocked vendor).
+
+Codex reports no dollar cost, so this run has token and time numbers only.
+
+The run data is in [results/continuity-codex-1](results/continuity-codex-1/).
+
 ## Why less text matters
 
 Memory results share the agent's context with code, diffs, and tool output, and agents search several times per task. About 6k characters per search is roughly 1.5k tokens; whole sessions would be about 16k. Fewer tokens make each turn cheaper and faster, and keep the relevant lines from being buried. Less text is only useful when the answer is still in it, which is why we report answer accuracy next to text size.
@@ -56,4 +106,6 @@ Memory results share the agent's context with code, diffs, and tool output, and
 
 `npm run eval:lagoon` runs the same kind of check, read-only, against your real Lagoon. It reads cases from `<Lagoon root>/evals/retrieval-cases.json`: a project, a query, and the event ids that answer it. An empty list means no memory should be returned. Keep the cases in Lagoon, not in this repo, because they quote private work.
 
+`npm run eval:continuity -- --systems cc-native,cc-hippocamp --run <name>` (or `codex-native,codex-hippocamp`, which needs a Codex ChatGPT sign-in) reruns the work continuity benchmark with your Claude subscription (`CLAUDE_CODE_OAUTH_TOKEN` in `.env`); `--report <name>` prints the tables, and `--dry-run` shows the cost estimate. A run stops cleanly at a usage limit and resumes where it stopped.
+
 `npm run eval:qa` measures answer accuracy on the same LongMemEval-S data with the official LongMemEval reader and grader prompts. It compares Hippocamp top 5 (`hippocamp`), top 10 (`hippocamp-k10`), and the labeled answer sessions (`oracle`). It calls an OpenAI-compatible endpoint from `MANIFEST_BASE_URL` and `MANIFEST_API_KEY`; set `QA_READER_MODEL` and `QA_GRADER_MODEL` to change models. Results are cached in `.context/qa-results.jsonl`, and `--dry-run` shows the jobs and token estimates without any API call.

package/docs/setup.md

@@ -16,7 +16,7 @@ npx hippocamp@latest install-codex
 npx hippocamp@latest install-grok
 ```
 
-Upgrade with the same command: `npx hippocamp@latest upgrade`. Installers default to `~/.lagoon` as the memory repo; pass `--global-root /absolute/path/to/lagoon` to use another clone.
+Upgrade with the same command: `npx hippocamp@latest upgrade`. When older events need it, an upgrade also brings them to the current keyword format, as one local Lagoon commit. Event files with uncommitted changes are left for a later upgrade. Restart your agents afterwards to load the new version. Installers default to `~/.lagoon` as the memory repo; pass `--global-root /absolute/path/to/lagoon` to use another clone.
 
 Installed through npx, the installer copies the package to `~/.hippocamp/<version>/` and registers that copy, so the server starts without npx or network and keeps working if npm cleans its cache. From a source checkout, `npm install` then `npm run install:claude` (or `install:codex`, `install:grok`) registers the checkout directly.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "hippocamp",
-  "version": "0.2.11",
+  "version": "0.2.13",
   "private": false,
   "description": "Local Git-backed memory for AI agents.",
   "license": "MIT",
@@ -30,6 +30,7 @@
   "scripts": {
     "dream": "node scripts/hippocamp-dream.cjs",
     "dream:railway": "bash assets/railway/run-dream.sh",
+    "eval:continuity": "node --env-file-if-exists=.env scripts/eval-continuity.cjs",
     "eval:lagoon": "node scripts/eval-lagoon.cjs",
     "eval:qa": "node scripts/eval-qa.cjs",
     "eval:retrieval": "node scripts/eval-retrieval.cjs",

package/scripts/hippocamp-memory.cjs

@@ -758,6 +758,72 @@ function parseEventBlocks(content) {
   });
 }
 
+// Older events used a "Cues:" header, inline comma lists, or plain bullets. Rewrite them in the
+// format append_event writes now: a "Keywords:" header and one [[wiki link]] bullet per keyword.
+// Only clear keyword bullets change: short, no spaces or backticks, before the first other line.
+function linkKeywordsInEventContent(content) {
+  const eol = content.includes("\r\n") ? "\r\n" : "\n";
+  const lines = content.split(/\r?\n/);
+  const output = [];
+  let inKeywords = false;
+
+  for (const line of lines) {
+    const header = line.match(/^(\s*)(?:Keywords|Cues):\s*(.*?)\s*$/i);
+
+    if (header) {
+      const inline = normalizeKeywordList(splitInlineKeywordValues(header[2]));
+      output.push(`${header[1]}Keywords:`, ...inline.map((keyword) => `${header[1]}- [[${keyword}]]`));
+      inKeywords = true;
+      continue;
+    }
+
+    const bullet = inKeywords && line.match(/^(\s*-\s+)(.+?)\s*$/);
+
+    if (!bullet) {
+      inKeywords = false;
+      output.push(line);
+      continue;
+    }
+
+    const value = bullet[2];
+    const keyword = normalizeKeyword(value);
+    const plain = !value.startsWith("[[") && value.length <= 60 && !/[\s`]/.test(value);
+    output.push(plain && keyword ? `${bullet[1]}[[${keyword}]]` : line);
+  }
+
+  const linked = output.join(eol);
+  const sameKeywords = (text) => JSON.stringify(parseEventBlocks(text).map((event) => event.keywords));
+
+  // Never change what search reads: keep the file as it is if any keyword would differ.
+  return linked !== content && sameKeywords(linked) === sameKeywords(content) ? linked : content;
+}
+
+// Link keywords in every event file under the Lagoon root, except paths `skip` returns true for.
+// Reads every file before writing any. Returns the changed paths, relative to the root.
+async function linkEventKeywords(globalRoot, skip = () => false) {
+  const rewrites = [];
+
+  for (const relativePath of await walkMarkdownFiles(globalRoot)) {
+    if (!/(?:^|\/)events\/\d{4}-\d{2}-\d{2}\.md$/.test(relativePath) || skip(relativePath)) {
+      continue;
+    }
+
+    const absolutePath = path.join(globalRoot, relativePath);
+    const content = await fs.readFile(absolutePath, "utf8");
+    const linked = linkKeywordsInEventContent(content);
+
+    if (linked !== content) {
+      rewrites.push({ relativePath, absolutePath, linked });
+    }
+  }
+
+  for (const rewrite of rewrites) {
+    await fs.writeFile(rewrite.absolutePath, rewrite.linked);
+  }
+
+  return rewrites.map((rewrite) => rewrite.relativePath);
+}
+
 function buildEventIndex(markdownContent, markdownPath) {
   return {
     version: EVENT_INDEX_VERSION,
@@ -1643,6 +1709,7 @@ module.exports = {
   readMemoryFile,
   writeMemoryFile,
   appendEvent,
+  linkEventKeywords,
   listDirectoryEntries,
   searchMemory,
   readRecentEvents,

package/scripts/install-claude.cjs

@@ -273,7 +273,11 @@ async function main() {
   printHints(lagoon);
 }
 
-main().catch((error) => {
-  console.error("Failed to install Hippocamp into Claude Code:", error.message);
-  process.exit(1);
-});
+if (require.main === module) {
+  main().catch((error) => {
+    console.error("Failed to install Hippocamp into Claude Code:", error.message);
+    process.exit(1);
+  });
+}
+
+module.exports = { buildClaudeManagedBlock, ensureClaudeInstructionsInstalled, ensureClaudeSkillInstalled };

package/scripts/install-common.cjs

@@ -4,6 +4,7 @@ const os = require("node:os");
 const path = require("node:path");
 const { execFile } = require("node:child_process");
 const { promisify } = require("node:util");
+const { linkEventKeywords } = require("./hippocamp-memory.cjs");
 
 const execFileAsync = promisify(execFile);
 
@@ -43,6 +44,7 @@ async function git(args, cwd) {
 
 // Make sure the Lagoon memory repo exists, and say what is still missing for sync.
 // It never creates a remote or pushes: those reach outside this machine.
+// It may commit a local format upgrade of older events.
 async function prepareLagoon(globalRoot) {
   const hints = [];
   let created = false;
@@ -75,6 +77,33 @@ async function prepareLagoon(globalRoot) {
     hints.push('Set a Git identity so memory can be committed: git config --global user.name "Your Name" && git config --global user.email you@example.com');
   }
 
+  // Upgrades bring older events to the current keyword format: [[wiki links]] for graph views.
+  // Best effort, committed locally only; the next memory sync pushes it. Files with uncommitted
+  // or untracked changes are left alone, and a failed commit is undone so the next install retries.
+  let linked = [];
+
+  try {
+    const changes = await Promise.all([
+      git(["ls-files", "-z", "--modified", "--others", "--exclude-standard"], globalRoot),
+      git(["diff", "--cached", "--name-only", "-z"], globalRoot),
+    ]);
+    const known = changes.every((change) => change.ok);
+    const dirty = new Set(changes.flatMap((change) => change.stdout.split("\0").filter(Boolean)));
+    linked = await linkEventKeywords(globalRoot, (relativePath) => !known || dirty.has(relativePath));
+
+    if (linked.length) {
+      const commit = await git(["commit", "-m", "hippocamp: link event keywords", "--", ...linked], globalRoot);
+
+      if (!commit.ok) {
+        await git(["checkout", "--", ...linked], globalRoot);
+        linked = [];
+        hints.push("Older event keywords were not linked because Git could not commit. Run the upgrade again once Git is set up.");
+      }
+    }
+  } catch (error) {
+    hints.push(`Older event keywords were not linked: ${error.message}`);
+  }
+
   const remote = await git(["remote", "get-url", "origin"], globalRoot);
   let push = "no_remote";
 
@@ -88,7 +117,7 @@ async function prepareLagoon(globalRoot) {
     }
   }
 
-  return { root: globalRoot, created, remote: remote.ok ? remote.stdout : null, push, hints };
+  return { root: globalRoot, created, remote: remote.ok ? remote.stdout : null, push, linkedEvents: linked.length, hints };
 }
 
 function printHints(lagoon) {

package/scripts/eval-lagoon.cjs

@@ -1,135 +0,0 @@
-#!/usr/bin/env node
-
-const fsSync = require("node:fs");
-const fs = require("node:fs/promises");
-const os = require("node:os");
-const path = require("node:path");
-const memory = require("./hippocamp-memory.cjs");
-
-const MAX_TOP_K = 20;
-const DEFAULT_CASES = "evals/retrieval-cases.json";
-
-function printHelp() {
-  console.log(`Hippocamp Lagoon eval
-
-Read-only retrieval check against your real Lagoon. No model or API key is used.
-Each case names a project, a query, and the event ids that answer it.
-An empty expect list means no memory should be returned.
-
-Usage:
-  npm run eval:lagoon -- [--cases PATH] [--top-k K]
-
-Options:
-  --cases PATH  Cases JSON. Default: <Lagoon root>/${DEFAULT_CASES}
-  --top-k K     Results retrieved per query, at most ${MAX_TOP_K}. Default: 5
-  --help        Show this help
-
-Cases file:
-  { "cases": [ { "project": "<slug>", "query": "...", "expect": ["<event id>"] } ] }
-`);
-}
-
-function parseArgs(argv) {
-  const options = { cases: path.join(memory.getGlobalRoot(), DEFAULT_CASES), topK: 5, help: false };
-
-  for (let index = 0; index < argv.length; index += 1) {
-    const token = argv[index];
-
-    if (token === "--help" || token === "-h") {
-      options.help = true;
-    } else if (token === "--cases") {
-      options.cases = argv[++index];
-    } else if (token === "--top-k") {
-      const number = Number(argv[++index]);
-
-      if (!Number.isInteger(number) || number < 1 || number > MAX_TOP_K) {
-        throw new Error(`--top-k must be an integer from 1 to ${MAX_TOP_K}.`);
-      }
-
-      options.topK = number;
-    } else {
-      throw new Error(`Unknown argument: ${token}`);
-    }
-  }
-
-  return options;
-}
-
-async function run(options) {
-  if (!fsSync.existsSync(options.cases)) {
-    throw new Error(`Missing ${options.cases}. Run with --help for the file format.`);
-  }
-
-  const { cases } = JSON.parse(await fs.readFile(options.cases, "utf8"));
-  // searchMemory derives the project slug from the folder name, so a temp path per
-  // slug reads that project's memory without touching any real repo. The folder is
-  // not created: an existing empty folder has no project and reads global memory.
-  const sandboxRoot = await fs.mkdtemp(path.join(os.tmpdir(), "hippocamp-eval-lagoon-"));
-  const answerable = { count: 0, hit: 0, mrr: 0, chars: 0, topScores: [] };
-  const empty = { count: 0, correct: 0, topScores: [] };
-
-  try {
-    for (const item of cases) {
-      if (!/^[a-z0-9][a-z0-9._-]*$/.test(String(item.project))) {
-        throw new Error(`Invalid project slug: ${item.project}`);
-      }
-
-      const projectRoot = path.join(sandboxRoot, item.project);
-
-      const search = await memory.searchMemory({
-        query: item.query,
-        scope: "project",
-        projectRoot,
-        maxResults: options.topK,
-      });
-      const topScore = search.results[0]?.score ?? 0;
-
-      if (!item.expect.length) {
-        empty.count += 1;
-        empty.correct += search.results.length ? 0 : 1;
-        empty.topScores.push(topScore);
-        console.log(`${search.results.length ? "MISS" : "ok  "}  empty   top ${topScore}  [${item.project}] ${item.query}`);
-        continue;
-      }
-
-      const rank = search.results.findIndex((result) => item.expect.includes(result.id));
-      answerable.count += 1;
-      answerable.hit += rank === -1 ? 0 : 1;
-      answerable.mrr += rank === -1 ? 0 : 1 / (rank + 1);
-      answerable.chars += search.results.reduce((sum, result) => sum + result.snippet.length, 0);
-      answerable.topScores.push(topScore);
-      console.log(
-        `${rank === -1 ? "MISS" : "ok  "}  rank ${rank === -1 ? "-" : rank + 1}  top ${topScore}  [${item.project}] ${item.query}`,
-      );
-    }
-  } finally {
-    await fs.rm(sandboxRoot, { recursive: true, force: true });
-  }
-
-  const percent = (part, whole) => `${((part / Math.max(whole, 1)) * 100).toFixed(1)}%`;
-  const range = (scores) => (scores.length ? `${Math.min(...scores)}-${Math.max(...scores)}` : "n/a");
-
-  console.log(`\nLagoon retrieval, K=${options.topK}, ${cases.length} cases`);
-  console.log(
-    `answerable (${answerable.count})  Hit@K ${percent(answerable.hit, answerable.count)}  MRR ${(answerable.mrr / Math.max(answerable.count, 1)).toFixed(3)}  context ${Math.round(answerable.chars / Math.max(answerable.count, 1))} chars  top score ${range(answerable.topScores)}`,
-  );
-  console.log(
-    `no memory  (${empty.count})  empty result ${percent(empty.correct, empty.count)}  top score ${range(empty.topScores)}`,
-  );
-}
-
-async function main() {
-  const options = parseArgs(process.argv.slice(2));
-
-  if (options.help) {
-    printHelp();
-    return;
-  }
-
-  await run(options);
-}
-
-main().catch((error) => {
-  console.error(error.message);
-  process.exit(1);
-});