@tpsdev-ai/flair-bench 0.22.0

package/LICENSE

@@ -0,0 +1,19 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   Copyright 2026 TPS Dev AI
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

package/README.md

@@ -0,0 +1,157 @@
+# @tpsdev-ai/flair-bench
+
+A standalone embedding recall benchmark for [flair](https://github.com/tpsdev-ai/flair). Run real recall numbers — precision@3, MRR — against any GGUF embedding model, batch-compare several, get a host-aware recommendation, and (optionally) save a redacted, shareable result. **No flair install required.**
+
+```
+npx @tpsdev-ai/flair-bench run --model-file ./nomic-embed-text-v1.5.Q4_K_M.gguf
+```
+
+## Why this exists
+
+flair ships nomic-embed-text-v1.5 (Q4_K_M by default) for local semantic memory search. The question "would a different model/quant actually recall better *for flair's use case*, on *my* hardware?" used to require a full flair checkout, its recall-eval harness, and an ephemeral Harper instance. flair-bench packages the same corpus and the same scoring math into a small, dependency-light CLI/library you can point at any GGUF, anywhere.
+
+## Install / run
+
+```bash
+npx @tpsdev-ai/flair-bench run --model-file /path/to/model.gguf
+```
+
+or install it:
+
+```bash
+npm install -g @tpsdev-ai/flair-bench
+flair-bench run --model-file /path/to/model.gguf
+```
+
+Requires Node.js >= 22. Downloads no models itself — point it at GGUF files you already have.
+
+## Commands
+
+### `run` — benchmark one or more models
+
+```bash
+flair-bench run --model-file a.gguf --model-file b.gguf --model-file c.gguf
+flair-bench run --manifest models.txt          # one path per line, # comments allowed
+flair-bench run --model-file a.gguf --json      # machine-readable output
+flair-bench run --model-file a.gguf --label "local-m4-mini"   # freeform host/infra tag
+```
+
+Reports, per model: p@3/MRR (aggregate and per-kind — stress/trap/hard/clean, see "The corpus" below), ms/embed (serial, warm, N≥64), peak RSS delta, GGUF file size/BPW, embedding dimensions, and load time. A batch (more than one `--model-file`) ends with a ranked comparison table.
+
+### `recommend` — pick a model for this host
+
+```bash
+flair-bench recommend --model-file a.gguf --model-file b.gguf --model-file c.gguf
+```
+
+Fingerprints the host (platform, arch, RAM, CPU model, and the **actual compute backend node-llama-cpp loaded** — Metal/CUDA/Vulkan/CPU, plus GPU device name(s) when present — not inferred from the OS), runs the batch, and recommends the best measured recall the host can plausibly afford. See "Recommend heuristic, and its limits" below — this is deliberately simple and says so.
+
+### `--share` — save a redacted, shareable result
+
+```bash
+flair-bench run --model-file a.gguf --share
+```
+
+Writes a canonical JSON file per model (see "Share schema" below) and prints where the eventual hosted-submission endpoint would receive it — **no network call is made**; the endpoint is a config placeholder (`SUBMISSION_ENDPOINT_PLACEHOLDER` in `src/share.ts`) for a hosted site that doesn't exist yet.
+
+## `--label`: benchmarking infra, not just models
+
+flair-bench's core use case extends past "which model" to "which model on which infra" — comparing a GGUF across, say, a free-tier Fabric host, a GPU-backed Fabric host, and a local Mac. `--label <string>` is a **freeform, user-chosen** string (e.g. `"fabric-free-gcp"`, `"fabric-gpu-a"`, `"local-m4-mini"`) that becomes the grouping key in `run`/`recommend` output and in `--share`'s `hardware.label` field. It is never auto-filled from the machine's real hostname — real hostnames never need to appear in a shared result. Combined with the measured `hardware.backend`/`hardware.gpu` fields (see below), a set of `--share` outputs across labeled hosts is a model × infra matrix.
+
+## Comparable to flair, with a caveat
+
+flair-bench uses **exactly the same corpus, the same p@3/MRR scoring math, and the same nomic search-prefix convention** flair's own isolated recall-eval harness (`test/bench/recall-harness/`) uses — see "What's shared vs. copied" below. The one deliberate difference: Harper's live `/SemanticSearch` endpoint ranks candidates through an **HNSW approximate index**; flair-bench has no HNSW graph, so it ranks every query against **every** corpus record by exact cosine similarity instead. For a corpus this size (251 records), exact search is strictly more informative, not less — but it means a number from this tool and a number from the harness aren't guaranteed to be bit-identical, only very close. See this repo's PR for the validation that quantifies exactly how close (spoiler: they match).
+
+## What's shared vs. copied (and how it's kept honest)
+
+- **Corpus** (`src/corpus-v2.ts`): a **build-time copy** of `test/bench/recall-harness/corpus-v2.ts`. A standalone, npx-able package can't import from the monorepo's `test/` directory at runtime (it isn't published), so the corpus is copied at commit time via `scripts/sync-corpus.mjs` and kept honest by `test/corpus-sync.test.ts`, which deep-equals this copy's exported `CORPUS`/`QUERIES` against the live harness source on every `bun test` run inside the monorepo checkout. Any drift fails CI loudly.
+- **Scorer** (`src/scorer.ts`): a **faithful hand-replication** of the harness's `statsFor()` (which isn't exported, so it can't be imported directly). `test/scorer-sync.test.ts` reads the harness source's raw text and asserts it still contains the exact formula fragments this package replicated — a tripwire against silent drift.
+- **Prefix convention** (`src/prefixes.ts`): re-implements the same `search_document: `/`search_query: ` string-prepend `resources/embeddings-provider.ts` gets from harper-fabric-embeddings' engine, since flair-bench talks to node-llama-cpp directly and has no HFE wrapper in the loop. Keyed on model filename today (see the file's own header) — there's an upstream proposal to carry this convention in the GGUF/HFE registration surface itself instead of every consumer re-deriving it from a filename pattern: `harper-fabric-embeddings#4`.
+
+## The corpus
+
+251 synthetic records across 30 topic clusters, 126 hand-written ground-truth queries in four kinds:
+
+- **stress** — durability/recency adversarial pairs (does a fresher-but-wrong record outrank an older-but-correct one?)
+- **trap** — cross-cluster lexical traps (the same ambiguous term used in two different domains, e.g. "transaction" in database internals vs. finance ops)
+- **hard** — near-duplicate-cluster disambiguation
+- **clean** — unambiguous single-best-answer sanity floor
+
+See `test/bench/recall-harness/corpus-v2.ts`'s header (in the main flair repo) for the full design rationale.
+
+## Recommend heuristic, and its limits
+
+`recommend` picks the model with the best measured MRR among those whose peak RSS delta fits within a RAM headroom budget (default: 50% of currently-available RAM) and whose ms/embed is under a latency ceiling (default: 500ms — deliberately generous). Ties broken by faster ms/embed. If nothing fits the budget, it falls back to ranking the full set and says so explicitly rather than returning nothing.
+
+**What it doesn't know**, stated plainly rather than hidden behind a confident-sounding number:
+
+- RSS is measured for a single model loaded and queried **serially, one request at a time**, in this one process. A real server holding the model resident under concurrent requests will use more memory than this measurement shows.
+- Latency is single-request serial ms/embed — not throughput under concurrency, not batched-request latency.
+- The RAM/latency thresholds are simple fixed fractions/ceilings (`--ram-headroom`, `--latency-threshold`), not a learned or host-class-aware model.
+- "Available RAM" comes from Node's `os.freemem()`, which on macOS in particular tends to report far less than what's actually usable — macOS treats most inactive/file-cache pages as reclaimable-but-not-"free", so `os.freemem()` can read a couple of GiB on a machine that's actually got plenty of headroom. On Linux, `os.freemem()` is closer to the truth (`MemFree`, not `MemAvailable`) but still not identical to it. Treat the RAM-budget gate as directionally useful, not authoritative — a "didn't fit the budget" fallback note is worth a second look on macOS specifically before concluding a model genuinely doesn't fit.
+
+The recommendation always cites the actual measured numbers it's based on (e.g. *"X because p@3 0.984 vs 0.976 (MRR 0.950 vs 0.946) at 22.1ms/embed and 612 MiB peak RSS on your 40.0GiB-available metal host"*) — never a bare model name with no evidence.
+
+## Share schema
+
+`--share` writes one JSON file per model:
+
+```jsonc
+{
+  "toolVersion": "0.1.0",
+  "timestamp": "2026-07-12T00:00:00.000Z",
+  "model": {
+    "name": "nomic-embed-text-v1.5",
+    "fileBasename": "nomic-embed-text-v1.5.Q4_K_M.gguf",
+    "sha256": "…",
+    "quant": "Q4_K_M",
+    "paramsApprox": 136731648,
+    "dims": 768
+  },
+  "hardware": {
+    "label": "local-m4-mini",
+    "platform": "darwin",
+    "arch": "arm64",
+    "cpuModel": "Apple M4 Pro",
+    "ramGiB": 48,
+    "gpu": "Apple M4 Pro",
+    "backend": "metal"
+  },
+  "results": {
+    "aggregate": { "n": 126, "p3": 0.976, "mrr": 0.946 },
+    "perKind": { "stress": { … }, "trap": { … }, "hard": { … }, "clean": { … } },
+    "msPerEmbedSerialWarm": 22.1,
+    "peakRssMiB": 612.1
+  }
+}
+```
+
+**Privacy**: this document NEVER includes a hostname, a filesystem path, or a username. `model.fileBasename` is a basename only (never the directory it lives in); `hardware.label` is whatever freeform string you passed to `--label` — it defaults to nothing, never your machine's real hostname. `test/share-schema.test.ts` gates this contract.
+
+**Stubbed**: the eventual hosted submission endpoint (a site to browse shared results — Nathan's "an option to share benchmarks") doesn't exist yet. `--share` writes the file locally and prints `submission endpoint not yet configured — file saved at <path>`; the endpoint URL in the code is a placeholder constant, and no network call is ever made.
+
+## Library use
+
+The public API (`src/index.ts`) has **no `process.exit`, no `console.*` calls** anywhere in its call graph — `runBenchmark()`/`recommend()` return structured data (`BatchResult`/`RecommendResult`); progress is reported via an optional `onProgress` callback; rendering (pretty text / JSON) lives separately in `src/format.ts`. `src/cli.ts` is a thin argv-parsing layer on top, and is the *only* place in the package that prints or sets an exit code.
+
+This shape is deliberate: a future `flair bench` subcommand on the main flair CLI is expected to import `runBenchmark`/`recommend` directly and drive its own UI, rather than shelling out to this package's bin.
+
+```ts
+import { runBenchmark, recommend } from "@tpsdev-ai/flair-bench";
+
+const batch = await runBenchmark({ modelFiles: ["./model.gguf"], label: "my-host" });
+console.log(batch.models[0].aggregate); // { n: 126, p3: 0.976, mrr: 0.946 }
+
+const picked = await recommend({ modelFiles: ["./a.gguf", "./b.gguf"] });
+console.log(picked.recommendation?.reason);
+```
+
+## Development
+
+```bash
+bun test                 # unit tests (scorer, prefixes, cosine, recommend heuristic, share schema, sync checks)
+bun run build             # tsc build to dist/
+bun run sync:corpus       # regenerate src/corpus-v2.ts from the harness source (run after editing the harness corpus)
+```
+
+`test/corpus-sync.test.ts` and `test/scorer-sync.test.ts` only pass inside a full monorepo checkout (they read `test/bench/recall-harness/` two levels up) — that's expected; they're the drift guard for maintainers, not something a consumer of the published package ever runs.

package/dist/cli-shim.cjs

@@ -0,0 +1,49 @@
+#!/usr/bin/env node
+"use strict";
+/**
+ * cli-shim.cts — Node-version PREFLIGHT for the flair-bench CLI.
+ *
+ * THIS IS THE BIN ENTRY (`package.json` "bin": { "flair-bench": "dist/cli-shim.cjs" }).
+ *
+ * Same pattern as the root package's src/cli-shim.cts and
+ * packages/flair-mcp/src/mcp-shim.cts — see either for the full rationale.
+ * Short version: the real CLI (dist/cli.js) is an ES module, and ESM hoists
+ * + links + evaluates the whole import graph before the first statement in
+ * the file body runs — so a Node-version check inside cli.ts itself would
+ * never execute on a too-old Node (node-llama-cpp requires a modern engine
+ * and fails to load first). This file is CommonJS, which evaluates
+ * top-to-bottom, so the version check below is guaranteed to run and print
+ * before anything tries to load the ESM CLI or node-llama-cpp.
+ *
+ * Deliberately ancient-safe syntax only — `var`, plain functions, string
+ * `.split`/`parseInt`, `console.error`, `process.exit`. No top-level await,
+ * no optional chaining reaching modern-only APIs.
+ */
+Object.defineProperty(exports, "__esModule", { value: true });
+var MIN_NODE_MAJOR = 22;
+function flairBenchCurrentNodeMajor() {
+    var raw = (process && process.versions && process.versions.node) || "0";
+    return parseInt(String(raw).split(".")[0], 10) || 0;
+}
+var flairBenchNodeMajor = flairBenchCurrentNodeMajor();
+if (flairBenchNodeMajor < MIN_NODE_MAJOR) {
+    console.error("");
+    console.error("  flair-bench requires Node.js >= " + MIN_NODE_MAJOR + ".");
+    console.error("  You are running Node.js " + (process.versions && process.versions.node ? process.versions.node : "(unknown)") + ".");
+    console.error("");
+    console.error("  Please upgrade Node and try again:");
+    console.error("    https://nodejs.org/  (or use nvm / fnm / volta)");
+    console.error("");
+    process.exit(1);
+}
+import("./cli.js")
+    .then(function (mod) {
+    if (mod && typeof mod.runCli === "function") {
+        return mod.runCli();
+    }
+    return undefined;
+})
+    .catch(function (err) {
+    console.error(err && err.stack ? err.stack : err);
+    process.exit(1);
+});

package/dist/cli-shim.d.cts

@@ -0,0 +1,21 @@
+#!/usr/bin/env node
+/**
+ * cli-shim.cts — Node-version PREFLIGHT for the flair-bench CLI.
+ *
+ * THIS IS THE BIN ENTRY (`package.json` "bin": { "flair-bench": "dist/cli-shim.cjs" }).
+ *
+ * Same pattern as the root package's src/cli-shim.cts and
+ * packages/flair-mcp/src/mcp-shim.cts — see either for the full rationale.
+ * Short version: the real CLI (dist/cli.js) is an ES module, and ESM hoists
+ * + links + evaluates the whole import graph before the first statement in
+ * the file body runs — so a Node-version check inside cli.ts itself would
+ * never execute on a too-old Node (node-llama-cpp requires a modern engine
+ * and fails to load first). This file is CommonJS, which evaluates
+ * top-to-bottom, so the version check below is guaranteed to run and print
+ * before anything tries to load the ESM CLI or node-llama-cpp.
+ *
+ * Deliberately ancient-safe syntax only — `var`, plain functions, string
+ * `.split`/`parseInt`, `console.error`, `process.exit`. No top-level await,
+ * no optional chaining reaching modern-only APIs.
+ */
+export {};

package/dist/cli.d.ts

@@ -0,0 +1,14 @@
+/**
+ * cli.ts — thin argv-parsing bin. Parses process.argv into the options
+ * objects index.ts's API expects, calls that API, formats the result
+ * (format.ts), and prints — this file is the ONLY place in the package that
+ * calls console.log/console.error/process.exit(code) (process.exitCode is
+ * used, not process.exit(), so a caller that imports runCli() programmatically
+ * doesn't get its own process killed).
+ *
+ * Deliberately thin: the future `flair bench` subcommand (see README
+ * "Library use") is expected to import index.ts's runBenchmark()/recommend()
+ * directly and do its own argv parsing / rendering inside the main `flair`
+ * CLI's existing command tree, not shell out to this file.
+ */
+export declare function runCli(argv?: string[]): Promise<void>;

package/dist/cli.js

@@ -0,0 +1,198 @@
+/**
+ * cli.ts — thin argv-parsing bin. Parses process.argv into the options
+ * objects index.ts's API expects, calls that API, formats the result
+ * (format.ts), and prints — this file is the ONLY place in the package that
+ * calls console.log/console.error/process.exit(code) (process.exitCode is
+ * used, not process.exit(), so a caller that imports runCli() programmatically
+ * doesn't get its own process killed).
+ *
+ * Deliberately thin: the future `flair bench` subcommand (see README
+ * "Library use") is expected to import index.ts's runBenchmark()/recommend()
+ * directly and do its own argv parsing / rendering inside the main `flair`
+ * CLI's existing command tree, not shell out to this file.
+ */
+import { readFileSync } from "node:fs";
+import { runBenchmark, recommend, buildShareDocument, writeShareDocument, SUBMISSION_ENDPOINT_PLACEHOLDER, TOOL_VERSION } from "./index.js";
+import { formatPretty, formatJson, formatRecommendPretty, formatRecommendJson } from "./format.js";
+const HELP = `flair-bench ${TOOL_VERSION} — standalone embedding recall benchmark for flair
+
+USAGE:
+  flair-bench run [options]
+  flair-bench recommend [options]
+
+OPTIONS:
+  --model-file <path>       GGUF file to benchmark. Repeatable for a batch.
+  --manifest <path>         Text file, one GGUF path per line (# comments, blank lines ignored).
+                             Combines with any --model-file flags.
+  --label <string>          Freeform host/infra label (e.g. "fabric-gpu-a", "local-m4-mini") — becomes
+                             the grouping key in output and in --share's hardware block. Never a
+                             hostname; nothing is auto-filled from this machine's actual hostname.
+  --warmup-n <int>          Untimed warmup embeds before ms/embed timing starts. Default 8.
+  --json                    Emit machine-readable JSON instead of the pretty text report.
+  --pretty                  Emit the pretty text report (default).
+  --share                   Also write a canonical, redacted result JSON per model to disk
+                             (see --share-out). Prints where the endpoint submission would go,
+                             once one exists — no network call is made.
+  --share-out <dir>         Directory for --share output. Default: current directory.
+
+RECOMMEND-ONLY OPTIONS:
+  --ram-headroom <fraction> Fraction of available RAM a model's peak RSS delta may use. Default 0.5.
+  --latency-threshold <ms>  ms/embed ceiling for a model to be considered usable. Default 500.
+
+PRIVACY NOTE:
+  --share writes model identity (name, file basename, sha256, quant, dims) and a host
+  fingerprint (platform, arch, CPU model, RAM, GPU backend/device string, and your --label)
+  plus the measured recall/latency/memory numbers. It NEVER includes a hostname, filesystem
+  path, or username — see the package README's "Share schema" section for the exact shape.
+`;
+function readManifest(path) {
+    const text = readFileSync(path, "utf8");
+    return text
+        .split("\n")
+        .map((l) => l.trim())
+        .filter((l) => l.length > 0 && !l.startsWith("#"));
+}
+function parseArgs(argv) {
+    const args = argv.slice(2);
+    const command = args[0] === "run" || args[0] === "recommend" ? args[0] : args[0] === "--help" || args[0] === "-h" || args[0] === undefined ? "help" : undefined;
+    if (command === undefined) {
+        throw new Error(`Unknown command "${args[0]}" — expected "run" or "recommend" (see --help)`);
+    }
+    const modelFiles = [];
+    let label;
+    let warmupN;
+    let json = false;
+    let share = false;
+    let shareOut;
+    let ramHeadroom;
+    let latencyThreshold;
+    for (let i = command === "help" ? 0 : 1; i < args.length; i++) {
+        const a = args[i];
+        const next = () => {
+            i++;
+            if (i >= args.length)
+                throw new Error(`${a} requires a value`);
+            return args[i];
+        };
+        switch (a) {
+            case "--model-file":
+                modelFiles.push(next());
+                break;
+            case "--manifest":
+                modelFiles.push(...readManifest(next()));
+                break;
+            case "--label":
+                label = next();
+                break;
+            case "--warmup-n":
+                warmupN = Number(next());
+                break;
+            case "--json":
+                json = true;
+                break;
+            case "--pretty":
+                json = false;
+                break;
+            case "--share":
+                share = true;
+                break;
+            case "--share-out":
+                shareOut = next();
+                break;
+            case "--ram-headroom":
+                ramHeadroom = Number(next());
+                break;
+            case "--latency-threshold":
+                latencyThreshold = Number(next());
+                break;
+            case "--help":
+            case "-h":
+                return { command: "help", modelFiles, json, share };
+            default:
+                throw new Error(`Unknown option "${a}" (see --help)`);
+        }
+    }
+    return { command, modelFiles, label, warmupN, json, share, shareOut, ramHeadroom, latencyThreshold };
+}
+function onProgress(event) {
+    switch (event.type) {
+        case "host-fingerprinted":
+            console.error(`[flair-bench] host: ${event.host.platform}/${event.host.arch} backend=${event.host.backend}`);
+            break;
+        case "model-start":
+            console.error(`[flair-bench] (${event.index + 1}/${event.total}) loading ${event.fileName}...`);
+            break;
+        case "model-loaded":
+            console.error(`[flair-bench] loaded ${event.fileName} in ${event.loadTimeMs.toFixed(0)}ms — embedding corpus...`);
+            break;
+        case "model-embedding":
+            if (event.done % 64 === 0 || event.done === event.total) {
+                console.error(`[flair-bench]   ${event.fileName}: ${event.done}/${event.total} embedded`);
+            }
+            break;
+        case "model-done":
+            console.error(`[flair-bench] ${event.fileName} done — p@3=${event.result.aggregate.p3.toFixed(3)} MRR=${event.result.aggregate.mrr.toFixed(3)}`);
+            break;
+    }
+}
+export async function runCli(argv = process.argv) {
+    let parsed;
+    try {
+        parsed = parseArgs(argv);
+    }
+    catch (err) {
+        console.error(String(err instanceof Error ? err.message : err));
+        console.error("");
+        console.error(HELP);
+        process.exitCode = 1;
+        return;
+    }
+    if (parsed.command === "help") {
+        console.log(HELP);
+        return;
+    }
+    if (parsed.modelFiles.length === 0) {
+        console.error(`"${parsed.command}" requires at least one --model-file (or --manifest). See --help.`);
+        process.exitCode = 1;
+        return;
+    }
+    try {
+        if (parsed.command === "run") {
+            const options = { modelFiles: parsed.modelFiles, label: parsed.label, warmupN: parsed.warmupN, onProgress };
+            const result = await runBenchmark(options);
+            console.log(parsed.json ? formatJson(result) : formatPretty(result));
+            if (parsed.share) {
+                for (const m of result.models) {
+                    const doc = buildShareDocument(m, result.host);
+                    const written = writeShareDocument(doc, parsed.shareOut);
+                    console.log(`\n${written.endpointNote}`);
+                    console.log(`(submission endpoint placeholder: ${SUBMISSION_ENDPOINT_PLACEHOLDER} — not live, no network call was made)`);
+                }
+            }
+        }
+        else {
+            const options = {
+                modelFiles: parsed.modelFiles,
+                label: parsed.label,
+                warmupN: parsed.warmupN,
+                ramHeadroomFraction: parsed.ramHeadroom,
+                latencyThresholdMsPerEmbed: parsed.latencyThreshold,
+                onProgress,
+            };
+            const result = await recommend(options);
+            console.log(parsed.json ? formatRecommendJson(result) : formatRecommendPretty(result));
+            if (parsed.share) {
+                for (const m of result.batch.models) {
+                    const doc = buildShareDocument(m, result.batch.host);
+                    const written = writeShareDocument(doc, parsed.shareOut);
+                    console.log(`\n${written.endpointNote}`);
+                    console.log(`(submission endpoint placeholder: ${SUBMISSION_ENDPOINT_PLACEHOLDER} — not live, no network call was made)`);
+                }
+            }
+        }
+    }
+    catch (err) {
+        console.error(`[flair-bench] error: ${err instanceof Error ? err.stack ?? err.message : String(err)}`);
+        process.exitCode = 1;
+    }
+}

package/dist/corpus-v2.d.ts

@@ -0,0 +1,102 @@
+/**
+ * corpus-v2.ts — SYNCED COPY of test/bench/recall-harness/corpus-v2.ts
+ * (the flair recall-eval harness's eval instrument v2).
+ *
+ * flair-bench ships standalone (npx-able, no monorepo checkout required at
+ * runtime), so it cannot import the harness's corpus at runtime from
+ * test/bench/ — that directory is dev-only and excluded from the published
+ * package. This file is a build-time copy kept faithful by
+ * test/corpus-sync.test.ts, which deep-equals this file's exported CORPUS
+ * and QUERIES against the live harness source on every `bun test` run
+ * inside the monorepo checkout — any drift fails CI loudly. Regenerate with
+ * `bun run sync:corpus` (packages/flair-bench/scripts/sync-corpus.mjs)
+ * whenever the harness corpus changes.
+ *
+ * Below this header, content is byte-for-byte the harness's corpus-v2.ts —
+ * see that file for the full design rationale (near-duplicate clusters,
+ * cross-cluster lexical traps, durability/recency stress pairs).
+ *
+ * ORIGINAL HEADER (test/bench/recall-harness/corpus-v2.ts):
+ *
+ *  * corpus-v2.ts — eval instrument v2: a larger, harder synthetic corpus +
+ * ground-truth query set for the isolated recall-eval harness
+ * (test/bench/recall-harness/run.ts, --corpus v2).
+ *
+ * WHY THIS EXISTS (vs. corpus.ts / v1): v1 is 87 records / 30 queries. A
+ * flair#504 Phase 2 prefix A/B on v1 measured a small regression (p@3
+ * 0.967→0.933) that is, at N=30, just 1-2 queries shifting rank — not enough
+ * queries for the instrument to tell "ship" from "park" apart from noise. v2
+ * exists to have enough queries per "kind" (see QueryKind below) that a
+ * kind-level delta is a real signal, not a coin flip. v2 does NOT replace v1
+ * — v1's frozen numbers (README.md) stay reproducible verbatim by never
+ * touching corpus.ts; v2 is additive and becomes the standing gate for
+ * future embedding/scoring changes (larger N, same design principles).
+ *
+ * DESIGN — same three axes as v1, just wider:
+ *   1. NEAR-DUPLICATE / ADJACENT DENSITY — every cluster below holds one or
+ *      two pairs of records about the SAME narrow sub-topic from different
+ *      angles (see each record's "Near-dup facet of X" note). A "hard" query
+ *      targets the specific facet, with the sibling as the plausible
+ *      rank-2/3 confusion.
+ *   2. CROSS-CLUSTER LEXICAL "TRAP" PAIRS — three genuinely different-domain
+ *      cluster pairs deliberately share one pivotal, ambiguous term used in
+ *      a different sense in each domain, with high token overlap:
+ *        - DBSTORE (database internals) ↔ FINOPS (finance operations): "transaction"
+ *        - GITWF (git branching workflows) ↔ HORTIC (gardening/horticulture): "branch"
+ *        - MUSICPROD (music production) ↔ SPORTAN (sports analytics): "score"
+ *      Every record in a trap-pair cluster uses its cluster's sense of the
+ *      shared term naturally and prominently (not shoehorned in for the
+ *      trap alone) — the trap only works if the token overlap is real.
+ *   3. VARIED DURABILITY + createdAt (recency) — 17 of the 30 clusters carry
+ *      a deliberate "stress pair": one older, standard/persistent-durability
+ *      CORRECT record vs. a same-topic, fresh (~2-4 day) permanent-durability
+ *      DISTRACTOR sibling — the same composite-vs-raw discriminator
+ *      mechanism v1 uses (see corpus.ts's header and flair#623). The
+ *      remaining records in every cluster (stress or not) still spread
+ *      naturally across all four durability levels and a wide age range
+ *      (~4-100 days), not just the engineered stress pairs.
+ *
+ * 30 topic clusters, genuinely different domains (distributed systems,
+ * medical scheduling, logistics, cooking, astronomy, legal contracts,
+ * fitness, team process, home renovation, marine biology, typography,
+ * database internals, finance ops, git workflow, horticulture, music
+ * production, sports analytics, cloud infra, winemaking, ornithology,
+ * carpentry, meteorology, chess, photography, aviation, insurance,
+ * archaeology, perfumery, ceramics, linguistics), 9 records each = 270
+ * records.
+ *
+ * ALL content below is authored synthetic prose — no real memories, no real
+ * project/people/host/key names, nothing copied from ~/ops or any
+ * production corpus. Every domain fact is invented-but-plausible, written
+ * fresh for this file.
+ *
+ * GROUND TRUTH: identical convention to v1 (see corpus.ts's header) — every
+ * query was written by hand against ONE specific record, then checked that
+ * no other record in the corpus answers the query's literal question as
+ * directly. See QueryKind below for what each kind tests.
+ *
+ * ageDays is relative to seed time (computed fresh by run.ts on every seed),
+ * matching v1.
+ */
+export type Durability = "permanent" | "persistent" | "standard" | "ephemeral";
+export interface CorpusRecord {
+    /** Unique marker, e.g. "DISTSYS::1". Also used to derive the record's id. */
+    marker: string;
+    cluster: string;
+    text: string;
+    durability: Durability;
+    /** Age in days at seed time (createdAt = now - ageDays). */
+    ageDays: number;
+    /** Free-form note on why this record's durability/age was chosen. */
+    note?: string;
+}
+export type QueryKind = "stress" | "trap" | "hard" | "clean";
+export interface GroundTruthQuery {
+    q: string;
+    expectMarker: string;
+    kind: QueryKind;
+    /** What this query is specifically testing, and against which distractor. */
+    note: string;
+}
+export declare const CORPUS: CorpusRecord[];
+export declare const QUERIES: GroundTruthQuery[];