pi-ui-extend 1.0.39 → 1.0.40

package/external/pi-tools-suite/docs/evals.md

@@ -0,0 +1,684 @@
+# Evaluation framework
+
+## Purpose
+
+The eval framework answers four different questions that ordinary unit tests do
+not answer equally well:
+
+1. Does every extension and model-facing tool still satisfy its deterministic
+   contract?
+2. Does a live model choose the right tool for the task instead of merely being
+   capable of calling it?
+3. Does the model follow a good coding workflow from investigation through
+   verification, rather than reaching the right answer by a fragile sequence of
+   guesses and patches?
+4. Does orchestration improve quality or cost without introducing unnecessary
+   delegation, retries, context growth, or latency?
+
+The framework deliberately does **not** reduce all of these questions to one LLM
+judge score. Correctness and safety invariants are machine-checked wherever
+possible. Live-model evals are reserved for choices and workflows that cannot be
+proved by deterministic tests alone.
+
+The implementation lives under `test/evals/` and complements the existing unit,
+integration, prompt-eval, browser-QA, and locate-benchmark suites.
+
+## Design principles
+
+### Deterministic facts are deterministic tests
+
+Schemas, state transitions, routing precedence, persistence, retries, fallbacks,
+security boundaries, parsing, cleanup, and other mechanical behavior should not
+depend on a probabilistic judge. Existing focused tests remain the source of
+truth for these contracts.
+
+### Live evals test model behavior
+
+Live cases are used for questions such as:
+
+- should the model choose `repo_search` or a direct read?
+- should a non-trivial task create a todo plan?
+- should an expensive Sol parent delegate substantial implementation work?
+- should Terra escalate a high-risk architecture decision?
+- should a narrow local failure stay local instead of calling an oracle?
+- did a coding agent reproduce the bug before editing and verify behavior after
+  editing?
+
+### Outcome beats narration
+
+Coding-quality cases use executable broken repositories and post-run checks.
+Passing typecheck, build, or lint alone is insufficient when the requested
+behavior is still wrong.
+
+### Negative controls are first-class
+
+The suite checks both when a tool **should** be called and when it **should
+not**
+be called. This is important for cost control: an agent that delegates every
+task can look capable while still being materially worse.
+
+### Cost is measured separately from correctness
+
+Correctness and safety are hard gates. Token usage, model cost, tool calls, and
+wall-clock time are comparison metrics. A cheap failure is still a failure, and
+a correct solution that becomes dramatically more expensive should be visible
+in the report rather than hidden inside one aggregate score.
+
+## Framework layout
+
+```text
+test/evals/
+  cases.ts
+  coverage-manifest.ts
+  extension-contracts.test.ts
+  harness.test.ts
+  live-evals.test.ts
+  run-evals.ts
+  fixtures/
+    coding-hypotheses/
+    coding-regression/
+    coding-async/
+  harness/
+    types.ts
+    runner.ts
+    assertions.ts
+    metrics.ts
+    report.ts
+```
+
+The responsibilities are intentionally separated:
+
+- `cases.ts` contains live behavioral scenarios and their assertions.
+- `coverage-manifest.ts` maps every extension and model-facing tool to the tests
+  that cover it.
+- `extension-contracts.test.ts` enforces coverage completeness and fills a few
+  previously missing deterministic registration contracts.
+- `harness.test.ts` tests the eval machinery itself.
+- `live-evals.test.ts` exposes the live cases through Bun's test runner.
+- `run-evals.ts` runs a matrix and writes JSON/Markdown comparison artifacts.
+- `runner.ts` creates isolated fixture projects, launches Pi, records events,
+  snapshots changed files, and collects metrics.
+- `assertions.ts` converts case expectations into machine-checkable results.
+- `metrics.ts` derives workflow, token, subagent, mutation, and timing metrics.
+- `report.ts` produces the cross-model comparison report.
+
+## Coverage gate
+
+`coverage-manifest.ts` is a registry of deterministic and optional live coverage
+for the suite.
+
+Two invariants are enforced by `extension-contracts.test.ts`:
+
+1. every extension registered in `src/index.ts` must have deterministic eval
+   coverage;
+2. every model-facing tool described by the suite must have deterministic eval
+   coverage.
+
+This is intentionally a maintenance gate. Adding a new extension or tool without
+assigning it a contract causes the eval contract test to fail.
+
+The current extension registry covers all 19 modules:
+
+| Extension | Deterministic coverage | Representative live coverage |
+| --- | --- | --- |
+| `coding-discipline` | coding-discipline tests | coding-quality workflows |
+| `ast-grep` | ast-grep tests | structural tool selection |
+| `async-subagents` | core/tools/UI tests | Sol/Luna/Terra orchestration |
+| `lsp` | LSP tests | deterministic only |
+| `comment-checker` | comment-checker tests | deterministic only |
+| `session-name` | session-name tests | deterministic only |
+| `session-recovery` | recovery tests | overview-first recovery |
+| `repo-discovery` | repo-discovery tests | search/architecture selection |
+| `antigravity-auth` | provider/auth tests | deterministic only |
+| `opencode-import` | import tests | deterministic only |
+| `todo` | todo + persistence tests | plan creation / negative control |
+| `model-tools` | alias/profile tests | direct small-task selection |
+| `usage` | eval extension contracts | deterministic only |
+| `web-search` | web-search tests | deterministic only |
+| `dcp` | DCP prompt/pruning/state tests | existing prompt evals |
+| `prompt-commands` | eval extension contracts | deterministic only |
+| `skill-installer` | eval extension contracts | deterministic only |
+| `credential-firewall` | firewall tests | deterministic only |
+| `codex-reasoning-fix` | reasoning-fix tests | deterministic only |
+
+The tool registry similarly covers `lookup`, `ast_grep`, `ast_apply`, all
+subagent actions, all `repo_*` tools, `todo`, session tools, web tools, Claude
+aliases, Codex aliases, and `compress`.
+
+## Live case model
+
+Each live case is an `EvalCase` with:
+
+- a stable `id`;
+- a category;
+- a human description;
+- a fixture project;
+- a user prompt;
+- optional model filters;
+- optional fake indexed-repository support;
+- optional tools that should be recorded but not executed;
+- machine-checkable assertions;
+- an optional custom validator for structured tool input.
+
+Conceptually:
+
+```ts
+{
+  id: "quality.behavior-over-structural-check",
+  category: "coding-quality",
+  fixture: "coding-regression",
+  prompt: "Fix the rounding bug, reproduce it first, then verify behavior...",
+  assert: {
+    requireReproBeforeMutation: true,
+    requireVerificationAfterMutation: true,
+    maxMutations: 2,
+    maxFilesChanged: 1,
+    postRunCommands: [
+      { command: "npm test" },
+      { command: "node ...hidden-style boundary check..." },
+    ],
+  },
+}
+```
+
+The prompt should describe user intent rather than naming the expected tool
+unless naming the tool is itself part of the contract. This prevents a case from
+passing merely because the answer was embedded in the prompt.
+
+## Initial 20-case corpus
+
+### Tool selection
+
+- `tool.semantic-repo-search`: an unknown behavior owner should select
+  `repo_search` before direct discovery.
+- `tool.architecture-first`: a broad unfamiliar-repository overview should
+  select `repo_architecture`.
+- `tool.exact-literal-direct`: an exact literal lookup should avoid semantic
+  and architecture-discovery overhead.
+- `tool.ast-structural`: a syntax-aware structural query should select
+  `ast_grep`.
+- `tool.todo-plan`: non-trivial four-stage work should initialize synchronized
+  todo state.
+- `tool.session-recovery-overview`: lost context with no reliable search phrase
+  should begin with `session_overview`.
+
+### Coding quality
+
+- `quality.two-hypotheses-before-fix`: reproduce and distinguish plausible
+  causes before mutation.
+- `quality.behavior-over-structural-check`: verify the behavioral claim instead
+  of stopping at structural validity.
+- `quality.async-stale-state`: cover stale async completion behavior, not only
+  the success path.
+- `quality.investigate-without-edit-probes`: investigate with evidence instead
+  of using edits as diagnostic probes.
+- `quality.counterexample-preserves-validation`: preserve nearby validation and
+  explicitly check a boundary or counterexample.
+
+### Orchestration and escalation
+
+- `orchestration.sol-delegates-substantial` (Sol): the expensive parent should
+  delegate substantial multi-file work.
+- `orchestration.sol-keeps-tiny-edit` (Sol): a tiny known-file edit should stay
+  in the parent.
+- `orchestration.luna-delegates-substantial` (Luna): substantial implementation
+  or research should use the configured worker tier.
+- `orchestration.luna-escalates-high-risk` (Luna): high-risk cross-module or
+  security work should trigger stronger review/deep escalation.
+- `orchestration.terra-escalates-high-risk` (Terra): high-risk architecture or
+  security uncertainty should escalate to stronger roles.
+
+### Negative controls
+
+- `negative.trivial-chat-no-tools`: prevents todo/compress/subagent overhead for
+  trivial chat.
+- `negative.known-file-read-no-oracle`: prevents planning/search/oracle
+  overhead for one known-file fact.
+- `negative.routine-local-failure-no-oracle`: prevents escalation for a
+  straightforward locally verifiable test failure.
+- `negative.small-known-edit-no-plan`: prevents todo/repo-search/subagent
+  overhead for a tiny known-file edit.
+
+## Coding-quality fixtures
+
+The quality fixtures are deliberately small enough to understand but broken in
+ways that distinguish disciplined debugging from lucky patching.
+
+### `coding-hypotheses`
+
+Symptom: a second checkout for the same user can reuse the first checkout's
+idempotency key.
+
+At least two plausible explanations exist from the task description:
+
+- the retry-key cache identity is too broad;
+- callers may be passing the wrong checkout identifier.
+
+The baseline test fails. The intended workflow is to inspect/reproduce, gather
+evidence that separates the hypotheses, make the smallest fix, and verify both
+stable retry behavior and separate checkout identities.
+
+### `coding-regression`
+
+Symptom: percentage discounts can produce a one-cent overcharge because the
+fixture rounds where the contract requires flooring.
+
+The function also contains input validation that must not be weakened. This
+fixture catches agents that make a superficially plausible arithmetic change
+without checking behavioral boundaries or regression safety.
+
+### `coding-async`
+
+Symptom: an older profile request can resolve after a newer request and
+overwrite
+the latest state.
+
+The public contract still requires each `load(userId)` call to resolve with its
+own profile. The fix therefore has to distinguish return-value correctness from
+shared-state freshness and verify the stale completion path explicitly.
+
+All three fixture test suites are expected to fail before the agent changes the
+repository. A fixture that starts green is not a valid bug-fix eval.
+
+## Isolation and runner lifecycle
+
+For each `(case, model)` pair the runner:
+
+1. copies the selected fixture into a fresh temporary project;
+2. snapshots ordinary project files before execution;
+3. creates an isolated `.pi` session directory;
+4. optionally marks the project as indexed and installs a deterministic fake
+   `idx` executable for repository-selection cases;
+5. writes a recorder extension that captures tool calls/results and final usage;
+6. starts a real `pi` subprocess with the requested model and suite extension;
+7. waits for completion or the per-case timeout;
+8. snapshots the project again and derives changed files;
+9. computes workflow/cost metrics;
+10. evaluates assertions and post-run behavioral checks;
+11. deletes the temporary project unless retention was requested.
+
+The subprocess disables unrelated extension/skill/template/theme discovery so
+the case measures this suite rather than ambient user configuration.
+
+### Recorded-but-blocked tools
+
+Some selection cases need to prove that a model **chose** a tool without paying
+for or executing the next layer of work. `blockTools` records the call and then
+returns a controlled block result.
+
+Examples:
+
+- `ast_grep` can be blocked after structural selection is observed;
+- `subagents` can be blocked after delegation/escalation selection is observed.
+
+This keeps selection evals focused and prevents nested model calls from making a
+simple routing case unnecessarily expensive.
+
+## Assertions
+
+The shared assertion layer currently supports:
+
+- required tools;
+- forbidden tools;
+- exact first tool;
+- first tool from an allowed set;
+- maximum tool-call count;
+- maximum mutation count;
+- maximum changed-file count;
+- reproduction/verification before the first mutation;
+- behavioral verification after the last mutation;
+- required/forbidden output text;
+- post-run commands with expected exit status;
+- required post-run output text;
+- custom case-specific validation.
+
+The todo-plan case uses custom validation to check structured todo input: one
+`batch_create`, four stages, exactly one `in_progress` item, and an explicit
+final
+report task.
+
+## Workflow inference
+
+The harness does not inspect private chain-of-thought. It infers observable
+workflow from tool events.
+
+Mutation tools currently include:
+
+```text
+edit / Edit
+write / Write
+apply_patch
+ast_apply
+```
+
+Verification calls are shell-like tool calls whose commands visibly run tests,
+checks, lint, typecheck, or similar verification commands.
+
+This makes assertions such as `requireReproBeforeMutation` and
+`requireVerificationAfterMutation` auditable without depending on model-written
+prose.
+
+The heuristic is intentionally conservative. If a new verification mechanism is
+introduced, extend `metrics.ts` and add a harness regression test instead of
+silently assuming the metric still means the same thing.
+
+## Metrics
+
+Each result records:
+
+| Metric | Meaning |
+| --- | --- |
+| `elapsedMs` | wall-clock runtime for the model case |
+| `toolCallCount` | total captured tool calls |
+| `toolCalls` | ordered tool names |
+| `failedToolResults` | tool results marked as errors |
+| `mutationCount` | number of recognized mutation tool calls |
+| `verificationCount` | number of recognized verification commands |
+| `changedFiles` | project files whose content changed |
+| `parentUsage` | parent token usage and provider-reported cost |
+| `subagentUsage` | aggregate worker/subagent usage |
+| `subagentCount` | number of detected subagent workspaces |
+
+Parent usage is captured from the final `agent_end` event when available, with
+session artifacts as a fallback. Subagent usage is collected from subagent
+session artifacts under `.pi/subagents/`.
+
+Provider cost fields are reported when the provider supplies them. A zero cost
+does not necessarily mean a request was free; it may mean that the active
+provider does not populate monetary cost metadata.
+
+## Reports
+
+`npm run evals:report` writes:
+
+```text
+test/evals/artifacts/<timestamp>/eval-report.json
+test/evals/artifacts/<timestamp>/eval-report.md
+```
+
+The default artifact directory is ignored by Git.
+
+The Markdown report contains a model summary table with:
+
+- passed cases / total cases;
+- parent tokens;
+- worker tokens;
+- provider-reported cost;
+- tool calls;
+- elapsed time.
+
+It also lists every case with its tool sequence, changed files, token split, and
+runtime. Failed assertions are expanded in a dedicated failure section.
+
+The JSON report preserves the structured results for later statistical or CI
+analysis.
+
+## Running evals
+
+### Deterministic coverage and harness tests
+
+Run these on normal development changes:
+
+```bash
+npm run test:evals:contracts
+```
+
+This runs both the extension/tool coverage gate and tests for the harness
+itself.
+It does not call a live model.
+
+The same tests are also included in the normal `npm test` traversal because they
+live under `test/`.
+
+### Live matrix through Bun tests
+
+Set one or more models:
+
+```bash
+PI_TOOLS_SUITE_EVAL_MODELS='zai/glm-5.3' \
+  npm run test:evals:live
+```
+
+Multiple models are comma-, semicolon-, or newline-separated:
+
+```bash
+MODELS='zai/glm-5.3,'\
+'openai-codex/gpt-5.6-luna,'\
+'openai-codex/gpt-5.6-terra,'\
+'openai-codex/gpt-5.6-sol'
+PI_TOOLS_SUITE_EVAL_MODELS="$MODELS" \
+  npm run test:evals:live
+```
+
+Cases with model filters only run for matching models. For example, Sol-specific
+orchestration cases do not run against GLM.
+
+`PI_TOOLS_SUITE_EVAL_CASES` and `PI_TOOLS_SUITE_EVAL_CATEGORIES` (documented
+below) filter the Bun live matrix the same way they filter the report runner.
+
+### Comparative report runner
+
+For model comparison, prefer the report runner:
+
+```bash
+MODELS='zai/glm-5.3,'\
+'openai-codex/gpt-5.6-terra,'\
+'openai-codex/gpt-5.6-sol'
+PI_TOOLS_SUITE_EVAL_MODELS="$MODELS" \
+  npm run evals:report
+```
+
+The process exits non-zero if any selected case fails.
+
+### Run only one category
+
+```bash
+PI_TOOLS_SUITE_EVAL_MODELS='zai/glm-5.3' \
+PI_TOOLS_SUITE_EVAL_CATEGORIES='coding-quality,negative' \
+  npm run evals:report
+```
+
+Valid current categories are:
+
+```text
+tool-selection
+coding-quality
+orchestration
+negative
+```
+
+### Run named cases
+
+```bash
+PI_TOOLS_SUITE_EVAL_MODELS='openai-codex/gpt-5.6-sol' \
+CASES='orchestration.sol-delegates-substantial,'\
+'orchestration.sol-keeps-tiny-edit'
+PI_TOOLS_SUITE_EVAL_CASES="$CASES" \
+  npm run evals:report
+```
+
+### Other runner controls
+
+| Variable | Purpose |
+| --- | --- |
+| `PI_TOOLS_SUITE_EVAL_TIMEOUT_MS` | per-case timeout; default 240 seconds |
+| `PI_TOOLS_SUITE_EVAL_STREAM_IO=1` | stream child stdout/stderr while running |
+| `PI_TOOLS_SUITE_EVAL_KEEP=1` | retain temporary projects for debugging |
+| `PI_TOOLS_SUITE_EVAL_OUTPUT_DIR=/path` | choose report output directory |
+
+## Existing prompt evals and benchmarks
+
+The unified harness does not replace the older focused suites.
+
+Use the existing prompt evals when changing the behavior they specifically
+cover:
+
+```bash
+npm run test:prompt-evals
+npm run test:prompt-evals:tool-selection
+npm run test:prompt-evals:async
+npm run test:prompt-evals:dcp
+```
+
+Use the locate benchmark for deeper repository-discovery efficiency comparisons:
+
+```bash
+npm run bench:locate
+npm run bench:locate:analyze
+```
+
+The long-term direction is to share infrastructure where useful, but not to
+discard specialized tests that measure a distinct contract well.
+
+## Recommended model matrix
+
+For changes to model discipline or orchestration, the useful comparison set is:
+
+```text
+zai/glm-5.3
+openai-codex/gpt-5.6-luna
+openai-codex/gpt-5.6-terra
+openai-codex/gpt-5.6-sol
+```
+
+This matrix exposes several important regressions:
+
+- a prompt improves GLM but makes Terra overthink simple tasks;
+- Luna stops delegating substantial work;
+- Terra escalates routine failures unnecessarily;
+- Sol stops delegating expensive implementation work;
+- Sol delegates tiny tasks and increases total cost;
+- a cheaper route saves parent tokens but increases total worker tokens or
+  wall-clock time enough to erase the benefit.
+
+Model availability and authentication are environment-dependent. Live evals are
+therefore opt-in rather than part of the normal deterministic gate.
+
+## Interpreting results
+
+Do not rank models by one scalar score alone. Read the report in this order:
+
+1. **Hard correctness:** did the required behavioral/post-run checks pass?
+2. **Safety and discipline:** were forbidden tools absent and file/mutation
+   budgets respected?
+3. **Workflow:** did the agent reproduce before editing and verify after editing
+   where the case requires it?
+4. **Delegation quality:** did orchestration occur only at the intended
+   boundary?
+5. **Cost:** how many parent and worker tokens were consumed?
+6. **Latency:** did extra workers or retries materially increase wall-clock
+   time?
+
+A useful before/after comparison looks like:
+
+```text
+success rate        85% -> 95%
+parent tokens       -38%
+worker tokens       +19%
+total tokens        -11%
+tool calls          -14%
+wall time           +6%
+unwanted escalation 3 -> 0 cases
+```
+
+The framework currently reports the raw ingredients rather than inventing a
+single weighted score. If an aggregate score is added later, hard correctness
+and security failures should remain non-negotiable gates outside that score.
+
+## Adding a new live case
+
+1. Decide whether the behavior actually requires a live model. If a
+   deterministic test can prove it, prefer the deterministic test.
+2. Add or reuse a small fixture. A bug-fix fixture should fail before the agent
+   acts.
+3. Add a stable case to `test/evals/cases.ts`.
+4. Express as much of the expected behavior as possible using shared assertions.
+5. Use `postRunCommands` for executable correctness and hidden-style regression
+   checks.
+6. Add a custom validator only when structured tool arguments need inspection.
+7. Add the live case ID to the relevant extension/tool entries in
+   `coverage-manifest.ts`.
+8. Run the deterministic harness tests before spending model tokens.
+9. Run the smallest relevant model/category subset first.
+10. Run the broader matrix before merging a prompt/routing change.
+
+Good cases have one clear behavioral question. Avoid giant prompts that test
+five unrelated policies at once because failures become hard to diagnose.
+
+## Adding a new extension or tool
+
+For a new extension:
+
+1. add focused deterministic tests;
+2. add the extension to `EXTENSION_EVAL_COVERAGE`;
+3. add live cases only if model choice/workflow matters.
+
+For a new model-facing tool:
+
+1. add deterministic registration/execution tests;
+2. add the tool to `TOOL_EVAL_COVERAGE`;
+3. add positive and negative tool-selection cases when selection is non-obvious.
+
+The coverage contract test intentionally fails until these steps are complete.
+
+## CI strategy
+
+A practical split is:
+
+### Every change / pull request
+
+```bash
+npm run typecheck
+npm test
+npm run test:evals:contracts
+```
+
+The last command is redundant with the broad `npm test` traversal but useful as
+a fast explicit gate in targeted jobs.
+
+### Changes to tool descriptions, prompts, routing, or discipline
+
+Run the relevant live category/model subset in addition to deterministic tests.
+
+### Nightly or manual comparison
+
+Run the full available GLM/Luna/Terra/Sol matrix and persist the JSON/Markdown
+report as a CI artifact. This is the right place to watch token/cost/latency
+drift because live-model variance and expense make it a poor default PR gate.
+
+## Known limitations
+
+- Live model behavior is probabilistic; one run is useful for regression
+  detection but not a statistically strong benchmark by itself.
+- The current runner records one execution per case/model. Repetition and
+  confidence intervals are natural future additions.
+- Verification detection is heuristic and based on observable shell commands.
+- Provider-reported monetary cost may be absent even when tokens are present.
+- Selection cases that block `subagents` measure the delegation decision, not
+  nested worker outcome quality.
+- The current orchestration cases validate whether delegation/escalation was
+  selected. Deeper end-to-end worker-quality cases should remain separate so
+  routing failures and worker failures are diagnosable independently.
+- No LLM judge currently grades prose quality. This is intentional: the initial
+  suite prioritizes objective behavior, tool discipline, executable outcomes,
+  and cost.
+
+## Future extensions
+
+Useful next additions include:
+
+- repeated runs with pass-rate confidence intervals;
+- baseline report comparison with explicit regression thresholds;
+- richer parent-versus-worker token attribution by role/model;
+- per-case normalized cost budgets;
+- automatic detection of equivalent repeated patch attempts;
+- DCP continuation evals that continue work after forced compaction;
+- comment-checker precision/recall corpora;
+- larger AST transformation fixtures with semantic hidden tests;
+- end-to-end orchestration cases that allow real workers to complete and compare
+  `Sol direct` vs `Sol + Terra` vs `Terra + Sol escalation`;
+- machine-readable CI summary output for trend dashboards.
+
+The important constraint is to keep correctness evidence objective whenever it
+can be objective, and use live models only for the decisions that genuinely
+depend on model behavior.

package/external/pi-tools-suite/package.json

@@ -29,6 +29,10 @@
     "test:prompt-evals:async": "PROMPT_EVAL_E2E=1 bun test --concurrent --max-concurrency=5 test/async-subagents/selection-e2e.test.ts test/prompt-evals/async-routing-e2e.test.ts",
     "test:prompt-evals:dcp": "PROMPT_EVAL_E2E=1 bun test --concurrent --max-concurrency=5 test/prompt-evals/dcp-summary-e2e.test.ts",
     "test:prompt-evals": "PROMPT_EVAL_E2E=1 bun test --concurrent --max-concurrency=5 test/tool-selection-e2e.test.ts test/async-subagents/selection-e2e.test.ts test/prompt-evals",
+    "test:evals:contracts": "bun test test/evals/extension-contracts.test.ts test/evals/harness.test.ts",
+    "test:evals:live": "PI_TOOLS_SUITE_EVALS_LIVE=1 bun test --concurrent --max-concurrency=4 test/evals/live-evals.test.ts",
+    "evals": "npm run test:evals:contracts && bun test test/evals/live-evals.test.ts",
+    "evals:report": "bun test/evals/run-evals.ts",
     "bench:locate": "PI_LOCATE_BENCH_ITERATIONS=5 PI_LOCATE_BENCH_FAKE_IDX=0 PI_LOCATE_BENCH_MODEL=zai/glm-5-turbo PI_LOCATE_BENCH_MODES=direct-read-grep,ast-structural,repo-search-hybrid,repo-discovery,subagent-search,unrestricted-suite node test/fixtures/hard-to-find-project/benchmark/run-locate-benchmark.mjs",
     "bench:locate:analyze": "node test/fixtures/hard-to-find-project/benchmark/analyze-locate-benchmark.mjs",
     "test:locate-benchmark-e2e": "PI_LOCATE_BENCH_E2E=1 PI_LOCATE_BENCH_MODEL=zai/glm-5-turbo bun test test/locate-benchmark-e2e.test.ts",
@@ -44,9 +48,9 @@
     "vscode-languageserver-protocol": "^3.17.5"
   },
   "peerDependencies": {
-    "@earendil-works/pi-ai": "0.85.0",
-    "@earendil-works/pi-coding-agent": "0.85.0",
-    "@earendil-works/pi-tui": "0.85.0",
+    "@earendil-works/pi-ai": "0.85.1",
+    "@earendil-works/pi-coding-agent": "0.85.1",
+    "@earendil-works/pi-tui": "0.85.1",
     "typebox": "*"
   },
   "devDependencies": {