pi-ui-extend 1.0.39 → 1.0.41

package/external/pi-tools-suite/docs/context-gateway-p00-adr.md

@@ -0,0 +1,216 @@
+# Context Gateway P00 ADR: tool-result stage ordering
+
+<!-- markdownlint-disable MD013 -->
+
+> Status: accepted implementation decision for future Context Gateway work.
+> This ADR does not enable Gateway, change user configuration, or certify rollout.
+> Evidence baseline: repository `daa1b06`, installed Pi SDK `0.85.1`, 7 September 2026.
+
+## Context
+
+The installed extension runner chains `tool_result` handlers in registration
+order. The suite currently has four independent `tool_result` registrations:
+LSP, comment-checker, DCP, and the opt-in credential firewall. Their module load
+order is LSP → comment-checker → DCP → credential firewall.
+
+That order is unsuitable for Gateway enforce mode:
+
+- mutation diagnostics must be present before Gateway renders a bounded result;
+- session-hygiene redaction, when enabled, must happen before durable capture;
+- Gateway must shape before DCP records the delivered result;
+- changing the global `MODULES` order would also change unrelated lifecycle and
+  provider hooks, including the intentionally late provider firewall.
+
+P00 tests also prove that throwing from a result handler is fail-open, and that
+an in-flight extension tool can cross an extension reload: the old wrapper then
+fails on its stale runner while the new runner handles the resulting error.
+Therefore current active-runner state is not an origin binding.
+
+## Decision
+
+For the current **P01-R storeless scope**, keep the existing independent
+`tool_result` chain. A suite-local coordinator is **not** introduced merely to
+make the architecture look uniform. It becomes conditional future work only if
+a store-backed/enforce stage is authorised and a concrete conflict between
+enabled suite-owned result handlers cannot be expressed safely through the
+verified event-specific order.
+
+The verified storeless order is:
+
+1. LSP and comment-checker result enrichers.
+2. Context Gateway `observe` (passive; no result patch).
+3. Optional `truncation-metadata-normalizer` (details-only duplicate cleanup).
+4. Other downstream result observers/modifiers; the current legacy DCP happens
+   to be in this position but P01-R does not depend on DCP state or persistence.
+5. Opt-in credential-firewall session-hygiene redaction.
+
+For provider hooks, credential-firewall redaction runs before
+`codex-reasoning-fix`, and `codex-reasoning-fix` remains the final
+`before_provider_request` sanitizer.
+
+This order is covered both by the `MODULES` contract and by an executable
+`ExtensionRunner` chain with a DCP-free fake downstream observer. The optional
+normalizer removes only redundant metadata; the later firewall still owns secret
+redaction. With firewall session hygiene disabled, the normalizer does not redact
+or otherwise alter visible content.
+
+If a future enforce/store stage is authorised, the previously proposed
+coordinator remains the candidate design: modules participating in it must not
+also register an independent `tool_result` handler, while their non-result hooks
+remain registered normally. That future decision must be revalidated against the
+then-current DCP/session-recovery implementation rather than copied mechanically
+from the legacy chain.
+
+The **future enforce coordinator candidate** order is:
+
+1. **Enrich** — LSP mutation diagnostics, then comment-checker output.
+2. **Session-hygiene sanitize** — the credential-firewall tool-result transform
+   when that existing opt-in policy is enabled.
+3. **Gateway sanitize/capture/render** — apply Gateway's archive-specific
+   permitted-snapshot sanitizer, consume any trusted pre-truncation capture
+   handle, publish if policy requires it, and return passthrough/exact/compact/
+   degraded delivery.
+4. **DCP observe** — record only the result that will be delivered after the
+   preceding stages.
+
+The credential firewall's `before_provider_request` hook stays in its existing
+late provider position. `codex-reasoning-fix` remains the last provider-payload
+sanitizer. The coordinator is event-specific; it is not a replacement for
+global module ordering.
+
+### P01-R storeless ordering before any coordinator exists
+
+The accepted coordinator above is conditional future **store-backed enforce**
+work. P01-R does not add it merely to clean redundant SDK metadata. The checked
+storeless path keeps independent handlers and the current suite registration
+order:
+
+1. suite-owned result enrichers such as LSP/comment-checker;
+2. Context Gateway `observe`, which records the original result boundary and
+   does not transform it;
+3. optional `truncation-metadata-normalizer`, which may remove only a proven
+   duplicate `details.truncation.content` while preserving the delivered body,
+   outcome and every other details field;
+4. downstream result observers. The R-B contract uses a fake observer with no
+   DCP imports so this boundary is not coupled to the current or future DCP
+   implementation;
+5. optional credential-firewall `tool_result` hygiene, which remains the
+   security transform for delivered result content/details when that module is
+   enabled;
+6. provider hooks later run in their normal order: credential firewall before
+   the final `codex-reasoning-fix` payload sanitizer.
+
+This ordering intentionally lets `observe` measure the pre-cleanup metadata
+boundary. The normalizer is not a security boundary and does not restore or
+create source bytes. If firewall hygiene is enabled after it, secrets still
+present in delivered content/remaining details are redacted before JSONL/next
+provider context. When hygiene is disabled, the normalizer must not silently
+pretend those bytes were redacted.
+
+The normalizer's tool-name/SDK-shape allowlist is a compatibility guard, not
+trusted provenance. `tool_result` carries no authenticated extension owner, so
+a replacement tool can reuse `Read`/shell/`ast_grep` names. Exact duplicate
+cleanup can remain semantics-preserving while all stronger capability/lifetime
+claims for such a replacement stay **limited**. A future capture/store path
+cannot use this name/shape test as permission to open a path or publish a
+snapshot.
+
+R-B does **not** introduce a coordinator because no conflict requiring one is
+present in the storeless path. Existing tests prove independent handler
+composition, exception fail-open behavior and the fact that a later extension
+can reinsert raw content. Therefore hard enforce remains unsupported; if a
+future store-backed stage needs different security ordering, it must implement
+the event-specific coordinator without leaving duplicate independent result
+registrations active.
+
+## P01-R same-name replacement limitation
+
+The metadata normalizer can conservatively reject unknown tool names and
+malformed/non-matching truncation metadata, but the generic `tool_result` event
+does not carry a cryptographic or host-owned identity proving which tool
+definition produced a result. A separately loaded replacement that deliberately
+uses a measured name such as `read` plus the exact SDK truncation shape is
+therefore **not independently provenance-certified** by name+shape alone. P01-R
+support claims are limited to the verified suite/SDK tool combinations. Strict
+enforce must not generalise this heuristic to arbitrary replacements without a
+host-owned tool-definition identity seam.
+
+## Failure contract
+
+Gateway stage failure must not rely on `throw`. A Gateway failure returns an
+explicit bounded degraded result that preserves the real execution outcome and
+states that archival/retrieval is unavailable. A successful mutation must never
+be reported as "not executed" merely because publication failed.
+
+Enrichment or optional session-hygiene failures keep their existing semantics
+until their coordinator adapters are implemented and tested. DCP observation
+must never be allowed to restore a raw source after Gateway shaping.
+
+## Origin binding and reload
+
+`toolCallId` is necessary but not sufficient. Gateway execution identity must
+also bind the originating session/workspace and attempt/runtime epoch before an
+await boundary. The active tab or current extension runner after execution is
+not authoritative.
+
+The installed SDK currently makes extension reload during an in-flight custom
+tool a limited path: `wrapRegisteredTool()` touches the old runner after the
+tool returns and can turn the original completion into a stale-runner error.
+Gateway strict-enforce support for such cross-reload executions is therefore
+**not claimed**. Wrapper-level capture may preserve permitted bytes as an
+orphaned source, but it must not fabricate a successful delivered result.
+
+At the app layer, tab ownership already uses runtime/session/generation guards.
+Gateway bindings should use equivalent host-owned identity rather than the
+currently active tab.
+
+## Capture implications
+
+- Built-in `read`: generic `tool_result` is after truncation and there is no
+  full-output handle. Treat as limited unless a supported execution wrapper is
+  introduced; exact native paging remains the primary path.
+- Built-in `bash`: generic result is bounded, but a trusted `fullOutputPath`
+  exists on successful truncated output. Timeout/abort throw paths preserve a
+  text prefix/status but lose structured details.
+- `repo_*`: add any future capture seam inside the suite wrapper before
+  `truncateOutput`; the generic result hook cannot recover omitted bytes.
+- `ast_grep`: the suite wrapper can capture before truncation and already
+  exposes a full-output temp path when truncated.
+- Unknown/custom tools remain limited unless their concrete execution path is
+  separately proven.
+
+## Unsupported integrations in the current evidence
+
+The current Pix ACP `session/new` implementation consumes `cwd` and `_meta` and
+does not plumb the protocol `mcpServers` field into a local MCP execution path.
+MCP result capture is therefore unsupported, not implicitly covered by the
+generic hook.
+
+Browser QA runs in a child Pi process launched with `--no-extensions` plus a
+restricted extension set. The parent Gateway cannot observe Playwright DOM,
+network, or console bytes as parent `tool_result` events. Future integration may
+reuse completed subagent artifacts; it is not a direct browser adapter.
+
+## Storage/platform boundary
+
+P00 chooses no durable publication primitive. Current tests run on macOS and do
+not certify directory rename/fsync/no-clobber or cross-process quota behavior on
+Linux or Windows. Until P02 fault/platform tests exist, Gateway must not describe
+its storage as crash-durable across the supported platform matrix.
+
+## Third-party extensions
+
+The suite coordinator controls only suite-owned stages. A separately loaded
+third-party extension may register a later `tool_result` modifier and reinsert
+large/raw data. Strict enforce is unsupported for an unverified extension
+combination until a final session/provider-boundary test proves that the raw
+sentinel cannot reappear.
+
+## Evidence
+
+- `test/context-gateway/sdk-pipeline.test.ts`
+- `test/context-gateway/capture-contracts.test.ts`
+- `test/context-gateway/lifecycle-contracts.test.ts`
+- `test/context-gateway/provider-serialization.test.ts`
+- `test/context-gateway/p00-capabilities.md`
+- root `tests/tabs-controller.test.ts`, late origin-tab result contract

package/external/pi-tools-suite/docs/context-gateway-p01n-gate-review.md

@@ -0,0 +1,122 @@
+# Context Gateway P01-N gate review
+
+<!-- markdownlint-disable MD013 -->
+
+> Decision date: 7 September 2026.
+> Baseline: repository `daa1b06`, Pi SDK `0.85.1`.
+> Decision: **Do not start P02 immutable-store implementation now. The independent P01-R storeless track in plan 31 was subsequently completed without unlocking P02.**
+
+## Scope of this decision
+
+This is a gate decision, not a claim that a Gateway store will never be useful. The question is narrower: does the current evidence justify paying the P02 storage/security/quota complexity now?
+
+It does not. Native Compact already removes a large fraction of deterministic repo-result delivery, while the remaining large result classes have not yet been shown to require durable artifact semantics rather than existing native paging/temp-output mechanisms. The authorized live paired gate also does not show a consistent total-cost win large enough to justify adding a store.
+
+## Current Native Compact evidence
+
+The existing paired synthetic corpus in `test/context-gateway/benchmark.test.ts` reports:
+
+| Metric | Baseline / Prompt runtime | Native Compact | Delta |
+| --- | ---: | ---: | ---: |
+| Repo prompt guidance chars | 3326 historical | 2294 current | -1032 (-31.0%) |
+| Delivered repo-result bytes | 71,025 | 27,519 | -43,506 (-61.25%) |
+| Tool calls | 6 | 8 | +2 |
+| Continuation calls | 1 | 3 | +2 |
+| Policy refusals | 0 | 0 | 0 |
+| Full overrides | 0 | 0 | 0 |
+| Critical facts recovered | 8 / 8 | 8 / 8 | unchanged |
+
+The byte ratio is `0.3874551214`. The reduction is therefore not free: structure/AST recovery uses additional native cursor calls. Those calls must remain in the total task cost instead of being hidden behind an initial-output-only metric.
+
+The policy function itself is not a material processing bottleneck in the current synthetic microbenchmark. After warm-up, seven runs of 100,000 policy applications measured roughly `0.255–0.338 µs/call`, with six of seven runs around `0.255–0.267 µs/call`. This is only policy CPU overhead, not end-to-end tool, filesystem, provider, or task latency.
+
+## Authorized live paired gate
+
+The live gate was run with the explicitly authorized configured model `zai/glm-5.3`. Every Prompt Compact and Native Compact arm passed its behavioral assertions.
+
+The first run exposed a real Native Compact recovery regression in `tool.architecture-first`: the model requested `repo_structure --max-files 50` twice in compact mode, received two `compact-limit-exceeded` refusals, retried with `20`, and also issued an unrelated `repo_architecture outputMode=full`. That arm used 10 tool calls versus 4 for Prompt Compact and 120,336 versus 64,731 parent tokens. This violated the P01-N acceptance rule against refusal/retry loops.
+
+Native Compact model-facing argument guidance was then tightened without changing policy ceilings or refusal semantics: schema descriptions now publish the compact/full native limits and tell the model to correct the same rejected argument rather than broaden an unrelated tool. Deterministic policy tests remained green.
+
+The authorized re-run removed the loop: all six arms passed with zero Native Compact refusals, zero full overrides, and zero retry-after-refusal events.
+
+| Re-run aggregate | Prompt Compact | Native Compact | Delta |
+| --- | ---: | ---: | ---: |
+| Repo-result bytes | 1,286 | 983 | -303 (-23.6%) |
+| Tool calls | 14 | 12 | -2 (-14.3%) |
+| Parent tokens | 179,855 | 207,611 | +27,756 (+15.4%) |
+| Elapsed time | 88.682 s | 96.219 s | +7.537 s (+8.5%) |
+| Passed arms | 3 / 3 | 3 / 3 | unchanged |
+| Refusals / full overrides / refusal retries | 0 / 0 / 0 | 0 / 0 / 0 | unchanged |
+
+The per-case result is mixed rather than uniformly negative or positive. `tool.architecture-first` improved repo bytes (903 → 600), calls (8 → 5), and elapsed time (38.9 s → 30.9 s), but parent tokens still rose 67,019 → 86,298. `tool.semantic-repo-search` delivered the same 383 repo bytes while Native Compact used one extra tool call, about 10.2% more parent tokens, and about 22.8% more elapsed time. The known-file negative case preserved the required direct `Read` behavior.
+
+These live numbers are model-run evidence, not a claim of deterministic causality: the same seeded case ordering produced materially different Prompt Compact call counts between runs. The correct conclusion is therefore that Native Compact is now behaviorally safe in this small gate, but not a proven total-cost win.
+
+## Residual result classes after repo Native Compact
+
+P00 capture fixtures prove that native truncation does not make every other result small. Using the same synthetic 2,100-line fixture shape:
+
+| Path | Delivered bytes | Truncated | Full-output handle |
+| --- | ---: | --- | --- |
+| Built-in `read` | 36,053 | yes | no |
+| Built-in `bash` | 36,152 | yes | yes, temp |
+| `ast_grep` | 36,194 | yes | yes, temp |
+
+These are real residual classes relative to an 8 KiB prospective Gateway inline budget, but size alone is not a P02 requirement.
+
+- Built-in `read` already has exact `offset`/`limit` continuation; the unresolved question is historical/stable recovery, not whether current bytes can be paged.
+- Built-in `bash` already preserves a temp full-output path on successful truncation; the unresolved question is whether task quality or resume semantics require a durable permitted snapshot instead.
+- `ast_grep` already preserves a complete temp output artifact when truncated and can be captured in its suite-owned wrapper before truncation; a new store must demonstrate incremental value over that path.
+- `repo_*` broad output is already the main Native Compact target and should not be counted again as Gateway savings.
+
+## Authorized observe residual gate
+
+The explicitly authorized `zai/glm-5.3` observe run used three newly-created synthetic sessions with `contextGateway.mode=observe`, `repoDiscovery.profile=native-compact`, and an 8 KiB observation budget. The persisted report contains only aggregate Context Gateway telemetry; raw result bodies, tool arguments, project paths, and archive references are not written to the report.
+
+| Observed class | Result content bytes | Details bytes | Upstream-truncated | Over 8 KiB | Potential content bytes over budget |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| Built-in `Read` / `code-read` | 52,153 | 52,280 | 1 | 1 | 43,961 |
+| Built-in shell / `shell` | 52,211 | 52,368 | 1 | 1 | 44,019 |
+| Real `ast_grep` / `ast-grep` | 52,040 | 52,531 | 1 | 1 | 43,848 |
+
+The `Read` and `ast_grep` task-level assertions passed. The shell task produced the expected single successful, upstream-truncated shell result and then one unrelated small `other` error result, so its strict "one tool call only" task assertion failed. That extra result is kept visible rather than hidden, but it does not invalidate the measured shell residual class. The observe runner now reports `taskPassed` and `observationValid` separately so future evidence cannot confuse prompt compliance with telemetry validity.
+
+The run also exposed a cheaper residual than a new store: current SDK truncation details retain `truncation.content`. Direct SDK contracts confirm that this field contains the large already-delivered prefix/tail again for built-in `read`, built-in `bash`, and the suite `ast_grep` wrapper. In the observe samples this makes `detailsBytes` roughly the same size as the delivered content. The checked OpenAI-completions serializer does not send tool-result details to the provider, but persisted session JSONL and in-memory downstream result metadata still carry that duplication. Current DCP sidecar persistence is explicitly not a decision input here: its serializer already strips output details, and DCP is scheduled for a separate redesign. Before P02, prefer a result-metadata normalization path that removes only redundant `truncation.content` while preserving structural truncation fields and generic downstream-observer behavior.
+
+## Why P02 does not start yet
+
+P02 introduces durable sensitive-data storage, session/workspace authorization, immutable publication, hash validation, cross-process quota reservations, platform-specific atomicity/sync behavior, and later reader lifecycle obligations. The current evidence does not yet show enough residual task-level value to justify those costs.
+
+The observe-residual requirement is satisfied for the measured synthetic `Read`, shell, and `ast_grep` sessions, not for all production workloads. The metadata-normalization follow-up is a separate **opt-in module disabled by default**. It removes `details.truncation.content` for the measured tool names when the SDK-shaped metadata and delivered prefix match. Unknown names and nonmatching shapes are skipped; a matching name/shape is not proof of the provenance of an arbitrary replacement tool, which remains a P01-R compatibility boundary. Existing contracts show more than 20 KiB of JSONL fixture reduction with unchanged visible content, structural truncation fields, and checked OpenAI-completions payload. This is serialized metadata savings, not demonstrated provider-token or heap savings. Gateway off/observe alone remain non-transforming; an explicitly enabled `observe + normalizer` arm intentionally changes details and must be labeled separately.
+
+The saved guarded recovery report has three passing task assertions and sequences `Read → Read → Read`, `Bash → Read`, and `ast_grep → Read`. It predates R-A validator/report v2. Although the Bash/ast-grep harness was configured to delete the producer sources and the old validator had a real `path` versus `file_path` bug, the aggregate report does not retain the transient producer handle, exact Read arguments/result correlation, or native continuation hint required by v2. Therefore **all three historical recovery strategies are unknown under the R-A provenance contract**, including the old Read-offset strategy that v1 labeled PASS. The original report remains unchanged.
+
+This supports investigating native recovery, not a universal conclusion that storage is unnecessary. A correct recovery can still incur expensive repeated delivery; three synthetic successes do not establish temporary-output durability, satisfactory first-result size, or total-cost improvement.
+
+The following evidence is still required before reversing this No-Go:
+
+1. Establish a reproducible task and reliable provenance/oracle where the checked storeless path is insufficient: correctness, required historical lifetime, or unacceptable total delivery/cost despite successful recovery. A recovery failure is not a mandatory prerequisite, and a large first result alone is not sufficient evidence.
+2. Explain why cheaper native scope/range controls, metadata cleanup, existing permitted outputs, and retrieval of actually persisted session text do not satisfy that task. Do not assume the future plan-32 runtime already exists or that JSONL recovers bytes never persisted there.
+3. Propose a limited P02 adapter/lifetime scope and evidence that new snapshot semantics add value. Compare against the checked storeless control, count all recovery/latency, and report model variance. Unsupported paths are listed explicitly, not claimed as protected or used as measured savings.
+4. Record the decision and obtain separate implementation authorization. Only relevant R-A/R-B/R-C/R-G contracts are prerequisites; P02 does not require completing every optional storeless investigation or the DCP rewrite.
+
+## Decision
+
+**P02 remains deferred; the P01-R storeless branch is complete.** Keep store-backed enforce disabled. Do not create the immutable store, artifact readers, quotas, catalogs, or retention machinery under a different name merely to advance the roadmap. The final P01-R release decision keeps Native Compact and the metadata normalizer explicit opt-in, test/build parsing observe-only, and the other measured surfaces passthrough/limited; see `context-gateway-p01r-rg-evidence.md`. None of that is a Go for P02.
+
+Plan 32 owns the new DCP journal, session-recovery pagination, and removal of the sidecar/legacy machinery. It is independent of the completed P01-R branch. P07 remains only a conditional integration gate for a future claimed Hybrid profile. No new validator-v2 live recovery run, user-config change, or manual synchronization was required for the P01-R release scope.
+
+## Verification used for this review
+
+- `PI_CONTEXT_GATEWAY_BENCHMARK_REPORT=1 bun test test/context-gateway/benchmark.test.ts` — 2 pass, 0 fail; report values above.
+- `bun test test/repo-native-compact.test.ts test/context-gateway/benchmark.test.ts` after the refusal-loop guidance fix — 12 pass, 0 fail, 108 assertions.
+- `npm run typecheck` in `external/pi-tools-suite` — pass.
+- Policy-only synthetic microbenchmark: seven rounds × 100,000 calls after warm-up; results stated above.
+- Authorized live reports: `test/evals/artifacts/p01n-2026-09-07T17-23-48-741Z/p01n-paired-report.json` (regression discovery) and `test/evals/artifacts/p01n-2026-09-07T17-30-21-479Z/p01n-paired-report.json` (post-fix gate).
+- Authorized observe report: `test/evals/artifacts/context-gateway-observe-2026-09-07T18-00-44-288Z/context-gateway-observe-report.json`; three expected residual classes were observed without persisted raw bodies.
+- Optional non-store normalizer: `src/truncation-metadata-normalizer/index.ts`, disabled by default. `test/context-gateway/metadata-normalization.test.ts` plus the SDK pipeline contract prove conservative matching, JSONL reduction, unchanged visible content, generic downstream-observer compatibility, and a byte-equivalent checked provider payload.
+- Guarded native-recovery report: `test/evals/artifacts/context-gateway-recovery-2026-09-07T18-32-36-215Z/context-gateway-recovery-report.json`; preserved unchanged and treated as v2-provenance `unknown` for all three recovery strategies.
+- R-A deterministic evidence: `docs/context-gateway-p01r-ra-evidence.md`; corpus/validator/report v2, transient exact-handle probe, source/run SHA-256 identity, negative fixtures and safe-report contract. Focused gate: 17 pass, 0 fail, 105 assertions; suite typecheck/diff-check pass; live runner remains fail-closed without an explicitly selected model.
+- R-F lifecycle/UI evidence: `docs/context-gateway-p01r-rf-evidence.md`; Context Gateway lifecycle/result gate 49 pass, 0 fail, 337 assertions; root TUI/session 98/98; ACP lazy current-result 4/4 plus persisted-history/request-boundary 6/6 and typecheck; Desktop transcript/client 28/28 with zero check errors (two pre-existing accessibility warnings). No native-temp reader endpoint was added.
+- R-G final storeless release evidence: `docs/context-gateway-p01r-rg-evidence.md`; external deterministic P01-R gate 144 pass, 0 fail, 1011 assertions, plus root schema/typecheck/diff hygiene. Storeless defaults remain conservative and no additional live model call was required for the accepted scope.

package/external/pi-tools-suite/docs/context-gateway-p01n-measurement.md

@@ -0,0 +1,133 @@
+# Context Gateway P01-N Native Compact measurement
+
+<!-- markdownlint-disable MD013 -->
+
+> Status: P01-N paired/observe evidence is recorded; native-recovery provenance has explicit validation limitations. **P01-R storeless hardening is planned independently; P02 is deferred**, not the entire roadmap. No new implementation or live evaluation is performed by this documentation revision.
+> Repository baseline: `daa1b06`, Pi SDK `0.85.1`, 7 September 2026.
+
+## What is measured
+
+P01-N separates three effects that must not be attributed to one another:
+
+1. **Historical Baseline prompt** — the pre-trim repo tool guidance measured before `daa1b06`.
+2. **Prompt Compact** — current repo descriptions/snippets/guidelines, with the historical runtime behavior (`repoDiscovery.profile=baseline`).
+3. **Native Compact** — the same current prompts plus `repoDiscovery.profile=native-compact`.
+
+The historical pre-trim Baseline prompt is not reconstructed by loading the entire previous extension commit in live evals. The previous commit differs in code outside repo prompt text, so doing that would confound the comparison. Historical prompt size is retained as a separate textual baseline; live paired evaluation compares current Prompt Compact against current Native Compact on the same codebase.
+
+## Deterministic offline paired corpus
+
+Executable evidence: `test/context-gateway/benchmark.test.ts`.
+
+The corpus exercises five repo paths with fixed critical facts:
+
+- semantic search followed by one narrow `--include-content --max-files 1` call;
+- structure with a native `--cursor` continuation;
+- AST with a native `--cursor` continuation;
+- signature-first explain;
+- shallow deps.
+
+It is intentionally synthetic. It proves runtime byte/cursor/refusal behavior and deterministic critical-fact recovery, not model judgment or production latency.
+
+| Metric | Historical/Prompt runtime baseline | Native Compact | Delta |
+| --- | ---: | ---: | ---: |
+| Repo prompt guidance chars | 3326 historical | 2294 current | -1032 (-31.0%) |
+| Delivered repo-result bytes | 71,025 | 27,519 | -43,506 (-61.25%) |
+| Tool calls | 6 | 8 | +2 |
+| Continuation calls | 1 | 3 | +2 |
+| Native policy refusals | 0 | 0 | 0 |
+| Full overrides | 0 | 0 | 0 |
+| Critical facts recovered | 8 / 8 | 8 / 8 | unchanged |
+
+The Native Compact byte ratio in this corpus is `0.3874551214` of baseline. The extra calls are the cost of using native continuations instead of receiving the broad structure/AST output in one response.
+
+Prompt Compact is runtime-equivalent to Baseline in this offline corpus by construction: prompt text is not a model and therefore cannot change tool selection in a deterministic wrapper-only test. That distinction is explicit so prompt savings are not misreported as runtime shaping.
+
+## Runtime policy evidence
+
+Executable evidence: `test/repo-native-compact.test.ts`.
+
+- default profile remains `baseline`;
+- Native Compact compact delivery is bounded to 400 lines / 12 KiB;
+- explicit `outputMode=full` is bounded to 2000 lines / 50 KiB;
+- `--flag=value` is normalized before validation;
+- duplicate, unknown, malformed, conflicting, and over-limit flags are refused before `idx` executes;
+- native result schema uses integer/min/max bounds and does not rely on `execute()` alone for basic type constraints;
+- final UTF-8 output is bounded after execution as well as native flags before execution;
+- refusal metadata contains only allowlisted policy fields, not query/argv/body.
+
+`test/context-gateway/observe.test.ts` additionally proves that observe telemetry accepts only allowlisted Native Compact policy outcomes and does not copy query/body or forged policy data from unrelated tools.
+
+## Live paired gate
+
+The eval harness now records result byte counts plus Native Compact refusals/full overrides/retry-after-refusal without recording result bodies. `test/evals/run-p01n-paired.ts` runs Prompt Compact and Native Compact in deterministic randomized order for the same model/case and reports:
+
+- existing behavioral/tool-selection assertions;
+- repo/tool-result bytes;
+- parent/worker tokens;
+- tool-call count;
+- elapsed time;
+- Native Compact refusals, full overrides, and retry-after-refusal count.
+
+Run it only with an explicitly selected already-configured live model, for example:
+
+```sh
+PI_TOOLS_SUITE_EVAL_MODELS='zai/glm-5.3' npm run evals:p01n
+```
+
+The command refuses to run when `PI_TOOLS_SUITE_EVAL_MODELS` is unset. Live model evaluation is intentionally not folded into ordinary deterministic tests.
+
+### Authorized `zai/glm-5.3` result
+
+The first authorized live run found a Native Compact refusal loop in `tool.architecture-first`: two over-limit compact `repo_structure --max-files 50` calls were refused and retried before the model settled on the compact limit. Model-facing schema/recovery guidance was tightened without changing the enforced ceilings, and deterministic policy tests stayed green.
+
+The post-fix paired run passed every arm with zero refusals/full overrides/retry-after-refusal events. Aggregate results across the three paired cases were:
+
+| Metric | Prompt Compact | Native Compact | Delta |
+| --- | ---: | ---: | ---: |
+| Repo-result bytes | 1,286 | 983 | -23.6% |
+| Tool calls | 14 | 12 | -14.3% |
+| Parent tokens | 179,855 | 207,611 | +15.4% |
+| Elapsed time | 88.682 s | 96.219 s | +8.5% |
+| Passed pairs | 3 / 3 | 3 / 3 | unchanged |
+| Refusals / full / refusal retries | 0 / 0 / 0 | 0 / 0 / 0 | unchanged |
+
+This closes the small live/task comparison gate, but it does not establish a universal token/latency effect: model behavior varied materially between repeated runs even with deterministic arm ordering. The evidence supports keeping Native Compact available and bounded, not claiming that it always lowers total task cost.
+
+## Authorized observe residual evidence
+
+Three newly-created synthetic `zai/glm-5.3` sessions ran with `contextGateway.mode=observe`, Native Compact enabled for `repo_*`, and `maxResultBytes=8192`. The test-only recorder uses the same `ContextGatewayTelemetry` implementation as the extension and persists only its aggregate snapshot.
+
+The expected residual classes were all observed as upstream-truncated and over budget: built-in `Read` delivered 52,153 content bytes, shell 52,211, and real `ast_grep` 52,040. Potential content bytes above the 8 KiB observation budget were 43,961, 44,019, and 43,848 respectively. No tool arguments or result bodies are stored in the observe report.
+
+Metadata is also material: the corresponding `detailsBytes` were 52,280, 52,368, and 52,531. Direct installed-SDK contracts show why: truncation details include a `truncation.content` string duplicating the already-delivered truncated text. This duplication is now handled by a cheaper optional result-metadata normalizer before any durable store work; the module is disabled by default and activates only through the existing suite `modules` opt-in surface.
+
+The current DCP sidecar is not used as evidence for this optimization. Its persistence format already compacts away tool output/details, and DCP is expected to be redesigned separately. The relevant surfaces here are Pi session JSONL, in-memory tool-result metadata, checked provider serialization, renderers, and generic downstream result observers.
+
+The Read and ast-grep task assertions passed. The shell session made one extra unrelated tool call after the valid shell observation, so task compliance failed while the shell observation itself remained valid. The observe runner now keeps those statuses separate.
+
+The follow-up `truncation-metadata-normalizer` is separate and disabled by default. It removes `details.truncation.content` only for the measured tool names when the SDK-shaped fields and delivered prefix match; unknown names and nonmatching metadata are skipped. Arbitrary tool overrides matching those names are not independently certified. In a persisted fixture the duplicate sentinel falls from two copies to one and JSONL shrinks by more than 20 KiB, preserving structural fields and visible content. The checked OpenAI-completions payload is exactly equal. These are serialized metadata savings, not measured provider-token or heap savings. Gateway off/observe alone remain byte-preserving; an explicitly enabled normalizer is a separate transforming arm. Renderer/final-pipeline/security/lifecycle checks were subsequently completed in P01-R R-B/R-F; the final storeless release scope and rollback are recorded in `context-gateway-p01r-rg-evidence.md`.
+
+## Guarded native recovery evidence
+
+The next synthetic gate asked whether a fact hidden by a large first result can be recovered without a new durable Gateway source. The saved guarded report `test/evals/artifacts/context-gateway-recovery-2026-09-07T18-32-36-215Z/context-gateway-recovery-report.json` records:
+
+| Case | Task assertion | Saved strategy assertion | Tool sequence | Aggregate parent tokens |
+| --- | --- | --- | --- | ---: |
+| Read offset | PASS | PASS (v1 validator) | Read → Read → Read | 213,135 |
+| Bash temp output | PASS | FAIL | Bash → Read | 75,253 |
+| ast_grep temp output | PASS | FAIL | ast_grep → Read | 81,861 |
+
+The producer source fixtures were configured for deletion after the initial Bash/ast-grep result, and the v1 evaluator had a real `Read.file_path` versus `path` bug. R-A now replaces that logic with validator/report v2: exact producer call/result correlation, exact issued native handle, result-before-Read order, successful producer/Read outcomes, fact-bearing native source/read result, exact native offset hint for Read continuation, verified synthetic cleanup, and explicit task/observation/availability/strategy statuses. However, the saved v1 aggregate lacks the transient handle/Read args/probes needed by v2, so it cannot be replayed into a corrected live result. All three saved recovery strategies are therefore **unknown under v2 provenance**. The old report is not rewritten; a new v2 live run would require separate authorization.
+
+R-A deterministic evidence is recorded in `docs/context-gateway-p01r-ra-evidence.md`. Focused offline validation passes 17 tests / 105 assertions and records source/harness/corpus/validator identities without raw paths/body in final reports.
+
+The evidence is consistent with useful native recovery but does not prove universal storage redundancy, cheap recovery, or durable temporary files. The token figures include aggregate parent usage, not a direct measure of recovery-only input or money. First-result size, repeated delivery, metadata/JSONL size, provider traffic, and historical availability remain separate questions.
+
+## P02 decision status
+
+**P02 remains deferred.** Existing evidence does not justify the new durable-store complexity. The independent P01-R storeless branch was subsequently completed: Native Compact and the metadata normalizer remain explicit opt-in features, test/build parsing remains observe-only, and the other measured surfaces remain passthrough/limited. Completion of P01-R does not unlock P02.
+
+Likely residual classes remain only candidates, not proven P02 requirements: generic built-in `read` after its own truncation boundary, shell output where the native temp handle is insufficient for desired history semantics, arbitrary custom tools, and child browser/subagent artifacts. MCP remains unsupported in the current Pix adapter and therefore cannot be used as evidence for a Gateway adapter.
+
+Revisit P02 only for a concrete task demonstrating incremental value from new snapshot/lifetime semantics over the checked storeless alternatives. Native recovery may be correct but still too costly; failure is not the sole qualifying condition, and high cost alone does not prove a durable store will help. Record the narrower scope, comparison, uncertainty, and separate implementation authorization. Plan 32 owns the new DCP/session-recovery implementation without sidecar or legacy; P07 is only a future Hybrid integration gate.

package/external/pi-tools-suite/docs/context-gateway-p01r-ra-evidence.md

@@ -0,0 +1,111 @@
+# Context Gateway P01-R / R-A evidence
+
+<!-- markdownlint-disable MD013 -->
+
+> Date: 7 September 2026.
+> Repository HEAD during deterministic gate: `daa1b06`.
+> Installed Pi SDK: `@earendil-works/pi-coding-agent` `0.85.1`.
+> Scope: deterministic recovery provenance, run identity and native capability accounting only. No new live model run was performed for R-A.
+
+## Result
+
+R-A's deterministic infrastructure is complete. Future recovery runs use corpus v2,
+validator v2 and report v2. Exact producer-native-handle provenance exists only in
+a disposable per-run probe file inside the synthetic fixture project. The final
+report contains only status enums/booleans/counts, tool names and source-code/runtime
+identity; it does not retain tool inputs, native paths, result bodies, artifact
+contents or hidden-fact fingerprints.
+
+This closes the validation flaw in the old runner, but it does **not** turn the
+existing live report into a new successful run.
+
+## Deterministic gate
+
+```text
+bun test test/evals/recovery-corpus.test.ts \
+  test/evals/recovery-run-identity.test.ts \
+  test/evals/recovery-validation.test.ts \
+  test/evals/recovery-report.test.ts \
+  test/evals/harness.test.ts
+```
+
+Result: **17 pass, 0 fail, 105 assertions**.
+
+Additional checks:
+
+- `npm run typecheck` — pass.
+- `git diff --check` — pass.
+- `npm run evals:context-gateway-recovery` without `PI_TOOLS_SUITE_EVAL_MODELS` — fail-closed with exit `2`.
+
+## What validator v2 proves
+
+For native temp-output recovery it requires one concrete producer call and matching
+successful result, a producer-details `fullOutputPath`, a fact-bearing readable
+native file, verified deletion of the synthetic source fixture, a later `Read`,
+exact equality between that Read path (`path` or `file_path`) and the issued handle,
+a successful Read result and the expected opaque fact in that Read result.
+
+For native Read continuation it requires the same source path and the exact numeric
+`Use offset=N to continue` hint from the preceding successful result. `offset > 1`
+alone is no longer sufficient.
+
+Negative fixtures reject:
+
+- Read before the producer result;
+- failed producer or failed recovery Read;
+- unrelated temp paths;
+- repeated producers;
+- direct reads of the original producer source;
+- full-output-looking text without a producer `details.fullOutputPath`;
+- direct answer/guess without a recovery Read;
+- unverified synthetic cleanup;
+- wrong Read source or wrong continuation offset;
+- unknown recovery case IDs.
+
+## Corpus and run identity
+
+The opaque expected values live in `recovery-corpus.ts`, not in prompts. Tests prove
+that each value exists in its intended synthetic source and does not appear in the
+case prompt or generated suite config.
+
+Future report identity records:
+
+- report/harness/validator/corpus versions;
+- Git HEAD plus dirty/not-dirty state;
+- SHA-256 of `index.ts`, the tested package source tree (`src/`, `index.ts`, `package.json`), harness runner, recovery runner, validator and corpus;
+- installed Pi SDK version;
+- Node/Bun/platform/architecture;
+- effective storeless config and non-interactive Pi flags;
+- model/provider and deterministic arm order.
+
+The identity uses relative logical names plus code hashes, not host absolute paths.
+This means a dirty source tree is identified by its tested bytes rather than by HEAD
+alone, and the user's watcher is not evidence of which source revision a run used.
+
+## Historical live report status
+
+The saved guarded report at
+`test/evals/artifacts/context-gateway-recovery-2026-09-07T18-32-36-215Z/context-gateway-recovery-report.json`
+is retained verbatim. It predates the v2 transient provenance probe and safe identity.
+
+Its task assertions/tool-name sequences remain historical observations, but exact
+native recovery provenance is **unknown under v2** for all three cases:
+
+| Case | Saved v1 task/strategy | R-A v2 interpretation |
+| --- | --- | --- |
+| `recovery.read-offset` | task PASS / strategy PASS | Unknown: saved report lacks the exact native offset hint and Read args/result chain needed by v2. |
+| `recovery.bash-temp-output` | task PASS / strategy FAIL | Unknown: saved report lacks producer `fullOutputPath` and exact Read-handle correlation. |
+| `recovery.ast-grep-temp-output` | task PASS / strategy FAIL | Unknown for the same reason. |
+
+The earlier `path` versus `file_path` unit fix explains one validator bug, but is not
+evidence that the old live Read used the exact producer-issued native handle. A new
+v2 live run, if needed later, requires separate authorization and creates a new
+timestamped report rather than rewriting the old one.
+
+## Capability boundary
+
+The current native recovery/lifetime matrix is recorded in
+`test/context-gateway/p00-capabilities.md`. It explicitly distinguishes current-file
+Read continuation, native index cursor, temporary full-output handle, persisted
+session text and a future durable snapshot. Error/timeout/abort and expiry edge cases
+that R-A does not certify remain R-C work.