openmerit 0.3.11 → 0.4.1

package/docs/scout.md

@@ -1,237 +0,0 @@
-# Scout by OpenMerit
-
-Scout helps you choose application-model candidates through the
-[website](https://openmerit.ai/scout/), CLI and coding-agent MCP. It recommends;
-it does not route inference, run candidates or prove their application quality.
-Use your own evals before adopting a model.
-
-## Connect a coding agent
-
-Use Node.js 22.19 or newer. Claude Code requires the `claude` CLI on PATH;
-Pi requires version 1.5.0. Run one installer from your project:
-
-```sh
-npx -y openmerit@latest install --client claude-code --scope project
-# Or:
-npx -y openmerit@latest install --client pi --scope project
-```
-
-Approve the matching device code in the browser, then start a new agent session.
-Only approve requests you initiated. Codes expire after 30 minutes; if the
-installer returns pending, approve its link and rerun the same command to resume.
-Use `--no-open` on a remote machine. Installing the npm package alone does not
-configure either agent.
-
-User scope applies across projects. Project credentials stay outside the
-repository. For Claude, private local MCP configuration follows the Git root,
-not necessarily the directory containing the skill. With npm 0.3.9 and Claude
-2.1.63 or newer, linked worktrees share that configuration under the main checkout;
-older Claude uses the worktree root. Use the same Claude CLI for setup and sessions.
-Existing edited or unowned entries are not overwritten or migrated.
-
-See [setup](getting-started.md), [Pi details](pi-extension.md) and
-[troubleshooting](troubleshooting.md). Package publication, hosted deployment and
-your installed instructions are separate: rerun setup to install a published fix.
-
-## Commands
-
-```sh
-openmerit status --client claude-code --scope project --check
-openmerit doctor --client claude-code --scope project
-openmerit scout "Extract invoice line items as structured JSON" --all-endpoints
-openmerit scout --conversation <id> --brief input="English invoice text"
-openmerit scout --conversation <id> --sync langfuse
-```
-
-Use `--client pi` for Pi checks. `--json` returns structured CLI results.
-`--plan` previews setup/removal without applying it. `remove` removes unchanged
-owned configuration and revokes that installation's key; edited files and project
-policy remain. `logout` revokes and deletes its key; rerun `login` and `install`
-to recover an expired or revoked setup.
-
-## Scope and model choice
-
-The published CLI uses `.openmerit/scout.json` to declare accessible public
-catalogue endpoint IDs. `project --policy existing-only --endpoint-id <id>` limits
-searches to that inventory. `existing-first` tries it first and requires approval
-before broadening. `--all-endpoints` explicitly requests the whole catalogue and
-cannot override existing-only. An empty inventory grants access to no endpoints.
-Never put credentials or private endpoint URLs in this file.
-
-Scout asks for missing task requirements. Answer in the same conversation or
-explicitly use `--skip` to continue with stated assumptions. A recommendation
-names exact model/endpoint IDs, quality evidence, estimated workload cost and gaps.
-Cheapest meeting the quality bar is the default; an explicit quality or recency
-priority changes the order. The bar is an estimate, not a measured accuracy floor.
-Under the quality priority, unmeasured quality estimates within three points of
-the best one not yet placed count as tied, and the cheaper model ranks first.
-Measured models keep their measured order.
-An unmeasured candidate that another unmeasured candidate beats on both quality
-estimate and estimated cost is left out, so a shortlist can have fewer than five
-models. Candidates at the same price stay, and measured candidates are never left
-out this way.
-Unknown processing location is not evidence of residency compliance.
-
-## Close the loop
-
-1. Install, authorize and consult Scout with non-confidential task requirements.
-2. Reuse suitable evals. If none exist or they are too weak, agree on a local
-   task-specific plan before creating customer-owned evals.
-3. Run approved candidates, changing one agent at a time. Keep the quality floor,
-   prompts, tools and controls fixed; judge that agent's own checks.
-4. Connect Langfuse with a separate Viewer key and confirmed score/model mappings,
-   or report aggregates from your own system. Never submit raw records.
-5. Refresh Scout, compare measured quality, cost and limitations, and save the
-   decision. All candidates passing a tiny eval is inconclusive, not equal quality.
-
-The [close-the-loop guide](scout-close-loop.md) covers readiness paths, protected
-connection setup, aggregate schemas, mappings, sample counts, regret and
-evidence-first cleanup. Langfuse is the supported connector; other platforms can
-report aggregates. Scout does not write customer datasets, runs or scores.
-New model spend, provider credentials, deployments and deletion need approval.
-
-## Using Scout from the agent
-
-The installed startup rule and skill ask the coding agent to consult Scout before
-application-model choices, including implicit pretrained-model defaults. They
-are guidance, not enforcement. Unrelated edits, ordinary inference and the coding
-agent's own model are outside scope. Pi owns tools, permissions and sessions.
-
-With a confirmed Langfuse connection and mappings, npm 0.3.8+ instructions request
-aggregate sync before selection. Sync does not execute candidates. A failed sync
-must be disclosed; retained cached evidence is not a fresh import. Reading only a
-conversation ID retrieves its saved answer; a follow-up or sync recomputes it.
-
-## Automatic instruction updates (version 0.1.10+)
-
-Claude setup defaults to automatic instruction updates through asynchronous
-startup/response-end hooks. Checks use npm's stable registry at most once daily;
-failed checks retain the installed version and retry after 15 minutes of eligible
-use. New sessions load new instructions reliably. Disabled hooks or an exiting
-one-shot process can prevent a check or notice.
-
-```sh
-openmerit update --client claude-code --scope project
-```
-
-Checks verify package integrity and read a bounded instruction bundle; they do
-not execute downloaded scripts or replace credentials/project policy. Edited
-files are retained. `OPENMERIT_AUTO_UPDATE=off` suppresses automatic checks in that
-process, not explicit updates. Pi has no update hooks: rerun its installer.
-
-## Device keys
-
-Each client/scope approval creates its own private recommendation key, accepted on
-Scout MCP and expiring after 90 days without use. The server stores its hash.
-Local credentials and Claude's private MCP headers contain the key; protect them
-and never paste it in chat. Revoke devices on the connection page or through the
-targeted `logout`/`remove` command. Langfuse management uses a separate approval
-and key; a recommendation key does not gain deletion permissions.
-
-## Account sign-in — published, 21 October 2026
-
-Browser recommendations require Clerk sign-in. Agent MCP uses the separate device
-flow above; a browser cookie is not an agent credential. Sign-in and provider
-processing have their own policies. Failed authorization does not grant access.
-
-## Product analytics
-
-Use [Privacy & analytics](https://openmerit.ai/scout/privacy/) and the
-[data policy](scout-data-policy.md) to review collection and retention. General
-opt-out, Do Not Track and Global Privacy Control suppress optional analytics,
-not necessary task processing. Redaction is best effort, not permission to send
-confidential data. There is no session recording.
-
-### Scout setup analytics default — version 0.1.10
-
-Setup usage analytics defaults on. `OPENMERIT_TELEMETRY=0` or `DO_NOT_TRACK=1`
-disables it. Installed MCP connections do not share prompt content for analytics.
-Rerun setup to apply changed installation preferences.
-
-### Coding-agent lifecycle instrumentation — npm 0.1.9
-
-Eligible setup and MCP usage can report bounded milestones and fixed error codes,
-not keys, source files, paths or transcripts. Linked setup uses random identifiers;
-request-only unlinking does not erase earlier account links.
-
-### Version-5 instrumentation
-
-Eligible web/API/MCP events can include allowlisted diagnostic categories, actual
-computation counts and latency. Unknown cost or usage stays unknown. Delivery is
-best effort; an accepted request does not prove downstream ingestion.
-
-### Unified product and search analytics — deployed, 22 October 2026
-
-Optional PostHog analytics is separate from necessary service processing and the
-historical Cloudflare prompt archive. Retention and deletion are not interchangeable
-between those systems. See the policy rather than assuming one universal expiry.
-
-### Account profile synchronization — deployed, 22 October 2026
-
-Enabled account-linked analytics can include name, verified email, signup time
-and sign-in providers. It is identifiable, not anonymous. Account deletion is
-asynchronous and broader deletion may require contacting the policy's address.
-
-## Refresh and data handling
-
-### Task-relevant catalogue evidence — deployed, 9 November 2026
-
-The agent's internal `get_model_details` tool accepts optional `benchmarkIds`:
-up to 40 benchmark or leaderboard IDs from task research, shared across up to
-eight requested models. When supplied, only those results are returned, even
-when their standings are weaker than unrelated evidence. Split larger selections
-across calls; an empty list selects no evidence. Without this option, the tool
-keeps its strongest-40 benchmark and strongest-30 leaderboard defaults.
-
-Each model's `evidenceCoverage` reports distinct result `total` and `omitted`
-counts separately for `benchmarks` and `leaderboards`. Omitted counts include
-results outside an explicit selection as well as results beyond default caps.
-`missingRequested` lists requested IDs with no result for that model, including
-unknown IDs; absence is not a zero score. Hard constraints and endpoint inventory
-still apply: details retain `constraintViolation` rather than hiding an
-ineligible model's record or making it eligible.
-
-Equal chat benchmark scores share competition ranks (1, 1, 3), including ranks
-shown in citations. Those ranks remain among all measured catalogue models, not
-just those meeting a filter. Leaderboards keep their own ranks and direction.
-Hosted Scout has served this since 9 November 2026; it does not by itself
-establish better shortlist quality. The homepage has no FAQ block; no homepage
-claim changes.
-
-Public benchmarks, catalogue prices and vendor metadata may be incomplete or
-outdated; citations do not certify model quality, access, residency or zero
-retention. Verify the exact endpoint in your application. No claim is made that
-all catalogue models can be run in your environment.
-
-**Deployed 6 November 2026:** decision models that OpenRouter routes (supplied
-choices, ordered rubric scores and boolean probabilities through its alpha
-Decisions API) are listed alongside Vercel AI Gateway's. An OpenRouter route
-whose expiration date has passed is no longer listed; a route with a removal
-date still ahead shows it as `retiresAt`, and Scout mentions it when it
-recommends a model through that route. Other hosts' deprecation dates do not
-remove a route. Moving aliases such as OpenRouter's `~lab/…-latest` IDs are
-never treated as one model.
-
-**Deployed 6 November 2026:** more host routes are matched to the model they
-serve when a host spells the version or names the lab differently, for example
-Amazon Bedrock's `eu.cohere.embed-v4:0`, Azure's `embed-v-4-0` or Fireworks'
-`glm-5p2`. A model that OpenRouter or Vercel AI Gateway listed under its own
-spelling, such as `cohere/embed-5-fast` beside `cohere/embed-v5.0-fast`, is
-merged into the existing model, so it no longer appears twice.
-
-On enabled deployments, reported and imported aggregates belong to your account
-and apply across conversations. They are not shared with other customers. The
-connector reads experiment headers/metadata and aggregate metrics, never trace
-or observation bodies. Source/scope identifier warnings are not guaranteed
-redaction. Retention, encrypted credentials, disconnect behavior and deletion
-limits are specified in the [data policy](scout-data-policy.md) and
-[security guide](security.md). Disconnecting does not erase past conversations
-or customer-platform records; deleting those is a separate approved action.
-
-## Run locally
-
-The CLI runs on your machine but calls hosted Scout; it does not install a local
-Scout server. Use `openmerit scout --help` for published options. The standalone
-[evaluation CLI](agent-evaluations.md), [Inference Terminal](inference-terminal.md)
-and legacy `/scout/api/recommend` interface are retired. Use the conversational
-agent, CLI or canonical MCP instead.

package/docs/security.md

@@ -1,107 +0,0 @@
-# Security
-
-**Pi adapter — npm 0.1.12 and later:** Pi 1.5.0 uses native MCP plus shared Scout rule/skill sections, not the former evaluation adapter. It never initiates login, changes the host model/tools/session, or provisions background jobs. Browser-approved credentials are independent per client/scope/project, stored privately outside the repository, and passed as MCP HTTP headers, never instructions. Pi's installed connection JSON contains only the private credential path. See [Pi paths and targeted removal](pi-extension.md). The standalone evaluation CLI remains separate.
-
-**Retirement:** npm 0.3.0 removes the standalone [agent-evaluation runner](agent-evaluations.md), including planning; the Scout serving deploy of 5 November 2026 removed `/scout/api/recommend`. Existing local reports, budgets, configuration and host-agent instructions are not deleted; review them separately. No provider billing control or remote data deletion is implied.
-
-The Scout agent, MCP, publisher and account analytics/deletion capabilities remain. Historical analytics and stored catalogue/research data are not erased by endpoint retirement. Necessary provider processing for the agent and publisher continues under the [Scout data policy](scout-data-policy.md).
-
-## Credential ownership
-
-OpenMerit accepts no model-provider or Jev credential. Credentials belong to the coding harness, its environment, or an external secret store.
-
-Never place secrets in evaluation configuration, task fixtures, Scout project policy, traces, or committed environment files.
-
-### Langfuse connector
-
-The [connection guide](scout-close-loop.md#connect-langfuse) requires npm 0.3.5
-and the matching hosted deployment. Only a separate
-Viewer key pair is accepted. A successful read plus a deliberately invalid
-dataset-create permission probe (HTTP 403 required) fails closed for Full keys;
-no valid write payload is sent. Live Viewer connection acceptance was verified
-on 6 November 2026; fresh mapped import/reranking remain separate release gates.
-Fixed US/EU/HIPAA hosts, manual redirects, timeouts and bounded reads
-prevent credential forwarding and unbounded response bodies. The connector reads
-only experiment headers/metadata and aggregate Metrics v2 values, never trace,
-observation, experiment-item or raw-score bodies. Extra metadata fields are
-discarded; experiment source labels are hashes, not names or record IDs.
-
-Credentials and mappings are AES-GCM encrypted in one Cloudflare Durable Object
-per authenticated user, bound to that object's ID with a dedicated Worker secret.
-Keys never enter browser storage, URLs or analytics. Keep that encryption secret
-stable: replacing it invalidates stored connections and requires reconnecting.
-With npm 0.3.6 and the matching deployment, authenticated settings-only updates
-reuse and reverify the saved encrypted key without exporting it. Supplying a
-partial key pair fails; first connection still requires a protected Viewer pair.
-Changing settings invalidates imported aggregates and requires syncing again.
-Disconnect deletes the key and imported aggregates; deleting all evaluation data
-also deletes reported aggregates. Neither erases conversation text. Retention,
-limits and processor caveats are in the [data policy](scout-data-policy.md).
-
-## Local state
-
-`.openmerit/` may hold evaluation configuration, task fixtures, reports and budget usage, and Scout project policy, plus files left by the retired Inference Terminal (`terminal.json`, `inference.jsonl`), which upgrades do not delete. It may reveal task names, model choices, budgets or endpoint identifiers. Treat it as sensitive project metadata even though it must not contain credentials.
-
-With native Scout, Pi manages its own session/tool records and provider context; the adapter adds no separate evaluation-history store. Task descriptions sent to hosted Scout follow its data policy. Local OpenMerit storage does not imply that all processing stays on your machine.
-
-## Evidence integrity
-
-OpenMerit does not establish the truth of imported or external evidence. Use authenticated observability, immutable artifacts, and digests where stronger provenance is required.
-
-Do not include credentials, private traces, or proprietary evaluation data in public issues.
-
-## Hosted Scout authentication
-
-Hosted Scout uses Clerk Google/GitHub sign-in. The page and status are public; JSON recommendations require a valid Clerk session bearer token, not just a cookie. MCP methods require a device key from the setup below. Missing/invalid credentials return HTTP 401, and missing authentication configuration fails closed with HTTP 503. Clerk remains the identity system; the analytics integration below can synchronize selected profile fields to PostHog. Raw account IDs are used transiently for the per-user edge limiter and are not joined to historical Cloudflare prompt records. A prompt draft is temporarily stored in the current browser tab during sign-in. No Clerk Billing or automatic agent OAuth is included. See [Scout account sign-in](scout.md#account-sign-in-published-21-october-2026) for token lifetime, rate limits, data handling and the remaining real-login validation limits.
-
-### Coding-agent device keys
-
-The [coding-agent setup](scout.md#connect-a-coding-agent) in npm 0.1.7 uses hosted device sign-in instead of OAuth or tokens in prompts. The copied prompt contains no secret. A signed-in person approves an eight-character code on `/connect`; codes expire after 30 minutes (ten before the Scout serving deploy of 5 November 2026) and are single-use. Approval mints one key per device and client, scoped to `scout:recommend` and accepted only on `/scout/mcp`. The server stores only SHA-256 hashes. Keys expire after 90 days without use and can be revoked on `/connect` or with `openmerit remove`/`logout`. Locally the key is in `~/.config/openmerit/credentials.json` (mode 600) and in Claude Code's plain-text MCP configuration; the CLI never prints it. As with any device-code flow, only approve a code you started.
-
-**Automatic instruction updates — version 0.1.10:** updater-enabled setup defaults on with an opt-out, installing asynchronous Claude Code startup/response-end hooks. Checks use npm's public stable registry without credentials, task content or installation IDs. SHA-512 verification checks package bytes against registry metadata; it does not defend against a compromised publisher/registry. The updater reads only a bounded instruction bundle, never executes downloaded code or package scripts, and leaves MCP configuration, keys, privacy settings and project policy unchanged. A supported release contract permits instruction updates; maintainers must change it when permissions or data handling change. This is a publisher assertion, not semantic validation of instructions. Edited files are retained, concurrent writes serialize, and journaled recovery stops on intervening edits. New sessions reliably load new instructions. See [automatic updates](scout.md#automatic-instruction-updates-version-0110) for cadence, offline retries, manual commands and the one-time bootstrap required for older installations. Availability follows npm publication independently of hosted releases.
-
-These update hooks are Claude Code only. The new Pi adapter installs none; updates require explicit targeted reinstall. Its `login`, `logout`, `status`, `remove` and `doctor` shell commands target the selected client/scope without revoking another installation's independent key.
-
-### OpenMerit coding-agent reset — 23 October 2026
-
-**Historical reset, superseded by npm 0.1.7 and hosted device setup.** The reset temporarily removed the installer and MCP transport. The current [Claude Code setup](scout.md#connect-a-coding-agent) uses browser-approved, device-specific keys scoped to MCP recommendations, not pasted session tokens or agent OAuth. The key is stored locally in mode-600 credentials and Claude Code's plain-text MCP configuration. Setup installs an owned Scout skill; conflicts are not overwritten, and removal revokes the device key. Browser Google/GitHub sign-in remains separate.
-
-**npm 0.1.8:** setup also installs an owned, always-loaded `rules/openmerit-scout.md` instruction requiring Scout before implicit or explicit application-model choices, including pretrained/local/browser models and SDK defaults. Edited or unowned rules are not overwritten or removed. Missing policy requires asking; unavailable Scout or insufficient coverage requires pausing model selection. This is model guidance, not an enforced tool gate or authorization to download weights, add providers, spend or deploy. Scope, device-key permissions and task-data handling are unchanged. New sessions are required after upgrading; older npm 0.1.7 installations remain skill-only until upgraded.
-
-Scout serving, main-site documentation and npm publication are separate releases. This reset does not revoke remote grants or clean up files previously installed on other machines. Earlier hosted MCP/authentication descriptions in this guide refer to past deployments, not this build. Browser Google/GitHub sign-in and its Clerk account lifecycle are separate and unchanged.
-
-## Hosted Scout analytics
-
-**Version 0.1.10:** [Scout setup analytics](scout.md#scout-setup-analytics-default-version-0110) defaults on without a prompt prefix; npm publication and hosted prompt deployment remain separate. `OPENMERIT_TELEMETRY=0` and `DO_NOT_TRACK=1` suppress setup analytics and propagate to approval and installed MCP usage. Browser privacy choices and account opt-outs still apply. Terminal/evaluation analytics remain opt-in, and installed MCP prompt-content capture remains off. This supersedes the setup opt-in default below for version 0.1.10 and later.
-
-**Coding-agent lifecycle instrumentation — npm 0.1.9 and matching Scout deployment:** setup clicks, clipboard/deep-link attempts, device authorization states, opted-in installation/component verification and each dispatched MCP invocation gain typed milestones and fixed error codes. Linked setup uses random setup/installation UUIDs, never paths, machine names, device codes, keys or transcript content. The browser handoff explicitly carries usage/unlink choices into setup; standalone CLI analytics remains opt-in. Setup/installation IDs are omitted in unlinked mode, DNT wins, and installed MCP prompt-content capture stays off. Request-only suppression no longer overwrites the saved account-wide preference. Explicit account opt-out/deletion gates remain in force. See the [lifecycle contract](scout.md#coding-agent-lifecycle-instrumentation-npm-019) for event meanings, bounded delivery retries and diagnostic counters. Existing installations require an installer rerun; release acceptance is separate, and no historical completeness, replay, transcript collection or skipped-call detection is claimed.
-
-**Version-5 instrumentation:** the [serving contract](scout.md#version-5-instrumentation) adds validated native web events and explicit CTA-only `$autocapture` mapping, not general DOM capture. Acquisition/UTM/device values are categorical allowlists; public URL properties are reconstructed without raw URLs. Browser performance is sampled and support-dependent. Linked same-tab signup/search identity is separate from unlinking's request-only identifiers. Logical MCP errors count even with HTTP 200; stage/provider codes, actual attempt tokens and feedback/retry traces exclude raw messages and free text. CLI diagnostics are opt-in in npm 0.1.7, adding sanitized version/platform/runtime and bounded application basename frames, never absolute paths or environment contents. Prompt sharing, retention, deletion gates and necessary provider processing are unchanged. Replay is a separate, inactive spike; no experiment is active. Main-site publication remains blocked by documented source divergence; the Scout rollout does not publish npm or activate alert notifications.
-
-**Beta evaluation rollout, 22 October 2026:** The serving configuration enables `SCOUT_BETA_EVALUATION_LOGGING=true`, alongside `POSTHOG_CAPTURE_CONTENT=true` and complete account-sync/deletion configuration. The browser defaults content on only without a stored content choice; saved off, general opt-out, DNT/GPC and unreadable storage keep it off. API/MCP still require `X-Scout-Analytics-Content: on`; CLI content is always off. This is product evaluation/debugging, not foundation model training. Deployment is owner-approved; this is not legal certification, and beta status establishes no consent or legal basis. Main-site `/docs/` publication remains blocked by source divergence; the [policy](/scout/data-policy/) is generated from the same Markdown and published with Scout instead.
-
-The schema confines all prompt and task excerpt/five-layer prose to `$ai_trace.$ai_input_state={prompt,tasks}`, never ordinary `scout_search_details`. Best-effort redaction retains the 4,000-character prompt, 1,000-character per excerpt/layer and three-task limits. [PostHog's official policy](https://posthog.com/docs/ai-observability/data-retention) stores these AI content properties in `posthog.ai_events` for 30 days, stripping them from normal events at ingestion; metadata follows normal retention. Synthetic live-project records confirmed storage separation and `retention_days=30`, not eventual expiry. Historical v3 details may still hold content. There is no custom retention cron or export, retroactive deletion claim or exact backup-erasure guarantee.
-
-The **Don’t link this browser’s activity to my account** control sends `X-Scout-Analytics-Account: off`. After account opt-out/deletion gates, the server uses fresh unrelated random identifiers, ignores previously linked browser IDs and supplied trace IDs, omits the response trace ID, and sends no profile/identify operation for that request. This is browser/request-specific, not account-wide: Clerk profile/lifecycle analytics can continue under general usage consent, past records remain linked, and copied coding-agent instructions do not inherit the setting. Unlinked events cannot support cross-request funnels or automatic account-based historical deletion; text itself may identify someone. Restrict provider access to authorized personnel and disclose processors, rather than claiming no third parties. The [Scout data policy](/scout/data-policy/) lists providers, deletion limitations and the verified data-request contact, [lazim@openmerit.ai](mailto:lazim@openmerit.ai).
-
-**Deployed, 22 October 2026:** [unified PostHog analytics](scout.md#unified-product-and-search-analytics-deployed-22-october-2026) replaces new Analytics Engine writes with account-linked recommendation metadata, bounded public evidence and sanitized exceptions. Prompt/breakdown text requires separate explicit client opt-in plus configured profile-deletion support; redaction is best effort, not a guarantee against personal or confidential data. General opt-out/DNT/GPC suppresses new analytics, but not necessary OpenAI processing. Existing Cloudflare archives remain under their original policy and are not exported, backfilled or erased.
-
-The historical Analytics Engine archive contains submitted task prompts, task categories and ordered shortlists with metrics and ranking evidence. Its prompt text is unredacted and may contain personal information or secrets; treat this dataset as potentially sensitive. No request object, prompt hash, additional free-text summary, error message, project inventory, IP address, cookie, user agent, visitor ID or session ID was separately recorded in this dataset. Requests still reach the hosting infrastructure and task descriptions still go to OpenAI for Decisions interpretation, Structured Outputs metadata extraction, transient query embeddings and relevant compatibility/applicability checks; public repository metadata and benchmark definitions/methodology also go to Decisions during hourly preparation. Their policies are separate. See [Scout data handling](scout.md#refresh-and-data-handling).
-
-Historical records are accessible only through authenticated Cloudflare account queries, with sampled three-month retention. Missing prompt/evidence fields cannot be recovered. Scout exposes no public history/query endpoint. Both historical and current source evidence and prompt text remain untrusted data when retrieved; never treat them as instructions.
-
-### Optional product analytics
-
-Scout has a visible privacy disclosure and controls. Public homepage/docs navigation events remain source-only; npm 0.1.7 includes opt-in local terminal/map events and aggregate counts, which npm 0.3.0 removes with the [Inference Terminal](inference-terminal.md). The terminal relay cannot mutate project or inference state, retains loopback Host/Origin guards, and forwards only bounded allowlisted telemetry. Terminal and CLI use a process-only random ID; website events use a separate tab ID. Neither is implicitly linked to Clerk. Application names, raw inputs/outputs, local identifiers and source files remain local. Sanitized error classes/categories (and Scout application file/line frames) exclude raw messages, credentials, source context, arbitrary URLs and filesystem paths. No replay is enabled. All expanded account-linked events pass the same preference/deletion gate as profile synchronization; ordinary event delivery remains best effort. New search details link to identified accounts when profile sync is active, unlike the legacy Cloudflare archive. No external retention setting is changed.
-
-The [PostHog integration](scout.md#product-analytics) uses the EU ingestion endpoint; unconfigured deployments remain off. Verified Clerk user IDs become namespaced SHA-256 pseudonyms for cross-client account retention. Profile sync can associate those identities with names and verified email addresses, so they are identifiable personal data. No identity is added to the historical Cloudflare prompt archive. The browser retains an anonymous ID and hashed account/session transition markers in tab storage, rotating the anonymous ID on account changes. There is no autocapture, replay or visitor-IP forwarding. PostHog has its own retention/access policies, not Cloudflare's three-month policy.
-
-Do Not Track, Global Privacy Control and the documented browser/API opt-outs suppress PostHog collection. Local CLI telemetry in npm 0.1.7 requires `OPENMERIT_TELEMETRY=1`; `DO_NOT_TRACK=1` overrides it. It uses a process-only random identity and creates no identity file. CLI planning requests propagate that consent to Scout while always disabling analytics content capture. These product opt-outs do not disable necessary provider calls or erase historical records. Operators remain responsible for retention, access, spend limits and any required visitor consent. Anonymous visits and rejected requests from the hosted rollout are confirmed in PostHog storage; authenticated-session tracking, profile sync and actual Clerk lifecycle delivery remain unverified.
-
-The deployed [account sync](scout.md#account-profile-synchronization-deployed-22-october-2026) supports identifiable PostHog profiles containing only name, verified primary email, signup time and connected Google/GitHub providers. Signed webhook verification, per-account Durable Objects and a project-scoped PostHog deletion credential are configured. Names/emails come from Clerk's current backend record, never client input. The most recent authenticated analytics preference controls background forwarding; opted-out browsers send a boolean privacy-control request, not a product event. Unknown preferences buffer only bounded pseudonymous lifecycle metadata for up to seven days. Cloudflare stores preference state, hashes, receipts and tombstones, not raw Clerk IDs or a copy of account profiles. No identity is joined to the historical prompt archive.
-
-Clerk webhook signatures are verified over bounded raw request bodies with the SDK's Svix-compatible verifier. A session bearer token cannot authorize webhook delivery. Stable delivery IDs and serialized account state protect against retries; current Clerk lookups prevent stale profile updates. Account deletion persists a tombstone before requesting asynchronous PostHog person/event/recording deletion, including for opted-out accounts. Daily alarms repeat deletion to reconcile already-accepted but delayed upstream ingestion; zero matching persons is a valid no-op, not proof of erasure. Deletion still works when new capture is disabled. Operators must inspect completion and replay failed deliveries. Old `$set` event properties are not erased merely by clearing a current profile field. The installed person-deletion key is restricted to the intended PostHog project and `person:write`; dashboard/query management credentials are not deployed. Production account deletion has not been exercised, and no retention settings or historical records were changed during rollout.
-
-## Scout research storage
-
-Scout's twice-daily refresh reads public sources directly and no longer uses a research service. Decisions processes task descriptions and relevant public evidence. Previously collected public research citations, semantic annotations, publication completion receipts and immutable catalogue versions are stored in Scout's private Cloudflare R2 bucket. Public endpoint/model facts and observations also enter the private D1 serving projection. Vectorize stores public benchmark methodology embeddings, not visitor query vectors. Query embeddings are used transiently; OpenAI has its own API retention policies. Scout uses the prompt-and-evidence analytics policy above. API credentials are held as Worker secrets and are not included in browser assets, catalogue objects or analytics. Source text is treated as evidence, not executable code or instructions. Citations and semantic validation do not independently verify that publisher claims are true.

package/docs/testing.md

@@ -1,114 +0,0 @@
-# Testing and validation
-
-**Cleanup:** npm 0.3.0 retires the standalone evaluation CLI and the Inference Terminal; the Scout serving deploy of 5 November 2026 retired the legacy JSON recommendation pipeline, and unreleased source retires the unpublished docs website shell. Dated evaluation/API acceptance records below describe historical releases, not current supported commands. Current verification retains agent, MCP, publisher, status, policy and protected-memo coverage; it does not publish or deploy anything.
-
-**Pi adapter — npm 0.1.12 and later:** the Pi 1.5.0 adapter uses native Scout MCP and shared guidance only. Dated acceptance records below describe the packages tested at the time; some mention package exports that have since been removed. Standalone evaluation tests remain separate.
-
-## Stable 0.1.7 acceptance — 24 October 2026
-
-Published `openmerit@0.1.7` to npm's `latest` channel after full repository verification: 757 passing tests, one existing skip, syntax/type checks, builds and documentation checks. The `preview` tag remains `0.1.7-preview.5`. A fresh registry installation on macOS ARM64 with Node.js 22.23.1 matched the tested 104-file tarball byte-for-byte. Packed core/protocol/Pi exports and the installed terminal runtime smoke checks passed, including passive collection, source removal, runtime outage/recovery and shutdown.
-
-The live Scout button opened a prefilled Claude Desktop Code session through Chrome. That session ran `npx -y openmerit@latest install --client claude-code`, opened the real device-approval page, and completed after approval of the matching code. This was a clean setup after removing and revoking this test's earlier source installation, not reuse of an existing key. Status then reported the MCP server and skill installed and connected; credentials had mode 600 and other MCP servers were preserved.
-
-A fresh Desktop session received an application-model request that never mentioned Scout. It loaded `openmerit-scout` and called the real `scout_models` tool with the saved empty inventory, returning zero candidates. It asked before broadening; after separate approval, its second MCP call returned five candidates and it produced a provisional recommendation. No application code, providers, candidate inference or deployments were performed by that session. Anonymous MCP access and use of the device key on the JSON API both returned 401, while authenticated tool discovery succeeded.
-
-These are integration observations, not a general guarantee that every agent follows the skill. Both search responses reported stale/degraded source data, including incomplete Parallel research. Their approximately 228,000- and 538,000-character payloads exceeded Desktop's inline tool limit; Claude recovered through saved tool-result files. The flow works, but these checks do not establish catalogue freshness, task accuracy or small response size. A separate recovery-path check exposed an obsolete preview install command in the MCP 401 message; its stable-command regression test failed before the source correction and passed afterward. Hosted correction is a separate deployment from the npm release.
-
-## Published preview.5 acceptance — 17 October 2026
-
-Published `0.1.7-preview.5` to npm's `preview` channel after all **679 repository tests**, syntax/type checks, builds and documentation checks passed, including 336 Scout tests. Registry integrity matches the reviewed tarball and stable `latest` remains `0.1.6`. All 101 packaged files are accounted for; runtime bytes, public exports and legal metadata are unchanged from preview.4. Version metadata and public guides carry this release. The frozen package contains the preparation record; this post-publication result is maintained here after the registry and VM checks.
-
-A fresh registry installation in the existing Ubuntu ARM64 Mac Lima VM passed **203/203 assertions** with Node.js 22.23.1 and the official MCP SDK. Package hashes, registry-lock integrity, public core/protocol/Pi exports, live hourly status, tool discovery, seven HTTP/MCP workloads, inventory restrictions and request/protocol errors passed. The seven workloads cover visual category selection, bird identification, safety labels, safety plus rewriting, SQL source generation, photographed invoice JSON and unsupported audio. SQL and safety rewriting each returned five candidates through both interfaces; unsupported audio correctly returned none. The installed CLI's email and SQL plans contained one and two candidates respectively, restricted to the two declared connections. The existing global installation was preserved, and planning executed no recommended models or evaluation budget.
-
-The seven HTTP workload samples had a median of **3.392 seconds** and a maximum of 3.682 seconds. The slowest of all fourteen HTTP/MCP workload requests took **4.376 seconds**; none timed out. Several exact MCP repeats used the result cache. These small samples establish tested integration and matching behavior, not a production latency percentile or exact task accuracy. Scout remains independently deployed with unchanged UI and hourly preparation. Earlier bird/invoice execution limitations and failed acceptance attempts below remain recorded.
-
-## Hosted matching repair — 17 October 2026
-
-The repair passed full repository verification, including 336 Scout tests and documentation checks, plus eight live-provider staging filter/ambiguity cases checked against the catalogue metadata oracle. The isolated Ubuntu ARM64 VM staging run passed 203/203 assertions using the installed npm preview.4 package, HTTP API, official MCP SDK and CLI planning. Both declared-connection CLI workloads had candidates: email returned one, SQL returned two. All original seven workloads passed through both interfaces. Package hashes/exports, inventories and request/protocol errors passed; no recommended models were executed. The package version and npm tags are unchanged.
-
-Earlier staged attempts are retained: the first run exposed an email retrieval gap after SQL was fixed; another reused the CLI project directory and failed the expected fresh-directory `eval init` assertion. The final run corrected test isolation without changing that assertion. This record does not replace the failed preview.4 post-publication acceptance below. Production rollout results are recorded after deployment. These are matching and integration checks, not proof of task accuracy or a production latency percentile.
-
-The first production rollout passed its 203 VM checks, but the separate filter sweep passed 6/8: impossible context returned unavailable once and explicit SVG produced contradictory task fields. A generic bounded Decisions consistency correction was then added. Full verification passed with 336 Scout tests; the revised staging run passed all 203 VM checks and all eight filter cases. The SVG case exercised the reconciliation path. These phases are recorded separately rather than treating the earlier failures as successful attempts.
-
-The final production deployment passed **203/203 VM assertions and 8/8 filter/ambiguity cases**. All seven HTTP/MCP workloads, both CLI plans, package hashes/exports, exact inventories and request/protocol errors passed. The seven HTTP workload samples had a median wall time of **3.665 seconds**, compared with **4.924 seconds** in the earlier published-preview failure run. Maximum HTTP time was 4.268 seconds; the slowest HTTP/MCP request was 4.840 seconds. Several exact repeated MCP requests used the isolate-local cache; the books/posters MCP request did fresh work, so cache availability is not assumed. No task timed out in this final run. These are small end-to-end observations under different load and cache conditions, not a production percentile or guaranteed response time. The hourly schedule and all 127 UI source files are unchanged; five checked live scripts/styles match source bytes. npm remains preview.4 and no new package was published.
-
-## Preview.4 release checks — 17 October 2026
-
-The published `0.1.7-preview.4` package passed the full repository verification, including 329 Scout tests and documentation checks. Its packaged runtime is byte-for-byte unchanged from preview.3; version metadata and public guides carry this release. Registry integrity matches the reviewed tarball, the `preview` tag resolves to this version, and stable `latest` remains `0.1.6`. Scout deploys independently and its UI files are unchanged. Broader evidence coverage, task calibration and complete quality/cost/latency validation remain follow-up work.
-
-The hosted beta's public post-deployment suite passed 9/11 cases. Explicit SVG generation returned no shortlist; a Qwen-family code-authoring request admitted tool-calling evidence. Restriction checks passed for creator, family, context and dates, including known matches and impossible-context no-match. The actual bird prompt also displayed five recommendations in a browser. Earlier candidate deadline and interpretation failures were recorded separately at the time. These checks do not establish universal semantic reliability or improved task accuracy.
-
-Fresh registry installation in the existing Ubuntu ARM64 Lima VM verified all 101 packaged file hashes, registry integrity and public core/protocol/Pi exports. Seven workloads were sent through both HTTP and the official MCP SDK. Six of those fourteen requests failed their functional expectations: books/posters and invoice HTTP requests returned unavailable responses, while SQL generation and safety classification with rewriting returned empty shortlists through both interfaces. The matching MCP books/posters and invoice requests succeeded; bird identification, label-only classification and unsupported audio behaved as expected. The first harness stopped when it assumed SQL had returned an endpoint. A separate continuation tested exact endpoint scope using the successful bird result, request/protocol errors and installed CLI planning, preserving all original failures. Together the runs recorded 187 passing checks and nine failed assertions, including duplicate agreement failures caused by unavailable responses and an installed-CLI candidate check. Both CLI email and SQL plans were empty. Planning created no execution or spend state, and the existing global VM installation was retained. This is not a clean acceptance pass. The seven first HTTP workload requests had a median wall time of 4.924 seconds and a maximum of 6.229 seconds, including unavailable responses; repeated cached requests were faster.
-
-Post-deployment refresh validation exposed duplicate stable research identities. The hosted integration now updates those exact identities without introducing model or benchmark exceptions. Full verification then passed 331 Scout tests. The normal recovery refresh reused all 24 saved research jobs, completed all 24 and published its catalogue with a matching durable receipt and empty backlog. Its interpretation and staging steps needed memory-limit and vector-readiness retries; successful publication does not establish sufficient memory headroom at larger scale. Live status reported neither stale nor degraded data after recovery. This repair deploys independently of the already published npm runtime.
-
-## Published preview.3 acceptance — 16 October 2026
-
-Published `openmerit@0.1.7-preview.3` under npm's `preview` tag after all 596 repository tests and the syntax, type, build, documentation and whitespace checks passed. Stable `latest` remains `0.1.6`. Registry integrity matches the tested tarball. Runtime files are unchanged from `0.1.7-preview.2`; this release updates packaged Scout guides and excludes internal website/deployment notes. Scout's hosted research and matching service deploys independently of npm.
-
-A registry installation in a separate directory in the existing Ubuntu ARM64 Lima VM, with Node 22.23.1, passed all 202 acceptance assertions, including SHA-256 verification of all 101 packaged files. Public core/protocol/Pi imports and the installed Pi extension load passed. The existing global installation was retained.
-
-Seven workload descriptions were tested through both the live HTTP API and the official MCP SDK: books/posters classification, open bird identification, safety labels, safety labels with explanation/rewrite, SQL generation with a runtime schema, photographed-invoice JSON extraction, and unsupported audio transcription. The operation, modality and output requirements agreed across channels. Inventory restrictions, invalid requests, MCP discovery/ping, and endpoint selection passed. Installed `eval init` and `eval plan --scope existing --budget 1` produced two frozen task shortlists limited to the declared connections; no execution, budget ledger or background work was started.
-
-Across these 14 prompt requests, observed median wall time was 1.43 seconds and the slowest was 4.14 seconds. Source status reported neither stale nor degraded data. This verifies packaging and API/MCP integration, not model task accuracy, universal semantic reliability, a guaranteed response time, or a new hourly-refresh trial. The earlier hosted refresh acceptance remains separate. No website deployment is included in this npm release.
-
-## Standard verification
-
-```sh
-npm ci
-npm run verify
-npm pack --dry-run
-```
-
-`verify` runs workspace tests, syntax checks, strict TypeScript checks and the TypeScript migration ratchet, the package build, memo tests, policy/memo build and Markdown-guide checks, Scout tests/build, and whitespace validation. It does not publish a package or deploy the website. `npm run check:worker-types` separately checks generated Worker types; CI also performs packed-install, CLI help and Pi load smoke checks.
-
-## Pi load and package smoke tests
-
-CI packs the package, installs it in a temporary directory, imports `openmerit/pi`, checks the installed `openmerit --help`, and loads the installed extension in Pi with `--list-models`. These checks establish packaging and loading, not successful provider authentication or Scout recommendations.
-
-## What evidence establishes
-
-- **Unit verified:** deterministic behavior has automated coverage.
-- **Runtime smoke verified:** the runtime loads the extension or executable.
-- **Product trial verified:** a representative application completes the workflow with inspected evidence.
-- **Production verified:** repeated representative runs support the claimed operating conditions and reliability.
-
-Implementation checks and public model benchmarks cannot substitute for target-product validation. See [current limitations](roadmap.md).
-
-
-## Agent-evaluation preview validation — 15 October 2026
-
-The npm preview `0.1.7-preview.1` package adds a separate agent-evaluation loop. `npm run verify` passes 393 tests, plus syntax/type/build/documentation checks. Fifteen evaluation-runner tests cover fresh workspaces, frozen input bytes, scope acknowledgement, final-output parsing, balanced/adaptive repeats, revision compatibility, process timeout, invalid graders, connection errors, cumulative unknown-cost reservations, corrupt ledgers, file collisions, locking and watch coalescing.
-
-An isolated installation of the actual npm tarball completed init, real hosted Scout planning and a disabled-budget check. Twelve simulated agent attempts through that installed command verified two CLI revisions, six attempts each, compatible deltas and persistent accounting. Simulated metrics are packaging tests, not model-performance evidence.
-
-Real macOS smoke runs used Claude Code 2.1.7, Codex CLI 0.162.0-alpha.17.2 and OpenCode 1.18.30. On one tiny JavaScript normalization fixture, with three fresh attempts per route, Claude/Haiku 4.5 and Codex/GPT-6 Luna passed all executable checks. OpenCode/Gemini 2.5 Flash Lite passed one of three. Costs remained unknown for Codex. These different model/harness combinations validate execution and grading paths; they are not an unbiased comparison of agents or a general model ranking. Earlier smoke runs deliberately exposed unsupported authentication/model access, native permission failures, a timeout and invalid JSON. Those cases informed connection-error handling and conservative partial-cost accounting. No model-specific ranking rule was added.
-
-Scout Worker `7ddeacd3-1310-48bd-b546-60fcab4fa4ef` passed live HTTP/MCP checks for pre-ranking model eligibility, empty inventory and duplicate-ID rejection. npm publication is separate from the successful tarball installation; the registry credential returned HTTP 401 during release preparation.
-
-
-### Published preview on Linux — 15 October 2026
-
-Installed `openmerit@0.1.7-preview.1` from npm into a separate prefix in the existing Ubuntu ARM64 VM. The live Scout API returned a frozen shortlist within the declared OpenRouter connections. A deterministic CSV grader scored quoted commas, escaped quotes, leading-zero identifiers and field types; the uncorrected and corrected CLI passed 8/12 and 12/12 checks respectively in grader sanity checks.
-
-Claude Code/Haiku 4.5 completed three attempts per revision at 66.7% then 100% mean field accuracy. The combined watcher stopped on budget admission before OpenCode’s last post-change attempt, so its report has no automatic comparison. A separate OpenCode/Gemini 2.5 Flash Lite watcher completed three attempts per revision at 66.7% then 100%, detected the source edit, emitted token/cost/time deltas, and exited cleanly. Earlier partial trials remain separate. Codex/GPT-4.1 Mini could not execute filesystem tools under this VM’s Linux sandbox; no passing Codex task result is claimed. Total account usage was approximately $0.26 including diagnostics, while native reported estimates differed. One authored fixture and three repetitions do not establish general performance or a cost improvement. Source and diagnostic artifacts are retained in the task’s `openmerit-vm-eval` directory; a scan found no temporary key in the 66 test-project files.
-
-### Accounting corrections on Linux — unreleased source, 15 October 2026
-
-The source package was packed and installed into a separate VM prefix; the registry preview and existing VM installations were left unchanged. All 400 repository tests pass, including 22 eval-runner tests. The added cases cover partial usage/cost recovery, missing prices, conservative old/new ledgers, scored wrong answers, native budget and permission blocks, zero-exit sandbox failures, and the free Linux sandbox probe. Replaying the previously captured Claude budget response recovers 30,820 input tokens and 196 output tokens instead of zero.
-
-The real Codex connection now stops during sandbox preflight with `environment_blocked`, zero attempts, no reserved allowance and no inference spend. A deliberately small Claude budget stops after one attempt with unknown quality, 21,532 input tokens, 190 output tokens and a retained $0.110705 partial native estimate. A second normal-run attempt also hit its native cap; its unscored record and $0.1505768 reservation were preserved. With sufficient per-attempt allowance, Claude completed three scored CSV attempts at 100%, a 12.82-second median, and a $0.06029 mean native estimate. This is not a billed-dollar measurement or a before/after Claude comparison.
-
-OpenCode's real watcher completed three attempts per revision. The earlier CLI averaged 44.4% field accuracy (one actual zero-score task output remained included); the corrected CLI averaged 100%. Median times were 5.05 and 5.79 seconds. One earlier and two later attempts had incomplete native step pricing, so the cost means and cost delta correctly remained unknown while partial estimates were retained. The cumulative reservation stayed at $0.03. These checks validate accounting and failure separation on the same integration fixture, not general task quality, model ordering or savings. Sanitized reports, package and verification output are retained in the task's `openmerit-eval-accounting` directory.
-
-### Budget setup — unreleased source, 15 October 2026
-
-The eval suite now has 29 tests. New coverage checks the three budget choices after scope, custom amount validation, explicit unlimited configuration, reuse of saved choices, automatic reservations, retained cumulative usage, bounded repetitions and cancellation. A real CLI subprocess uses a loopback Scout fixture to verify persistence across replans and failed requests, rejection of missing or invalid noninteractive choices, and lock cleanup. An interactive terminal smoke test also verifies scope-before-budget, custom input, no-repeat questions, no fixed cap and EOF cancellation. These are offline workflow tests; no model calls, provider charges or npm publication were required.
-
-### Published preview.2 acceptance — 15 October 2026
-
-Published `0.1.7-preview.2` to the npm `preview` tag after all 407 verification tests passed. Stable `latest` remains `0.1.6`. A fresh registry installation in a separate Ubuntu ARM64 VM directory matched the tested package integrity and all 124 packaged files. The installed command rejected a missing budget, asked scope before the custom-budget question, saved all three budget modes, reused saved choices, and preserved the ledger across limit changes. No install-time evaluation or background service was added.
-
-On a fresh CSV fixture, the real foreground watcher completed three attempts per agent per revision: twelve scored attempts total. With Claude Code/Haiku 4.5, mean field accuracy changed from 66.7% to 100%, mean input/output tokens from 87,826/666 to 87,781/653, native estimated cost from $0.09333 to $0.06534 per attempt, and median elapsed time from 13.87 to 13.42 seconds. With OpenCode/Gemini 2.5 Flash Lite, accuracy changed from 44.4% to 100%, tokens from 20,093/306 to 22,726/511, estimated cost from $0.001501 to $0.001896, and median time from 4.97 to 5.72 seconds. One baseline OpenCode output correctly remained a zero-score task failure. Both revised tools passed all checks; better quality did not imply lower cost or faster execution on every route. These are native estimates and one integration fixture, not reconciled per-attempt bills or a general model ranking.
-
-A deliberate native Claude budget stop retained 21,516 input tokens and a $0.0286866 partial estimate, with unknown quality and no revision comparison. Codex's Linux sandbox probe stopped before inference with zero attempts and no reserved spend. The initial sandbox fixture had an empty route list, which planning correctly rejected; declaring the intended Codex route allowed the separate sandbox check without repeating paid trials. Across the acceptance run, provider-reported account usage increased by approximately $0.17, below the $1 test allowance. All three projects released their locks, the watcher exited, and a scan found no temporary credential in 57 test files. Reports and registry verification are retained in the task's `openmerit-preview2-release` directory. The website was not deployed as part of this npm release.

package/docs/troubleshooting.md

@@ -1,50 +0,0 @@
-# Troubleshooting
-
-The Pi Scout adapter requires OpenMerit **0.1.12** or newer and targets Pi **1.5.0**. Check the resolved package version when using `@latest`. The removed Pi 0.87 `/openmerit` workflow is not a diagnostic path for the replacement.
-
-## Scout is missing in Pi
-
-Run targeted checks in your shell:
-
-```sh
-npx -y openmerit@latest status --client pi --scope user --check
-npx -y openmerit@latest doctor --client pi --scope user
-```
-
-Use `--scope project` from the original project root for project setup. In a project inside a larger git repository, Claude Code project setup up to npm 0.3.1 ends with "Claude Code did not save the openmerit MCP server" although Claude Code saved it under the repository root; Scout still works in that repository, and npm 0.3.2 reads the entry correctly; rerun the install with it. npm 0.3.1 and 0.3.2 also end every Claude Code install with `ENOENT … dist/terminal/src/updates.ts`: the MCP server, skill and rule are installed, but automatic instruction updates stay off. Rerun the install with npm 0.3.3 or later. Check Pi and Node.js versions against [setup](getting-started.md), complete browser device approval, then start a **new session**. Installing the npm package or reconnecting an old session does not load new extension instructions. For SDK embedding, load Pi's `createMcpExtension()` explicitly alongside `createScoutPiExtension`; the Scout factory alone is not an MCP runtime.
-
-## The connection configuration is invalid
-
-The installed extension reads `openmerit-scout.json` beside its bundle; that file points to a private credential file outside the repository. Do not put the key in the extension or prompt. Check the scope and configuration-directory overrides against [installed paths](pi-extension.md#installed-files-and-credentials). Reinstall explicitly to repair missing owned files:
-
-```sh
-npx -y openmerit@latest install --client pi --scope user
-```
-
-Expired or revoked credentials require targeted `login` followed by reinstall. The adapter reports a sanitized setup diagnostic; it never starts sign-in on its own. Approve only a browser code you initiated.
-
-## Installation reports a conflict
-
-The installer does not overwrite edited or unowned files. Inspect and preserve your changes before resolving the conflict. Also check for overlapping Pi user/project installations or an existing `openmerit` MCP entry. Remove the unwanted owned installation using its own scope; do not revoke a different client's key as a repair. Disable the old `npm:openmerit` Pi registration when migrating from 0.87.
-
-## The adapter did not update automatically
-
-That is expected. Pi has no OpenMerit auto-update hooks. Explicitly rerun `install --client pi` with the original scope, then start a new session. Claude Code's update settings do not configure Pi.
-
-## `/openmerit` or evaluation tools are missing
-
-They were removed, not renamed. The Pi adapter now supplies only native Scout MCP and shared guidance. It does not set up evaluations, recover intents, schedule work, swap models, or install a LaunchAgent. Existing legacy background jobs must be removed separately; installing the new adapter is not evidence that they stopped. Preserve old evaluation state and audit files if needed. For new evaluations use [the standalone CLI](agent-evaluations.md).
-
-## Scout was not consulted or its evidence is insufficient
-
-The rule and skill are model guidance, not a guaranteed tool gate. Confirm they loaded in a new session and that the host permits the MCP tools. When project model policy is missing, ask the user; when Scout is unavailable or lacks suitable evidence, pause the model choice rather than invent a result. Host model/tool/session restrictions remain authoritative. Scout candidates are not measurements of your application's quality.
-
-## `openmerit dash` or `openmerit eval` is an unknown command
-
-Both are retired in npm 0.3.0; see the [Inference Terminal](inference-terminal.md) and [evaluation CLI](agent-evaluations.md) retirement notes. Existing local files are not deleted. Never mark missing or rejected evidence successful merely to advance a workflow.
-
-## Report a reproducible problem
-
-Include OpenMerit, Pi and Node.js versions, the client/scope, the triggering command, and a sanitized error. Remove credentials, private paths, prompts and proprietary data before sharing. Do not attach credential or connection files containing private configuration.
-
-[Open an issue on GitHub](https://github.com/openmerit-ai/openmerit-core/issues)