openmerit 0.1.11 → 0.2.0

package/docs/inference-terminal.md

@@ -20,8 +20,6 @@ Transport is best-effort, silent, has no retries, and is bounded to about 750 ms
 
 The same opt-in enables local terminal events using that process-only UUID: `terminal_visited` (sample/live mode), `terminal_page_viewed` (fixed page category, including the connections map), `terminal_interaction` (inspect, filter, search, map expansion, mode or command action and entity kind only), `terminal_snapshot` (retained model, record, source, host, deployment and distinct observed-agent counts), and `terminal_error` (fixed collection/snapshot/render/uncaught error code and standard error class only). Counts are integers from 0 through 1,000,000,000; they describe the retained view, not complete environment totals. No entity keys, names, search values, source paths, raw source, prompts, credentials, hostname, arbitrary error messages or stacks are transmitted. No session replay or login is added.
 
-This instrumentation covers the local terminal, not the hosted fictional preview at `dash.openmerit.ai`. The hosted preview remains uninstrumented and deploys separately.
-
 `GET /api/info` exposes the boolean `telemetry` capability. Browser events additionally respect browser Do Not Track, Global Privacy Control and `localStorage['scout-analytics-opt-out'] === '1'`. They go only to the same-origin `POST /api/analytics` relay; the browser CSP still permits connections only to self. The loopback Host/Origin/cross-site guards apply to the relay, which projects every accepted event onto a fixed allowlist, limits JSON bodies to 2 KiB and body arrival to one second. Forwarding has no queue or retries, permits at most 60 terminal events per minute across the process and four in flight, and times out after 750 ms. Browser submission is separately capped at 60 events per minute and two in flight with a one-second timeout. Events can be dropped without affecting collection or UI requests. Collection errors follow process consent independently of browser controls. Outbound analytics carries `X-Scout-Client: cli` and `X-Scout-Analytics: on`.
 
 ## Install
@@ -40,16 +38,10 @@ The CLI starts a server bound to `127.0.0.1` and opens the browser before initia
 
 If npm reports `EACCES`, use a user-owned Node installation/npm prefix rather than `sudo`. If the command is not found after installation, reopen the shell and check that npm's global executable directory is on `PATH`.
 
-The **terminal and hosted dashboard** follow the device's light/dark preference, including charts and inspectors, and update when that preference changes. There is no theme control or saved override. The npm terminal includes this appearance behavior in 0.1.7.
+The **terminal** follows the device's light/dark preference, including charts and inspectors, and update when that preference changes. There is no theme control or saved override. The npm terminal includes this appearance behavior in 0.1.7.
 
 Keep the CLI running while using the dashboard. Files unreadable by your user are reported as unavailable. Configuration can identify services but cannot supply missing request history.
 
-## Hosted samples
-
-[dash.openmerit.ai](https://dash.openmerit.ai/) shows 200 fictional agents, 699 task IDs, 21 models, and 9.6 million requests over 30 days. The [before/after comparison](https://dash.openmerit.ai/demo) uses a separate fictional insurance scenario with 280 workers and 980 task IDs across fourteen workflows. See [the comparison details](#hosted-enterprise-sample-and-comparison) for selection rules, preference overrides, prices, and limitations.
-
-Hosted pages do not connect to your machine, accept project uploads, or run inference. Their public JSON endpoints return fictional data; the local snapshot API is not exposed. They request no search-engine indexing. The local CLI's sample selector uses a smaller 40-agent, 140-task example described below.
-
 ## Run from a source checkout
 
 With dependencies installed in the OpenMerit checkout:
@@ -79,7 +71,7 @@ openmerit dash --no-open
 | `--demo` | Use only a fictional local sample; no local collectors run |
 | `--help` | Show usage |
 
-Ctrl+C stops collection. The terminal has no persistent daemon and is independent of Pi's pause and scheduling commands.
+Ctrl+C stops collection. The terminal has no persistent daemon. The Pi 1.5.0 Scout adapter exposes no pause or scheduling commands.
 
 ## Repository indexing
 
@@ -234,59 +226,6 @@ Torph animates changed totals, table measurements, status labels, and source cou
 
 The environment selector opens `inference-sample`, a fictional enterprise workspace with 40 agents, 140 tasks, 21 models, and 9,600 requests over 24 hours. It includes all five path classifications, traffic bursts, running and failed requests, missing measurements, partial and unavailable sources, three self-hosted runtimes, 27 distinct deployments, 14 shared capacity pools, and an hour of host history. Its six hosts include a workstation and five rack servers across two racks, with eleven accelerators listed. Four hosts have recent readings, one is stale, and one has no measurement. Seven loaded models are placed explicitly across three hosts. Six cloud models have separate `us-1` and `eu-1` example deployments. The capacity sample includes twelve recent readings, one stale reading, and one missing reading. Shared gateway pools connect consumers across reported agent IDs. Names resemble exported configuration keys (for example, `support.review` and `llm-gateway`); some agent/task display names are absent and the UI uses their IDs. These identifiers are fictional, not discovered in the current repository. The sample workstation has no site label; server sites and racks are fictional exporter fields. Baseten, Groq, and Cerebras illustrate serving services distinct from model creators; their sample configuration evidence, deployment modes, and quotas are fictional. Model rates, latency, allocation, and utilization values are illustrative; the sample is for inspecting the interface at scale, not a performance benchmark. Sample data never enters the real snapshot or project files. A command started with `--demo` only exposes a sample workspace; restart without that flag to collect real data.
 
-### Hosted enterprise sample and comparison
-
-The hosted sample at [dash.openmerit.ai](https://dash.openmerit.ai/) has a separate enterprise scenario: 9.6 million fictional requests over 30 days, represented by 9,600 groups of 1,000 requests, with 200 agent IDs, 699 task IDs and 21 models. Tokens and costs are **group totals**. Overview, model/service counts, activity, maps and inspectors use the represented request count. These sample weights never affect live request records. The local `--demo` and sample selector keep the smaller scenario described above.
-
-The comparison at [`dash.openmerit.ai/demo`](https://dash.openmerit.ai/demo) is an **insurance-tech-demo**: 280 fictional workers, 980 task identifiers, and fourteen insurance application workflows. Claim intake, policy extraction, coverage review support, document verification, renewals and correspondence use plausible application declarations. These names were not discovered from a real insurance repository. Both stages preserve the configured service set, workloads, request counts and token volume. Qualified assignments can move between those services; their request paths and service attribution update accordingly. No real claims, underwriting decisions or inference calls are involved.
-
-The comparison is the leading panel, with larger Before/After token costs, a prominent stage switch, and the modeled difference beside its evidence link. Overrides remain visible below the totals; workload decisions expand within the panel. On narrow screens the controls and result stack without hiding the evidence. This visual hierarchy does not change selection, pricing or request totals.
-
-**After OpenMerit** includes a preference selector: default within the existing stack, bias ZDR, bias open weights, no open weights, bias USA models, or bias Chinese models. Every workload searches the full reviewed catalog across the six configured services. Default chooses the lowest modeled token cost among qualifying replacements and retains the baseline if none qualify.
-
-Other preferences first select the lowest-cost qualifying match. If none exists, an already matching baseline is retained. Otherwise, a compatible matching offer may override the benchmark and efficiency criteria. These **Preference overrides** are labeled in the comparison, workload decisions and evidence drawer. The drawer lists every unmet criterion and preserves missing comparisons and known regressions. Overrides are ranked by fewest axes with known regressions, then fewest axes with missing evidence, presence of a verified efficiency improvement, and modeled cost; offer ID breaks ties. This is a disclosed demo fallback, not a measured claim that the selected model is best for the task. Missing evidence never becomes a passing score.
-
-Configured services, compatible output contracts and known token prices remain required. **No open weights** still requires every assigned model to be explicitly closed-weight. Only when no compatible closed offer exists does a workload remain unassigned and the full-plan cost stay unavailable. Other biases fall back to the default qualifying choice or baseline if no compatible preference match exists, with the mismatch visible. Overrides are included in all After views and modeled costs; request and token volumes stay fixed.
-
-The primary criteria require equal or better selected public benchmark scores for tool use, capabilities and reasoning, plus lower modeled token cost or a lower paired latency measurement. The catalog records benchmark suite, metric, score, source and publisher configuration. Missing or non-comparable evidence prevents qualification even when a preference override selects the model. Decision workflows additionally require accuracy and calibration evidence; an override must disclose missing or regressing decision metrics too. Jev is a typed-decision model, not a general text generator; the configured-service and output-contract constraints keep it out of the After plan.
-
-The reviewed catalog contains 26 model/service offers across OpenAI, Anthropic, Google, Groq, Baseten and Mistral; Jev is separately reviewed as an out-of-stack candidate. Model identity is distinct from serving service. Multiple benchmark cohorts remain separate, and every matched metric on each required axis is checked: a passing score cannot hide a matched regression. Model-card scores refer to their published configurations, not measured endpoint performance.
-
-Nine workflows qualify for a change, including four application workers that generate actuarial/reporting SQL and reconciliation/reserve-analysis scripts. These are inference inside the fictional insurance product, not coding-harness activity. Their selected capability screen is code generation: LiveCodeBench v6 in the [GLM-4.7 comparison](https://huggingface.co/zai-org/GLM-4.7), or LiveCodeBench Pro in the [Gemini 3 Flash comparison](https://storage.googleapis.com/deepmind-media/gemini/gemini_3_flash_model_evaluation.pdf). Each comparison also supplies tool-use and GPQA reasoning scores for its candidate and Claude Sonnet 4.5 baseline. Percentage and Elo scores remain separate, and these code-generation results are not substituted for general knowledge or visual-document capability.
-
-Both GLM-4.7 on Baseten and Gemini 3 Flash qualify for those four workloads. The scenario declares fixed fictional profiles: SQL workers average 14K/700 and 18K/900 input/output tokens per request; script workers average 5K/2K and 6K/2.4K, with deterministic ±15% variation. Profiles include billed reasoning output and remain identical across stages and preferences. Published rates favor Gemini for these input-heavy SQL profiles and GLM for the longer script outputs. No preference changes prices, token volumes or recorded benchmark scores. Exceptions to the benchmark and efficiency requirements remain explicit.
-
-The fixed 30-day scenario calculates $344,689.33 Before. Default selects eight model changes and one additional service-only change, with five retained baselines, producing $177,125.39 After (48.6% lower modeled token cost). Under preference overrides the complete After projections are:
-
-| Preference | Modeled token cost | Workloads with an override |
-| --- | ---: | ---: |
-| Default | $177,125.39 | 0 |
-| ZDR | $38,281.42 | 8 |
-| Open weights | $50,147.49 | 7 |
-| No open weights | $171,146.32 | 3 |
-| USA | $162,436.44 | 2 |
-| Chinese | $40,953.91 | 9 |
-
-Lower cost in a plan containing overrides is not a quality-preserving improvement claim. All six plans now cover fourteen workloads and 9.6 million requests. Closed weights assigns GPT-4.1 mini to claim tagging, fraud review support and broker correspondence with missing-comparison exceptions; claim tagging also loses the verified efficiency improvement and costs more than its baseline. The other eleven closed assignments need no override. Equivalent model mixes, if any, are disclosed instead of changed artificially.
-
-Public benchmark scores are publisher-reported point estimates in the linked configurations, **not an OpenMerit insurance-task evaluation**, statistical proof of parity, or a claim of superiority across every benchmark. For example, GPT-4.1's ComplexFuncBench score is below GPT-4o's even though it passes the selected τ-bench retail, MMLU and AIME 2024 screen. The DeepSeek comparison uses AutomationBench, Agent’s Last Exam and GPQA Diamond from its published instruct-model comparison; it is not a claim about every reasoning benchmark. Application quality and latency remain untested. The dated catalog is not exhaustive: retained means no qualifying alternative in the reviewed evidence.
-
-ZDR means an endpoint/account retention policy, not a property of weights. This scenario only treats Baseten's documented synchronous default policy as confirmed; other providers' conditional ZDR eligibility is not assumed enabled. Router, gateway and application logging require separate checks. USA and Chinese refer to the model developer's origin, not hosting location or data residency. Open weights does not imply a particular license or privacy policy.
-
-Both sides calculate token cost from published standard USD input/output rates checked on 10 October 2026. **Workload decisions** shows each Before/After model, serving service, qualifying-alternative count, per-workload Before/After token cost for the selected window, and preference match. Choices that differ from Default appear first, followed by workloads with more qualifying alternatives. **Swaps & evidence** exposes selected and rejected candidates, rates, public benchmark comparisons and methodology. Inputs are below 33K tokens per represented request; the scenario assumes uncached text, no batch discount, and no regional premium, tool charge, gateway fee, tax or infrastructure cost. Token counts, including billed reasoning output where applicable, are held fixed for comparison; actual tokenization, reasoning budgets and generated output can differ. The broader sample leaves costs unknown when no verified token price is included. It never treats local infrastructure as free.
-
-The hosted browser builds deterministic fictional records from a small same-origin seed response. `/api/sample` and `/api/demo?stage=before|after&preference=default|zdr|open|closed|usa|china` expose complete sample snapshots; `/api/sample-seed` supplies the recipe version and timestamp. The comparison metadata includes plan feasibility, model-versus-service changes, preference matches, `equivalentPreferences` for identical model/service assignments, and window totals. An incomplete After snapshot keeps `simulation.feasible: false`, contains request/model/deployment rows only for qualifying assignments, and has no projected complete-plan After cost. Its unresolved assignments and full Before totals remain in comparison metadata. Each window reports both baseline `requests` and `qualifiedRequests`; individual qualifying records and workload totals retain their After costs. Unresolved assignments retain `after: null`, without substituting their open-weight baseline. Unknown preferences fall back to default. Stage and preference survive tab navigation and are shareable in the page URL. No collector, local files, credentials or inference provider is contacted. The comparison holds its dataset fixed while you explore; reload to start a fresh scenario. The hosted scenarios are separate from local collection; exploring them does not change your installed package or application.
-
-### Optional map preview
-
-The terminal includes an opt-in React Flow + ELK map experiment. Add `?map=flow` before the page hash in your local terminal URL (for example, `http://127.0.0.1:4310/?map=flow#overview`). To try fictional enterprise data instead, use `?sample=1&map=flow#overview`. The existing map remains the default; **Original map** removes the preview flag immediately. No data migration or server restart is needed to switch views.
-
-The preview keeps the existing card dimensions and uses ELK to arrange all collapsed groups. Layout is recalculated only when the set of nodes or relationships changes, not when request counts refresh. The graph scrolls at a fixed scale rather than automatically zooming to fit.
-
-**Browse all** opens a searchable inventory of every workload, entry connection, and model/service pair in the selected scope. Selecting a stack filters that inventory to its exact members; the visible result count always shows the filter coverage. Selecting a member spotlights its actual recorded paths and brings a matching member to the front of each related stack without expanding the diagram. This does not change the total request counts. **Details** opens the existing inspector; **Clear spotlight** or Escape clears the selection. The browser sits beside the map on wide screens and below it on narrower screens, with independently scrolling lists. Keyboard users can navigate its tabs with arrows, Home, and End.
-
-This is an optional presentation experiment, not a new data source. Its separately bundled scripts and styles are loaded from the local server only when the preview is selected. It makes no inference calls and adds no collection permissions. There is additional download and initial layout cost; very large and densely connected environments still need user evaluation. The original map remains available for comparison and rollback.
-
 ## Collection sources
 
 ### OpenMerit evidence

package/docs/pi-extension.md

@@ -2,53 +2,57 @@
 
 ## Supported version
 
-The adapter targets Pi 0.87 and declares `>=0.87.0 <0.88.0`. Development uses 0.87.0 without modifying a globally installed Pi.
+OpenMerit **0.1.12** adds a Scout adapter for Pi **1.5.0**. It replaces, rather than upgrades in place, the Pi 0.87 evaluation adapter. The `@latest` commands below require OpenMerit 0.1.12 or newer. Package publication and website deployment remain independent releases.
 
-## Loading
+## Installation
 
-Install the published package with `pi install npm:openmerit`. For development
-from a checkout, load the source extension with:
+With Node.js 22.19 or newer, npm, and Pi 1.5.0 installed:
 
 ```sh
-./node_modules/.bin/pi --no-extensions -e ./packages/pi/src/index.ts
+npx -y openmerit@latest install --client pi --scope user
 ```
 
-The isolation flag avoids duplicate `openmerit_complete_intent` and `openmerit_report_signal` registrations when npm `openmerit` is also installed in Pi's user settings.
+Approve the matching browser device code and start a new Pi session. Scope defaults to `user`; use `--scope project` from the project's root for project-local setup. Each client and scope gets an independent key and approval; project keys are also separated by project. The npm package alone does not configure Pi.
 
-Use `/openmerit doctor` to confirm that the adapter and intent executor are available.
+The website's **Add to Pi** action is copy-only and sits below the Claude action, with the Pi logo before the label. It copies the installer command; it does not open a Pi deep link or prove installation. The button requires the corresponding Scout website deployment.
 
-Use `/openmerit logs` to locate the project-local audit stream, exporter failures, evidence manifests, and state while testing an end-to-end run.
+## Installed files and credentials
 
-On `session_start`, an interactive unconfigured project receives a bounded source preflight. The adapter also repeats that bounded detection after a settled Pi turn. This lets Pi build an application from an empty directory and then receive an automatic `establish_evals` intent without requiring `/openmerit setup`. Durable active-intent state prevents duplicate setup nudges. Non-interactive runs do not attempt a confirmation flow.
+| File | User scope | Project scope |
+| --- | --- | --- |
+| Native extension bundle | `~/.pi/agent/extensions/openmerit-scout.ts` | `.pi/extensions/openmerit-scout.ts` |
+| Credential-path reference | `~/.pi/agent/extensions/openmerit-scout.json` | `.pi/extensions/openmerit-scout.json` |
+| Shared Scout skill | `~/.pi/agent/skills/openmerit-scout/SKILL.md` | `.pi/skills/openmerit-scout/SKILL.md` |
+| Installer ownership record | `~/.pi/agent/openmerit-install.json` | `.pi/openmerit-install.json` |
 
-If Pi exits while an intent is active, the adapter restores its completion tool and resends that same persisted intent on the next interactive session so the harness can continue it.
+`PI_CODING_AGENT_DIR` overrides the user agent directory. The connection JSON contains a `credentialsPath`, not the key. The default private key paths are `~/.config/openmerit/pi/user/credentials.json` for user scope and `~/.config/openmerit/pi/<project-id>/credentials.json` for project scope. The project ID derives from the resolved project root; run targeted commands from that root. `OPENMERIT_CONFIG_DIR` or `XDG_CONFIG_HOME` can override the private configuration base. Do not copy credentials into the project or model instructions.
 
-For configured projects with a confirmed application target, session start reports the available model catalogue and a scheduled tick. Application instrumentation reports idempotent completed-task and metric-window signals. OpenMerit evaluates the confirmed policy before issuing work. On macOS, Pi can install a per-project user LaunchAgent after you confirm persistent execution and authorize infrastructure changes in the policy. It uses the selected authenticated Pi model, checks for due work locally every five minutes, and starts a headless Pi only when the due core check finds downstream harness work. If installation or background authentication fails, the adapter reports a scheduling gap. Other platforms still need an external scheduler.
+The extension reads credentials at session startup and registers the `openmerit` HTTP MCP server with direct tool exposure. Authorization is an HTTP header, not prompt content. Missing or invalid configuration produces a sanitized setup diagnostic, not an automatic login. The adapter performs no per-turn HTTP connection checks or credential reads.
 
-The macOS background runner is not yet product-validated for candidate work. A local runner test verifies that an insufficient baseline is checked and recorded without starting Pi. A model-driven runner is capped at 15 minutes and records a nonzero exit and active intent ID on failure. Do not infer that unattended candidate discovery or swapping has passed end-to-end validation.
+## Native ownership and shared guidance
 
-The adapter does not convert Pi's own provider usage into application metrics. Application-task signals must carry the application model, target ID, and durable evidence; application telemetry may include provider-reported USD cost, token counts, and wall-clock latency. If automatic setup is active when interactive input arrives, the input is queued and released after setup completes instead of racing the setup turn.
+Pi supplies the MCP connection, tool discovery and execution, permissions, and session management. The thin adapter adds the same Scout rule and skill used by the shared installer as structured system-prompt sections. It leaves the host's model, tool allow/exclude lists, and other prompt sections intact. This is guidance to consult Scout before application-model choices, not enforcement of tool invocation or permission to download, spend, deploy, or change models. Scout's [data-handling policy](scout.md#refresh-and-data-handling) applies to submitted tasks.
 
-Pi also exposes `openmerit_report_signal` for verified `product_task_completed`, `metric_window_available`, and `verification_window_completed` events. Evals or observability work performed by Pi can submit measured application-target metric records through this typed ingress using stable signal IDs. For baseline checks, send cumulative metric windows with sample counts and evidence references. The background wakeup only checks whether a time-based action is due; it does not collect application results by itself. `/openmerit assess` checks those stored windows locally and starts candidate discovery only when all required metrics are ready.
+## Updates and targeted lifecycle commands
 
-## Intent delivery
-
-The extension persists each intent as a custom Pi session entry, sends private extension-authored context with `display: false`, and triggers a follow-up Pi turn. The intent describes the outcome, constraints, evidence, task profile, and authorization boundary without prescribing Pi's method.
-
-Pi must finish by calling `openmerit_complete_intent` exactly once. Success without evidence is rejected. Setup, assessment, and frontier intents receive additional structured-output checks. During setup, Pi first presents its inferred metrics and sampling plan to the user. The accepted task profile records how many representative cases and repetitions each metric needs and how results are aggregated; sample counts are kept separate from optional performance thresholds.
-
-If Pi exits while an intent is active, the next session reports the recovered intent. Use `/openmerit cancel` to record an explicit cancellation before retrying; OpenMerit does not silently reinterpret interrupted work as success.
+```sh
+npx -y openmerit@latest status --client pi --scope user --check
+npx -y openmerit@latest doctor --client pi --scope user
+npx -y openmerit@latest login --client pi --scope user
+npx -y openmerit@latest logout --client pi --scope user
+npx -y openmerit@latest remove --client pi --scope user
+```
 
-The extension replaces the completion tool definition whenever an intent becomes active. Its `outputs` property therefore exposes the canonical schema for that exact intent instead of untyped JSON. Pi validates the arguments before execution and requests provider-side JSON Schema constrained sampling when supported. OpenMerit validates the canonical schema and semantic rules again when accepting the result. OpenMerit records the exact incumbent during setup, freezes accepted candidates after discovery, and carries accepted assessments and the experiment manifest into frontier calculation so these transitions do not rely on conversational memory.
+Replace `user` with `project` for a project installation. Removal and logout target that client's scope/key, not a Claude Code installation or another project. Edited or unowned files are not silently overwritten or deleted. Resolve duplicate user/project adapters or existing `openmerit` MCP entries before installing.
 
-## Model changes
+Updates are **explicit reinstall only**: rerun `install --client pi` with the same scope and start a new session. There are no Pi auto-update hooks; Claude Code update-hook settings do not apply to Pi.
 
-The extension never treats Pi's model as the application target. After policy authorization, it sends a bounded, target-bound `apply_model_swap` intent. Pi performs the application-route change and returns configuration proof; the target ID must match the confirmed application LLM target, and Pi's own model must remain unchanged.
+## SDK embedding
 
-## Jev System One
+`openmerit/pi` exports `createScoutPiExtension({ credentialsPath })` and a default extension created by that factory. SDK hosts must explicitly load Pi's own **`createMcpExtension()`** alongside the Scout extension; the Scout factory does not create an MCP runtime. The Pi CLI already supplies native MCP. The explicit credential path takes precedence over the installed adjacent connection JSON, with the shared Pi user credential path as the fallback when that JSON is absent. The host supplies its own model, tools, session and permission configuration.
 
-Jev is not integrated into OpenMerit. If configured in Pi, the intent asks Pi to use harness-configured fast decision resources aggressively for bounded ranking, triage, and judgment. Credentials remain outside OpenMerit.
+## Migration from Pi 0.87
 
-## Non-interactive modes
+Remove or disable the old `npm:openmerit` Pi package registration before loading this adapter, and explicitly remove any previously authorized background job through its existing management path. New setup does not promise to uninstall a legacy LaunchAgent. Preserve `.openmerit/` evaluation state and evidence if needed; do not delete them as a setup repair.
 
-A harness must not attempt interactive confirmation in a mode without dialog-capable UI. It should return a waiting-for-user or failed result with an actionable reason.
+The replacement has no `/openmerit` commands, `openmerit_complete_intent` or `openmerit_report_signal` tools, source preflight, evaluation lifecycle, intent recovery, scheduling, automatic swaps, or LaunchAgent. The [standalone evaluation CLI](agent-evaluations.md) is unchanged.

package/docs/roadmap.md

@@ -2,27 +2,15 @@
 
 ## Current status
 
-The Inference Terminal provides a read-only view of existing application evidence. The Pi integration coordinates supervised model-comparison trials. Neither a populated dashboard nor passing implementation tests establishes that an application has better models, trustworthy evaluations, or reliable unattended automation.
+The Inference Terminal provides a read-only view of existing application evidence. The Pi **1.5.0** adapter in OpenMerit **0.1.12** provides only native Scout MCP and shared guidance, not evaluation coordination. The standalone evaluation CLI remains separate. Neither a populated dashboard nor passing implementation tests establishes that an application has better models, trustworthy evaluations, or reliable unattended automation.
 
 ## Known limitations
 
 - The terminal does not instrument the application, call inference providers, route traffic, run evaluations, or apply model changes. It needs supported logs or exports for actual request history. Code/configuration declarations alone cannot supply past requests or spend.
 - History and inventory can be incomplete because of source availability and read/parser limits. “All time” means retained supported evidence, not a complete lifetime archive. Cloud account usage is not fetched.
 - Machine and runtime readings describe shared resources; they are not per-request resource measurements. Static endpoints do not prove deployment, availability, or physical location.
-- Hosted samples and the before/after dashboard demo are fictional. Their cost projections and public benchmark screens do not establish application quality or latency.
-- Pi 0.87 is the only implemented coding-harness adapter. Jev must be configured and invoked by the harness; OpenMerit provides no Jev client.
-- Setup asks the harness to establish task-specific evaluations and observability. Generic coding checkpoints cannot replace product telemetry, and structured-result validation cannot prove external evidence is truthful.
-- Background wakeups require a logged-in macOS user, working headless Pi authentication, and explicit persistent-execution/infrastructure permission. Other platforms require an external scheduler. Deterministic baseline checks have automated coverage; unattended candidate discovery and model changes still need representative product validation.
-- Pause persists and gates automatic work, but does not cancel active intents, stop explicit commands, or stop the standalone terminal.
-- Challenger accounting depends on reported costs. It cannot independently meter provider billing or stop an in-flight request. Use provider spending controls as well as the confirmed evaluation policy.
-- The source interface and hosted dashboard follow the device's light/dark preference. That UI change is not included in npm 0.1.6.
-
-## Next validation milestones
-
-1. Establish working evaluations and observability in a representative product.
-2. Inspect generated artifacts, measurements, and side effects.
-3. Collect repeated evidence for the confirmed profile.
-4. Complete a budgeted challenger cycle and inspect the frontier and recommendation.
-5. Perform a supervised swap, post-swap verification, and rollback drill.
-6. Validate background candidate work before relying on unattended operation.
-7. Repeat with materially different workloads before making broad reliability claims.
+- The local sample workspace is fictional. Its measurements and prices illustrate the interface, not application quality or latency.
+- The Pi 0.87 evaluation adapter is removed in source. Pi 1.5.0 setup needs explicit browser approval, a new session and an explicit installer rerun for updates. It has no lifecycle commands, scheduler, LaunchAgent or auto-update hooks. The host owns tools, model, permissions and sessions; SDK hosts must load Pi's native MCP extension explicitly.
+- Scout rule/skill availability does not guarantee invocation or measured application quality. Missing project policy requires asking; unavailable or insufficient Scout evidence requires pausing the model choice.
+- Existing legacy Pi 0.87 background jobs require separate cleanup during migration.
+- Agent-evaluation budget admission depends on reported costs. It cannot independently meter provider billing or stop an in-flight request. Use provider spending controls as well.