openmerit 0.1.12 → 0.2.0

package/docs/scout.md

@@ -2,7 +2,7 @@
 
 Scout is OpenMerit's model search agent. Describe your task and get model recommendations. Scout returns up to five model-and-endpoint candidates per task. Comparable measurements support performance ranking; narrow decision and vision tasks use an explicitly labelled task-fit shortlist when quality is unmeasured. The `/scout/` page and JSON API use the same index and ranking.
 
-Scout is a separate hosted beta product. Npm 0.1.7 introduced optional Claude Code setup for it. OpenMerit 0.1.12 adds native Scout access for Pi 1.5.0, replacing the old Pi evaluation adapter without changing the core coordinator, standalone evaluation CLI or Inference Terminal. Recommendations are evidence-based candidates for your own evaluation, not a guarantee of task accuracy or an independently validated state-of-the-art ranking.
+Scout is a separate hosted beta product. Npm 0.1.7 introduced optional Claude Code setup for it. OpenMerit 0.1.12 adds native Scout access for Pi 1.5.0, replacing the old Pi evaluation adapter without changing the standalone evaluation CLI or Inference Terminal. Recommendations are evidence-based candidates for your own evaluation, not a guarantee of task accuracy or an independently validated state-of-the-art ranking.
 
 **Website rebuild — published 24 October 2026:** the rebuild preserves the existing landing page at `/` and Scout at `/scout/`, and takes all hosted `/docs/` pages offline, including this guide. `/connect/`, the JSON API/MCP and the standalone `/scout/data-policy/` remain available. The guide remains maintained in Markdown; this does not remove the local CLI or its documentation from the package. See the [cutover notes](site-domain-migration.md) for publication and verification details.
 
@@ -34,6 +34,20 @@ Pointer navigation uses a short, non-looping opacity/position transition. Keyboa
 
 Design references: [Linear's welcome screen](https://mobbin.com/screens/62773d0f-3574-4191-ac91-eb8a51697a95), [Linear's onboarding flow](https://mobbin.com/flows/cbaf58a9-fd84-4afe-b214-e65b24484013), [Notion's focused setup screen](https://mobbin.com/screens/99396b7b-c21d-4cea-8956-4942c2b9aae6), [Neon's integration walkthrough](https://mobbin.com/screens/8b8b58fe-86fd-4a2a-a5ad-51b84614d7b2), and [Torph's text-continuity examples](https://torph.lochie.me/). The implementation keeps Scout's own graphite palette and system typography; it does not embed Mobbin or generated mockup assets or add an animation dependency. The Claude Code image comes from the owner's 24 October recorded session: two unaltered content crops of `claude-automatic-expanded-full.png`, `(1330, 1029)–(2810, 1103)` and `(1330, 1552)–(2810, 1770)`, stacked with a 42-pixel neutral gap and encoded as WebP. OS chrome, account information and unrelated session content are excluded.
 
+## Tasks per model, and one agent for every client — npm 0.2.0
+
+Once the brief is settled, Scout splits the work into tasks, one per model it needs (for example transcribing calls, then tagging the transcripts), and recommends up to five models for each task with its own summary. Steps one model call performs well stay one task. The page allows at most three tasks, so its answer stays readable; Scout groups steps to fit. A single-task answer shows one table without a task heading; several tasks show one numbered table each.
+
+Coding agents now use the same Scout agent as the page, instead of the older single-call search. Claude Code and Pi reach it through the `scout_models` MCP tool, which runs one conversation turn per call and returns `status`:
+
+- `needs_answers`: Scout's `brief` and `questions`. The agent shows them to the user, then calls again with the `conversationId` and `brief` set to the rewritten components, or `skip: true`.
+- `recommended`: `tasks[]`, each with `title`, `work`, `input`, `output`, `summary`, `workload`, `priority` (`cost` by default, or `quality`, `latency` or `recency`), `qualityBar` and up to five ranked `models`. Coding-agent conversations have no task limit. Each model's `cost` names the cheapest eligible route's `endpointId` and `providerModelId`.
+- `working`: the call ended before the turn did. Calling again with only the `conversationId` reads the outcome.
+
+The tool takes `prompt`, `conversationId`, `brief`, `skip` and, on the first call, `endpointIds`. The inventory limits every search and shortlist in that conversation, whatever filters the agent's model chooses; an empty list allows none, and later turns cannot change it. `POST /scout/api/agent` accepts the same `endpointIds`. `modelIds` is no longer accepted, and the response no longer has `tasks[].slots`. A turn usually takes one to three minutes, much longer than a search. Conversations are keyed by the device key's account. The local `npm run scout` preview has no agent and reports MCP calls as unavailable.
+
+npm 0.2.0 adds `openmerit scout` for agents without MCP and for people at a terminal. It calls the same tool with this device's key and applies the project's saved endpoint inventory. Without `--json` it prints each task's models in ranked order with their quality estimate and list prices. As on the page, the default cheapest-first order is unlabelled; when the user asks for the best quality, the fastest or the newest model, a line names that order, and fastest and newest lists also show each model's time per task or release date (unknown when missing). The installed instructions tell the agent to relay Scout's questions rather than answer them itself.
+
 ## Every model kind in the agent — source only, not deployed
 
 This describes source changes that are not yet published or deployed. Until both the publisher and the serving Worker are deployed and a refresh has published a new catalogue version, the agent behaves as described in the next section.
@@ -65,7 +79,7 @@ In the shortlist, the Input and Output columns keep per-million-token prices; a
 
 The conversational agent is the Scout page at `/scout/`, replacing the search box and its results table. The page stays open to everyone; sending a message requires sign-in, as searching did, and a message typed while signed out is kept while you sign in. The JSON search API (`/scout/api/recommend`), the MCP server (`/scout/mcp`), coding-agent setup at `/connect/` and the catalogue refresh are unchanged, so the CLI and installed coding agents keep their current behaviour. Sections below that describe the search page's prompt examples, results table and loading messages are historical; the Claude Code setup handoff remains available in the new page.
 
-You describe the work; Scout writes a task brief with five components (input, work, output, criteria and constraints), each marked stated, assumed or missing. Each component is editable in place: correct anything Scout got wrong, or fill in a missing one. All five fields share the same editable background, including Work; the page does not show per-component question text. Once sent, the brief collapses. **Continue** sends only the components you changed and requests recommendations immediately, using stated assumptions for missing details. It also works without edits; there is no separate Skip questions button. Pressing Enter in a brief field does the same; Shift+Enter adds a line break. The message box stays at the bottom of the page. Each turn ends with an editable brief or a shortlist of five models, followed by Scout's summary. While a turn runs, its steps are listed with the current one last; afterwards they stay as a collapsed record of the work. The shortlist reports Scout's 0–100 quality estimate with the cited benchmark ranks, input and output prices per million tokens on its cheapest eligible route and release date. Expanding a model shows its rationale, benchmark ranks, route and reference latency. A missing release date is shown as unknown. Reply in the same conversation to change constraints or trade-offs; **New** starts over.
+You describe the work; Scout writes a task brief with five components (input, work, output, criteria and constraints), each marked stated, assumed or missing. A component holds only what is known, without listing unknowns. Criteria and constraints are filled only from what you explicitly say, such as "best quality output" or "open weights only"; otherwise they stay empty and missing. Each component is editable in place: correct anything Scout got wrong, or fill in a missing one. All five fields share the same editable background, including Work; the page does not show per-component question text. Once sent, the brief collapses. **Continue** sends only the components you changed and requests recommendations immediately, using stated assumptions for missing details. It also works without edits; there is no separate Skip questions button. Pressing Enter in a brief field does the same; Shift+Enter adds a line break. The message box stays at the bottom of the page. Each turn ends with an editable brief or a shortlist of five models, followed by Scout's summary. While a turn runs, its steps are listed with the current one last; afterwards they stay as a collapsed record of the work. By default each shortlist is the five cheapest models that maintain the task's output quality. Scout sets a 0–100 quality bar for the task and estimates each candidate it assessed (up to ten) on the same scale. Candidates below the bar are dropped, the rest are ranked by estimated cost per task (Scout's estimate of the task's workload at the route's list prices) and the cheapest five are shown, so a costlier model appears only when fewer than five cheaper ones reach the bar. A cost that leaves part of the workload unpriced ranks after complete ones. Only when the user explicitly asks for the best quality, the fastest or the newest model are the qualifying candidates ranked by quality estimate, reference time or release date instead. The default ranking is not labelled on the page; an explicit priority is named in a line above the table. The bar and the estimates are Scout's judgement from benchmark standings, not measured accuracy. The shortlist reports Scout's 0–100 quality estimate with the cited benchmark ranks, input and output prices per million tokens on its cheapest eligible route and release date; a latency ranking adds a time per task column. Expanding a model shows its rationale, benchmark ranks, route and reference latency. A missing release date is shown as unknown. Reply in the same conversation to change constraints or trade-offs; **New** starts over. When Scout is updated while the page is open, a notice at the bottom of the sidebar asks you to refresh, and clicking anywhere on it reloads the page; an open page notices on its next turn or when you return to its tab, and checks `GET /scout/api/version`, which returns the deployed version ID.
 
 The agent runs in a Cloudflare Durable Object, one per conversation, using Pi Durable and the Agents SDK. Its model is set by `SCOUT_AGENT_MODEL`: `openai/<model>` calls OpenAI's Responses API directly with the Worker's `OPENAI_API_KEY` (default `openai/gpt-6-sol`), and any other value runs on Workers AI, such as `@cf/zai-org/glm-5.3`. Its tools search benchmark descriptions by embedding (OpenAI embeddings, as in search), read leaderboards and model records from the published catalogue, and use Workers AI web search when the catalogue cannot answer something decisive. Cost, latency and recency figures come from catalogue data, not from the model's text, and a shortlist that cites models or benchmarks outside the catalogue, or breaks a stated hard constraint, is rejected and resubmitted. A turn usually takes one to three minutes.
 
@@ -220,23 +234,9 @@ The existing edge limiter applies separately to IP addresses and authenticated u
 
 Local development now also needs `CLERK_PUBLISHABLE_KEY` and `CLERK_SECRET_KEY` in the server environment. Configure a Clerk development instance with Google/GitHub sign-in and sign-up; disable other sign-in methods and unnecessary required profile fields. Use the production instance and its own OAuth applications for `openmerit.ai`. Never expose the secret key in client configuration or commit it. The deployed production modal was checked at desktop/mobile sizes, and both buttons reached provider authorization redirects requesting basic identity/email scopes only. Signed-out submissions opened sign-in without a recommendation request and preserved the tab's prompt draft. Hosted API/MCP checks rejected anonymous, invalid-token and cookie-only requests. Tests stopped before provider authentication: complete sign-in/sign-up, callback return and real-user authenticated searches remain unverified. Offline signature tests are not evidence of a complete Google/GitHub login.
 
-## Canonical promotion — published, 21 October 2026
-
-The [canonical Scout](https://openmerit.ai/scout/) now replaces the older interface and matcher with the former alpha: original prompt excerpts, one shared task-sequence table, automatic ambiguity resolution and cost-first selection after the relative quality screen. Canonical endpoints remain `/scout/api/status`, `/scout/api/recommend` and `/scout/mcp`. Existing `/scout/alpha/` page, asset and API/MCP URLs redirect to the corresponding canonical paths with HTTP 308, preserving query strings and POST methods/bodies. Clients should update their configured endpoint rather than rely on redirects. There is no separate alpha interface after promotion; retired `/scout/v1/` and `/scout/v2/` paths remain unavailable.
-
-Only serving changes. The existing hourly catalogue publisher, research budget, data and secrets remain intact; the unreleased collection changes below are not activated. Search retains the alpha analytics dataset and existing three-month data-handling policy. The homepage, terminal and protected main application are outside this deployment. Candidate counts can remain below five per task, timing may be unknown, and searches can exceed five seconds. Promotion does not establish task accuracy or jointly evaluated workflows. This promotion supersedes the alpha-only scope in the dated release records below.
+## Former alpha URLs
 
-Fresh canonical browser searches returned five, nine and thirteen candidates for one, two and three tasks in 6.91, 6.44 and 6.71 seconds, using interpreter 37 without result-cache hits or clarification questions. A real MCP request through the old alpha URL redirected successfully to `/scout/mcp`. Redirects retained queries and POST bodies; untrusted origins remained rejected. Deployed page/script/style hashes matched the build, and desktop/mobile renders were inspected. The publisher's deployment and hourly schedule, main/router/terminal deployments and unrelated routes remained unchanged. Full verification passed 692 tests. These smoke checks establish the canonical routing and UI contracts, not a latency or task-accuracy guarantee.
-
-## Task sequences — published alpha, 21 October 2026
-
-The [alpha](https://openmerit.ai/scout/alpha/) UI uses a normal Model/Quality/Cost/Time table for one task. For two or three tasks, each task becomes a column in one shared table. Read each row left to right as a sequence of model choices, with quality, reference cost and time beneath each model. Column headings quote short excerpts from the submitted prompt rather than rewriting the work. The five-layer breakdown, role labels, frontier badges and capability tags remain hidden. The table scrolls internally when needed; narrow screens retain the side-by-side order instead of stacking separate task tables.
-
-Rows align each task's independently ranked alternatives; they are **not jointly evaluated workflows**, a global ranking of model combinations, or verified model-to-model dependencies. The top row takes each task's first choice under its existing ranking policy. Five is still a maximum, not a quota: a shorter shortlist leaves an explicit gap, and no model is repeated to complete a sequence. The same model can legitimately appear in different task columns with different evidence and workloads. Expand any model for its task-specific evidence. **Assumptions & limitations** retains quality-screen status, workload, warnings and specialist discovery. Unknown values remain unknown; calibration accuracy is distinct from relative benchmark scores. Costs are USD per 1,000 reference tasks, not exact dataset charges, and are not added into a workflow quote.
-
-Interpreter version 37 requests a short `tasks[].sourceText` excerpt in the existing decomposition call and checks that it occurs verbatim in the prompt. When every excerpt has a unique position, those positions establish task order while keeping restrictions and models attached; otherwise the interpreter's task order is retained. An invalid excerpt becomes `null`, without blocking the shortlist or asking a follow-up question; the UI then uses an unquoted interpreted title (or the original full prompt for a single task). Long headings are visually shortened, with their full text in the hover hint, accessible text and expanded model details. The existing five-layer data and `tasks[].slots[0]` remain; no additional model call, dependency analysis, recommended-model execution or ranking-policy change is introduced. This change is published only to alpha; stable Scout is unchanged.
-
-Fresh hosted browser checks returned five, ten and fourteen candidates for one, two and three tasks in 7.66, 7.43 and 7.12 seconds. All three searches used interpreter 37 without result-cache hits, preserved original prompt excerpts and cost-ordered model lists, and displayed no clarification questions. A separate uncached MCP request returned five coding models. The deployed HTML, JavaScript and styles matched the checked build; desktop and 390px mobile captures were inspected, including horizontal scrolling and model evidence expansion, with no browser errors or page overflow. Full verification passed 692 tests before publication; deployment repeated 349 Scout tests and 13 documentation tests. Stable Scout, the main site, domain routing and the hourly publisher were unchanged. These checks do not establish recommended-model task accuracy, a fixed five candidates per task, or the under-five-second target.
+The former alpha now serves canonical Scout at `/scout/`, `/scout/api/status`, `/scout/api/recommend` and `/scout/mcp`. Old `/scout/alpha/` page and API URLs redirect there with HTTP 308, preserving queries and POST bodies. The retired `/scout/alpha/mcp` path returns 404; reinstall coding-agent setup to use `/scout/mcp`.
 
 ## Newer document evidence — deployed, 22 October 2026
 
@@ -279,7 +279,7 @@ Similar tasks can still receive identical shortlists when one benchmark is their
 
 ## Common quality estimate and labelled tradeoffs — deployed, 22 October 2026
 
-This change replaces the cost-first policy below. Quality screening previously compared a model only with models that had exactly the same set of accepted benchmarks. That split the catalogue into many two- or three-model groups, so the best of a small, weak group passed the screen and cost-first ordering promoted it.
+This change replaced an earlier cost-first policy. Quality screening previously compared a model only with models that had exactly the same set of accepted benchmarks. That split the catalogue into many two- or three-model groups, so the best of a small, weak group passed the screen and cost-first ordering promoted it.
 
 Scout now fits **one latent quality estimate across every connected evaluation** for the task. Each benchmark has its own difficulty offset, score scale and noise, so a near-zero benchmark and a widely spread one contribute on comparable terms. Correlated series in one benchmark family share that family's weight. Copies of one evaluation from different publishers, such as GPQA Diamond reported by Epoch AI, Artificial Analysis and model developers, form one family and count as one evaluation; their scores stay separate series. Evidence from the task's own domain carries at least three quarters of the weight when present, and evidence that scores the task's own kind of work counts twice as much as other domain evidence. General proxies fill in the rest and connect models that lack domain results. Each model receives an estimate with a standard error; a benchmark run by only one model links nothing. Applicable calibration observations join as one further evaluation using effective quality. Models with no evidence connected to the main group receive no estimate and remain provisional; they are never compared numerically. The estimate is reported as `comparison.qualityEstimate` and `comparison.qualityStandardError`; `quality` is its relative percentile among the estimated models.
 
@@ -289,215 +289,6 @@ Reference time previously required measurements bound to the selected endpoint.
 
 These remain relative benchmark signals, not task accuracy. An offline replay of the bundled 15 October catalogue, with simulated evidence retrieval, returned frontier coding and reasoning models first instead of the lowest-cost members of small cohorts; it did not execute any recommended model. Ranking added about 6 ms per search locally.
 
-## Cost-first selection — published alpha, 21 October 2026
-
-The source change above supersedes this policy once deployed.
-
-The [alpha](https://openmerit.ai/scout/alpha/) ranker puts the **lowest known reference cost first after a relative quality screen**. It keeps models within two proxy-score points of the strongest model with the same evaluation coverage, using the existing quality-indifference band. At least two distinct models must share that scope; duplicate routes do not create an independent comparison. Supported Pareto dominance still removes inferior configurations before the list is limited to five distinct models. Latency breaks price ties, followed by optional metadata preferences and quality. Extra quality, a soft preference or an inferred priority does not justify a higher price rank. Explicit metadata restrictions remain mandatory.
-
-This is a conservative interim policy, **not a verified “good enough for your task” threshold**. A best-in-cohort benchmark score may still be inadequate for the task. Unrelated evaluation scales are not compared numerically, and the relative display score is not used as an accuracy floor. Even a user-stated target such as 95% accuracy remains unverified without matching task evaluations; Scout neither invents such a target nor claims that the proxy band satisfies it.
-
-Missing or disconnected quality evidence cannot displace or pad a qualifying shortlist. If no comparable cohort exists, Scout retains an explicitly unverified discovery list rather than treating unknown quality as a pass. `ranking.frontier.qualityGate` reports `proxy-qualified` or `unverified`, the margin, and configuration counts. Unknown prices sort after known prices; unknown latency is not zero. Costs still use the disclosed reference token workload, not a quote for the user's exact images, execution loop or dataset. Native task-fit specialist lists remain unmeasured, not quality-qualified recommendations. No additional model calls, collection jobs or UI controls are introduced.
-
-Local real-provider browser checks returned five, eight and thirteen candidates for one-, two- and three-task searches in 6.69, 4.70 and 7.57 seconds, all cache misses. The invoice task retained three distinct models after screening and dominance; it was not padded to five. Every returned benchmark shortlist was ordered by reference cost. A single-model inventory displayed an unverified shortlist, not a quality pass. Desktop and narrow light-theme renders were inspected. These tests use the local catalogue and do not execute the recommended models, establish task accuracy, validate hosted storage or meet the under-five-second target universally.
-
-After deployment, hosted browser searches returned three, nine and fourteen candidates for one, two and three tasks in 5.50, 6.59 and 6.26 seconds. An additional MCP image-classification request returned four models in 5.08 seconds. All four requests used real OpenAI calls and production R2/D1/Vectorize without result-cache hits; every benchmark shortlist was cost-ordered and contained no clarification. A single-model inventory remained explicitly unverified. Desktop and mobile captures were inspected. The hosted catalogue and retrieved evaluation scopes differ from local checks; candidate counts and identities are not invariant across interpretations. Full verification passed 691 tests. Stable Scout, the main site and the hourly publisher were unchanged. These samples do not establish exact task accuracy, universal reliability or the five-second target.
-
-## Direct model lists — published alpha, 21 October 2026
-
-Alpha resolves ambiguity internally and returns one ranked shortlist per independent task, without clarification questions or answer buttons. One, two and three tasks target five, ten and fifteen candidates respectively. Each list contains distinct models; the same model may suit more than one task. Explicit restrictions and evidence requirements still apply, so fewer candidates can be returned when the verified frontier or selected inventory is too small. Scout does not invent models or measurements to fill a list.
-
-Steps toward one deliverable remain one task. Scout chooses the most likely intended output when it is unstated, marks inferred task layers, and leaves optional handling policies unspecified. It preserves explicit labels, boxes, masks, formats, provider restrictions and exclusions. Missing runtime data does not trigger a conversation. More than three independent tasks still returns HTTP 422; provider failures remain errors rather than fabricated recommendations.
-
-HTTP and MCP retain the existing response envelope: each `tasks[]` entry has its five layers and exactly one `slots[0]` containing the ranked models. The top-level compatibility shortlist remains available for a single task. `task.clarification` is always `null` in the evidence engine. Any residual semantic ambiguity is reconciled internally before evidence retrieval. The browser no longer displays model-role counts or clarification controls. The 30-second server and 35-second browser timeouts are unchanged; this is not a latency improvement.
-
-Before the cost-first quality screen, local browser checks with real OpenAI calls returned five, ten and fifteen models for one-, two- and three-task requests in 4.77, 5.42 and 6.48 seconds, without result-cache hits. Explicit bounding-box and pixel-mask requests retained their requested output types. One of the first eleven real-provider checks returned HTTP 503; twenty subsequent fresh repetitions of that prompt succeeded, but the intermittent failure's cause is unconfirmed. These checks use the local catalogue, not hosted storage, and do not establish universal search reliability or the under-five-second target.
-
-## Multi-task discovery — alpha, 21 October 2026
-
-The `scout-one` interface and evidence matcher are published as an experimental [alpha](https://openmerit.ai/scout/alpha/). The stable `/scout/` service is unchanged; the dated hosted-beta records below describe its earlier releases. This section records the initial role-based alpha contract; the direct task lists and cost-first policy above supersede it.
-
-Alpha serves `/scout/alpha/`, with `/scout/alpha/api/status`, `/scout/alpha/api/recommend` and `/scout/alpha/mcp`. It uses a separate Worker and reads the existing published catalogue; it does not run a second refresh or publish the expanded research policy. Alpha analytics uses a separate private dataset with the same data-handling policy. Initial publication checks confirmed the page, assets, status and MCP tool listing, but all three recommendation smoke checks failed: invoice and two-task HTTP requests returned unavailable in approximately 4.8 seconds, and the MCP summary request returned a tool error in 5.1 seconds. The subsequent timeout mitigation passed the four targeted hosted checks documented below; the latency target and nine-role limitation remain unresolved.
-
-Submit text or Markdown, up to 4,000 characters and three independent tasks. Scout describes each task as **input**, **work/steps**, **deliverable**, **criteria/evals**, and **constraints/guardrails**. Each layer distinguishes explicit information, necessary inference and missing information. Missing details are not automatically hard requirements. Steps toward one deliverable normally remain one task. Only separate model roles or incompatible endpoint needs produce additional roles, up to three per task. Requests exceeding either cap return HTTP 422 rather than silently losing work.
-
-The prompt field has no formatting toolbar or task-count hint; task decomposition happens automatically. The header shows catalogue counts without an hourly collection/status line. Source disclosures retain collection dates and refresh-failure details, and the status API still reports freshness.
-
-Each role has its own shortlist of up to five distinct models. Two roles can therefore display ten candidates, with overlap between roles. These are independent alternatives, not ten jointly evaluated combinations or a ranking of every possible model pairing. Comparable measured objectives support Pareto selection across quality, cost and latency. Dominated or unsupported candidates do not pad the list to five; unknown quality, cost and timing remain unknown. Provisional frontiers and task-fit shortlists are labelled separately. Benchmark proxies do not establish exact task accuracy or a global optimum.
-
-### API and MCP response
-
-The input remains `{ "prompt": "...", "modelIds": [...], "endpointIds": [...] }`, with both inventories optional and enforced on every role. Read `tasks[].layers` and `tasks[].slots[]`. Each slot contains `label`, `task`, `recommendations`, `ranking`, `warnings`, `specialists`, and diagnostic timings. Layers contain `text` and `origin` (`explicit`, `inferred`, or `missing`); missing text is `null`. A single-role response also retains the previous top-level result. For multiple roles, top-level `ranking.mode` is `multi-task` and `recommendations` is empty; clients must read the nested lists. MCP limits evidence excerpts per candidate as before. An ambiguous role can request clarification without discarding the other tasks in the UI.
-
-One Structured Outputs call decomposes the request and extracts scoped metadata restrictions. A single embedding batch and concurrent role-specific Decisions, eligibility and evidence checks follow. Keyword methodology retrieval supplements semantic vectors and prepared capability memberships; retrieval similarity never becomes a quality score. All roles share a 30-second server matching budget; the browser allows 35 seconds for transport overhead. A required provider failure or deadline returns unavailable, not a partially successful multi-task response. Exact-result caching remains bounded to 30 seconds and preserves evaluation dates.
-
-The under-five-second end-to-end target is **not met**. Before mitigation, the first six-case local check with real OpenAI calls returned five HTTP 503 deadline failures at approximately 4.5 seconds; only the four-task overflow case returned its expected HTTP 422. Task decomposition consumed most of the budget. Separate decomposition-only diagnostics with a longer timeout took approximately 5.5 seconds for one invoice task and 17.9 seconds for three tasks with nine roles. Those initial diagnostics did not change the serving deadline. These are local observations against the bundled snapshot, not hosted acceptance or recommended-model accuracy measurements. Fixture-based UI and contract checks alone do not establish successful live search.
-
-### Timeout mitigation — published 21 October 2026
-
-Alpha now separates the five-second performance target from a 30-second total matching timeout, with a 35-second browser timeout to allow for transport overhead. Every role still shares one deadline, and required interpretation, eligibility and evidence checks remain unchanged. Slow valid work can finish instead of being rejected at 4.5 seconds; this does not make searches faster or satisfy the latency target.
-
-The reported request to separate cats, dogs and snakes from 1,000 images returned HTTP 503 on alpha in 4.95 seconds. With the mitigation, the local browser returned five candidates in 6.23 seconds using real OpenAI calls and no result-cache hit. Local invoice and two-task checks returned candidates in 5.32 and 5.81 seconds; the same image task through MCP succeeded in 5.75 seconds. Four-task overflow still returned HTTP 422. A nine-role request still failed; a follow-up diagnostic found that a role's primary evidence capability could not be established after reconciliation. That semantic failure is not bypassed by the timeout change.
-
-After deployment, the exact image-classification request returned five candidates in the hosted browser in 7.00 seconds. Hosted invoice and two-task HTTP checks succeeded in 5.59 and 6.08 seconds, with five candidates per task; the image task through MCP succeeded in 4.75 seconds. All four checks used the existing R2/D1/Vectorize catalogue and real OpenAI calls without result-cache hits. Stable Scout was unchanged. These targeted checks establish successful search for those requests, not task accuracy, a universal success rate or compliance with the under-five-second target.
-
-### Hourly collection and coverage
-
-The collection changes below remain **unreleased**; alpha reads the existing stable catalogue. The branch retains the `7 * * * *` UTC trigger, six-way refresh concurrency and 24-job Parallel budget. Pending paid jobs resume first. Up to one quarter of the budget is reserved for six discovery categories: gateways (OpenRouter and Vercel AI Gateway); lab launches and updates; independent evaluations (Artificial Analysis and Arena); editorial leads (SemiAnalysis and Citrini); Hugging Face, hosted inference and self-hosted engines; and industry-specialist labs. Smaller budgets rotate categories. These are research starting points, not dedicated integrations or guaranteed hourly coverage of each site.
-
-Research requests cited lifecycle/expiry information, capability updates, deployment regions, fine-tuning scope and documented hardware requirements. Standard rates remain separate from promotional offers, subsidies and expiry conditions. Editorial claims supply leads rather than numerical benchmark observations. Provider-specific facts must retain their scope; OpenRouter expiry is stored on its route rather than on every deployment of a model. Unknown or unsupported facts remain unavailable. No new paid collection run has validated this expanded coverage yet.
-
-OpenAI receives the submitted request for decomposition and derived work descriptions for interpretation and embedding. Private hosted analytics records each model role separately with the original prompt, role position and allowlisted supporting evidence; the existing sampled three-month retention applies. Do not put secrets or personal information into prompts. Keys stay server-side.
-
-For development, `npm run scout` serves the page on port 45833 and requires `OPENAI_API_KEY` for matching. It refreshes structured feeds locally each hour; it does not run the hosted R2/D1/Vectorize publication pipeline. Local evidence retrieval uses lexical matches and prepared memberships without a Vectorize binding. Production-like validation needs an isolated Worker with the storage bindings and `OPENAI_API_KEY`. Never put its value in browser assets, Git or a chat message.
-
-## Evidence architecture — hosted beta, 16 October 2026
-
-Scout separates semantic interpretation from exact metadata checks and numerical selection. The hosted service uses this evidence architecture after publication of its validated serving generation. The prompt and results interface is unchanged; no npm release is included.
-
-| Stage | Hosted implementation |
-| --- | --- |
-| Hourly discovery | Structured feeds and incremental Parallel research; resumable Hub discovery archives public listings without inventing usable endpoints or scores. |
-| Hourly evidence | Source-backed identities, deployment facts, benchmark methodology and specialist profiles; cached summaries fall back to the original source when support is uncertain. |
-| Hourly publication | Private R2 records, an indexed D1 fact/observation projection and Vectorize benchmark embeddings, stored once per content and listed per version. The active pointer changes only after the complete generation is ready. |
-| Task interpretation | Structured Outputs extracts metadata restrictions, alternatives, preferences and the downstream work. Decisions interprets that work for capabilities, operations and output serialization. No prompt keywords or provider/family exception lists are used. |
-| Eligibility | Generic typed equality, membership and range comparisons against documented facts, intersected with explicit inventories. Missing or incorrectly typed facts cannot establish a restriction. |
-| Evidence retrieval | A query embedding retrieves benchmark methodology, supplemented by hourly capability memberships. Semantic similarity supplies evidence candidates, never a model's score. |
-| Selection | All eligible verified catalogue models are considered before selecting up to five distinct models. One quality estimate spans connected evaluations; Pareto comparisons require known quality, cost and latency. |
-
-Mandatory restrictions use alternatives of complete condition groups. Soft preferences affect ordering, not eligibility. Relative dates use a supplied current UTC day; arbitrary numeric bounds are retained, including impossible requirements. An unsupported mandatory condition cannot silently disappear. A no-match result describes the available verified catalogue and inventory, not every model in existence.
-
-Quality remains a relative benchmark signal, not predicted accuracy on an arbitrary task. Correlated benchmark families are grouped. Incomplete or incomparable objectives remain provisional and cannot establish a global three-objective frontier. Narrow typed tasks can still use a task-fit shortlist with unknown quality. Comparable direct calibration observations supplement quality evidence without excluding other eligible models. Calibration episode charges and timing do not replace the common token-workload reference costs and endpoint-bound timing; unknown values remain unknown.
-
-The UI and HTTP/MCP inputs remain unchanged. This path has no request-time web research, catalogue refresh, evidence preparation or recommended-model execution. Interpretation, query embedding and bounded Decisions checks are external calls; latency is measured end to end rather than assumed instantaneous. The matching budget is six seconds. A provider failure returns an unavailable response instead of silently relaxing restrictions.
-
-OpenAI receives the submitted task for structured interpretation, Decisions and query embeddings. Query vectors are used transiently; only public benchmark embeddings enter Vectorize. Public model/endpoint facts and observations enter private D1, with source records in private R2. Parallel receives public research subjects, not visitor prompts. Existing private prompt-and-evidence analytics and three-month sampled retention remain unchanged. Credentials remain server-side Cloudflare secrets.
-
-The canonical paths remain `/scout/`, `/scout/api/recommend`, `/scout/api/status` and `/scout/mcp`. There are no versioned preview pages or API aliases. The dated sections below retain historical implementation and validation records; this architecture and the current Ranking section supersede conflicting earlier policies.
-
-Acceptance retained the earlier provider failures rather than counting them as successes. The final remote filter/control run initially passed 18 of 27 cases; nine reached the six-second deadline. Each of those nine passed one focused recheck without changing the matcher. Ten additional ambiguous-output cases passed on the revised interpreter. These checks establish interpretation, eligibility and API/MCP behavior for the tested prompts, not universal reliability or actual recommended-model accuracy. The isolated real-feed refresh completed R2/D1/Vectorize publication with new Parallel purchases disabled; the normal production workflow was then exercised separately. The production refresh published successfully after a staging-memory correction. Public HTTP/MCP checks initially passed six of eight cases; the provider and invoice timeouts both passed one focused recheck. Successful live observations were approximately 2.4–6.4 seconds. Six research jobs remain queued for later enrichment. These are separate attempts, not an eight-case clean first pass.
-
-## Task relevance corrections — hosted beta, 17 October 2026
-
-These corrections are deployed as a limited hosted beta. Hourly evidence documents retain verified research descriptions and directly selected methodology alongside feed metadata; embedding caches change when that evidence changes. The already verified descriptions are indexed directly, without a second model-generated summary; original citation excerpts remain in the source records rather than the semantic search text. Structured extraction separates the downstream work from model restrictions before Decisions interprets its capabilities and output contract. Name resolution can run alongside Decisions. This avoids interpreting model selection itself as a downstream capability; it does add a sequential interpretation dependency within the existing six-second budget.
-
-The initial task-relevance beta used query vectors to retrieve up to sixteen methodologies, then required both a general-relevance predicate and a scored-ability predicate for each. Although intended to admit component proxies, these overlapping questions could disagree and reject a benchmark because its domain differed from the requested task. The matching repair below replaces that assessment path; earlier acceptance results remain historical records.
-
-Classification and judgment searches consider applicable benchmark evidence before an unmeasured native endpoint contract. Without comparable candidates they retain an explicitly unmeasured task-fit fallback; contract compatibility alone does not demonstrate accuracy. Spatial pipelines keep their separate compatibility checks. Ambiguous output representations can use the existing clarification response, with model restrictions preserved. No task-name, object-name or model-specific exception is added.
-
-The Artificial Analysis parser locates serialized records by their data shape rather than a component-property name and never executes downloaded scripts. Omitted historical observations are retained only for models still listed and unchanged benchmark definitions, keeping their original provenance and historical status. Missing endpoint-specific timing and incomparable quality scopes remain unknown or provisional; restoring a feed does not authorize copying model-level timing to an unverified route. Matching retains its six-second deadline, and the hourly schedule and UI are unchanged. Default selection orders supported frontier points by quality and known cost/time, without reserving slots for the cheapest or fastest point. Explicit cost and latency priorities and metadata preferences still apply.
-
-The initial release was withheld after real-task validation; the owner subsequently approved a limited beta release with the evidence and accuracy limitations retained. Earlier development attempts exposed source-relevance, retrieval and interpretation errors; a passing interpretation/filter run alone did not authorize deployment. A fresh execution set made **38 OpenRouter requests**, with **$0.223226** in reported charges and no unknown-charge calls. Five shortlisted models per task received two public examples each; bird and invoice tasks also included two comparison models from the earlier shortlist. The source images, answer keys, scorer and selected pool were frozen before dispatch. These executions used recorded OpenRouter routes, which do not establish latency or cost on a different recommended host.
-
-| Fresh task check | Shortlisted models | Comparison models | Release gate |
-| --- | --- | --- | --- |
-| Bird species names in alphabetical order | 19/20 required items correct | 8/8 correct | Lower sampled accuracy |
-| Book versus poster labels | 20/20 labels correct | No separate comparison pool | Passed |
-| Invoice number and total, strict JSON | 18/20 fields correct | 8/8 correct | Lower sampled accuracy |
-
-One model confused two bird species; two invoice candidates added an extra zero to a printed total. All outputs parsed successfully, so schema compliance did not establish accuracy. These failures remain recorded; no model-specific exception, held-out score or answer key was added to ranking. The final downstream-interpretation correction passed 11/11 matching cases, including exact provider/family/context/date filters, known matches, no-match and ambiguous-output cases. Observed client durations were approximately 2.2–5.9 seconds, with no percentile guarantee. That correction changed some bird/category shortlist membership, so the frozen execution results do not certify the final pool; the invoice pool remained the same and failed the original accuracy-only gate. That gate compared five shortlisted models with two comparison models rather than evaluating quality, cost and latency together. In this sample the comparison models had approximately 4.1 times the mean request charge for birds and 3.4 times for invoices. Lower mean accuracy alone does not establish a worse Pareto shortlist. The beta makes no claim of improved task accuracy or globally optimal five-model membership. Broader coverage, comparable deployment measurements and independent validation of all three objectives remain follow-up work. The held-out outcomes remain excluded from ranking.
-
-Publication retained the latest completed hourly research and a healthy refreshed feed set. The final rollout check passed 9/11 cases on its first attempt: bird matching reached the deadline and explicit SVG generation returned no shortlist. Both passed one focused recheck without a matcher change. After deployment, the public HTTP suite again passed 9/11: the explicit SVG request returned no shortlist, and a Qwen-family code-authoring request admitted tool-calling evidence. Creator/family/context/date eligibility checks and impossible-context no-match passed. The live browser bird prompt displayed five recommendations without overflow. These are retained semantic and latency limitations of the beta, not a clean first-pass acceptance or proof of best-five performance.
-
-## Matching reliability repair — 17 October 2026
-
-The task Decisions batch now distinguishes the primary quality target from endpoint requirements such as image input or serialization. Retrieval and evidence assessment use that same target. Verified hourly memberships backstop the vector results even when some primary evidence was already retrieved, so an incomplete vector result cannot suppress other prepared evidence that fits the selected inventory. The existing sixteen-methodology request bound remains; this is not exhaustive methodology retrieval.
-
-If the primary target contradicts the required abilities, Scout makes one bounded Decisions reconciliation using the quoted task and previous answers. It has at most 750 milliseconds within the existing matching deadline. An unresolved contradiction returns unavailable instead of being presented as a verified no-match. This is a general consistency check, without task-name or format keywords.
-
-One Decisions batch identifies each proposed benchmark's deliberately scored ability from its own methodology and the registry's evaluation definitions. It does not require a general component benchmark to reproduce the task's exact domain, labels or output artifact. Only a recognized ability with sufficient confidence contributes proxy quality; incidental operations, unknown scope, rejected answers and unavailable checks do not. A primary-target proxy does not establish the quality of every component in a combined task or exact task accuracy. Endpoint modality, format and tool restrictions still apply independently. No model, provider or task-object exceptions are used.
-
-Exact metadata selection and query embedding start after extraction and canonical-name resolution, alongside task Decisions. Model evidence loads alongside the benchmark assessment; unaccepted observations are removed before ranking. These changes reuse the existing hourly catalogue, D1 projection and vectors, add no background service and retain the six-second matching deadline. The UI and API/MCP request inputs are unchanged. No new npm publication is included.
-
-Diagnostic stage timings overlap and must not be added together. `interpretationMs` includes the concurrently prepared eligibility/embedding result needed to complete interpretation; individual Decisions, extraction and database timings remain available in `runtime`. Cache hits retain the original evaluation time and report zero new interpretation time as before.
-
-Before deployment, full repository verification passed, including 336 Scout tests. Eight filter/ambiguity checks passed against the prepared metadata oracle: provider, family, context, date, impossible context, an incidental provider mention, an ambiguous diagram and explicit SVG source generation. The isolated Ubuntu VM run passed 203/203 assertions against staging. All seven workloads passed through both HTTP and the official MCP SDK; installed CLI email and SQL plans contained only the declared connections. An earlier staged run exposed the remaining email retrieval gap. A subsequent test setup reused an initialized directory and correctly failed `eval init`; the final isolated run used a fresh directory. Those attempts remain recorded separately. Final production acceptance also passed 203/203 VM assertions and all eight filter/ambiguity cases after the consistency correction. The HTTP workload median was 3.665 seconds versus the earlier 4.924-second sample; no final request timed out. These checks establish tested matching and integration contracts, not exact task accuracy or a latency guarantee. The [testing guide](testing.md#hosted-matching-repair-17-october-2026) retains the earlier failed rollout and separate final results.
-
-## Latency improvements — hosted beta, 17 October 2026
-
-The hosted implementation keeps the existing task and format Decisions, exact metadata comparisons and numerical selection. Numeric and date vocabularies are omitted from interpretation input because their bounds are arbitrary; small canonical vocabularies remain in the first call. Larger vocabularies are deferred until an extracted restriction references them; a model-assisted resolution checks against the complete published vocabulary without changing unrelated conditions. Unknown mandatory names remain unresolved and cannot widen the result set. Stable reference data has an explicit provider prompt-cache boundary separate from the prompt and UTC reference time. Cache availability is measured rather than guaranteed.
-
-Hourly staging prepares content-addressed specialist packs alongside profile indexes. Searches select the same scoped profiles and use those packs instead of many repository reads. A version-checked derived pack index can also be prepared for an existing immutable catalogue without repeating research. Existing generations remain readable through the original shard path when no matching pack index is available. Source research and the hourly trigger are unchanged.
-
-Successful searches may be reused within a Worker isolate for up to 30 seconds, bounded to 32 entries and approximately 4 MiB. Exact prompt text, model/endpoint inventories, publication, interpretation version and UTC calendar date identify a cache entry. Cache hits preserve the original `evaluatedAt` time; date conditions remain evaluated as of that time for at most 30 seconds, and results are not reused across UTC midnight. Identical concurrent requests share computation. Errors and results missing a specialist or applicability check are not cached. This temporary memory cache is not semantic similarity caching. Since 26 October 2026, a separate seven-day answer cache in D1 replays the AI answers of an identical completed search; see the repeatable-searches section above.
-
-Private search records now include fixed stage durations, pending stages at a deadline, cache status and allowlisted provider status/error codes. They do not store raw errors, stack traces, headers or credentials. API results include `runtime` timings/cache status; cache hits report their current serving duration and zero new interpretation time. The page remains unchanged. A six-second matching deadline remains a reliability boundary; these changes do not establish a latency guarantee. The full repository release checks passed, including 318 Scout tests. An initial candidate timed out on books/posters; that failure was retained, and the interpretation was then reduced further. The revised semantic suite passed 17 of 17 cases, including cold repetitions and exact-cache reuse, and four final cold checks passed after the independent database reads were parallelized. These small samples do not establish a production percentile or a latency guarantee. Post-deployment HTTP/MCP checks passed five of five on their first attempts: supplied-label vision, creator restriction, impossible context, JSON invoice extraction through MCP and an exact repeat. Observed end-to-end durations were approximately 2.0–4.3 seconds for new searches and 146 ms for the repeated prompt served from the same isolate cache. The 127 UI source files and five live scripts/styles were unchanged. These are individual observations, not percentile guarantees.
-
-## Default architecture — hosted beta, 16 October 2026
-
-Scout maintains separate model, endpoint, benchmark and observation records. Structured feeds, resumable discovery and bounded Parallel research prepare source-backed public metadata each hour. Unreviewed discovery listings remain outside the executable catalogue. New or changed methodology, metadata and embeddings use source-addressed caches. An incomplete generation cannot replace the active catalogue; searches continue against the last committed version.
-
-A search runs Decisions task interpretation, Structured Outputs metadata extraction and a transient query embedding. It applies exact restrictions in D1, retrieves prepared evidence and relevant specialist profiles, and compares verified eligible configurations locally. There is no request-time web research, source preparation or model execution. The model pool has no pre-frontier top-K cap; evidence retrieval and specialist checks remain bounded. Missing objectives, incomplete benchmark coverage and approximate evidence retrieval prevent a claim of a globally verified optimum.
-
-The trigger remains hourly at `7 * * * *` UTC, with up to 24 Parallel jobs, refresh concurrency six and seven-day research rechecks. Discovery and enrichment are incremental; the trigger does not promise complete world coverage or a revisit to every source each hour. Relevant specialist checks remain bounded to 64 prepared profiles. Matching has a six-second budget and includes external provider/storage calls. Successful focused remote checks were approximately 2.4–5.4 seconds; earlier timeouts are disclosed above. Instantaneous results are not guaranteed.
-
-## Matching corrections — hosted beta, 15 October 2026
-
-The hosted beta includes the following corrections. This update does not publish a new npm package:
-
-- Benchmark, measured and task-fit candidates share the same endpoint requirements for image input, structured output and tools. Reviewed typed endpoints must support the requested operation. General catalogue entries require declared text-only output; text accompanying audio, image or video generation does not establish a general text contract. Mixed-output endpoints need a separately reviewed operation contract. Image inputs with text outputs remain supported. These are metadata eligibility checks, not task-accuracy guarantees.
-- Unknown input/output prices and the compatibility blended price remain `null`, including when only one rate is known. Actual advertised zero rates remain zero. The existing page renders unknown prices with a dash.
-- Either `modelIds` or `endpointIds` suppresses separate specialist discovery, including empty and unknown-only endpoint inventories. Explicitly selected compatible specialist endpoints can still enter a task-fit shortlist.
-- When selecting among routes for the same model, an explicit latency priority compares known endpoint-bound completion times first, then cost. Without comparable timing, known timing precedes unknown timing; if neither route has timing, known cost and stable endpoint IDs decide. Other priorities retain cost-first route selection. Task-fit candidates retain their contract-fit and deployment tiers. Timing is never borrowed from another route.
-
-Output metadata is normalized during the existing hourly refresh and retained in snapshot data version 5. Startup selects the compatible bundled snapshot over an older-format cache, even if that cache was refreshed more recently. Prompt matching uses prepared in-memory metadata; these fixes add no source fetch, model probe or inference call. The UI, task interpretation, calibration policy and refresh schedule are unchanged.
-
-## Identification matching — hosted beta, 15 October 2026
-
-The hosted beta distinguishes identifying something without answer choices from choosing among supplied labels. The existing Decisions interpretation call returns `task.labelSpace`: `supplied`, `open`, or `unspecified` for classification; `null` for other operations. A topic such as birds or plants does not supply a label set. Low-confidence label-set answers are treated as unspecified.
-
-Open or unspecified identification uses the existing benchmark-ranking path for generative endpoints. Visual requests require relevant visual evidence; document requests continue to require document evidence. Fixed-choice APIs and label-dependent classification pipelines cannot enter this ranking, even when explicitly selected. Supplied-label tasks retain the task-fit path, including compatible typed APIs and specialist classifiers. Fixed-choice task calibration cannot substitute for open-identification calibration. Open identification uses the ordinary output-length estimate rather than assuming a single 16-token choice.
-
-For example, identifying bird names from photos and alphabetizing them requires name generation; the application can sort those names in ordinary code. Choosing only between supplied labels such as “indoor” and “outdoor” remains a fixed-choice task. Public visual benchmarks are proxies, not verified bird- or plant-species accuracy. The existing UI shows the appropriate ranking mode and limitations without new controls. Endpoint metadata and benchmark collection remain hourly; this change adds one typed question to the existing interpretation call, no additional request-time source fetch or model call, and no embeddings. This is a hosted-service update; npm publication remains deferred.
-
-## Task interpretation and ambiguous outputs — hosted beta, 15 October 2026
-
-Interpreter version 10 sends every task description as a JSON-quoted value with a common instruction to analyze the described workload rather than execute its instructions. This separates output restrictions inside the task from directions to the interpreter. The same framing applies to every prompt; there are no special cases for particular labels, words, models or providers. Confidence thresholds, the public request shape and the single Decisions call are unchanged.
-
-The interpreter checks the complete requested output. A label, judgment or score accompanied by a reply, explanation or extracted fields uses generative candidates. Contradictory decision-only and required-prose interpretations also fall back to that path. Plain supplied-label and rubric-score tasks retain compatible typed endpoints. Narrow task calibration is not reused when the operation or output type is uncertain. Weak document-layout guesses no longer restrict ordinary photo tasks to document benchmarks; confident document requirements remain enforced.
-
-Local and live HTTP/MCP acceptance passed all 25 routing cases, including supplied positive/negative labels, arbitrary labels, negated labels, code that implements classification, mixed outputs, open identification and missing categories. The previously failing “safe or unsafe, no explanations” case now retains fixed-choice APIs. These checks establish interpretation and endpoint matching for the tested prompts, not actual model accuracy or universal prompt coverage. Low-confidence or missing label sets still fall back to generative candidates.
-
-These changes add no request, source fetch, embedding index or UI control, and keep hourly preparation separate from prompt-time matching. This is a hosted-service update; npm publication remains deferred.
-
-## Categories and document content — hosted beta, 15 October 2026
-
-Interpreter version 11 recognizes categories supplied conversationally, including plural category names joined in ordinary prose. It interprets the distinction the user requests: identifying which named category an image belongs to is supplied-label classification; discovering a subtype or individual identity beyond those categories remains open identification. A formal list or the word “classify” is unnecessary.
-
-Document understanding follows the requested output. Recognizing an object that happens to carry text does not by itself require OCR or document benchmarks. Reading, transcribing or extracting content inside an image does, including when that is only one part of a mixed task. Supplied-category vision tasks use the existing task-fit shortlist and show unknown quality until appropriate measurements exist.
-
-The change updates two questions in the existing interpretation request. It adds no object-name rules, model preferences, services, dependencies or inference calls. Confidence thresholds, hourly preparation and the UI are unchanged. Local and live HTTP/MCP semantic acceptance passed 43 cases, including classification/reading contrasts and the prior 25-case suite. These are routing checks, not executions of the recommended models or a guarantee for every possible prompt. This is a hosted-service update; npm publication remains deferred.
-
-## Semantic formats and SQL interpretation — hosted beta, 15 October 2026
-
-Interpreter version 13 includes removal of the English keyword scans for strict output formats and structured-data capability. The existing Decisions questions now supply those judgments directly: a prescribed record or delimited layout for software to parse can require structured output even without a named format. The capability question distinguishes extraction and categorization from merely organizing prose or reading already formatted input. The same 21 questions run in one request; no UI control, source fetch, new dependency or additional inference call is added. Hourly catalogue preparation is unchanged.
-
-The response decoder consumes typed answers without rereading the prompt. It retains response validation and general capability-consistency rules. The existing 0.4 confidence threshold now also guards structured-data capability and strict-format requirements. A low-confidence format answer in either direction remains marked uncertain, does not impose a hard format requirement, and cannot match narrow calibration evidence. This threshold is a heuristic, not a calibrated probability of correctness. Semantic interpretation can still misread a request, including unfamiliar formats or languages; catalogue format support is not proof that a model will satisfy an exact output schema.
-
-The operation question recognizes writing, revising or debugging a read-only SQL query even when concrete schema details are omitted from the task description. Those details can be supplied when the model performs the task. Expressions that produce records or serialized values remain query-writing work; they do not make the model's answer a serialized data result. Executing a query, modifying stored data, explaining a query and generating another program remain distinct operations. This changes the general Decisions definition, with no query-text matching, function-name exceptions, dialect lookup or extra question.
-
-This hosted update combines the format changes with SQL interpretation. Real Decisions interpretation passed all 91 cases locally and through live HTTP/MCP: the prior 75 cases, including the previously failing SQL request, twelve additional SQL boundaries and four fresh holdouts. Tests include query source versus serialized output, omitted/runtime schema details, recursive queries, execution loops, data modifications and multilingual requests. npm publication remains deferred. Semantic checks establish tested interpretation and endpoint matching, not query execution, exact-schema compliance or universal prompt accuracy.
-
-## Semantic specialist matching — hosted beta, 15 October 2026
-
-Interpreter version 14 and snapshot data version 6 replace Hugging Face prompt keyword dictionaries, language-name lookup and partial label overlap with Decisions compatibility checks. The hourly refresh prepares each repository's specialist operation and subject scope from public metadata, caches that analysis by revision and relevant metadata, and groups equivalent profiles. Changed repositories are analyzed in batches of up to twenty; unchanged profiles are reused. Preparation uses the server's Decisions API key and incurs provider usage. If preparation fails, Scout retains the previous complete specialist snapshot and its original timestamp while other sources can update.
-
-A search still starts with the existing 21-question task interpretation. When eligible Hub profiles are relevant, a further batched Decisions request checks their subject, explicit language requirements and complete fixed-label coverage. Shared checks are evaluated once; all applicable checks must pass with the existing 0.4 confidence threshold. Requests contain at most 180 compatibility checks plus one language-requirement question; larger sets are split into bounded batches. Open identification, unsupported work, tool/memory tasks and inventories without eligible Hub routes skip this work. There are no repository fetches, model executions or per-repository calls during matching. A failed compatibility request omits Hub candidates for that search and leaves ordinary eligible results available with a warning. There is no keyword fallback.
-
-Compatibility is eligibility, not predicted accuracy. Scout still enforces endpoint contracts, preserves unknown quality/cost/time, and orders discovery components by metadata completeness and diversity. Metadata and semantic interpretation can be wrong, and the existing bounded catalogue can omit suitable specialists. The UI and hourly schedule are unchanged. This revision is deployed on the hosted beta; npm publication remains deferred. All 107 semantic cases passed locally and through live HTTP/MCP: the prior 91 regression cases plus twelve specialist cases and four fresh holdouts. They check domain paraphrases, background context, negation, language requirements, multilingual instructions, full versus partial fixed-label coverage and open identification. These are interpretation and matching checks, not executions of recommended models.
-
-## Semantic benchmark interpretation — hosted beta, 15 October 2026
-
-Snapshot data version 7 prepares benchmark capabilities and correlated benchmark families with Decisions during the hourly refresh. It replaces benchmark-name regexes, manually assigned capability tags and the automatic reasoning fallback. Source adapters retain metric identifiers, score units and parsing contracts. Epoch descriptions and methodology are read from its published catalogue; Artificial Analysis and BFCL methodology are read from their public pages. Unavailable definitions remain explicit limitations, and names alone do not establish an opaque benchmark's capabilities.
-
-Each changed benchmark receives an isolated interpretation request so another benchmark's description cannot supply its capabilities. For shared publisher methodology, a preceding Decisions request selects the directly applicable definition for that benchmark and evaluated subset; unrelated sections are omitted from capability interpretation. A section titled with the benchmark's own name, such as "GPQA Diamond" on Artificial Analysis's methodology page, is always kept and listed first, ahead of shared sections such as the Intelligence Index overview. The publisher's methodology is used even when an earlier research description exists. Publisher-declared suite identities are retained as source facts; other family relationships are interpreted by Decisions. Capability choices and family references are validated, and positive choices below the existing 0.4 confidence threshold are not used. Family links group related evidence; numerical comparisons still require the exact same benchmark series. Unknown scopes receive no capability tags and cannot contribute capability-ranking evidence. This is semantic metadata interpretation, not a measurement of model quality or a guarantee that the classifications are correct.
-
-Interpretations are cached by definition, relevant publisher methodology, family inventory and interpreter version. Score, model, price and collection-time changes alone do not invalidate that cache. Family inventory changes trigger rechecking its references. If an interpretation fails, unchanged cached records remain usable, affected new or changed series stay unclassified, and source failure status is exposed until a later refresh succeeds. A methodology download failure retains the prior methodology when available and marks its source degraded. There is no keyword fallback.
-
-Searches read the prepared tags and family IDs without fetching methodology or making benchmark interpretation calls. The existing prompt interpretation and optional specialist compatibility calls remain. `benchmarkInterpretation` in status/recommendation responses reports prepared, classified, unclassified and unavailable series counts; each returned benchmark observation includes its capability tags, family and interpretation version/time. The UI and hourly schedule are unchanged. npm publication remains deferred.
-
-The initial deployed snapshot processed all 146 benchmark series: 136 have supported capability tags and ten remain unclassified. Individual weak capability judgments are also omitted, even when other capabilities for that series are supported. A repeat preparation with unchanged metadata made zero Decisions calls. All 24 semantic benchmark cases and 107 task-interpretation/matching cases passed both locally and through live HTTP/MCP, alongside 546 automated checks. These checks cover interpretation, caching, source failure and matching; they do not execute recommended models or establish universal semantic accuracy.
-
 ## Use Scout
 
 **Source only, not deployed:** results use **Model Choices** in their count and no longer display rank numbers, the Rank column or the “Read left to right” instruction. The Assumptions & limitations disclosure, including its supplementary specialist list, is replaced with “Results could be wrong. Please verify before choosing a model.” Model evidence remains expandable; ranking, API evidence and the methodology documented here are unchanged. Feedback's reason selector and action buttons share a 40px height.
@@ -538,7 +329,9 @@ Setup takes four steps, and you act in each one only once:
 3. **Click Approve** on `https://openmerit.ai/connect`, which the command opens for you. Sign in first if needed. The page shows the client, the device name, the scope, an eight-character code that matches the command's output, and the only permission granted: Scout recommendations through MCP. Only approve a request you started yourself.
 4. **Done.** The command has kept waiting, so nothing needs rerunning. It stores the device key, adds the `openmerit` MCP server with `claude mcp add-json`, writes the Scout skill and startup rule, and checks the connection with a real MCP `initialize` and tool discovery (not a search). The approval page switches to *Claude Code is connected* once the command has collected its key, and Claude Code reports the result. **Start a new Claude Code session** to load the startup instructions; reconnecting `openmerit` in `/mcp` alone is not enough.
 
-The prompt contains no key, account data or task draft. If the Claude desktop app isn't installed or your browser blocks the link, paste the copied prompt into Claude Code instead. The command needs `npx` (Node.js 22.19 or newer) and the `claude` CLI on the PATH of that session; the installer runs `claude mcp add-json`. Having the desktop app alone does not establish that those commands are available. See [Claude Desktop](https://code.claude.com/docs/en/desktop).
+The prompt contains no key, account data or task draft. Its 10-minute timeout is an upper bound for the command, not a delay: the installer can wait up to nine minutes for browser approval. The short shared setup prompt retains that timeout, human approval, pending-link recovery, stopping on errors and installer-owned configuration; Pi reports the installer’s next steps and Claude asks for a new session. The copied text is released through Scout website deployment, independently of npm publication.
+
+If the Claude desktop app isn't installed or your browser blocks the link, paste the copied prompt into Claude Code instead. The command needs `npx` (Node.js 22.19 or newer) and the `claude` CLI on the PATH of that session; the installer runs `claude mcp add-json`. Having the desktop app alone does not establish that those commands are available. See [Claude Desktop](https://code.claude.com/docs/en/desktop).
 
 When the command can open a browser, it waits up to nine minutes for the Approve click. When it can't, for example over SSH, it prints the link and code and returns after 30 seconds with exit code 10 so the agent can show them. Rerunning the same command resumes the same request; it does not issue a new code. Codes expire after ten minutes and can be used once. `OPENMERIT_LOGIN_WAIT` sets the wait in seconds.
 
@@ -575,8 +368,12 @@ openmerit login   # replace an expired or revoked key, then rerun install
 openmerit logout  # revoke and delete this device's key
 openmerit project --policy existing-only|existing-first [--endpoint-id <id> ...] [--plan]
 openmerit doctor  # Node.js, claude, reachability, key, MCP check and install state
+openmerit scout "<task>" [--all-endpoints] [--json]   # ask Scout from the terminal
+openmerit scout --conversation <id> [--brief <component>=<text> ...] [--skip] ["<follow-up>"]
 ```
 
+`openmerit scout` limits a new conversation to the endpoint IDs in `.openmerit/scout.json`. `--all-endpoints` searches the whole catalogue; it is refused under existing-only and needed when no policy is saved.
+
 Removal deletes the MCP server, the unchanged skill and startup rule, and the install record, and revokes the key. It keeps edited files and `.openmerit/scout.json`.
 
 ### Using Scout from the agent
@@ -710,7 +507,7 @@ Some specialist models have useful evaluations outside general LLM leaderboards.
 
 Each refresh's discovery sample takes up to 30 repositories sorted by downloads for each of fourteen pipelines: document question answering, token classification, text classification, sentence similarity, text-to-text generation, text generation, image-to-text, image-text-to-text, image classification, zero-shot image classification, object detection, zero-shot object detection, image segmentation and mask generation. The two image-to-text queries remain OCR-filtered. Bounded pinned configs add label vocabularies for up to six repositories in each fixed-label vision pipeline. Private/disabled repositories, adapters and quantization packages are excluded; fine-tunes remain separate. Downloads select coverage only, never quality. This bounded sample can miss capable low-download specialists.
 
-For document, code and retrieval components, **Specialist models** expands beneath the five recommendations when compatible prepared profiles match. For classification, detection and segmentation, compatible native pipelines may enter the five **Candidates to evaluate** instead. Fixed-label pipelines need complete semantic coverage of the requested categories; one overlapping label is insufficient. Zero-shot pipelines accept supplied labels; masks can require points or boxes. Pipeline metadata is not an accuracy guarantee. Up to three components are ordered by metadata completeness and component diversity after compatibility checks. Domain and language requirements come from Decisions, with no prompt keyword dictionary. Tool execution and persistent-memory tasks do not receive unvalidated specialist matches. Entity taggers, OCR and embedding models are components requiring integration, not interchangeable chat APIs. Repeated repository names are de-emphasized for display diversity without merging their identities. See the release status in [semantic specialist matching](#semantic-specialist-matching-hosted-beta-15-october-2026).
+For document, code and retrieval components, **Specialist models** expands beneath the five recommendations when compatible prepared profiles match. For classification, detection and segmentation, compatible native pipelines may enter the five **Candidates to evaluate** instead. Fixed-label pipelines need complete semantic coverage of the requested categories; one overlapping label is insufficient. Zero-shot pipelines accept supplied labels; masks can require points or boxes. Pipeline metadata is not an accuracy guarantee. Up to three components are ordered by metadata completeness and component diversity after compatibility checks. Domain and language requirements come from Decisions, with no prompt keyword dictionary. Tool execution and persistent-memory tasks do not receive unvalidated specialist matches. Entity taggers, OCR and embedding models are components requiring integration, not interchangeable chat APIs. Repeated repository names are de-emphasized for display diversity without merging their identities.
 
 Cards show declared licenses, access gates, language, library, parameter count when known, and live hosting routes listed by the Hub. Missing hosting metadata does not prove that no provider exists. Size is not a hardware-fit estimate. Public weights do not establish an open-source license or permission for a particular use; check the pinned model card and actual license. Scout neither downloads weights nor executes repository code, and it does not send task descriptions to Hugging Face.
 
@@ -773,24 +570,6 @@ The expanded nine-model tool profile's final check used **two new cases × five
 
 The candidate action grader was corrected from calibration outputs after test dispatch began but before any held-out outputs were inspected. Final predictions and grader hashes were frozen before inspection; no test outcomes entered calibration. This was not a fully preregistered experiment. Tool action scoring does not assess final prose precision. The release still does **not** establish the correct order of all five models across tasks.
 
-The older results below describe ranking version 3 at that release, not a fresh validation of the current version.
-
-The 14 October development audit used 438 calibration observations, an initial 358-call test, then 159 fresh validation calls after a general reliability correction. The unchanged visual shortlist retained its original 40-call test; it was not rerun. The final validation has three fresh cases per text task and five document photos. The candidate pools were selected before testing and do not cover the whole catalogue.
-
-| Task | Top-five membership, before → after | Ordered pairs, before → after | Field accuracy, before → after |
-| --- | --- | --- | --- |
-| Text extraction | 5/5 → 4/5 | 6/10 → 6/10 | 43/45 → 42/45 |
-| Classification | 4/5 → 4/5 | 4/10 → 9/10 | 30/30 → 30/30 |
-| Single-tool dispatch | 1/5 → 5/5 | 6/10 → 8/10 | 15/15 → 15/15 |
-| SQL generation | 4/5 → 4/5 | 3/10 → 9/10 | 15/15 → 15/15 |
-| Numerical reasoning | 3/5 → 3/5 | 3/10 → 10/10 | 15/15 → 15/15 |
-| Long-text retrieval | 3/5 → 4/5 | 8/10 → 6/10 | 45/45 → 43/45 |
-| Document photos | 4/5 → 4/5 | 8/10 → 7/10 | 135/150 → 134/150 |
-
-Membership compares against the measured-policy top five within each tested pool. Ordered pairs compare all ten pairs within each shortlist, using the same 60/20/20 policy on actual test accuracy, charges and request time. Overall membership improved from 24/35 to 28/35 and pair agreement from 38/70 to 55/70, but none of the seven final lists matched the entire measured order. Text extraction, long-text retrieval and visual extraction still showed small accuracy regressions. These results do **not** establish that the text-extraction regression is fully resolved or that all five recommendations are optimal.
-
-The first test exposed cross-task request failures; those test outcomes were not ingested as calibration. The reliability correction uses calibration observations only. Provider rate limits and timeouts count as failures, and latency can change between runs. The final sample is small and the visual test contains four payment slips and one invoice, including missing identifiers and tax fields. Reported improvements are development evidence, not a SOTA claim.
-
 ## Refresh and data handling
 
 The task description is sent to Scout and then to OpenAI for Decisions interpretation, Structured Outputs metadata extraction, a transient query embedding and, when relevant, specialist compatibility/applicability checks. Hourly preparation also sends public repository metadata and benchmark definitions/methodology to Decisions; no user prompts are involved in that refresh. New search analytics use [PostHog](#product-analytics), with the separate browser/API content choices and beta defaults described above. Runtime request logging remains disabled. The public index and published calibration aggregates are persisted; prompts are not sent to benchmark sites or recommended-model providers. [OpenAI's API data controls and retention policies](https://developers.openai.com/api/docs/guides/your-data) apply to those provider calls. Scout does not claim zero retention by OpenAI. Hosting infrastructure can have separate request-metadata policies. Clipboard actions occur only when selected. Describe the task rather than pasting actual invoices, credentials or private customer records.
@@ -820,7 +599,7 @@ Open `http://127.0.0.1:45833/scout/`. This lightweight local preview uses the sh
 
 ## Hosting and verification
 
-`wrangler.scout.jsonc` defines the Scout Worker, a private R2 catalogue, a twice-daily Workflow and a SQLite Durable Object coordinating publication. The cron runs at 00:07 and 12:07 UTC (`7 0,12 * * *`). Configure `OPENAI_API_KEY` and `PARALLEL_API_KEY` as Worker secrets. Refreshes build immutable catalogue versions and prepared candidate pools; searches read those pools directly and never pass through the publication coordinator. A running refresh does not block searches against the active catalogue. The first hosted publication must be validated before serving searches; a missing catalogue fails explicitly. The Worker is `openmerit-scout`, the private catalogue bucket is `openmerit-scout-catalogue`, and the Workflow is `openmerit-scout-refresh`. Both API keys are stored as Cloudflare Worker secrets and read from server-side bindings; they are not shipped to the browser. Separate workers.dev and preview URLs are disabled.
+`wrangler.scout.jsonc` defines the Scout Worker, a private R2 catalogue, a twice-daily Workflow and a SQLite Durable Object coordinating publication. The cron runs at 00:07 and 12:07 UTC (`7 0,12 * * *`). Configure `OPENAI_API_KEY` as a Worker secret. Refreshes build immutable catalogue versions and prepared candidate pools; searches read those pools directly and never pass through the publication coordinator. A running refresh does not block searches against the active catalogue. The first hosted publication must be validated before serving searches; a missing catalogue fails explicitly. The Worker is `openmerit-scout`, the private catalogue bucket is `openmerit-scout-catalogue`, and the Workflow is `openmerit-scout-refresh`. Both API keys are stored as Cloudflare Worker secrets and read from server-side bindings; they are not shipped to the browser. Separate workers.dev and preview URLs are disabled.
 
 Publication records a durable completion receipt after updating the active pointer, pending-research queue and health status. Interrupted publication resumes unfinished work; a lost success reply can be acknowledged again after the lease is released. Replaying a completed publication cannot roll back a newer catalogue or clear a later refresh's lease. Completed research receipts leave the queue only when publication commits; unfinished research retains its original paid job for a later hourly run.
 
@@ -830,7 +609,7 @@ The 17 October post-deployment recovery reused all 24 saved research jobs and pu
 
 Prepared capability pools can overlap. Searches use their prepared ID lists to choose a smaller covering set of pools, preserving the selected candidates and complete evidence while avoiding repeated downloads of the same model records. Explicit model and endpoint inventories retain their existing scope checks. These metadata reads do not fetch source pages or redo hourly research.
 
-`demo/domain-router.mjs` routes the landing page, Scout and `/connect/` to the Scout serving service. Hosted docs, demos, engineering articles and terminal pages/APIs are offline during the rebuild. The separately protected memo was restored through the router on 25 October 2026; see the [cutover record](site-domain-migration.md). There is no Scout admin/report route. Do not deploy the older root `wrangler.jsonc` to publish Scout. `npm run build:scout` packages the landing page, Scout, `/connect/`, the standalone Scout data policy and required shared assets; generated docs, memo and demo applications are excluded.
+`demo/domain-router.mjs` routes the landing page, Scout and `/connect/` to the Scout serving service. Hosted docs and engineering articles are offline during the rebuild. The former demos and hosted terminal sample have been removed. The separately protected memo was restored through the router on 25 October 2026; see the [cutover record](site-domain-migration.md). There is no Scout admin/report route. Do not deploy the older root `wrangler.jsonc` to publish Scout. `npm run build:scout` packages the landing page, Scout, `/connect/`, the standalone Scout data policy and required shared assets; generated docs, memo and demo applications are excluded.
 
 ```sh
 npm run check:scout

package/docs/security.md

@@ -1,6 +1,6 @@
 # Security
 
-**Pi replacement — unreleased, unpublished source:** Pi 1.5.0 uses native MCP plus shared Scout rule/skill sections, not the former evaluation adapter. It never initiates login, changes the host model/tools/session, or provisions background jobs. Browser-approved credentials are independent per client/scope/project, stored privately outside the repository, and passed as MCP HTTP headers, never instructions. Pi's installed connection JSON contains only the private credential path. See [Pi paths and targeted removal](pi-extension.md). The standalone evaluation CLI and core coordinator remain separate.
+**Pi adapter — npm 0.1.12 and later:** Pi 1.5.0 uses native MCP plus shared Scout rule/skill sections, not the former evaluation adapter. It never initiates login, changes the host model/tools/session, or provisions background jobs. Browser-approved credentials are independent per client/scope/project, stored privately outside the repository, and passed as MCP HTTP headers, never instructions. Pi's installed connection JSON contains only the private credential path. See [Pi paths and targeted removal](pi-extension.md). The standalone evaluation CLI remains separate.
 
 The [preview agent-evaluation runner](agent-evaluations.md) is a separate execution path: it runs trusted local agents/graders, passes only basic OS variables and explicitly named environment variables, and preserves agent-owned authentication. It sends task descriptions and optional model IDs to Scout. Local metric reports and budget reservations persist; temporary task outputs are removed. Fresh directories are not security sandboxes. Its budget admission controls do not enforce provider billing limits.
 
@@ -10,29 +10,17 @@ The [budget selector](agent-evaluations.md#budget-selection) in `0.1.7-preview.2
 
 OpenMerit accepts no model-provider or Jev credential. Credentials belong to the coding harness, its environment, or an external secret store.
 
-Never place secrets in profiles, policies, intent constraints, summaries, evidence URIs, audit events, fixtures, traces, or committed environment files.
-
-## Mutation authorization
-
-This section describes core/custom evaluation harness policy. Pi-specific lifecycle and background-runner behavior below is **historical Pi 0.87 functionality**, removed from the new Scout adapter. Installing the replacement does not establish that an old background job was removed; manage that job separately.
-
-Most intents carry `modelMutationAllowed: false`. Only application-target apply-swap and rollback intents can authorize a mutation, limited to the relevant proposal, prior configuration, and confirmed target ID. Pi's model is explicitly excluded. Automatic swaps require explicit policy opt-in and meeting the configured verified-swap threshold, which can be zero. Otherwise a proposal requires approval. Post-swap evaluation depends on the harness performing the checks and reporting their results. OpenMerit is not a sandbox around the harness's tools.
-
-Persistent wakeup provisioning is a separate authorization boundary. An adapter that needs to install a system scheduler, create a paid cloud agent, or change infrastructure must declare `wakeupProvisioning: "infrastructure_change"`; OpenMerit will not request that provisioning unless the confirmed check policy sets `infrastructureChangesAllowed: true`.
-
-The macOS Pi adapter installs a user LaunchAgent only under that authorization and persistent execution mode. The plist stores the project path, selected Pi model identifier, and executable paths, but no API key. It verifies that Pi authentication works without shell-only credentials before installing. The agent launches Pi headlessly only for due elapsed-time checks, so the configured model provider may receive project context and incur charges under the confirmed policy. Pause unloads the agent; already-running work is not cancelled.
-
-The background runner imposes a 15-minute wall-clock limit and records a nonzero status if Pi fails to complete a due intent. This is a safety bound, not a provider-spend cap. Unattended candidate discovery and model changes still need representative product validation.
+Never place secrets in evaluation configuration, task fixtures, Scout project policy, terminal configuration, activity exports, traces, or committed environment files.
 
 ## Local state
 
-`.openmerit/` may reveal task names, candidate identifiers, evidence locations, budgets, or model choices. Treat it as sensitive project metadata even though it must not contain credentials.
+`.openmerit/` may hold evaluation configuration, task fixtures, reports and budget usage, Scout project policy, and terminal configuration or activity exports. It may reveal task names, model choices, budgets or endpoint identifiers. Treat it as sensitive project metadata even though it must not contain credentials.
 
-The historical evaluation adapter also recorded OpenMerit requests and results in Pi history. With native Scout, Pi manages its own session/tool records and provider context; the adapter adds no separate evaluation-history store. Task descriptions sent to hosted Scout follow its data policy. Local OpenMerit storage does not imply that all processing stays on your machine.
+With native Scout, Pi manages its own session/tool records and provider context; the adapter adds no separate evaluation-history store. Task descriptions sent to hosted Scout follow its data policy. Local OpenMerit storage does not imply that all processing stays on your machine.
 
 ## Evidence integrity
 
-OpenMerit validates structure and deterministic Pareto conformance, not the truth of arbitrary external evidence. Use authenticated observability, immutable artifacts, and digests where stronger provenance is required.
+OpenMerit does not establish the truth of imported or external evidence. Use authenticated observability, immutable artifacts, and digests where stronger provenance is required.
 
 Do not include credentials, private traces, or proprietary evaluation data in public issues.