openmerit 0.1.10 → 0.1.12

package/CHANGELOG.md

@@ -1,5 +1,18 @@
 # Changelog
 
+## 0.1.12 — 2026-11-03
+
+- Replace the obsolete Pi 0.87 evaluation adapter with native Scout support for Pi 1.5.0. Share browser device approval, private client/scope/project credentials, installer lifecycle and Scout instructions with Claude Code; keep Pi's native MCP, tools, model, permissions and session ownership intact.
+- Add targeted Pi setup, status, diagnostics, login, logout and removal, preserving edited files and independent credentials. Install a self-contained automatically discovered extension; updates require an explicit installer rerun and a new session. No legacy evaluation commands, scheduler, LaunchAgent or auto-update hooks remain in the Pi adapter.
+- Add the logo-first Add to Pi action below Claude Code, with copy-only setup and a clipboard-denied fallback. Scout serving and npm publication remain separate releases.
+- Keep the standalone evaluation CLI and core coordinator; move the prepared demo transport out of the removed Pi adapter and declare its Pi runtime dependency explicitly.
+
+## 0.1.11 — 2026-11-01
+
+- Refresh packaged Scout guides and include the Saga articles maintained in Markdown.
+- Recognize the Claude Code instruction updater when invoked through a symbolic link, so linked hook entry points run their update checks.
+- Preserve the existing CLI, Claude Code instruction-update contract, Pi integration and public exports. Scout, Saga and homepage source changes require separate website deployments; this npm release does not publish them.
+
 ## 0.1.10 — 2026-10-26
 
 - Add default-on Claude Code background checks for compatible Scout skill/startup-rule updates, with daily checks, 15-minute failure backoff, non-blocking failure notices with the “update openmerit” fallback, manual `openmerit update`, persistent opt-out and a natural-language update instruction.

package/README.md

@@ -1,8 +1,8 @@
 # OpenMerit
 
-OpenMerit provides a local Inference Terminal for understanding an application's models, services, recorded activity, and compute, plus a Pi integration for evidence-driven model comparisons.
+OpenMerit provides a local Inference Terminal for understanding an application's models, services, recorded activity, and compute, plus Scout guidance for application-model choices.
 
-The terminal reads existing evidence. The Pi integration asks a coding harness to build evaluations, measure candidates, and carry out authorized changes. OpenMerit coordinates that workflow and validates its structured results; the terminal does not route requests, run evaluations, or swap models.
+The terminal reads existing evidence; Scout recommends candidates to evaluate. Neither is a request router or proof of application-quality improvements. A separate standalone evaluation CLI and reusable core coordinator support explicit evaluation workflows.
 
 ## Open the terminal
 
@@ -14,7 +14,7 @@ cd /path/to/your/application
 openmerit dash
 ```
 
-Stable `openmerit@0.1.7` includes the terminal. No Pi installation or provider API key is needed. It opens a browser connected to a read-only loopback service. Supported source/configuration declarations establish inventory; existing application logs or exports supply request history. Prompts, responses, and credentials are not retained. Unsupported or missing evidence remains unknown.
+No Pi installation or provider API key is needed. The terminal opens a browser connected to a read-only loopback service. Supported source/configuration declarations establish inventory; existing application logs or exports supply request history. Prompts, responses, and credentials are not retained by terminal collection. Unsupported or missing evidence remains unknown.
 
 The terminal and hosted dashboard follow the device's light/dark preference without a theme control. CLI and terminal product analytics are off unless explicitly enabled; see [the analytics contract](docs/inference-terminal.md#optional-cli-product-analytics).
 
@@ -22,63 +22,38 @@ Explore the [hosted sample](https://dash.openmerit.ai/) or [before/after dashboa
 
 ## Connect Scout to Claude Code
 
-Run `npx -y openmerit@latest install --client claude-code`, then approve the device in your browser. The command installs the MCP server and Scout skill and checks the connection. New Claude Code sessions can consult Scout before choosing application models. Requires the `claude` CLI on PATH; see [setup, project policy and removal](docs/scout.md#connect-a-coding-agent). Installing the npm package alone makes no agent configuration changes.
+Run `npx -y openmerit@latest install --client claude-code`, then approve the device in your browser. The command installs the MCP server and shared Scout guidance and checks the connection. New Claude Code sessions can consult Scout before choosing application models. Requires the `claude` CLI on PATH; see [setup, project policy and removal](docs/scout.md#connect-a-coding-agent). Installing the npm package alone makes no agent configuration changes.
 
-## Compare models with Pi
+## Connect Scout to Pi
 
-Stable **npm 0.1.7** provides `openmerit eval` for evaluating Claude Code, Codex and OpenCode with your CLI. Choose model scope and a $10, custom or no-fixed-cap budget, freeze Scout's shortlist, and compare 3–10 fresh attempts after source changes. Budget and sandbox blocks stay separate from task scores; unknown costs stay unknown. Install with `npm install -g openmerit@latest`. See [agent evaluations](docs/agent-evaluations.md) for setup, graders, budgets and availability. Scout's hosted API and MCP service update independently of the npm package. This is separate from the Pi workflow below.
-
-With Pi 0.87 and a model provider configured:
+OpenMerit **0.1.12** adds a thin Scout adapter for Pi **1.5.0**, replacing the old Pi 0.87 evaluation extension:
 
 ```sh
-pi install npm:openmerit
-pi list
+npx -y openmerit@latest install --client pi --scope user
 ```
 
-Open the product repository in interactive Pi. An unconfigured project is checked for an application LLM call at session start and after settled build turns. Pi confirms the application target and incumbent model, proposes task-specific metrics and sampling, explains estimated evaluation cost, and requests confirmation of the plan, budget, and permissions. `/openmerit setup` can recover or rerun setup. Pi's own model is never the application target.
-
-Core checks stored baseline windows against the confirmed requirements. When ready, it asks Pi to discover candidates, run controlled comparisons, and calculate a frontier. Core validates the returned evidence contracts and Pareto conformance. The harness must execute the work and provide honest evidence.
-
-Model changes normally require approval. Explicit automatic-swap opt-in can allow changes once the configured verified-swap threshold is reached, including a threshold of zero. Post-swap evaluation and conditional rollback depend on the harness performing and reporting them.
+Approve the matching device code in your browser, then start a new Pi session. Scope defaults to `user`; run with `--scope project` from a project's root for project-only setup. Each client/scope installation has its own key; project credentials stay outside the repository. The installed `openmerit-scout.ts` extension references private configuration and uses Pi's native MCP with the shared Scout rule and skill. Pi owns tools, model, permissions and sessions.
 
-Read [getting started](docs/getting-started.md) and [your first supervised trial](docs/first-trial.md).
+There are no `/openmerit` lifecycle commands, evaluation setup, scheduler, LaunchAgent, or automatic update hooks. To update, explicitly rerun the installer with the same scope. Shell `login`, `logout`, `status`, `remove` and `doctor` support `--client pi --scope user|project`. See [Pi setup and migration](docs/pi-extension.md) for paths, conflicts, old background-job removal, and SDK embedding with `createScoutPiExtension` plus Pi's explicit `createMcpExtension()`.
 
-## Implemented orchestration
+Package publication and website deployment are separate releases. Pi setup requires OpenMerit 0.1.12 or newer; check the resolved package version when using `@latest`.
 
-- Task profiles with confirmed metrics, sampling plans, constraints, budgets, and permissions.
-- Preliminary model research, incumbent identity, durable candidate handoffs, and controlled challenger experiments.
-- Deterministic baseline checks and uncertainty-aware frontier validation.
-- Versioned result schemas for all ten intents and a reusable harness adapter boundary.
-- Challenger-spend accounting, bounded intent admission, approval, verification, and rollback requests.
-- Project-local state, redacted audit events, evidence manifests, and optional event exporters.
-- Task-count, elapsed-time, catalogue-change, regression, and post-swap signals.
-- Durable pause and authorized macOS user LaunchAgents; other platforms require an external scheduler.
+## Evaluate application changes
 
-These implementations have automated coverage. They do not establish application-quality improvements or reliable unattended model changes. See [current limitations](docs/roadmap.md).
+The standalone `openmerit eval` CLI is unchanged by the Pi replacement. Evaluate Claude Code, Codex and OpenCode with explicit tasks, model scope, budget and graders. Freeze Scout's shortlist and compare fresh repeated attempts after source changes. Budget and sandbox blocks stay separate from task scores; unknown costs stay unknown. See [agent evaluations](docs/agent-evaluations.md) for setup, graders, budgets and availability.
 
-## Pi commands
-
-- `/openmerit` or `/openmerit status` — inspect readiness, active work, and lifecycle state.
-- `/openmerit setup` — request or recover setup.
-- `/openmerit assess` — check stored baseline windows; discover candidates only when ready.
-- `/openmerit frontier` — request a harness-calculated comparison for validation.
-- `/openmerit approve` — approve the current verified proposal.
-- `/openmerit pause` / `/openmerit resume` — persist or clear the automatic-work pause and reconcile authorized scheduling; active work is not cancelled.
-- `/openmerit cancel` — record abandonment of an interrupted intent; it does not terminate a provider request.
-- `/openmerit logs` — locate local state, audit, and evidence files.
-- `/openmerit doctor` — inspect adapter and scheduler diagnostics.
+The core coordinator remains available for custom harnesses: task profiles, controlled experiments, baseline and frontier validation, evidence contracts, budgets, approvals and rollback requests. The harness performs the work and supplies evidence; core validation does not establish that external measurements are truthful. The older Pi lifecycle walkthroughs are retained as historical/core references, not supported commands in the new Scout adapter.
 
 ## Documentation
 
-The [documentation overview](docs/index.md) links all current guides. The website at [openmerit.ai/docs](https://openmerit.ai/docs/) is deployed separately from the package and this checkout.
+The [documentation overview](docs/index.md) links current guides and historical references. The website at [openmerit.ai/docs](https://openmerit.ai/docs/) is deployed separately from the package and this checkout.
 
-- [Inference Terminal](docs/inference-terminal.md)
-- [Metrics and evidence](docs/metrics-and-evidence.md)
-- [Lifecycle](docs/lifecycle.md), [automation](docs/automation.md), and [budgets](docs/budgets.md)
-- [Commands](docs/commands.md), [operations](docs/operations.md), and [troubleshooting](docs/troubleshooting.md)
+- [Getting started](docs/getting-started.md) and [troubleshooting](docs/troubleshooting.md)
+- [Scout](docs/scout.md) and [Pi adapter](docs/pi-extension.md)
+- [Inference Terminal](docs/inference-terminal.md) and [agent evaluations](docs/agent-evaluations.md)
 - [Security and data handling](docs/security.md)
-- [Architecture](docs/architecture.md), [Pi adapter](docs/pi-extension.md), and [adapter development](docs/adapter-guide.md)
-- [Pareto specification](docs/pareto-spec.md) and [testing](docs/testing.md)
+- [Architecture](docs/architecture.md), [adapter development](docs/adapter-guide.md), and [Pareto specification](docs/pareto-spec.md)
+- [Testing](docs/testing.md) and [current limitations](docs/roadmap.md)
 
 ## Development
 
@@ -88,14 +63,6 @@ npm run verify
 npm run dash -- /path/to/application
 ```
 
-For Pi extension development:
-
-```sh
-./node_modules/.bin/pi --no-extensions -e ./packages/pi/src/index.ts
-```
-
-`--no-extensions` avoids registering an installed npm copy alongside the source extension. Credentials belong to the harness or its environment and must never be committed. Jev, if configured, is invoked by the harness; OpenMerit provides no Jev client.
-
-`packages/terminal` contains the CLI, collectors, hosted sample, and browser interface. `packages/protocol`, `packages/core`, and `packages/pi` provide the protocol, coordinator, and Pi adapter. The package exports `openmerit/core`, `openmerit/protocol`, and `openmerit/pi`.
+`packages/terminal` contains the CLI, collectors, hosted sample, and browser interface. `packages/protocol` and `packages/core` provide evaluation contracts and coordination; `packages/pi` supplies only the native Scout adapter. The package exports `openmerit/core`, `openmerit/protocol`, and `openmerit/pi`. See the [Pi package README](packages/pi/README.md) for bounded adapter tests. Provider credentials belong to the harness or its environment and must never be committed.
 
 Edit documentation in `docs/`, then run `npm run check:docs`. The website is generated from those Markdown files. See [documentation maintenance](docs/site-documentation.md) and the [production-source reconciliation note](docs/site-domain-migration.md) before deployment; a local build does not publish changes.

package/dist/pi/scout-extension.mjs

@@ -0,0 +1,177 @@
+// packages/pi/src/index.ts
+import { readFileSync as readFileSync2, statSync } from "node:fs";
+
+// packages/terminal/src/scout-credentials.ts
+import { createHash } from "node:crypto";
+import { readFileSync } from "node:fs";
+import { homedir } from "node:os";
+import { dirname, join, resolve } from "node:path";
+function credentialsPath(client, scope, root) {
+  const base = resolve(process.env.OPENMERIT_CONFIG_DIR ?? join(process.env.XDG_CONFIG_HOME ?? join(homedir(), ".config"), "openmerit"));
+  if (client === "claude-code" && scope === "user") return join(base, "credentials.json");
+  const installation = scope === "project" ? createHash("sha256").update(resolve(root)).digest("hex").slice(0, 24) : "user";
+  return join(base, client, installation, "credentials.json");
+}
+function readCredentials(path) {
+  let value;
+  try {
+    value = JSON.parse(readFileSync(path, "utf8"));
+  } catch (error) {
+    if (error instanceof Error && "code" in error && error.code === "ENOENT") return null;
+    throw new Error("Scout credentials could not be read. Run the targeted install command again.");
+  }
+  if (!value || typeof value !== "object" || !("version" in value) || value.version !== 1 || !("server" in value) || typeof value.server !== "string" || !("key" in value) || typeof value.key !== "string" || !/^om_[A-Za-z0-9_-]{43}$/.test(value.key)) return null;
+  const url = new URL(value.server);
+  if (url.username || url.password || url.protocol !== "https:" && !(url.protocol === "http:" && ["127.0.0.1", "localhost", "[::1]"].includes(url.hostname))) return null;
+  const result = { version: 1, server: value.server.replace(/\/+$/, ""), key: value.key };
+  if ("keyId" in value && typeof value.keyId === "string") result.keyId = value.keyId;
+  if ("headers" in value && value.headers && typeof value.headers === "object") {
+    result.headers = Object.fromEntries(Object.entries(value.headers).filter(([key, v]) => /^X-Scout-[A-Za-z-]+$/.test(key) && typeof v === "string"));
+  }
+  return result;
+}
+
+// packages/terminal/src/scout-instructions.ts
+var rule = `# Application model decisions \xB7 OpenMerit Scout
+
+Before choosing, recommending, adding, downloading, replacing or configuring
+ANY application inference model, load the \`openmerit-scout\` skill and consult
+\`scout_models\` on the \`openmerit\` MCP server before committing to a model.
+This applies even when the user only asks you to build a feature, never mentions
+models or Scout, or names a model themselves (assess it; do not silently replace it).
+
+A pretrained model, local weights, browser inference, an SDK's default model,
+embeddings, OCR, speech, image classification, detection and segmentation all
+count as model choices, not implementation details. Identify this decision before
+writing model-specific code, installing a model package or downloading weights.
+
+Follow the skill's project access policy. If it is missing, ask and save it first.
+If Scout is unavailable, fails or has no suitable evidence/candidate, disclose the
+gap and pause the model decision; do not fall back to a remembered or SDK-default
+model. You may continue work that does not commit to a model.
+
+Do not consult Scout for unrelated edits, normal inference calls using an unchanged
+approved model, or this coding agent's own model. Do not reselect approved models
+unless the task introduces or changes a model decision.
+
+When the user says "update openmerit", run
+\`npx -y openmerit@latest update --scope user\` (use \`--scope project\` if this
+rule is in the project's .claude/rules directory). Report the result, including
+any conflict or manual-upgrade requirement. Do not delete edited files, bypass
+permissions or rerun login automatically. A new session loads updated instructions.
+`;
+var skill = `---
+name: openmerit-scout
+description: Consults OpenMerit Scout before choosing, adding, downloading, replacing or configuring any application inference model, including implicit choices while building features, pretrained browser/local models, SDK defaults, embeddings, OCR, speech, classification, detection, segmentation, LLMs and agents. Not for unrelated edits, ordinary inference calls or this coding agent's own model.
+---
+
+# OpenMerit \xB7 Scout
+
+Before introducing, recommending, replacing or configuring an application
+inference model, call the \`scout_models\` tool on the \`openmerit\` MCP server.
+This applies even when the user never mentions models or Scout. Downloading local
+weights, using a pretrained browser model or accepting an SDK default is a model
+decision: consult Scout before model-specific code, installs or downloads.
+Preserve the user's explicit model choice; assess it rather than silently replacing
+it. Leave unchanged, approved application selections in place.
+
+## Project policy
+
+Read \`.openmerit/scout.json\` before searching. If it is absent, ask once whether this
+project uses **existing-only** (only endpoints it can already reach) or
+**existing-first** (prefer them, ask before searching the whole catalogue), and
+which public catalogue endpoint IDs it can access. Save the answer:
+
+\`\`\`sh
+npx -y openmerit@latest project --policy existing-only --endpoint-id <id> ...
+\`\`\`
+
+Never infer access from a model name, SDK or environment-variable name. Never
+record credentials, private URLs, prompts or customer data. An empty inventory
+is not permission to search all models.
+
+## Searching
+
+Call \`scout_models\` with a non-confidential task description and
+\`endpointIds\` set to the saved inventory, including \`[]\` when empty.
+Existing-only never broadens. For existing-first, if no existing candidate fits,
+explain the gap and ask before a separate search without \`endpointIds\`. Send
+only task requirements, never source files, secrets or real customer records.
+
+Compare model AND endpoint, evidence, reference cost/time, unknowns and
+warnings. For multiple tasks read \`tasks[].slots[0].recommendations\`; the
+top-level list is empty. Prefer the cheapest usable option the quality evidence
+supports, without claiming benchmarks prove application accuracy. Use the
+returned provider model ID and the project's actual SDK and protocol.
+
+## Boundaries
+
+Ask before adding providers or credentials, spending, deploying or external
+writes. Report recommendation, endpoint connectivity and application checks as
+separate results, each only after it succeeds. Do not switch this coding agent's
+own model.
+
+If Scout fails or has no suitable candidate/evidence for the required deployment,
+say so and pause the model decision; do not substitute a remembered recommendation
+or SDK default. Continue only model-independent work. Local/browser models are not
+exempt when the catalogue lacks coverage. If it reports that the device key is missing,
+expired or revoked, run \`npx -y openmerit@latest login\` and then
+\`npx -y openmerit@latest install\` again. Never ask the user to paste a key.
+`;
+function scoutRule(client) {
+  if (client === "claude-code") return rule;
+  return rule.replace("load the `openmerit-scout` skill and consult", "follow the Scout project policy below and consult").replaceAll("`scout_models`", "`mcp__openmerit__scout_models`").replace("Follow the skill's project access policy.", "Follow the project access policy below.").replace("update --scope user", "install --client pi --scope user").replace("rule is in the project's .claude/rules directory", "Pi installation is project-scoped");
+}
+function scoutSkill(client) {
+  if (client === "claude-code") return skill;
+  return skill.replaceAll("`scout_models`", "`mcp__openmerit__scout_models`").replace("openmerit@latest login", "openmerit@latest login --client pi").replace("openmerit@latest install`", "openmerit@latest install --client pi`").replace("again. Never ask the user to paste a key.", "again, using `--scope project` on both commands for a project-scoped installation. Never ask the user to paste a key.");
+}
+
+// packages/pi/src/index.ts
+function configuredCredentialsPath() {
+  const config = new URL("./openmerit-scout.json", import.meta.url);
+  let value;
+  try {
+    if (statSync(config).size > 16384) throw new Error("Invalid Scout connection config");
+    value = JSON.parse(readFileSync2(config, "utf8"));
+  } catch (error) {
+    if (error instanceof Error && "code" in error && error.code === "ENOENT") {
+      return credentialsPath("pi", "user", process.cwd());
+    }
+    throw new Error("Invalid Scout connection config");
+  }
+  if (!value || typeof value !== "object" || !("credentialsPath" in value) || typeof value.credentialsPath !== "string" || !value.credentialsPath.trim()) {
+    throw new Error("Invalid Scout connection config");
+  }
+  return value.credentialsPath;
+}
+function createScoutPiExtension(options = {}) {
+  return (pi) => {
+    pi.on("session_start", (_event, ctx) => {
+      try {
+        const credentials = readCredentials(options.credentialsPath ?? configuredCredentialsPath());
+        if (!credentials) throw new Error("Scout credentials missing");
+        pi.registerMcpServer("openmerit", {
+          type: "http",
+          url: `${credentials.server}/scout/mcp`,
+          headers: { ...credentials.headers, Authorization: `Bearer ${credentials.key}` },
+          exposure: "direct"
+        });
+      } catch {
+        ctx.ui.notify(
+          "OpenMerit Scout could not load its connection. Run `npx -y openmerit@latest install --client pi --scope user` (use `--scope project` for a project installation), then start a new Pi session. SDK hosts must also load Pi's createMcpExtension().",
+          "warning"
+        );
+      }
+    });
+    pi.on("before_agent_start", (event) => {
+      event.systemPromptOptions.sections.openmerit_scout_rule = scoutRule("pi");
+      event.systemPromptOptions.sections.openmerit_scout_skill = scoutSkill("pi");
+    });
+  };
+}
+var index_default = createScoutPiExtension();
+export {
+  createScoutPiExtension,
+  index_default as default
+};

package/dist/pi/src/index.d.ts

@@ -1,32 +1,6 @@
-import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
-import { type HarnessDescriptor, type ImprovementStage } from "../../protocol/src/index.ts";
-export declare const PI_HARNESS_DESCRIPTOR: HarnessDescriptor;
-export interface OpenMeritUiState {
-    proactiveNudgesPaused: boolean;
-    applicationTarget: "not_detected" | "detected" | "confirmed";
-    taskProfile: "not_configured" | "pending" | "configured";
-    metricReadiness: "not_assessed" | "pending" | "assessed";
-    evaluationReadiness: "not_assessed" | "pending" | "assessed";
-    activeIntent: string | null;
-    nextAssessment: string | null;
-    lifecycleStage: ImprovementStage | "not_initialized";
-    incumbentCandidate: string | null;
-    candidateCount: number;
-    assessmentCount: number;
-    evaluationSpend: number;
-    proposedCandidate: string | null;
-}
-export declare function createInitialUiState(): OpenMeritUiState;
-export declare function renderStatus(state: OpenMeritUiState): string;
-export interface DetectedApplicationLlmTarget {
-    readonly id: string;
-    readonly kind: "application_llm_call";
-    readonly name: string;
-    readonly applicationId: string;
-    readonly callSites: readonly string[];
-    readonly routeKey: string;
-    readonly configuredModelId?: string;
-}
-/** Bounded, adapter-local detection. It requires an actual application call pattern, not only an SDK dependency. */
-export declare function detectApplicationLlmTargets(projectRoot: string): readonly DetectedApplicationLlmTarget[];
-export default function openMeritExtension(pi: ExtensionAPI): void;
+import type { ExtensionFactory } from "@earendil-works/pi-coding-agent";
+export declare function createScoutPiExtension(options?: {
+    credentialsPath?: string;
+}): ExtensionFactory;
+declare const _default: ExtensionFactory;
+export default _default;