openmerit 0.3.7 → 0.3.9

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,6 +1,14 @@
1
1
  # Changelog
2
2
 
3
- ## 0.3.7 — 2026-11-06 (release preparation)
3
+ ## 0.3.9 — 2026-11-06 (release preparation)
4
+
5
+ - Fix Claude Code project setup, status and conflict detection in linked git worktrees. Claude Code 2.1.63 and newer share private project MCP configuration under the main checkout; older versions use the linked worktree root. The installer follows the Claude CLI on PATH, leaves the project's skill and rule in place, and does not migrate or delete existing MCP entries.
6
+
7
+ ## 0.3.8 — 2026-11-06
8
+
9
+ - The Scout skill tells coding agents to refresh an already approved Langfuse connection on the first consultation and before a final model decision, without asking for new candidate-spend or customer-platform-write approval for that read. Old project notes are not a substitute for connector refresh; failed sync must be disclosed as stale evidence. The server still requires an explicit sync request, and candidate runs, platform writes and deletion still require approval.
10
+
11
+ ## 0.3.7 — 2026-11-06
4
12
 
5
13
  - The installed Scout skill distinguishes no eval setup, insufficient evals and suitable existing evals. Langfuse is the default customer-owned platform suggestion, not a requirement; local/custom evals can report aggregates. The customer's coding agent implements evals, while OpenMerit remains Viewer-only. Instructions require a local Markdown decision record before proposing explicitly approved cleanup of newly created comparison runs; existing records and reusable fixtures/definitions stay intact. This adds guidance, not an eval runner, automatic deletion or another platform connector. Fresh-agent acceptance remains a release gate.
6
14
 
@@ -191,11 +191,20 @@ A separate trusted MCP connection with the management key exposes
191
191
  \`scout_connection\` (status/configure/sync/disconnect/clear). For configure, keys
192
192
  must come from client-controlled HTTP headers outside model context; if the
193
193
  client cannot do this securely, use protected CLI input or the web UI instead.
194
- After they confirm connection and score/model mappings, call \`scout_models\` with
195
- \`sync: "langfuse"\` to refresh aggregate measurements before recommending. The CLI
196
- equivalent is \`openmerit scout --conversation <id> --sync langfuse\`. Never read
197
- trace bodies to work around missing mappings or untagged runs; use the aggregate
198
- report path instead. Results are stored per user, used in new conversations and
194
+ Once the user has confirmed the connection and score/model mappings (including
195
+ a connection already recorded in the project), call \`scout_models\` with
196
+ \`sync: "langfuse"\` on the first consultation and before a final model decision.
197
+ The CLI equivalent is \`openmerit scout --conversation <id> --sync langfuse\`.
198
+ Refreshing that approved connection reads aggregates and updates Scout's cache;
199
+ it does not run candidates or create/change records in Langfuse. Do not ask for
200
+ new spend or platform-write approval just to refresh it. Do not reconstruct and
201
+ re-report old measurements from project notes instead of using the connection.
202
+ If connection status is unknown, check \`openmerit connect langfuse --status\`
203
+ before proposing setup; application tracing keys alone do not prove a Scout
204
+ connection. If sync fails, disclose the failure and any use of cached evidence;
205
+ do not describe it as fresh. Never read trace bodies to work around missing
206
+ mappings or untagged runs; use the aggregate report path for those gaps instead.
207
+ Results are stored per user, used in new conversations and
199
208
  expire after 90 days. Disconnect deletes the Viewer key and imported aggregates;
200
209
  Delete all evaluation data also deletes manually reported aggregates. Obtain
201
210
  consent before deleting: headless CLI requires --yes, MCP requires confirm: true.
@@ -1 +1 @@
1
- {"format":1,"contract":"scout-recommend-v2","version":"0.3.7","skill":"---\nname: openmerit-scout\ndescription: Consults OpenMerit Scout before choosing, adding, downloading, replacing or configuring any application inference model, including implicit choices while building features, pretrained browser/local models, SDK defaults, embeddings, OCR, speech, classification, detection, segmentation, LLMs and agents. Not for unrelated edits, ordinary inference calls or this coding agent's own model.\n---\n\n# OpenMerit · Scout\n\nBefore introducing, recommending, replacing or configuring an application\ninference model, call the `scout_models` tool on the `openmerit` MCP server.\nThis applies even when the user never mentions models or Scout. Downloading local\nweights, using a pretrained browser model or accepting an SDK default is a model\ndecision: consult Scout before model-specific code, installs or downloads.\nPreserve the user's explicit model choice; assess it rather than silently replacing\nit. Leave unchanged, approved application selections in place.\n\n## Project policy\n\nRead `.openmerit/scout.json` before searching. If it is absent, ask once whether this\nproject uses **existing-only** (only endpoints it can already reach) or\n**existing-first** (prefer them, ask before searching the whole catalogue), and\nwhich public catalogue endpoint IDs it can access. Save the answer:\n\n```sh\nnpx -y openmerit@latest project --policy existing-only --endpoint-id <id> ...\n```\n\nNever infer access from a model name, SDK or environment-variable name. Never\nrecord credentials, private URLs, prompts or customer data. An empty inventory\nis not permission to search all models.\n\n## Searching\n\nCall `scout_models` with a non-confidential task description as `prompt` and\n`endpointIds` set to the saved inventory, including `[]` when empty. Send\nonly task requirements, never source files, secrets or real customer records.\nIf the MCP tool is unavailable, run\n`npx -y openmerit@latest scout \"<task>\" --json`; it applies the saved inventory.\n\nScout is a conversation with the agent behind openmerit.ai/scout. When\n`status` is `needs_answers`, fill in what the user or project already states,\nshow the user Scout's brief and remaining questions, and wait for their answers.\nThen call again with the returned `conversationId` and `brief` set to the\nrewritten components, or `skip: true` if the user wants Scout to assume. Never\nanswer Scout's questions with guesses. Send later refinements to the same\n`conversationId`. A call with only a `conversationId` reads an interrupted turn.\n\nWhen `status` is `recommended`, Scout has split the work into `tasks`, one\nper model, each with up to five ranked `models`; cover every task. Each\nmodel's `cost` names the `endpointId` and `providerModelId` to call.\nExisting-only never broadens. For existing-first, if no existing candidate fits a\ntask, explain the gap and ask before a new conversation without `endpointIds`.\nCompare quality evidence, reference cost/time and watch-outs. Prefer the cheapest\nusable option the evidence supports, without claiming benchmarks prove\napplication accuracy. Use the project's actual SDK and protocol.\n\n## Measuring candidates\n\nEach task also has an `evaluation`: the checks (per model call, agent run or\nend-to-end outcome), labelled examples per candidate and when to stop a losing\ncandidate. Offer to run it before the user commits to a model. First assess the\nproject's readiness for this task:\n\n- **No eval setup:** propose a customer-owned setup. Langfuse is the default\n suggestion for its supported aggregate connector, not a requirement: existing\n local/custom eval scripts can report aggregates without a hosted platform.\n Phoenix/Arize and Braintrust are alternatives, not supported Scout connectors.\n Ask which system the user wants before creating accounts or resources.\n- **Setup exists but evidence is insufficient:** keep that system. Write a\n task-specific eval plan in the project's local Markdown decision record for\n the user or their coding agent to implement. Name exact candidates/endpoints,\n quality floor and task/agent/outcome checks, labelled boundary cases and\n controls, sample size, early-stop rule, spend cap, aggregate mappings and\n temporary run retention. Explain what the current evidence cannot decide.\n- **Suitable evals exist:** reuse their fixtures and definitions; add only the\n approved candidate runs. Do not build a second eval or copy their datasets.\n\nOpenMerit does not host or run these evals. The customer or their coding agent\nimplements and maintains them with the customer's tools; Scout keeps Viewer-only\naccess. Ask before model spend or any platform write. Without measured evidence,\nScout may suggest candidates, but do not claim validated model quality.\n\nReport what you measured with `scout_models`: the same `conversationId` and\n`results`, one aggregate per model and metric (`level`, `task` title, `scope`,\n`model` endpoint or model ID, `metric`, `value`, `higherIsBetter`,\n`samples`, `source`). Set `kind` to `usd` or `seconds` for measured cost\nor time (per call at the task level, per run at the agent level), so Scout uses\nyour cost instead of its estimate. Report each agent's own checks at the agent\nlevel: an end-to-end result also measures the agents you did not swap. Send\naggregates only, never inputs, outputs or records.\nScout ranks them above public benchmarks for this user only; models whose\n`quality.measured` is true rest on them.\n\nSwap one agent at a time, holding the other agents, prompts, fixtures and controls\nfixed. Use each agent's own checks to attribute failures. If all candidates score\n100%, the eval cannot separate their quality: add harder labelled cases before\nclaiming equivalent quality. Cost may break the observed tie, not prove equality.\n\n## Preserve evidence before cleanup\n\nBefore removing temporary comparison runs, save a local Markdown decision record\nwith the package/service versions, date, task and policy, exact candidates,\nunchanged controls, metric definitions/directions, sample counts, aggregate\nquality/cost/time, failures and exclusions, chosen model, limitations and next\nreview. Record regret against the cheapest measured candidate meeting the same\nquality floor; write \"not measurable\" when comparable evidence is missing.\nExclude secrets, private record IDs, prompts, outputs and trace bodies.\n\nTemporary comparison traces/records are not permanent: after the record is saved\nand checked, propose the exact newly created comparison runs to remove. Obtain\nexplicit scoped approval, including any cascading deletion, before the customer\nor their coding agent deletes them with customer-controlled write access. Scout's\nViewer key cannot do this. Never delete pre-existing customer records, reusable\nfixtures, dataset definitions or eval definitions. Verify the approved cleanup\nand retained controls, then record the outcome locally. If not approved, leave\nrecords unchanged and state cleanup is pending; do not imply it happened.\n\nIf the user has Langfuse, use `openmerit connect langfuse --help`: the web UI is\noptional. Headless setup requires a separate connection-manager approval with\n`openmerit connect langfuse --authorize --no-open`. Relay its approval link and\nexpiry; after approval, rerun that command. Existing recommendation keys cannot\nmanage connections. The user creates a separate Viewer key; receive it through\nprotected stdin or a secret-manager-injected environment, never chat, tool\narguments, shell literals, traced commands or tracked files. The CLI accepts\nnon-secret settings via `--config mappings.json` and reads LANGFUSE_PUBLIC_KEY\nand LANGFUSE_SECRET_KEY from its environment, or reads full settings via --stdin.\nA separate trusted MCP connection with the management key exposes\n`scout_connection` (status/configure/sync/disconnect/clear). For configure, keys\nmust come from client-controlled HTTP headers outside model context; if the\nclient cannot do this securely, use protected CLI input or the web UI instead.\nAfter they confirm connection and score/model mappings, call `scout_models` with\n`sync: \"langfuse\"` to refresh aggregate measurements before recommending. The CLI\nequivalent is `openmerit scout --conversation <id> --sync langfuse`. Never read\ntrace bodies to work around missing mappings or untagged runs; use the aggregate\nreport path instead. Results are stored per user, used in new conversations and\nexpire after 90 days. Disconnect deletes the Viewer key and imported aggregates;\nDelete all evaluation data also deletes manually reported aggregates. Obtain\nconsent before deleting: headless CLI requires --yes, MCP requires confirm: true.\nDeleting the connection does not revoke the separate management device key.\nPast conversation text is not erased. Never put record IDs, emails or secrets in\n`scope` or `source`; use labels such as \"coding agent\" and \"held-out eval\".\n\n## Boundaries\n\nAsk before adding providers or credentials, spending, deploying or external\nwrites. Report recommendation, endpoint connectivity and application checks as\nseparate results, each only after it succeeds. Do not switch this coding agent's\nown model.\n\nIf Scout fails or has no suitable candidate/evidence for the required deployment,\nsay so and pause the model decision; do not substitute a remembered recommendation\nor SDK default. Continue only model-independent work. Local/browser models are not\nexempt when the catalogue lacks coverage. When an install or login waits for approval (exit code\n10), show the user the approval link and its UTC expiry in your reply, not only in\ncommand output, then rerun the same command once they approve. If it reports that the device key is missing,\nexpired or revoked, run `npx -y openmerit@latest login` and then\n`npx -y openmerit@latest install` again. Never ask the user to paste a key.\n","rule":"# Application model decisions · OpenMerit Scout\n\nBefore choosing, recommending, adding, downloading, replacing or configuring\nANY application inference model, load the `openmerit-scout` skill and consult\n`scout_models` on the `openmerit` MCP server before committing to a model.\nThis applies even when the user only asks you to build a feature, never mentions\nmodels or Scout, or names a model themselves (assess it; do not silently replace it).\n\nA pretrained model, local weights, browser inference, an SDK's default model,\nembeddings, OCR, speech, image classification, detection and segmentation all\ncount as model choices, not implementation details. Identify this decision before\nwriting model-specific code, installing a model package or downloading weights.\n\nFollow the skill's project access policy. If it is missing, ask and save it first.\nScout may answer with clarifying questions: relay them to the user and continue\nthe same Scout conversation with their answers before choosing.\nIf Scout is unavailable, fails or has no suitable evidence/candidate, disclose the\ngap and pause the model decision; do not fall back to a remembered or SDK-default\nmodel. You may continue work that does not commit to a model.\n\nDo not consult Scout for unrelated edits, normal inference calls using an unchanged\napproved model, or this coding agent's own model. Do not reselect approved models\nunless the task introduces or changes a model decision.\n\nWhen the user says \"update openmerit\", run\n`npx -y openmerit@latest update --scope user` (use `--scope project` if this\nrule is in the project's .claude/rules directory). Report the result, including\nany conflict or manual-upgrade requirement. Do not delete edited files, bypass\npermissions or rerun login automatically. A new session loads updated instructions.\n"}
1
+ {"format":1,"contract":"scout-recommend-v2","version":"0.3.9","skill":"---\nname: openmerit-scout\ndescription: Consults OpenMerit Scout before choosing, adding, downloading, replacing or configuring any application inference model, including implicit choices while building features, pretrained browser/local models, SDK defaults, embeddings, OCR, speech, classification, detection, segmentation, LLMs and agents. Not for unrelated edits, ordinary inference calls or this coding agent's own model.\n---\n\n# OpenMerit · Scout\n\nBefore introducing, recommending, replacing or configuring an application\ninference model, call the `scout_models` tool on the `openmerit` MCP server.\nThis applies even when the user never mentions models or Scout. Downloading local\nweights, using a pretrained browser model or accepting an SDK default is a model\ndecision: consult Scout before model-specific code, installs or downloads.\nPreserve the user's explicit model choice; assess it rather than silently replacing\nit. Leave unchanged, approved application selections in place.\n\n## Project policy\n\nRead `.openmerit/scout.json` before searching. If it is absent, ask once whether this\nproject uses **existing-only** (only endpoints it can already reach) or\n**existing-first** (prefer them, ask before searching the whole catalogue), and\nwhich public catalogue endpoint IDs it can access. Save the answer:\n\n```sh\nnpx -y openmerit@latest project --policy existing-only --endpoint-id <id> ...\n```\n\nNever infer access from a model name, SDK or environment-variable name. Never\nrecord credentials, private URLs, prompts or customer data. An empty inventory\nis not permission to search all models.\n\n## Searching\n\nCall `scout_models` with a non-confidential task description as `prompt` and\n`endpointIds` set to the saved inventory, including `[]` when empty. Send\nonly task requirements, never source files, secrets or real customer records.\nIf the MCP tool is unavailable, run\n`npx -y openmerit@latest scout \"<task>\" --json`; it applies the saved inventory.\n\nScout is a conversation with the agent behind openmerit.ai/scout. When\n`status` is `needs_answers`, fill in what the user or project already states,\nshow the user Scout's brief and remaining questions, and wait for their answers.\nThen call again with the returned `conversationId` and `brief` set to the\nrewritten components, or `skip: true` if the user wants Scout to assume. Never\nanswer Scout's questions with guesses. Send later refinements to the same\n`conversationId`. A call with only a `conversationId` reads an interrupted turn.\n\nWhen `status` is `recommended`, Scout has split the work into `tasks`, one\nper model, each with up to five ranked `models`; cover every task. Each\nmodel's `cost` names the `endpointId` and `providerModelId` to call.\nExisting-only never broadens. For existing-first, if no existing candidate fits a\ntask, explain the gap and ask before a new conversation without `endpointIds`.\nCompare quality evidence, reference cost/time and watch-outs. Prefer the cheapest\nusable option the evidence supports, without claiming benchmarks prove\napplication accuracy. Use the project's actual SDK and protocol.\n\n## Measuring candidates\n\nEach task also has an `evaluation`: the checks (per model call, agent run or\nend-to-end outcome), labelled examples per candidate and when to stop a losing\ncandidate. Offer to run it before the user commits to a model. First assess the\nproject's readiness for this task:\n\n- **No eval setup:** propose a customer-owned setup. Langfuse is the default\n suggestion for its supported aggregate connector, not a requirement: existing\n local/custom eval scripts can report aggregates without a hosted platform.\n Phoenix/Arize and Braintrust are alternatives, not supported Scout connectors.\n Ask which system the user wants before creating accounts or resources.\n- **Setup exists but evidence is insufficient:** keep that system. Write a\n task-specific eval plan in the project's local Markdown decision record for\n the user or their coding agent to implement. Name exact candidates/endpoints,\n quality floor and task/agent/outcome checks, labelled boundary cases and\n controls, sample size, early-stop rule, spend cap, aggregate mappings and\n temporary run retention. Explain what the current evidence cannot decide.\n- **Suitable evals exist:** reuse their fixtures and definitions; add only the\n approved candidate runs. Do not build a second eval or copy their datasets.\n\nOpenMerit does not host or run these evals. The customer or their coding agent\nimplements and maintains them with the customer's tools; Scout keeps Viewer-only\naccess. Ask before model spend or any platform write. Without measured evidence,\nScout may suggest candidates, but do not claim validated model quality.\n\nReport what you measured with `scout_models`: the same `conversationId` and\n`results`, one aggregate per model and metric (`level`, `task` title, `scope`,\n`model` endpoint or model ID, `metric`, `value`, `higherIsBetter`,\n`samples`, `source`). Set `kind` to `usd` or `seconds` for measured cost\nor time (per call at the task level, per run at the agent level), so Scout uses\nyour cost instead of its estimate. Report each agent's own checks at the agent\nlevel: an end-to-end result also measures the agents you did not swap. Send\naggregates only, never inputs, outputs or records.\nScout ranks them above public benchmarks for this user only; models whose\n`quality.measured` is true rest on them.\n\nSwap one agent at a time, holding the other agents, prompts, fixtures and controls\nfixed. Use each agent's own checks to attribute failures. If all candidates score\n100%, the eval cannot separate their quality: add harder labelled cases before\nclaiming equivalent quality. Cost may break the observed tie, not prove equality.\n\n## Preserve evidence before cleanup\n\nBefore removing temporary comparison runs, save a local Markdown decision record\nwith the package/service versions, date, task and policy, exact candidates,\nunchanged controls, metric definitions/directions, sample counts, aggregate\nquality/cost/time, failures and exclusions, chosen model, limitations and next\nreview. Record regret against the cheapest measured candidate meeting the same\nquality floor; write \"not measurable\" when comparable evidence is missing.\nExclude secrets, private record IDs, prompts, outputs and trace bodies.\n\nTemporary comparison traces/records are not permanent: after the record is saved\nand checked, propose the exact newly created comparison runs to remove. Obtain\nexplicit scoped approval, including any cascading deletion, before the customer\nor their coding agent deletes them with customer-controlled write access. Scout's\nViewer key cannot do this. Never delete pre-existing customer records, reusable\nfixtures, dataset definitions or eval definitions. Verify the approved cleanup\nand retained controls, then record the outcome locally. If not approved, leave\nrecords unchanged and state cleanup is pending; do not imply it happened.\n\nIf the user has Langfuse, use `openmerit connect langfuse --help`: the web UI is\noptional. Headless setup requires a separate connection-manager approval with\n`openmerit connect langfuse --authorize --no-open`. Relay its approval link and\nexpiry; after approval, rerun that command. Existing recommendation keys cannot\nmanage connections. The user creates a separate Viewer key; receive it through\nprotected stdin or a secret-manager-injected environment, never chat, tool\narguments, shell literals, traced commands or tracked files. The CLI accepts\nnon-secret settings via `--config mappings.json` and reads LANGFUSE_PUBLIC_KEY\nand LANGFUSE_SECRET_KEY from its environment, or reads full settings via --stdin.\nA separate trusted MCP connection with the management key exposes\n`scout_connection` (status/configure/sync/disconnect/clear). For configure, keys\nmust come from client-controlled HTTP headers outside model context; if the\nclient cannot do this securely, use protected CLI input or the web UI instead.\nOnce the user has confirmed the connection and score/model mappings (including\na connection already recorded in the project), call `scout_models` with\n`sync: \"langfuse\"` on the first consultation and before a final model decision.\nThe CLI equivalent is `openmerit scout --conversation <id> --sync langfuse`.\nRefreshing that approved connection reads aggregates and updates Scout's cache;\nit does not run candidates or create/change records in Langfuse. Do not ask for\nnew spend or platform-write approval just to refresh it. Do not reconstruct and\nre-report old measurements from project notes instead of using the connection.\nIf connection status is unknown, check `openmerit connect langfuse --status`\nbefore proposing setup; application tracing keys alone do not prove a Scout\nconnection. If sync fails, disclose the failure and any use of cached evidence;\ndo not describe it as fresh. Never read trace bodies to work around missing\nmappings or untagged runs; use the aggregate report path for those gaps instead.\nResults are stored per user, used in new conversations and\nexpire after 90 days. Disconnect deletes the Viewer key and imported aggregates;\nDelete all evaluation data also deletes manually reported aggregates. Obtain\nconsent before deleting: headless CLI requires --yes, MCP requires confirm: true.\nDeleting the connection does not revoke the separate management device key.\nPast conversation text is not erased. Never put record IDs, emails or secrets in\n`scope` or `source`; use labels such as \"coding agent\" and \"held-out eval\".\n\n## Boundaries\n\nAsk before adding providers or credentials, spending, deploying or external\nwrites. Report recommendation, endpoint connectivity and application checks as\nseparate results, each only after it succeeds. Do not switch this coding agent's\nown model.\n\nIf Scout fails or has no suitable candidate/evidence for the required deployment,\nsay so and pause the model decision; do not substitute a remembered recommendation\nor SDK default. Continue only model-independent work. Local/browser models are not\nexempt when the catalogue lacks coverage. When an install or login waits for approval (exit code\n10), show the user the approval link and its UTC expiry in your reply, not only in\ncommand output, then rerun the same command once they approve. If it reports that the device key is missing,\nexpired or revoked, run `npx -y openmerit@latest login` and then\n`npx -y openmerit@latest install` again. Never ask the user to paste a key.\n","rule":"# Application model decisions · OpenMerit Scout\n\nBefore choosing, recommending, adding, downloading, replacing or configuring\nANY application inference model, load the `openmerit-scout` skill and consult\n`scout_models` on the `openmerit` MCP server before committing to a model.\nThis applies even when the user only asks you to build a feature, never mentions\nmodels or Scout, or names a model themselves (assess it; do not silently replace it).\n\nA pretrained model, local weights, browser inference, an SDK's default model,\nembeddings, OCR, speech, image classification, detection and segmentation all\ncount as model choices, not implementation details. Identify this decision before\nwriting model-specific code, installing a model package or downloading weights.\n\nFollow the skill's project access policy. If it is missing, ask and save it first.\nScout may answer with clarifying questions: relay them to the user and continue\nthe same Scout conversation with their answers before choosing.\nIf Scout is unavailable, fails or has no suitable evidence/candidate, disclose the\ngap and pause the model decision; do not fall back to a remembered or SDK-default\nmodel. You may continue work that does not commit to a model.\n\nDo not consult Scout for unrelated edits, normal inference calls using an unchanged\napproved model, or this coding agent's own model. Do not reselect approved models\nunless the task introduces or changes a model decision.\n\nWhen the user says \"update openmerit\", run\n`npx -y openmerit@latest update --scope user` (use `--scope project` if this\nrule is in the project's .claude/rules directory). Report the result, including\nany conflict or manual-upgrade requirement. Do not delete edited files, bypass\npermissions or rerun login automatically. A new session loads updated instructions.\n"}
@@ -10,6 +10,7 @@ import { configureUpdates, inspectUpdates, recoverUpdates, removeUpdates, runUpd
10
10
  import { credentialsPath as deviceCredentialsPath, pendingPath as devicePendingPath } from './scout-credentials.js';
11
11
  import { rule, skill } from './scout-instructions.js';
12
12
  import { piPaths, inspectPi, configurePi, removePi, piConflicts, piVersion } from './scout-pi.js';
13
+ import { claudeProject } from './scout-claude.js';
13
14
  export { rule, skill } from './scout-instructions.js';
14
15
 
15
16
  // One-command coding-agent setup: device sign-in in the browser, then the MCP
@@ -109,12 +110,6 @@ function claudePaths(root, scope) {
109
110
  mcpScope: scope === 'user' ? 'user' : 'local',
110
111
  };
111
112
  }
112
- // Claude Code keys its private local scope by the project's git repository root, so a
113
- // project in a subdirectory of a repository shares the entry of the whole repository.
114
- async function claudeProject(root) {
115
- try { return (await exec('git', ['rev-parse', '--show-toplevel'], { cwd: root, timeout: 10_000, windowsHide: true })).stdout.trim() || root; }
116
- catch { return root; }
117
- }
118
113
  const configuredServer = (config, project, scope) => scope === 'user' ? config.mcpServers?.openmerit : config.projects?.[project]?.mcpServers?.openmerit;
119
114
  const serverConfig = (key, telemetry) => ({ type: 'http', url: `${server()}/scout/mcp`, headers: { Authorization: `Bearer ${key}`, ...setupHeaders(telemetry.properties, 'agent') } });
120
115
  const canonical = value => Array.isArray(value) ? value.map(canonical) : value && typeof value === 'object' ? Object.fromEntries(Object.keys(value).sort().map(k => [k, canonical(value[k])])) : value;
@@ -0,0 +1,25 @@
1
+ import { execFile } from 'node:child_process';
2
+ import { promisify } from 'node:util';
3
+ const exec = promisify(execFile);
4
+ // Claude's private local scope belongs to the repository, not its subdirectories.
5
+ export async function claudeProject(root) {
6
+ const options = { cwd: root, timeout: 10_000, windowsHide: true };
7
+ let project;
8
+ try {
9
+ project = (await exec('git', ['rev-parse', '--show-toplevel'], options)).stdout.trim() || root;
10
+ }
11
+ catch {
12
+ return root;
13
+ }
14
+ const { stdout: worktrees } = await exec('git', ['worktree', 'list', '--porcelain', '-z'], options);
15
+ const main = worktrees.split('\0')[0]?.slice('worktree '.length);
16
+ if (!main || main === project)
17
+ return project;
18
+ const { stdout } = await exec('claude', ['--version'], options);
19
+ const version = stdout.match(/^(\d+)\.(\d+)\.(\d+)/);
20
+ if (!version)
21
+ throw new Error('Could not determine Claude Code version for worktree setup. Check claude --version, then retry.');
22
+ // Claude Code 2.1.63 began sharing project configuration with the main checkout.
23
+ const major = Number(version[1]), minor = Number(version[2]), patch = Number(version[3]);
24
+ return major > 2 || major === 2 && (minor > 1 || minor === 1 && patch >= 63) ? main : project;
25
+ }
@@ -157,11 +157,20 @@ A separate trusted MCP connection with the management key exposes
157
157
  \`scout_connection\` (status/configure/sync/disconnect/clear). For configure, keys
158
158
  must come from client-controlled HTTP headers outside model context; if the
159
159
  client cannot do this securely, use protected CLI input or the web UI instead.
160
- After they confirm connection and score/model mappings, call \`scout_models\` with
161
- \`sync: "langfuse"\` to refresh aggregate measurements before recommending. The CLI
162
- equivalent is \`openmerit scout --conversation <id> --sync langfuse\`. Never read
163
- trace bodies to work around missing mappings or untagged runs; use the aggregate
164
- report path instead. Results are stored per user, used in new conversations and
160
+ Once the user has confirmed the connection and score/model mappings (including
161
+ a connection already recorded in the project), call \`scout_models\` with
162
+ \`sync: "langfuse"\` on the first consultation and before a final model decision.
163
+ The CLI equivalent is \`openmerit scout --conversation <id> --sync langfuse\`.
164
+ Refreshing that approved connection reads aggregates and updates Scout's cache;
165
+ it does not run candidates or create/change records in Langfuse. Do not ask for
166
+ new spend or platform-write approval just to refresh it. Do not reconstruct and
167
+ re-report old measurements from project notes instead of using the connection.
168
+ If connection status is unknown, check \`openmerit connect langfuse --status\`
169
+ before proposing setup; application tracing keys alone do not prove a Scout
170
+ connection. If sync fails, disclose the failure and any use of cached evidence;
171
+ do not describe it as fresh. Never read trace bodies to work around missing
172
+ mappings or untagged runs; use the aggregate report path for those gaps instead.
173
+ Results are stored per user, used in new conversations and
165
174
  expire after 90 days. Disconnect deletes the Viewer key and imported aggregates;
166
175
  Delete all evaluation data also deletes manually reported aggregates. Obtain
167
176
  consent before deleting: headless CLI requires --yes, MCP requires confirm: true.
@@ -28,13 +28,12 @@ Each recommended model names an exact endpoint and provider model ID.
28
28
 
29
29
  ## Measure with your own evals
30
30
 
31
- **Eval-readiness and evidence guidance — npm 0.3.7 release preparation:** this
31
+ **Eval-readiness and evidence guidance — npm 0.3.7:** this
32
32
  package adds the readiness and cleanup workflow below to the installed Scout
33
33
  skill. Existing npm 0.3.6 instructions cover measuring and aggregate reporting,
34
34
  but do not yet prescribe all three readiness paths or an evidence-first cleanup
35
- record. Registry publication is separate from these source changes. This is
36
- guidance for the customer's coding agent, not an automated eval runner, platform
37
- provisioner or deletion service.
35
+ record. This is guidance for the customer's coding agent, not an automated eval
36
+ runner, platform provisioner or deletion service.
38
37
 
39
38
  ### Choose the readiness path
40
39
 
@@ -300,6 +299,18 @@ resending it and retained the one-sample caveat. This focused check does not
300
299
  establish fresh-coding-agent usability; full fresh-agent and two-release
301
300
  acceptance remain open.
302
301
 
302
+ **npm 0.3.8 release preparation (unreleased guidance):** when your project records a confirmed
303
+ Langfuse connection and mappings, the coding agent should request aggregate sync
304
+ on its first consultation and before a final model decision. This refresh reads
305
+ Langfuse aggregates and updates Scout's cache; it does not execute candidates or
306
+ create customer-platform records, so it does not require new model-spend or
307
+ platform-write approval. Do not retype old measurements from project notes in
308
+ place of refreshing the connection. If sync fails, disclose the failure and
309
+ distinguish retained cached evidence from a fresh import. A tracing key in the
310
+ application does not establish that Scout is connected: check connection status
311
+ before proposing another setup. This guidance does not make the server sync
312
+ implicitly; the agent still requests it through the existing command or tool.
313
+
303
314
  Click **Sync aggregates**, or request a fresh recommendation with:
304
315
 
305
316
  ```sh
package/docs/scout.md CHANGED
@@ -389,6 +389,17 @@ When the command can open a browser, it waits up to nine minutes for the Approve
389
389
 
390
390
  `--client` defaults to the agent running the command (Claude Code sets `CLAUDECODE=1`). `--scope` defaults to `user`, which makes Scout available in every project. `--scope project` writes the skill to `.claude/skills/` and startup rule to `.claude/rules/` in the current project and stores the MCP server in Claude Code's private per-project (`local`) configuration, so the key never enters the repository. Claude Code keys that configuration by the git repository root, so for a project inside a larger repository (such as a monorepo package) the MCP server applies to every Claude Code session in that repository, while the skill and rule stay in the project; npm 0.3.2 and later say so, and earlier versions report a failed install in this case. `--plan` lists the changes without making them. `--json` prints one result object for agents.
391
391
 
392
+ **Linked worktrees — npm 0.3.9 release preparation:** the installer follows
393
+ the `claude` version on PATH. Before Claude Code 2.1.63, private local MCP setup
394
+ belongs to the linked worktree root. From 2.1.63, Claude shares it under the main
395
+ checkout root across that repository's worktrees; the skill and rule still stay
396
+ in the project where you ran setup. This follows [Claude's configuration-sharing
397
+ change](https://github.com/anthropics/claude-code/blob/main/CHANGELOG.md#2163).
398
+ Use the same Claude CLI for installation and agent sessions. Existing entries
399
+ are not migrated or deleted automatically. Published npm 0.3.8 and earlier
400
+ resolve only the current worktree root and can incorrectly report missing setup
401
+ with newer Claude versions.
402
+
392
403
  | Exit code | Meaning |
393
404
  |---|---|
394
405
  | 0 | Done |
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "openmerit",
3
- "version": "0.3.7",
3
+ "version": "0.3.9",
4
4
  "description": "Evidence-driven model decisions for coding harnesses.",
5
5
  "type": "module",
6
6
  "bin": {