openmerit 0.1.7 → 0.1.9

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,18 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.1.9 — 2026-10-24
4
+
5
+ - Add opt-in coding-agent installation milestones, component/connection diagnostics and consent-gated setup/installation correlation retained across resumed approval and upgrades. Carry browser usage and unlink choices into setup; keep standalone CLI analytics default-off and MCP prompt-content capture off.
6
+ - Add bounded installation-event retries with stable delivery IDs. Report verification and remote revocation failures even when existing local command exit semantics remain successful.
7
+ - Coordinate the independently released Scout browser/device/MCP lifecycle telemetry, correct HTTP-200 logical-error accounting, and distinguish key authorization from verified installation. These are observed events, not skipped-call detection, guaranteed delivery or downstream model-execution tracking.
8
+ - Preserve existing package exports, model-decision guidance and legal metadata. Existing installations need an installer rerun; npm publication does not deploy Scout or the main website.
9
+
10
+ ## 0.1.8 — 2026-10-24
11
+
12
+ - Install an owned startup rule requiring Scout before explicit or implicit application-model choices, including pretrained/local/browser models, weight downloads and SDK defaults. Keep detailed access-policy and search instructions in the skill. These instructions are guidance, not hard enforcement.
13
+ - Upgrade unchanged skill-only installations without another device approval; include rule ownership, conflict detection, interrupted-update recovery, status, diagnostics and safe removal.
14
+ - Require a new Claude Code session after setup or upgrade; reconnecting `/mcp` alone does not reload startup instructions. Pause model selection when Scout is unavailable or lacks suitable candidates rather than falling back to a remembered model.
15
+
3
16
  ## 0.1.7 — 2026-10-24
4
17
 
5
18
  - Release coding-agent device sign-in, Claude Code MCP/Scout skill installation, status, diagnostics, removal and project endpoint policy on the stable `latest` channel. Setup requires browser approval; it never requests a pasted key.
@@ -1,13 +1,14 @@
1
1
  import { readFile, writeFile, mkdir, rm } from 'node:fs/promises';
2
2
  import { dirname, join, resolve } from 'node:path';
3
3
  import { homedir, hostname } from 'node:os';
4
- import { createHash } from 'node:crypto';
4
+ import { createHash, randomUUID } from 'node:crypto';
5
5
  import { execFile } from 'node:child_process';
6
6
  import { promisify } from 'node:util';
7
7
  import { openBrowser } from './browser.mjs';
8
+ import { clientVersion, setupHeaders, setupUUID, sendInstallationEvent, setupCorrelationEnabled } from './telemetry.mjs';
8
9
 
9
10
  // One-command coding-agent setup: device sign-in in the browser, then the MCP
10
- // server and Scout skill. Built to be run by a person or by a coding agent:
11
+ // server, startup rule and Scout skill. Built to be run by a person or by a coding agent:
11
12
  // no interactive prompts, bounded waits, resumable logins and explicit exit codes.
12
13
  export const exitCodes = { ok: 0, error: 1, pending: 10, denied: 11, conflict: 20, unreachable: 30 };
13
14
  const exec = promisify(execFile);
@@ -15,18 +16,45 @@ const hash = value => createHash('sha256').update(value).digest('hex');
15
16
  const policyPath = '.openmerit/scout.json';
16
17
  const clients = { 'claude-code': 'Claude Code' };
17
18
 
19
+ // No paths frontmatter: Claude loads this before the first model decision,
20
+ // including user-scope sessions in Desktop's automatically created folders.
21
+ export const rule = `# Application model decisions · OpenMerit Scout
22
+
23
+ Before choosing, recommending, adding, downloading, replacing or configuring
24
+ ANY application inference model, load the \`openmerit-scout\` skill and consult
25
+ \`scout_models\` on the \`openmerit\` MCP server before committing to a model.
26
+ This applies even when the user only asks you to build a feature, never mentions
27
+ models or Scout, or names a model themselves (assess it; do not silently replace it).
28
+
29
+ A pretrained model, local weights, browser inference, an SDK's default model,
30
+ embeddings, OCR, speech, image classification, detection and segmentation all
31
+ count as model choices, not implementation details. Identify this decision before
32
+ writing model-specific code, installing a model package or downloading weights.
33
+
34
+ Follow the skill's project access policy. If it is missing, ask and save it first.
35
+ If Scout is unavailable, fails or has no suitable evidence/candidate, disclose the
36
+ gap and pause the model decision; do not fall back to a remembered or SDK-default
37
+ model. You may continue work that does not commit to a model.
38
+
39
+ Do not consult Scout for unrelated edits, normal inference calls using an unchanged
40
+ approved model, or this coding agent's own model. Do not reselect approved models
41
+ unless the task introduces or changes a model decision.
42
+ `;
43
+
18
44
  export const skill = `---
19
45
  name: openmerit-scout
20
- description: Use when choosing, adding, replacing or configuring an application inference model (an LLM or other model the user's application calls), for example when building extraction, classification, generation or agent features. Consults OpenMerit Scout for an evidence-based shortlist. Not for unrelated edits, ordinary inference calls or this coding agent's own model.
46
+ description: Consults OpenMerit Scout before choosing, adding, downloading, replacing or configuring any application inference model, including implicit choices while building features, pretrained browser/local models, SDK defaults, embeddings, OCR, speech, classification, detection, segmentation, LLMs and agents. Not for unrelated edits, ordinary inference calls or this coding agent's own model.
21
47
  ---
22
48
 
23
49
  # OpenMerit · Scout
24
50
 
25
51
  Before introducing, recommending, replacing or configuring an application
26
52
  inference model, call the \`scout_models\` tool on the \`openmerit\` MCP server.
27
- This applies even when the user never mentions Scout. Preserve the user's explicit
28
- model choice; assess it rather than silently replacing it. Leave unchanged,
29
- approved application selections in place.
53
+ This applies even when the user never mentions models or Scout. Downloading local
54
+ weights, using a pretrained browser model or accepting an SDK default is a model
55
+ decision: consult Scout before model-specific code, installs or downloads.
56
+ Preserve the user's explicit model choice; assess it rather than silently replacing
57
+ it. Leave unchanged, approved application selections in place.
30
58
 
31
59
  ## Project policy
32
60
 
@@ -64,24 +92,26 @@ writes. Report recommendation, endpoint connectivity and application checks as
64
92
  separate results, each only after it succeeds. Do not switch this coding agent's
65
93
  own model.
66
94
 
67
- If Scout fails, say so and pause the model decision; do not substitute a
68
- remembered recommendation. If it reports that the device key is missing,
95
+ If Scout fails or has no suitable candidate/evidence for the required deployment,
96
+ say so and pause the model decision; do not substitute a remembered recommendation
97
+ or SDK default. Continue only model-independent work. Local/browser models are not
98
+ exempt when the catalogue lacks coverage. If it reports that the device key is missing,
69
99
  expired or revoked, run \`npx -y openmerit@latest login\` and then
70
100
  \`npx -y openmerit@latest install\` again. Never ask the user to paste a key.
71
101
  `;
72
102
 
73
103
  export const help = `OpenMerit · coding-agent setup
74
104
 
75
- openmerit install [--client claude-code] [--scope user|project] [--plan] [--json] [--no-open]
105
+ openmerit install [--client claude-code] [--scope user|project] [--setup-id UUID] [--plan] [--json] [--no-open]
76
106
  openmerit status [--client claude-code] [--scope user|project] [--check] [--json]
77
107
  openmerit remove [--client claude-code] [--scope user|project] [--plan] [--json]
78
- openmerit login [--json] [--no-open] Replace this device's key
108
+ openmerit login [--setup-id UUID] [--json] [--no-open] Replace this device's key
79
109
  openmerit logout [--json] Revoke this device's key
80
110
  openmerit project --policy existing-only|existing-first [--endpoint-id <id> ...] [--plan]
81
111
  openmerit doctor [--json]
82
112
 
83
113
  install signs this device in through the browser (one Approve click), adds the
84
- OpenMerit MCP server and the Scout skill, then verifies the connection.
114
+ OpenMerit MCP server, startup rule and Scout skill, then verifies the connection.
85
115
  It opens the approval page and waits up to 9 minutes for the Approve click
86
116
  (30 seconds when no browser could be opened). If it exits with approval
87
117
  pending, rerun the same command: it resumes the same request.
@@ -115,12 +145,12 @@ async function save(path, value) {
115
145
  await writeFile(path, typeof value === 'string' ? value : `${JSON.stringify(value, null, 2)}\n`, { mode: 0o600 });
116
146
  }
117
147
 
118
- async function request(path, { method = 'GET', body, key } = {}) {
148
+ async function request(path, { method = 'GET', body, key, telemetry } = {}) {
119
149
  let response;
120
150
  try {
121
151
  response = await fetch(`${server()}${path}`, {
122
152
  method, redirect: 'error', signal: AbortSignal.timeout(15_000),
123
- headers: { ...(body ? { 'Content-Type': 'application/json' } : {}), ...(key ? { Authorization: `Bearer ${key}` } : {}), 'X-Scout-Client': 'cli', 'X-Scout-Analytics-Content': 'off' },
153
+ headers: { ...(body ? { 'Content-Type': 'application/json' } : {}), ...(key ? { Authorization: `Bearer ${key}` } : {}), ...setupHeaders(telemetry?.properties) },
124
154
  ...(body ? { body: JSON.stringify(body) } : {}),
125
155
  });
126
156
  } catch { throw new Stop(`Could not reach ${server()}. Check your connection and retry.`, exitCodes.unreachable); }
@@ -134,12 +164,13 @@ function detectClient() {
134
164
  }
135
165
 
136
166
  // Claude Code locations. Project scope keeps the key out of the repository:
137
- // the MCP entry uses Claude's private "local" scope, only the skill is in the project.
167
+ // the MCP entry uses Claude's private "local" scope; rule and skill are in the project.
138
168
  function claudePaths(root, scope) {
139
169
  const config = resolve(process.env.CLAUDE_CONFIG_DIR || join(homedir(), '.claude'));
140
170
  const base = scope === 'user' ? config : join(root, '.claude');
141
171
  return {
142
172
  skill: join(base, 'skills/openmerit-scout/SKILL.md'),
173
+ rule: join(base, 'rules/openmerit-scout.md'),
143
174
  legacyRule: join(base, 'rules/openmerit.md'),
144
175
  record: join(base, 'openmerit-install.json'),
145
176
  mcpFile: process.env.CLAUDE_CONFIG_DIR ? join(config, '.claude.json') : join(homedir(), '.claude.json'),
@@ -147,7 +178,7 @@ function claudePaths(root, scope) {
147
178
  };
148
179
  }
149
180
  const configuredServer = (config, root, scope) => scope === 'user' ? config.mcpServers?.openmerit : config.projects?.[root]?.mcpServers?.openmerit;
150
- const serverConfig = key => ({ type: 'http', url: `${server()}/scout/mcp`, headers: { Authorization: `Bearer ${key}`, 'X-Scout-Analytics-Content': 'off' } });
181
+ const serverConfig = (key, telemetry) => ({ type: 'http', url: `${server()}/scout/mcp`, headers: { Authorization: `Bearer ${key}`, ...setupHeaders(telemetry.properties, 'agent') } });
151
182
  const canonical = value => Array.isArray(value) ? value.map(canonical) : value && typeof value === 'object' ? Object.fromEntries(Object.keys(value).sort().map(k => [k, canonical(value[k])])) : value;
152
183
  const serverHash = value => value ? hash(JSON.stringify(canonical(value))) : null;
153
184
 
@@ -164,16 +195,18 @@ async function claude(args, root, key) {
164
195
  // Inspect the current Claude Code installation without changing anything.
165
196
  async function inspect(root, scope) {
166
197
  const paths = claudePaths(root, scope);
167
- const [record, config, installedSkill, legacyRule] = await Promise.all([object(paths.record), object(paths.mcpFile), optional(paths.skill), optional(paths.legacyRule)]);
168
- const owned = record.version === 2 && record.scope === scope;
198
+ const [record, config, installedSkill, installedRule, legacyRule] = await Promise.all([object(paths.record), object(paths.mcpFile), optional(paths.skill), optional(paths.rule), optional(paths.legacyRule)]);
199
+ const owned = [2, 3].includes(record.version) && record.scope === scope;
169
200
  const current = configuredServer(config, root, scope);
170
201
  const otherScope = configuredServer(config, root, scope === 'user' ? 'project' : 'user');
171
202
  const projectFile = await object(join(root, '.mcp.json'));
172
203
  return {
173
- paths, record, owned, current, installedSkill,
174
- // A pending header hash covers an install interrupted between add and record.
175
- serverOwned: !current || owned && (serverHash(current) === record.serverHash || !!record.pendingAuth && hash(current.headers?.Authorization ?? '') === record.pendingAuth),
204
+ paths, record, owned, current, installedSkill, installedRule,
205
+ // A full pending hash covers interruption without accepting edited headers.
206
+ // Keep pendingAuth compatibility for records from the earlier installer.
207
+ serverOwned: !current || owned && (serverHash(current) === record.serverHash || serverHash(current) === record.pendingServerHash || !!record.pendingAuth && hash(current.headers?.Authorization ?? '') === record.pendingAuth),
176
208
  skillOwned: installedSkill === null || owned && [record.skillHash, record.pendingSkill].includes(hash(installedSkill)),
209
+ ruleOwned: installedRule === null || owned && record.version === 3 && [record.ruleHash, record.pendingRule].includes(hash(installedRule)),
177
210
  // The pre-release installer recorded its rule hash; only an unchanged rule is removed.
178
211
  legacyRule: legacyRule !== null && record.version === 1 && hash(legacyRule) === record.ruleHash ? 'remove' : legacyRule !== null ? 'keep' : 'none',
179
212
  shadowed: !!(otherScope || projectFile.mcpServers?.openmerit),
@@ -184,10 +217,20 @@ function conflicts(state) {
184
217
  const found = [];
185
218
  if (!state.serverOwned) found.push('Claude Code already has an MCP server named "openmerit" that this installer did not create or that was edited. Remove it with "claude mcp remove openmerit", then retry.');
186
219
  if (!state.skillOwned) found.push(`${state.paths.skill} exists and was not created by this installer or was edited. Move or delete it, then retry.`);
220
+ if (!state.ruleOwned) found.push(`${state.paths.rule} exists and was not created by this installer or was edited. Move or delete it, then retry.`);
187
221
  if (state.shadowed) found.push('An "openmerit" MCP server exists in another Claude Code scope (or in .mcp.json). Remove it, then retry.');
188
222
  return found;
189
223
  }
190
224
 
225
+ function diagnostics(state) {
226
+ return {
227
+ installed: state.owned && !!state.current && state.serverOwned && state.installedSkill !== null && state.skillOwned && state.installedRule !== null && state.ruleOwned && !state.shadowed,
228
+ server_status: state.current ? state.serverOwned ? 'matches' : 'modified_or_unowned' : 'missing',
229
+ skill_status: state.installedSkill === null ? 'missing' : state.skillOwned ? 'matches' : 'modified_or_unowned',
230
+ rule_status: state.installedRule === null ? 'missing' : state.ruleOwned ? 'matches' : 'modified_or_unowned',
231
+ };
232
+ }
233
+
191
234
  async function credentials() {
192
235
  const saved = await object(credentialsPath());
193
236
  return saved.version === 1 && saved.server === server() && typeof saved.key === 'string' ? saved : null;
@@ -195,33 +238,56 @@ async function credentials() {
195
238
 
196
239
  // Device login. Starts or resumes a login, waits a bounded time, and stores the
197
240
  // key only on disk. Never prints the key.
198
- async function login({ client, scope, open, log, wait }) {
241
+ async function login({ client, scope, open, log, wait, telemetry }) {
242
+ await telemetry.step('login', 'started');
199
243
  let pending = await object(pendingPath());
200
244
  if (!(pending.server === server() && pending.expiresAt > Date.now() && typeof pending.deviceCode === 'string')) {
201
245
  const deviceName = (process.env.OPENMERIT_DEVICE_NAME ?? hostname()).replace(/[^\x20-\x7e]/g, '').slice(0, 64) || 'this device';
202
- const { status, data } = await request('/connect/api/device', { method: 'POST', body: { client, scope, device_name: deviceName } });
246
+ const { status, data } = await request('/connect/api/device', { method: 'POST', body: { client, scope, device_name: deviceName }, telemetry });
203
247
  if (status !== 200 || !data.device_code) throw new Stop(data.error ?? `Could not start sign-in (HTTP ${status}).`, status === 429 ? exitCodes.unreachable : exitCodes.error);
204
- pending = { server: server(), deviceCode: data.device_code, userCode: data.user_code, url: data.verification_uri_complete, interval: data.interval, expiresAt: Date.now() + data.expires_in * 1000, opened: false };
248
+ pending = { server: server(), deviceCode: data.device_code, userCode: data.user_code, url: data.verification_uri_complete, interval: data.interval, expiresAt: Date.now() + data.expires_in * 1000, opened: false, setup_id: telemetry.properties.setup_id, installation_id: telemetry.properties.installation_id };
205
249
  await save(pendingPath(), pending);
206
250
  }
251
+ const approvalURL = new URL(pending.url);
252
+ // Rebuild consent on every resume, so old query parameters cannot override a
253
+ // later refusal. Only opt-in setup IDs are ever sent to the approval page.
254
+ for (const key of ['setup_id', 'analytics', 'analytics_account']) approvalURL.searchParams.delete(key);
255
+ if (setupCorrelationEnabled() && setupUUID(telemetry.properties.setup_id)) approvalURL.searchParams.set('setup_id', telemetry.properties.setup_id);
256
+ // Browser usage consent is independent of CLI opt-in. Only an explicit
257
+ // cross-surface refusal disables it; account unlinking must not disable usage.
258
+ if (process.env.DO_NOT_TRACK === '1') approvalURL.searchParams.set('analytics', 'off');
259
+ if (process.env.OPENMERIT_ANALYTICS_ACCOUNT === 'off' || process.env.DO_NOT_TRACK === '1') approvalURL.searchParams.set('analytics_account', 'off');
260
+ pending = { ...pending, url: approvalURL.href, setup_id: telemetry.properties.setup_id, installation_id: telemetry.properties.installation_id };
261
+ await save(pendingPath(), pending);
207
262
  log(`Approve this device in your browser (opening it now):\n\n ${pending.url}\n\n Code: ${pending.userCode} (expires ${new Date(pending.expiresAt).toLocaleTimeString()})\n`);
263
+ await telemetry.step('browser_open', 'started');
208
264
  if (open && !pending.opened && !pending.openFailed) {
209
265
  // Only the approval page on the configured OpenMerit server may be opened.
210
- const approvalPage = url => { const u = new URL(url); return `${u.origin}` === server() && u.pathname === '/connect' && [...u.searchParams.keys()].join() === 'code' && !u.username && !u.password && !u.hash; };
266
+ const approvalPage = url => {
267
+ const u = new URL(url);
268
+ return u.origin === server() && u.pathname === '/connect' && !!u.searchParams.get('code')
269
+ && [...u.searchParams.keys()].every(key => ['code', 'setup_id', 'analytics', 'analytics_account'].includes(key) && u.searchParams.getAll(key).length === 1)
270
+ && (!u.searchParams.has('setup_id') || setupUUID(u.searchParams.get('setup_id')))
271
+ && ['analytics', 'analytics_account'].every(key => !u.searchParams.has(key) || u.searchParams.get(key) === 'off')
272
+ && !u.username && !u.password && !u.hash;
273
+ };
211
274
  const opened = await openBrowser(pending.url, { allowed: approvalPage }).catch(() => false);
212
275
  pending = { ...pending, ...(opened ? { opened: true } : { openFailed: true }) };
213
276
  await save(pendingPath(), pending);
214
277
  }
278
+ await telemetry.step('browser_open', pending.opened ? 'ok' : pending.openFailed ? 'error' : 'pending');
279
+ await telemetry.step('login', 'started');
215
280
  // With the approval page open, keep waiting for the Approve click (inside a
216
281
  // coding agent's 10-minute command limit) so nobody has to rerun anything.
217
282
  // Without a browser, return soon so the agent can show the link.
218
283
  const deadline = Date.now() + (wait ?? (pending.opened ? 540 : 30)) * 1000;
219
284
  let interval = (pending.interval ?? 5) * 1000;
220
285
  for (;;) {
221
- const { status, data } = await request('/connect/api/token', { method: 'POST', body: { device_code: pending.deviceCode } });
286
+ const { status, data } = await request('/connect/api/token', { method: 'POST', body: { device_code: pending.deviceCode }, telemetry });
222
287
  if (status === 200 && data.access_token) {
223
- await save(credentialsPath(), { version: 1, server: server(), key: data.access_token, keyId: data.key_id, createdAt: new Date().toISOString() });
288
+ await save(credentialsPath(), { version: 1, server: server(), key: data.access_token, keyId: data.key_id, setup_id: telemetry.properties.setup_id, createdAt: new Date().toISOString() });
224
289
  await rm(pendingPath(), { force: true });
290
+ await telemetry.step('login', 'ok');
225
291
  return data.access_token;
226
292
  }
227
293
  if (data.error === 'slow_down') interval += 5000;
@@ -236,80 +302,110 @@ async function login({ client, scope, open, log, wait }) {
236
302
  }
237
303
 
238
304
  // A real MCP initialize and tool discovery with the stored key; no search.
239
- async function verify(key) {
240
- const call = (id, method, params = {}) => request('/scout/mcp', { method: 'POST', key, body: { jsonrpc: '2.0', id, method, params } });
241
- const init = await call(1, 'initialize', { protocolVersion: '2025-06-18', capabilities: {}, clientInfo: { name: 'openmerit-cli', version: '1' } });
242
- if (init.status === 401) return 'key_rejected';
243
- if (init.status !== 200 || !init.data.result) return 'failed';
244
- const tools = await call(2, 'tools/list');
245
- return tools.data.result?.tools?.some(tool => tool.name === 'scout_models') ? 'connected' : 'failed';
305
+ async function verify(key, telemetry) {
306
+ await telemetry.step('verification', 'started');
307
+ const call = (id, method, params = {}) => request('/scout/mcp', { method: 'POST', key, telemetry, body: { jsonrpc: '2.0', id, method, params } });
308
+ const init = await call(1, 'initialize', { protocolVersion: '2025-06-18', capabilities: {}, clientInfo: { name: 'openmerit-cli', version: clientVersion ?? 'unknown' } });
309
+ let connection = init.status === 401 ? 'key_rejected' : 'failed';
310
+ if (init.status === 200 && init.data.result) {
311
+ const tools = await call(2, 'tools/list');
312
+ connection = tools.status === 401 ? 'key_rejected' : tools.status === 200 && tools.data.result?.tools?.some(tool => tool.name === 'scout_models') ? 'connected' : 'failed';
313
+ }
314
+ await telemetry.step('verification', connection === 'connected' ? 'ok' : 'error', { connection, ...(connection === 'connected' ? {} : { code: connection === 'key_rejected' ? 'access_denied' : 'command_error' }) });
315
+ return connection;
246
316
  }
247
317
 
248
- async function install(root, { client, scope, plan, open, log, wait }) {
318
+ async function install(root, { client, scope, plan, open, log, wait, telemetry }) {
249
319
  const state = await inspect(root, scope);
250
320
  const found = conflicts(state);
251
321
  const changes = [
252
322
  `${state.current ? 'Update' : 'Add'} MCP server "openmerit" in Claude Code (${state.paths.mcpScope} scope, ${state.paths.mcpFile})`,
253
323
  `${state.installedSkill === null ? 'Create' : 'Update'} ${state.paths.skill}`,
324
+ `${state.installedRule === null ? 'Create' : 'Update'} ${state.paths.rule} (loaded at session startup)`,
254
325
  ...(state.legacyRule === 'remove' ? [`Remove the unchanged earlier instructions at ${state.paths.legacyRule}`] : []),
255
326
  ...(await credentials() ? [] : [`Sign this device in through the browser and store its key in ${credentialsPath()} (mode 600)`]),
256
327
  ];
257
328
  if (plan) return { action: 'install', applied: false, client, scope, changes, conflicts: found };
258
329
  if (found.length) throw new Stop(found.join('\n'), exitCodes.conflict, 'Nothing was changed.');
259
330
  await claude(['--version'], root);
331
+ if (setupCorrelationEnabled()) telemetry.properties.installation_id ??= randomUUID();
332
+ await telemetry.step('preflight', 'ok');
260
333
  let key = (await credentials())?.key;
261
334
  // A revoked or expired key gets a fresh sign-in instead of a broken install.
262
- if (key && await verify(key) === 'key_rejected') { await rm(credentialsPath(), { force: true }); key = undefined; log('The saved device key is no longer valid; signing in again.'); }
263
- key ??= await login({ client, scope, open, log, wait });
264
- const next = serverConfig(key);
335
+ if (key && await verify(key, telemetry) === 'key_rejected') { await rm(credentialsPath(), { force: true }); key = undefined; log('The saved device key is no longer valid; signing in again.'); }
336
+ key ??= await login({ client, scope, open, log, wait, telemetry });
337
+ await telemetry.step('configuration', 'started');
338
+ const next = serverConfig(key, telemetry);
339
+ const ids = { setup_id: telemetry.properties.setup_id, installation_id: telemetry.properties.installation_id };
265
340
  // Record intent first so an interrupted install is repaired by rerunning.
266
- await save(state.paths.record, { version: 2, scope, client, serverHash: serverHash(state.current), skillHash: state.installedSkill === null ? null : hash(state.installedSkill), pendingAuth: hash(next.headers.Authorization), pendingSkill: hash(skill) });
267
- if (state.current?.url !== next.url || state.current?.headers?.Authorization !== next.headers.Authorization) {
341
+ await save(state.paths.record, { version: 3, scope, client, ...ids, serverHash: serverHash(state.current), skillHash: state.installedSkill === null ? null : hash(state.installedSkill), ruleHash: state.installedRule === null ? null : hash(state.installedRule), pendingServerHash: serverHash(next), pendingSkill: hash(skill), pendingRule: hash(rule) });
342
+ telemetry.properties.applied = true;
343
+ if (serverHash(state.current) !== serverHash(next)) {
268
344
  if (state.current) await claude(['mcp', 'remove', '--scope', state.paths.mcpScope, 'openmerit'], root, key);
269
345
  await claude(['mcp', 'add-json', '--scope', state.paths.mcpScope, 'openmerit', JSON.stringify(next)], root, key);
270
346
  }
271
347
  await save(state.paths.skill, skill);
348
+ await save(state.paths.rule, rule);
272
349
  if (state.legacyRule === 'remove') await rm(state.paths.legacyRule, { force: true });
273
350
  // Record what Claude actually wrote, so later edits are detected precisely.
274
351
  const written = configuredServer(await object(state.paths.mcpFile), root, scope);
275
352
  if (!written) throw new Stop('Claude Code did not save the "openmerit" MCP server.', exitCodes.error, 'Run "openmerit doctor".');
276
- await save(state.paths.record, { version: 2, scope, client, serverHash: serverHash(written), skillHash: hash(skill), installedAt: new Date().toISOString() });
277
- const connection = await verify(key);
353
+ await save(state.paths.record, { version: 3, scope, client, ...ids, serverHash: serverHash(written), skillHash: hash(skill), ruleHash: hash(rule), installedAt: new Date().toISOString() });
354
+ await telemetry.step('configuration', 'ok', diagnostics(await inspect(root, scope)));
355
+ const connection = await verify(key, telemetry);
356
+ if (connection !== 'connected') telemetry.outcome = 'error';
278
357
  if (connection === 'key_rejected') throw new Stop('OpenMerit rejected this device key (expired or revoked).', exitCodes.denied, 'Run "openmerit login", then rerun install.');
279
358
  return { action: 'install', applied: true, client, scope, changes, connection,
280
359
  next: connection === 'connected'
281
- ? `Scout is ready in new ${clients[client]} sessions. In a session that is already open, run /mcp and reconnect "openmerit".`
360
+ ? `Scout is ready in new ${clients[client]} sessions. Start a new session to load the startup rule and updated skill; reconnecting /mcp alone does not reload instructions.`
282
361
  : 'Installed, but the connection check failed. Run "openmerit doctor".' };
283
362
  }
284
363
 
285
- async function status(root, { client, scope, check }) {
364
+ async function status(root, { client, scope, check, telemetry }) {
365
+ await telemetry.step('status', 'started');
286
366
  const state = await inspect(root, scope), saved = await credentials();
287
- const installed = state.owned && !!state.current && state.serverOwned && state.installedSkill !== null && state.skillOwned && !state.shadowed;
288
- return { action: 'status', client, scope, installed, signedIn: !!saved,
289
- server: state.current ? state.serverOwned ? 'matches' : 'modified_or_unowned' : 'missing',
290
- skill: state.installedSkill === null ? 'missing' : state.skillOwned ? 'matches' : 'modified_or_unowned',
291
- conflicts: conflicts(state), connection: check && saved ? await verify(saved.key) : 'not_checked' };
367
+ const details = diagnostics(state), found = conflicts(state);
368
+ const connection = check && saved ? await verify(saved.key, telemetry) : 'not_checked';
369
+ telemetry.outcome = ['failed', 'key_rejected'].includes(connection) ? 'error' : found.length ? 'conflict' : details.installed ? 'ok' : 'not_installed';
370
+ await telemetry.step('status', telemetry.outcome, { ...details, connection });
371
+ return { action: 'status', client, scope, installed: details.installed, signedIn: !!saved,
372
+ server: details.server_status, skill: details.skill_status, rule: details.rule_status,
373
+ conflicts: found, connection };
292
374
  }
293
375
 
294
- async function revokeKey() {
376
+ async function revokeKey(telemetry) {
295
377
  const saved = await credentials();
296
378
  if (!saved) return false;
297
- const { status } = await request('/connect/api/keys/self', { method: 'DELETE', key: saved.key }).catch(() => ({ status: 0 }));
379
+ await telemetry.step('remove', 'started');
380
+ const { status } = await request('/connect/api/keys/self', { method: 'DELETE', key: saved.key, telemetry }).catch(() => ({ status: 0 }));
298
381
  await rm(credentialsPath(), { force: true });
382
+ if (status !== 200) telemetry.outcome = 'error';
383
+ await telemetry.step('remove', status === 200 ? 'ok' : 'error', status === 200 ? {} : {
384
+ code: status === 0 ? 'openmerit_unreachable' : status === 401 || status === 403 ? 'access_denied' : 'command_error',
385
+ });
299
386
  return status === 200;
300
387
  }
301
388
 
302
- async function remove(root, { client, scope, plan }) {
389
+ async function remove(root, { client, scope, plan, telemetry }) {
390
+ await telemetry.step('remove', 'started');
303
391
  const state = await inspect(root, scope);
304
- if (!state.owned) return { action: 'remove', applied: false, client, scope, changes: [], note: 'No OpenMerit installation by this installer in this scope.' };
392
+ if (!state.owned) {
393
+ telemetry.outcome = 'not_installed';
394
+ await telemetry.step('remove', 'not_installed');
395
+ return { action: 'remove', applied: false, client, scope, changes: [], note: 'No OpenMerit installation by this installer in this scope.' };
396
+ }
305
397
  const changes = [...(state.current && state.serverOwned ? [`Remove MCP server "openmerit" (${state.paths.mcpScope} scope)`] : []),
306
- ...(state.installedSkill !== null && state.skillOwned ? [`Delete ${state.paths.skill}`] : []), 'Revoke and delete this device key'];
307
- const kept = [...(state.serverOwned ? [] : ['the edited "openmerit" MCP server']), ...(state.skillOwned ? [] : [state.paths.skill]), policyPath];
398
+ ...(state.installedSkill !== null && state.skillOwned ? [`Delete ${state.paths.skill}`] : []),
399
+ ...(state.installedRule !== null && state.ruleOwned ? [`Delete ${state.paths.rule}`] : []), 'Revoke and delete this device key'];
400
+ const kept = [...(state.serverOwned ? [] : ['the edited "openmerit" MCP server']), ...(state.skillOwned ? [] : [state.paths.skill]), ...(state.ruleOwned ? [] : [state.paths.rule]), policyPath];
308
401
  if (plan) return { action: 'remove', applied: false, client, scope, changes, keeps: kept };
402
+ telemetry.properties.applied = true;
309
403
  if (state.current && state.serverOwned) await claude(['mcp', 'remove', '--scope', state.paths.mcpScope, 'openmerit'], root);
310
404
  if (state.installedSkill !== null && state.skillOwned) await rm(dirname(state.paths.skill), { recursive: true, force: true });
405
+ if (state.installedRule !== null && state.ruleOwned) await rm(state.paths.rule, { force: true });
311
406
  await rm(state.paths.record, { force: true });
312
- const revoked = await revokeKey();
407
+ const revoked = await revokeKey(telemetry);
408
+ await telemetry.step('remove', telemetry.outcome, diagnostics(await inspect(root, scope)));
313
409
  return { action: 'remove', applied: true, client, scope, changes, keeps: kept, keyRevoked: revoked };
314
410
  }
315
411
 
@@ -322,26 +418,29 @@ async function project(root, { policy, ids, plan }) {
322
418
  return { action: 'project', applied: !plan, path: policyPath, policy: value, note: 'Endpoint IDs limit discovery only; they do not prove access or authorize spending.' };
323
419
  }
324
420
 
325
- async function doctor(root, { scope }) {
421
+ async function doctor(root, { scope, telemetry }) {
422
+ await telemetry.step('status', 'started');
326
423
  const checks = [];
327
424
  const add = (name, ok, detail) => checks.push({ name, ok, detail });
328
425
  const [major, minor] = process.versions.node.split('.').map(Number);
329
426
  add('node', major > 22 || major === 22 && minor >= 19, `Node.js ${process.versions.node} (22.19+ required)`);
330
427
  try { const { stdout } = await exec('claude', ['--version'], { timeout: 10_000 }); add('claude', true, stdout.trim().slice(0, 80)); }
331
428
  catch { add('claude', false, 'Claude Code (claude) is not on PATH'); }
332
- const reach = await request('/scout/api/status').catch(error => ({ status: 0, error }));
429
+ const reach = await request('/scout/api/status', { telemetry }).catch(error => ({ status: 0, error }));
333
430
  add('openmerit', reach.status === 200, reach.status === 200 ? `${server()} reachable` : `${server()} unreachable`);
334
431
  const saved = await credentials();
335
432
  add('key', !!saved, saved ? `Device key ${saved.keyId ?? ''} stored in ${credentialsPath()}` : 'Not signed in. Run "openmerit install" or "openmerit login".');
336
- if (saved && reach.status === 200) { const connection = await verify(saved.key); add('mcp', connection === 'connected', `MCP check: ${connection}`); }
433
+ if (saved && reach.status === 200) { const connection = await verify(saved.key, telemetry); add('mcp', connection === 'connected', `MCP check: ${connection}`); }
337
434
  const state = await inspect(root, scope).catch(() => null);
338
- if (state) add('install', state.owned && !conflicts(state).length, state.owned ? conflicts(state).join(' ') || `Installed (${scope} scope)` : `Not installed in ${scope} scope`);
435
+ if (state) add('install', state.owned && !!state.current && state.installedSkill !== null && state.installedRule !== null && !conflicts(state).length, state.owned ? conflicts(state).join(' ') || (state.current && state.installedSkill !== null && state.installedRule !== null ? `Installed (${scope} scope)` : 'Installation incomplete. Rerun "openmerit install".') : `Not installed in ${scope} scope`);
436
+ telemetry.outcome = checks.every(check => check.ok) ? 'ok' : 'error';
437
+ await telemetry.step('status', telemetry.outcome, { ...(state ? diagnostics(state) : {}), ...(reach.status === 200 ? {} : { code: 'openmerit_unreachable', connection: 'failed' }) });
339
438
  return { action: 'doctor', ok: checks.every(check => check.ok), checks };
340
439
  }
341
440
 
342
441
  function render(result) {
343
442
  if (result.action === 'doctor') return result.checks.map(c => `${c.ok ? '✓' : '✗'} ${c.name}: ${c.detail}`).join('\n');
344
- if (result.action === 'status') return [`OpenMerit in ${clients[result.client]} (${result.scope} scope): ${result.installed ? 'installed' : 'not installed'}`, `Signed in: ${result.signedIn ? 'yes' : 'no'} · MCP server: ${result.server} · skill: ${result.skill} · connection: ${result.connection}`, ...result.conflicts].join('\n');
443
+ if (result.action === 'status') return [`OpenMerit in ${clients[result.client]} (${result.scope} scope): ${result.installed ? 'installed' : 'not installed'}`, `Signed in: ${result.signedIn ? 'yes' : 'no'} · MCP server: ${result.server} · skill: ${result.skill} · rule: ${result.rule} · connection: ${result.connection}`, ...result.conflicts].join('\n');
345
444
  if (result.action === 'project') return `${result.applied ? 'Saved' : 'Would save'} ${result.path}: ${result.policy.policy}, ${result.policy.endpointIds.length} endpoint ID(s). ${result.note}`;
346
445
  if (result.action === 'login') return result.note;
347
446
  if (result.action === 'logout') return result.note;
@@ -352,12 +451,39 @@ function render(result) {
352
451
  return lines.join('\n');
353
452
  }
354
453
 
355
- export async function main(args, { root = process.cwd(), print = console.log, printError = console.error } = {}) {
454
+ export async function main(args, dependencies = {}) {
455
+ const telemetry = {
456
+ stage: 'started', outcome: 'ok',
457
+ properties: { command: ['install', 'status', 'remove', 'login', 'logout', 'project', 'doctor'].includes(args[0]) ? args[0] : 'unknown', agent: 'claude-code', scope: 'user', applied: false, installed: false, connection: 'not_checked' },
458
+ async step(stage, outcome, properties = {}) {
459
+ this.stage = stage;
460
+ if (outcome === 'ok') delete this.properties.code;
461
+ Object.assign(this.properties, properties);
462
+ await sendInstallationEvent({ ...this.properties, stage, outcome });
463
+ },
464
+ async fail(error) {
465
+ this.outcome = ({ 10: 'pending', 11: 'denied', 20: 'conflict' })[error.exit] ?? 'error';
466
+ const code = ({ 10: 'approval_pending', 11: 'access_denied', 20: 'install_conflict', 30: 'openmerit_unreachable' })[error.exit] ?? 'command_error';
467
+ await this.step(this.stage, this.outcome, { code, ...(this.stage === 'verification' ? { connection: error.exit === exitCodes.denied ? 'key_rejected' : 'failed' } : {}) });
468
+ },
469
+ };
470
+ try {
471
+ const code = await runMain(args, dependencies, telemetry);
472
+ await telemetry.step('completed', telemetry.outcome);
473
+ return code;
474
+ } catch (error) {
475
+ await telemetry.fail(error);
476
+ await telemetry.step('completed', telemetry.outcome);
477
+ throw error;
478
+ }
479
+ }
480
+
481
+ async function runMain(args, { root = process.cwd(), print = console.log, printError = console.error }, telemetry) {
356
482
  const [action, ...rest] = args;
357
483
  if (!action || rest.includes('--help') || rest.includes('-h')) { print(help); return exitCodes.ok; }
358
- const options = { client: undefined, scope: 'user', plan: false, check: false, json: false, open: true, policy: undefined, ids: [] };
359
- const allowed = { install: ['--client', '--scope', '--plan', '--json', '--no-open'], status: ['--client', '--scope', '--check', '--json'], remove: ['--client', '--scope', '--plan', '--json'],
360
- login: ['--json', '--no-open'], logout: ['--json'], project: ['--policy', '--endpoint-id', '--plan', '--json'], doctor: ['--scope', '--json'] }[action];
484
+ const options = { client: undefined, scope: 'user', plan: false, check: false, json: false, open: true, policy: undefined, ids: [], telemetry };
485
+ const allowed = { install: ['--client', '--scope', '--setup-id', '--plan', '--json', '--no-open'], status: ['--client', '--scope', '--check', '--json'], remove: ['--client', '--scope', '--plan', '--json'],
486
+ login: ['--setup-id', '--json', '--no-open'], logout: ['--json'], project: ['--policy', '--endpoint-id', '--plan', '--json'], doctor: ['--scope', '--json'] }[action];
361
487
  if (!allowed) throw new Error('Unknown command. Run openmerit --help.');
362
488
  for (let i = 0; i < rest.length; i++) {
363
489
  const option = rest[i];
@@ -365,6 +491,7 @@ export async function main(args, { root = process.cwd(), print = console.log, pr
365
491
  const value = () => { const v = rest[++i]; if (v === undefined || v.startsWith('--')) throw new Error(`${option} needs a value.`); return v; };
366
492
  if (option === '--client') options.client = value();
367
493
  else if (option === '--scope') options.scope = value();
494
+ else if (option === '--setup-id') { options.setup_id = value(); if (!setupUUID(options.setup_id)) throw new Error('--setup-id must be a UUID v4.'); }
368
495
  else if (option === '--policy') options.policy = value();
369
496
  else if (option === '--endpoint-id') options.ids.push(value());
370
497
  else if (option === '--no-open') options.open = false;
@@ -381,17 +508,41 @@ export async function main(args, { root = process.cwd(), print = console.log, pr
381
508
  const wait = process.env.OPENMERIT_LOGIN_WAIT === undefined ? undefined : Number(process.env.OPENMERIT_LOGIN_WAIT);
382
509
  const log = text => (options.json ? printError : print)(text);
383
510
  try {
511
+ telemetry.properties.scope = options.scope;
512
+ // Hydrate correlation without creating identities or writing anything. A
513
+ // pending approval owns its setup ID even when the resumed command omits it.
514
+ if (['install', 'status', 'remove', 'doctor'].includes(action)) {
515
+ telemetry.stage = 'preflight';
516
+ const state = await inspect(root, options.scope);
517
+ Object.assign(telemetry.properties, diagnostics(state));
518
+ if (state.owned) for (const key of ['setup_id', 'installation_id']) if (setupUUID(state.record[key])) telemetry.properties[key] = state.record[key];
519
+ }
520
+ if (options.setup_id) telemetry.properties.setup_id = options.setup_id;
521
+ if (['install', 'login'].includes(action)) {
522
+ const saved = await credentials();
523
+ if (!telemetry.properties.setup_id && setupUUID(saved?.setup_id)) telemetry.properties.setup_id = saved.setup_id;
524
+ const pending = await object(pendingPath());
525
+ if (pending.server === server() && pending.expiresAt > Date.now() && typeof pending.deviceCode === 'string') {
526
+ if (setupUUID(pending.setup_id)) telemetry.properties.setup_id = pending.setup_id;
527
+ if (!telemetry.properties.installation_id && setupUUID(pending.installation_id)) telemetry.properties.installation_id = pending.installation_id;
528
+ }
529
+ }
530
+ await telemetry.step('started', 'started');
531
+ await telemetry.step('preflight', 'started');
384
532
  const result = action === 'install' ? await install(root, { ...options, log, wait })
385
533
  : action === 'status' ? await status(root, options)
386
534
  : action === 'remove' ? await remove(root, options)
387
535
  : action === 'project' ? await project(root, options)
388
536
  : action === 'doctor' ? await doctor(root, options)
389
- : action === 'logout' ? { action, revoked: await revokeKey(), note: 'Signed out. This device key was revoked and deleted. Agents configured with it stop working until you run install again.' }
390
- : (await revokeKey(), await rm(pendingPath(), { force: true }), await login({ client: 'claude-code', scope: 'user', open: options.open, log, wait }), { action, signedIn: true, note: 'Signed in. Run "openmerit install" again to give your coding agents the new key.' });
537
+ : action === 'logout' ? { action, revoked: await revokeKey(telemetry), note: 'Signed out. This device key was revoked and deleted. Agents configured with it stop working until you run install again.' }
538
+ : (await revokeKey(telemetry), await login({ client: 'claude-code', scope: 'user', open: options.open, log, wait, telemetry }), { action, signedIn: true, note: 'Signed in. Run "openmerit install" again to give your coding agents the new key.' });
539
+ if (result.conflicts?.length && telemetry.outcome !== 'error') { telemetry.outcome = 'conflict'; telemetry.properties.code = 'install_conflict'; }
540
+ if (options.plan) await telemetry.step(telemetry.stage, telemetry.outcome, { applied: false });
391
541
  print(options.json ? JSON.stringify({ ok: true, ...result }, null, 2) : render(result));
392
542
  return action === 'doctor' && !result.ok ? exitCodes.error : exitCodes.ok;
393
543
  } catch (error) {
394
544
  if (!(error instanceof Stop)) throw error;
545
+ await telemetry.fail(error);
395
546
  if (options.json) print(JSON.stringify({ ok: false, action, code: Object.keys(exitCodes).find(k => exitCodes[k] === error.exit), message: error.message, next: error.next }, null, 2));
396
547
  else printError([error.message, error.next].filter(Boolean).join('\n'));
397
548
  return error.exit;
@@ -1,7 +1,7 @@
1
1
  import { randomUUID } from 'node:crypto';
2
2
  import { readFileSync } from 'node:fs';
3
3
 
4
- let clientVersion;
4
+ export let clientVersion;
5
5
  for (const path of ['../../../package.json', '../../../../package.json']) {
6
6
  try { const pkg = JSON.parse(readFileSync(new URL(path, import.meta.url), 'utf8')); if (pkg.name === 'openmerit') { clientVersion = pkg.version; break; } } catch {}
7
7
  }
@@ -31,6 +31,74 @@ const environment = () => ({ ...(clientVersion ? { client_version: clientVersion
31
31
 
32
32
  const anonymousId = randomUUID();
33
33
  export const telemetryEnabled = () => process.env.OPENMERIT_TELEMETRY === '1' && process.env.DO_NOT_TRACK !== '1';
34
+ export const setupCorrelationEnabled = () => telemetryEnabled() && process.env.OPENMERIT_ANALYTICS_ACCOUNT !== 'off';
35
+ export const setupUUID = value => typeof value === 'string' && /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i.test(value);
36
+ // Device/verifier traffic is CLI opt-in. Installed MCP usage retains its existing
37
+ // default, but explicit privacy refusals must survive into future agent sessions.
38
+ export function setupHeaders(ids = {}, client = 'cli') {
39
+ return {
40
+ 'X-Scout-Client': client,
41
+ 'X-Scout-Agent': 'claude-code',
42
+ ...(clientVersion ? { 'X-Scout-Client-Version': clientVersion } : {}),
43
+ 'X-Scout-Analytics-Content': 'off',
44
+ ...(client === 'cli' ? { 'X-Scout-Analytics': telemetryEnabled() ? 'on' : 'off' } : process.env.DO_NOT_TRACK === '1' ? { 'X-Scout-Analytics': 'off' } : {}),
45
+ ...(process.env.OPENMERIT_ANALYTICS_ACCOUNT === 'off' || process.env.DO_NOT_TRACK === '1' ? { 'X-Scout-Analytics-Account': 'off' } : {}),
46
+ ...(setupCorrelationEnabled() && setupUUID(ids.setup_id) ? { 'X-Scout-Setup-Id': ids.setup_id } : {}),
47
+ ...(setupCorrelationEnabled() && setupUUID(ids.installation_id) ? { 'X-Scout-Installation-Id': ids.installation_id } : {}),
48
+ };
49
+ }
50
+ export function installationProperties(input) {
51
+ if (!input || typeof input !== 'object' || Array.isArray(input)) return null;
52
+ const out = {};
53
+ const enums = {
54
+ stage: ['started', 'preflight', 'login', 'browser_open', 'configuration', 'verification', 'status', 'remove', 'completed'],
55
+ outcome: ['started', 'ok', 'error', 'pending', 'denied', 'conflict', 'not_installed'],
56
+ command: ['install', 'status', 'remove', 'login', 'logout', 'project', 'doctor', 'unknown'],
57
+ agent: ['claude-code'], scope: ['user', 'project'],
58
+ rule_status: ['matches', 'missing', 'modified_or_unowned'],
59
+ skill_status: ['matches', 'missing', 'modified_or_unowned'],
60
+ server_status: ['matches', 'missing', 'modified_or_unowned'],
61
+ connection: ['connected', 'failed', 'key_rejected', 'not_checked'],
62
+ code: ['command_error', 'approval_pending', 'access_denied', 'install_conflict', 'openmerit_unreachable'],
63
+ };
64
+ for (const [key, values] of Object.entries(enums)) {
65
+ if (input[key] === undefined && !['stage', 'outcome'].includes(key)) continue;
66
+ if (!values.includes(input[key])) return null;
67
+ out[key] = input[key];
68
+ }
69
+ for (const key of ['setup_id', 'installation_id']) if (setupUUID(input[key])) out[key] = input[key];
70
+ for (const key of ['applied', 'installed']) if (typeof input[key] === 'boolean') out[key] = input[key];
71
+ return out;
72
+ }
73
+ export async function sendInstallationEvent(input) {
74
+ if (!telemetryEnabled()) return;
75
+ const properties = installationProperties(input);
76
+ if (!properties) return;
77
+ if (!setupCorrelationEnabled()) { delete properties.setup_id; delete properties.installation_id; }
78
+ // One immutable envelope across a transient retry. A hard total deadline
79
+ // bounds awaited milestones even if the transport ignores cancellation.
80
+ const body = JSON.stringify({ event: 'scout_installation_step', event_id: randomUUID(), anonymousId, properties: { ...properties, ...environment() } });
81
+ const controller = new AbortController();
82
+ let timer;
83
+ try {
84
+ await Promise.race([
85
+ (async () => {
86
+ for (let attempt = 0; attempt < 2 && !controller.signal.aborted && telemetryEnabled(); attempt++) {
87
+ try {
88
+ const response = await fetch('https://openmerit.ai/scout/api/analytics', {
89
+ method: 'POST', headers: { 'Content-Type': 'application/json', ...setupHeaders(properties) },
90
+ body, signal: controller.signal, redirect: 'error',
91
+ });
92
+ await response.body?.cancel();
93
+ if (response.ok || response.status !== 429 && response.status < 500) return;
94
+ } catch { /* Only a bounded retry; never export error text. */ }
95
+ }
96
+ })(),
97
+ new Promise(resolve => { timer = setTimeout(() => { controller.abort(); resolve(); }, 1500); }),
98
+ ]);
99
+ } catch { /* Never affect setup, even if the transport ignores abort. */ }
100
+ finally { clearTimeout(timer); }
101
+ }
34
102
  export const errorType = error => ['Error', 'TypeError', 'RangeError', 'SyntaxError', 'ReferenceError', 'URIError', 'EvalError', 'AggregateError', 'AbortError'].includes(error?.name) ? error.name : 'Error';
35
103
  export function terminalProperties(event, input) {
36
104
  if (!input || typeof input !== 'object' || Array.isArray(input)) return null;
@@ -65,12 +133,13 @@ export function sendTerminalEvent(event, input) {
65
133
  }
66
134
  async function sendAnalytics(event, properties) {
67
135
  const controller = new AbortController();
136
+ const body = JSON.stringify({ event, event_id: randomUUID(), anonymousId, properties: { ...properties, ...environment() } });
68
137
  let timer;
69
138
  try {
70
139
  await Promise.race([
71
140
  Promise.resolve().then(() => fetch('https://openmerit.ai/scout/api/analytics', {
72
- method: 'POST', headers: { 'Content-Type': 'application/json', 'X-Scout-Client': 'cli', 'X-Scout-Analytics': 'on' },
73
- body: JSON.stringify({ event, anonymousId, properties: { ...properties, ...environment() } }), signal: controller.signal, redirect: 'error',
141
+ method: 'POST', headers: { 'Content-Type': 'application/json', 'X-Scout-Client': 'cli', 'X-Scout-Analytics': 'on', ...(process.env.OPENMERIT_ANALYTICS_ACCOUNT === 'off' ? { 'X-Scout-Analytics-Account': 'off' } : {}) },
142
+ body, signal: controller.signal, redirect: 'error',
74
143
  })).then(response => response.body?.cancel()),
75
144
  new Promise(resolve => { timer = setTimeout(() => { controller.abort(); resolve(); }, 750); timer.unref(); }),
76
145
  ]);
@@ -78,7 +147,8 @@ async function sendAnalytics(event, properties) {
78
147
  finally { clearTimeout(timer); }
79
148
  }
80
149
 
81
- // Consent is independent of terminal discovery. Never persist an identity.
150
+ // Consent is independent of terminal discovery. Command identity is ephemeral;
151
+ // the installer separately manages the owned installation's correlation IDs.
82
152
  export function commandTelemetry(args) {
83
153
  if (process.env.OPENMERIT_TELEMETRY !== '1' || process.env.DO_NOT_TRACK === '1') return async () => {};
84
154
  const started = performance.now();
@@ -90,12 +160,13 @@ export function commandTelemetry(args) {
90
160
  async function send(event, properties) {
91
161
  if (!telemetryEnabled()) return;
92
162
  const controller = new AbortController();
163
+ const body = JSON.stringify({ event, event_id: randomUUID(), anonymousId, properties: { ...properties, ...environment(), command_id } });
93
164
  let timer;
94
165
  try {
95
166
  await Promise.race([
96
167
  Promise.resolve().then(() => fetch('https://openmerit.ai/scout/api/analytics', {
97
- method: 'POST', headers: { 'Content-Type': 'application/json', 'X-Scout-Client': 'cli', 'X-Scout-Analytics': 'on' },
98
- body: JSON.stringify({ event, anonymousId, properties: { ...properties, ...environment(), command_id } }),
168
+ method: 'POST', headers: { 'Content-Type': 'application/json', 'X-Scout-Client': 'cli', 'X-Scout-Analytics': 'on', ...(process.env.OPENMERIT_ANALYTICS_ACCOUNT === 'off' ? { 'X-Scout-Analytics-Account': 'off' } : {}) },
169
+ body,
99
170
  signal: controller.signal, redirect: 'error',
100
171
  })).then(response => response.body?.cancel()),
101
172
  new Promise(resolve => { timer = setTimeout(() => { controller.abort(); resolve(); }, 750); }),
@@ -6,6 +6,8 @@ It reads existing evidence independently of inference. It does not call a model,
6
6
 
7
7
  ## Optional CLI product analytics
8
8
 
9
+ **Coding-agent setup telemetry — npm 0.1.9:** opted-in setup adds stage/outcome/component/connection diagnostics and random setup/installation correlation, described in the [Scout lifecycle contract](scout.md#coding-agent-lifecycle-instrumentation-npm-019). Unlike the process-only terminal identity below, an opted-in setup stores its installation UUID in the existing ownership record and installed MCP headers for later calls. No paths, credentials, device names or task content are exported. The copied website prompt explicitly carries usage/unlink preferences; standalone commands still require `OPENMERIT_TELEMETRY=1`, and `DO_NOT_TRACK=1` overrides it. Plans and status checks do not create an installation identity. Older npm 0.1.8 installations need an installer rerun; hosted browser/server collection requires the matching independent Scout deployment.
10
+
9
11
  Available in npm 0.1.7. CLI product analytics are off unless `OPENMERIT_TELEMETRY=1`; `DO_NOT_TRACK=1` always disables them. Terminal `discovery` configuration and `--no-discovery` are unrelated to analytics consent.
10
12
 
11
13
  When explicitly enabled, the CLI posts `cli_command_started` and `cli_command_completed` to `https://openmerit.ai/scout/api/analytics`, with a fresh `crypto.randomUUID()` identity for that process only. No identity file, cookie, or installation ID is stored. Properties contain only a command category (`help`, `dash`, `eval_init`, `eval_doctor`, `eval_plan`, `eval_run`, `eval_watch`, `eval_report`, or `unknown`), plus completion outcome (`ok` or `error`) and integer elapsed milliseconds capped at one day. Paths, arguments, prompts, configuration, model IDs, error text, environment contents, machine names, and email are not included.
@@ -1,8 +1,10 @@
1
1
  # Scout beta data policy
2
2
 
3
- Updated 23 October 2026.
3
+ Updated 24 October 2026.
4
4
 
5
- **Coding-agent reset, 23 October 2026:** this build removes Scout's coding-agent backend and setup/connection tracking. The Claude Code panel is a web-only placeholder with no installation or authentication flow. MCP and agent OAuth metadata routes return HTTP 404 in this build; the website, browser sign-in and JSON API remain. Earlier MCP references below describe the policy before this reset. Existing analytics records and remote grants are not deleted by this reset. A source-only replacement adds device sign-in: Scout stores the approving account, device name you chose (the host name by default), client, scope, timestamps and hashes of device codes and keys, but never a key itself. It is not deployed yet.
5
+ **Coding-agent setup, 24 October 2026:** hosted Scout supports device sign-in and MCP; stable npm 0.1.8 installs Claude Code's MCP configuration, startup rule and Scout skill after approval. This supersedes the 23 October web-placeholder reset. Scout stores the approving account, device name you chose (the host name by default), client, scope, timestamps and hashes of device codes and keys, but never a plaintext key. Local setup stores the key in private configuration; external coding agents have their own data handling. Installation and startup guidance do not guarantee an agent will invoke Scout.
6
+
7
+ **Lifecycle telemetry — npm 0.1.9 and matching Scout deployment:** this revision adds consent-gated setup-click, clipboard/launch-request, device-authorization, installation/component-verification and MCP-invocation diagnostics. Linked setup uses random setup and installation UUIDs to correlate those milestones; the local ownership record and installed MCP headers retain them across sessions/upgrades. Unlinked mode omits these IDs. Copied setup prompts carry usage/unlink preferences into installation, while standalone CLI telemetry remains opt-in and DNT overrides it. Installed MCP prompt-content capture remains off. The revision also adds bounded delivery retries and sanitized Cloudflare delivery counters, without identity, content, URLs, device codes or credentials in those counters. Existing installations need an installer rerun; these changes do not backfill earlier activity. See the [lifecycle contract](scout.md#coding-agent-lifecycle-instrumentation-npm-019). Release acceptance is recorded separately.
6
8
 
7
9
  **Version-5 diagnostics:** Scout serving builds collect page views/leaves, bounded engagement, allowlisted acquisition/UTM and browser/device categories, and sampled browser performance observations where supported and usage analytics is enabled. Native URL properties use only reconstructed public paths, never raw visitor URLs, queries or fragments. Linked activity can join same-tab visits, signup and searches; unlinking retains request-only server identity, not a cross-request funnel. Feedback records a useful/not-useful rating and optional fixed reason, with linked trace/retry correlation, not free text. Server diagnostics include bounded stage/provider failure codes and actual pipeline attempt tokens when reported, not invented charges or executions of recommended models. See the [technical contract](scout.md#version-5-instrumentation). Main-site/docs collection and CLI publication have separate release boundaries. No replay or ranking experiment is enabled, and none of the prompt-content or retention controls below change.
8
10
 
@@ -22,7 +24,7 @@ Scout processes your submitted task to generate recommendations. With usage anal
22
24
  | --- | --- |
23
25
  | Submitted task and model restrictions | Required to interpret the task, find suitable models and return recommendations. Scout sends task information to OpenAI even when analytics is off. |
24
26
  | Recommendations and search diagnostics | When usage analytics is enabled, PostHog receives task categories, model IDs, ranks, available quality/cost/latency evidence, timings, cache status and outcomes. These describe recommendations, not execution of those models on your data. |
25
- | Product interactions and errors | Enabled analytics includes visits, preset selections, sign-in activity, results viewed, evidence opened, copies, provider-link clicks and coding-agent setup activity. API/MCP events include method, outcome and confirmed connections. Error reports use bounded categories and application stack locations, not raw error messages. |
27
+ | Product interactions and errors | Enabled analytics includes visits, preset selections, sign-in activity, results viewed, evidence opened, copies, provider-link clicks and coding-agent setup activity. API/MCP events record request outcomes. Lifecycle-enabled builds add allowlisted method/tool, reported agent/harness and version, setup/installation correlation and verified-connection diagnostics. A launch request is not proof an app opened, and key authorization is not proof of installation. Error reports use bounded categories and application stack locations, not raw error messages. |
26
28
  | Account details | Clerk handles Google/GitHub login. Enabled account analytics can synchronize your name, verified primary email, signup time and connected providers to PostHog. Account and session lifecycle events help us understand onboarding and returning use. |
27
29
  | Optional prompt content | Only when prompt sharing is enabled, PostHog also receives the submitted prompt and written task breakdown, for at most three tasks. Prompt text is limited to 4,000 characters, with up to 1,000 characters per excerpt or five-layer field. Common sensitive patterns are removed where detected; this is not guaranteed anonymization. All such text is confined to `$ai_trace.$ai_input_state={prompt,tasks}`; new ordinary `scout_search_details` events contain no prompt, excerpt or layer prose. |
28
30
 
@@ -39,7 +41,7 @@ Open **Privacy & analytics** in the [Scout footer](/scout/#analytics-settings):
39
41
 
40
42
  Scout respects Do Not Track and Global Privacy Control for these analytics. An authenticated browser also communicates its usage preference to the server; the latest received account preference controls background synchronization. A browser that is offline or not signed in cannot update that account preference until it reconnects and authenticates.
41
43
 
42
- API/MCP clients can send `X-Scout-Analytics: off` to disable analytics. Prompt text still requires explicit `X-Scout-Analytics-Content: on`; general opt-out and supported privacy signals take precedence. CLI content capture is always off. Copied agent setup instructions disable prompt-content sharing and do not inherit this browser's account-linkage setting. See the [technical analytics guide](scout.md#product-analytics) for the full contract.
44
+ API/MCP clients can send `X-Scout-Analytics: off` to disable analytics. Prompt text still requires explicit `X-Scout-Analytics-Content: on`; general opt-out and supported privacy signals take precedence. CLI content capture is always off. Older 0.1.8 setup does not inherit this browser's account-linkage setting; the 0.1.9 lifecycle revision explicitly carries usage and unlink preferences into the command and installed headers. Later browser changes do not rewrite existing agent configuration; rerun setup to update it. Request-only refusals in that revision do not overwrite saved account-wide preferences. See the [technical analytics guide](scout.md#product-analytics) for the full contract.
43
45
 
44
46
  Enabled signed-in analytics normally uses a stable account identifier linked to the profile above; hashing does not make it anonymous, and a pre-sign-in visit can be joined after login. In unlinking mode the server uses fresh unrelated random identifiers, sends no profile/identify operation, ignores previously linked browser identifiers and supplied trace IDs, and does not return an analytics trace ID for later interactions to reuse. Account opt-out and deletion gates still apply before sending. Clerk profile/lifecycle analytics can continue separately under general usage consent. Past records remain linked; signing out or selecting this option does not unlink them. Unlinked request events cannot support cross-request funnels or automatic account-based historical deletion. Prompt text itself may identify a person: unlinking is not anonymization.
45
47
 
package/docs/scout.md CHANGED
@@ -24,6 +24,28 @@ Examples go through the same interpretation and ranking as typed prompts, with n
24
24
 
25
25
  ## Product analytics
26
26
 
27
+ ### Coding-agent lifecycle instrumentation — npm 0.1.9
28
+
29
+ Npm 0.1.9 includes these installer changes; browser/server instrumentation requires the matching independent Scout deployment. Release acceptance is recorded separately. Rerun the installer to update existing 0.1.8 installations; nothing backfills historical events. This contract supersedes the no-retry and process-only setup-identity descriptions below for lifecycle-enabled builds. Homepage/docs deployment remains separate.
30
+
31
+ | Touchpoint | Event and meaning |
32
+ |---|---|
33
+ | Add to coding agent | `scout_agent_setup_started` records the click. `scout_agent_launch_requested` records an attempted Claude deep link, **not proof the app opened**. `scout_agent_setup_copied` separately records clipboard success/failure. |
34
+ | Device authorization | Server-owned `scout_connect_step` records device creation, approval-page requests, pending/slow polling, approval or denial, expiry, key collection, device listing and revocation. Expected pending polls are not exceptions. Browser failures use `scout_connect_client_error` with a fixed stage/code. |
35
+ | Installation | Opt-in `scout_installation_step` records preflight, login/browser opening, configuration, verification, status, removal and completion. `applied`, component states and `connection` distinguish a plan, incomplete installation and a verified connection. A command exit of zero alone does not establish installation success. |
36
+ | MCP connection | `scout_request_completed` records every handled MCP request, including initialization/discovery and authentication failures. Allowlisted `mcp_method`, `mcp_tool`, reported agent and package version distinguish verifier probes from agent calls. Unrecognized names become `other`/`unknown`, never raw strings. |
37
+ | Each invocation/task | `scout_tool_invocation_started` and `scout_tool_invocation_completed` bracket authenticated tool dispatch. `scout_search_completed` counts an accepted recommendation attempt; `scout_search_details` retains each task's shortlist. All share the request trace. Protocol/tool errors are failures even with HTTP 200. |
38
+
39
+ When browser usage analytics is enabled, the copied setup prompt explicitly sets `OPENMERIT_TELEMETRY=1`; usage-disabled prompts instead set `DO_NOT_TRACK=1`. Account-unlinked prompts also set `OPENMERIT_ANALYTICS_ACCOUNT=off`. A linked setup gets a random `--setup-id` UUID that joins the click, resumed login, installation and later invocations. An opted-in installation stores a separate random `installation_id` in its existing ownership record and MCP headers; upgrades reuse it. These identifiers contain no account, hostname or project path. They are correlation hints, not authentication. Plans and status checks do not create an installation identity. Standalone CLI telemetry remains default-off, and `DO_NOT_TRACK=1` wins. Installed MCP content capture remains off.
40
+
41
+ The approval page initializes the same browser privacy controls and authenticated account preference synchronization as Scout, without inventing a Scout pageview. Installer URL refusals (`analytics=off`, `analytics_account=off`) remain effective without changing the browser's stored settings; a request-only refusal does not overwrite the account preference. A collected key is labelled **authorized**, not a verified installation. Account-wide opt-out still requires the authenticated preference endpoint and continues to gate linked and unlinked account requests. Unlinking removes setup/installation identifiers as well as browser/trace linkage. A privacy change in a browser does not rewrite an already installed agent's headers; rerun setup with the desired settings. Existing account opt-outs are not automatically reset.
42
+
43
+ First trigger and first task mean **first observed** tool call and task for that installation/account. They are derived from the same event stream, not additional search counts. Discovery is not a trigger. Recommendation completion is not downstream model execution or application-task success. No transcript is read, and this instrumentation cannot prove a startup rule loaded, a model decision occurred without calling Scout, or a native app actually opened. Missing consent, storage, connectivity or abrupt termination can leave a funnel incomplete; missing events must not be presented as a verified zero or a proven skipped call.
44
+
45
+ Initialization can record an allowlisted harness name and semver-shaped reported harness version; the installed headers report the npm installer version, not Claude's current version or invoking model. The stateless service cannot infer the coding agent's model from Scout's own provider generation events. Installer milestones include a fixed `command` category so status/doctor/project completion is never counted as a new installation. Setup milestone submission allows one transient retry within a hard 1.5-second total deadline; general CLI/terminal delivery retains the separately documented behavior below.
46
+
47
+ Browser event submission retries one transient failure with a stable event UUID. Server delivery retries transport failures, HTTP 429 and 5xx up to three total attempts, with the same event UUIDs/timestamps and a 1.5-second timeout per attempt. Permanent 4xx responses do not retry. Product responses do not wait for delivery. There is no durable product-event queue or exactly-once guarantee; bounded retries can still exhaust. The serving Worker configuration enables sanitized Cloudflare console counters (`scout_analytics_delivery`: delivered, failed, request-suppressed or account-suppressed), with query redaction and invocation logs disabled. Counters contain no identity, trace, device code, URL, content or credential and remain available independently of PostHog delivery. They are operational diagnostics, not PostHog service-log ingestion or session replay. `X-Scout-Analytics-Status` reports only whether the request permits capture, never a receipt of ingestion.
48
+
27
49
  ### Version-5 instrumentation
28
50
 
29
51
  This contract applies to version-5 Scout serving builds. The serving rollout is independent of main-site publication and the npm package; npm 0.1.7 includes the opt-in CLI additions, while the website collector remains source-only. Prompt-content choices, redaction, retention and account deletion controls are unchanged. Main-site publication remains blocked by [source divergence](site-documentation.md#website-analytics-source-only-unreleased). Earlier dated receipts below describe earlier releases, not acceptance of version 5.
@@ -396,7 +418,7 @@ The capability index lists each area with its benchmark-series and model counts.
396
418
 
397
419
  ## Connect a coding agent
398
420
 
399
- **Device sign-in setup — npm 0.1.7:** use the stable `latest` tag for the commands below. Hosted Scout serves `/connect` and `/scout/mcp` independently of npm. Claude Code is the only supported client; other agents are not yet supported. The older `0.1.7-preview.5` package does not contain these commands.
421
+ **Device sign-in setup — npm 0.1.8:** use the stable `latest` tag for the commands below. Version 0.1.8 adds the startup model-decision rule to the device setup introduced in 0.1.7. Hosted Scout serves `/connect` and `/scout/mcp` independently of npm. Claude Code is the only supported client; other agents are not yet supported. The older `0.1.7-preview.5` package does not contain these commands.
400
422
 
401
423
  Setup takes four steps, and you act in each one only once:
402
424
 
@@ -408,13 +430,13 @@ Setup takes four steps, and you act in each one only once:
408
430
  ```
409
431
 
410
432
  3. **Click Approve** on `https://openmerit.ai/connect`, which the command opens for you. Sign in first if needed. The page shows the client, the device name, the scope, an eight-character code that matches the command's output, and the only permission granted: Scout recommendations through MCP. Only approve a request you started yourself.
411
- 4. **Done.** The command has kept waiting, so nothing needs rerunning. It stores the device key, adds the `openmerit` MCP server with `claude mcp add-json`, writes the Scout skill and checks the connection with a real MCP `initialize` and tool discovery (not a search). The approval page switches to *Claude Code is connected* once the command has collected its key, and Claude Code reports the result. Scout is available in new Claude Code sessions; in a session that is already open, reconnect `openmerit` in `/mcp`.
433
+ 4. **Done.** The command has kept waiting, so nothing needs rerunning. It stores the device key, adds the `openmerit` MCP server with `claude mcp add-json`, writes the Scout skill and startup rule, and checks the connection with a real MCP `initialize` and tool discovery (not a search). The approval page switches to *Claude Code is connected* once the command has collected its key, and Claude Code reports the result. **Start a new Claude Code session** to load the startup instructions; reconnecting `openmerit` in `/mcp` alone is not enough.
412
434
 
413
435
  The prompt contains no key, account data or task draft. If the Claude desktop app isn't installed or your browser blocks the link, paste the copied prompt into Claude Code instead. The command needs `npx` (Node.js 22.19 or newer) and the `claude` CLI on the PATH of that session; the installer runs `claude mcp add-json`. Having the desktop app alone does not establish that those commands are available. See [Claude Desktop](https://code.claude.com/docs/en/desktop).
414
436
 
415
437
  When the command can open a browser, it waits up to nine minutes for the Approve click. When it can't, for example over SSH, it prints the link and code and returns after 30 seconds with exit code 10 so the agent can show them. Rerunning the same command resumes the same request; it does not issue a new code. Codes expire after ten minutes and can be used once. `OPENMERIT_LOGIN_WAIT` sets the wait in seconds.
416
438
 
417
- `--client` defaults to the agent running the command (Claude Code sets `CLAUDECODE=1`). `--scope` defaults to `user`, which makes Scout available in every project. `--scope project` writes the skill to `.claude/skills/` in the current project and stores the MCP server in Claude Code's private per-project (`local`) configuration, so the key never enters the repository. `--plan` lists the changes without making them. `--json` prints one result object for agents.
439
+ `--client` defaults to the agent running the command (Claude Code sets `CLAUDECODE=1`). `--scope` defaults to `user`, which makes Scout available in every project. `--scope project` writes the skill to `.claude/skills/` and startup rule to `.claude/rules/` in the current project and stores the MCP server in Claude Code's private per-project (`local`) configuration, so the key never enters the repository. `--plan` lists the changes without making them. `--json` prints one result object for agents.
418
440
 
419
441
  | Exit code | Meaning |
420
442
  |---|---|
@@ -431,6 +453,8 @@ When the command can open a browser, it waits up to nine minutes for the Approve
431
453
 
432
454
  Each approval creates one key for one device and client, limited to `scout:recommend`. It is accepted only on `/scout/mcp`; the JSON API and browser still require a Clerk session. The server stores only SHA-256 hashes of keys and device codes. A key expires after 90 days without use; each use extends it. The usual per-user rate limits and analytics preferences apply.
433
455
 
456
+ Missing or invalid MCP credentials return HTTP 401 with `device_key_required`. The recovery command is `npx -y openmerit@latest install`; the older preview package does not include the installer.
457
+
434
458
  On the device, the key is stored in `~/.config/openmerit/credentials.json` (mode 600; `OPENMERIT_CONFIG_DIR` or `XDG_CONFIG_HOME` override the location) and in Claude Code's MCP header configuration, which Claude Code stores in plain text. The CLI never prints the key and scrubs it from any failing `claude` output. Do not paste keys into a chat.
435
459
 
436
460
  Revoke devices under **Connected devices** on `/connect`. `openmerit remove` and `openmerit logout` also revoke this device's key.
@@ -447,14 +471,24 @@ openmerit project --policy existing-only|existing-first [--endpoint-id <id> ...]
447
471
  openmerit doctor # Node.js, claude, reachability, key, MCP check and install state
448
472
  ```
449
473
 
450
- Removal deletes the MCP server, the unchanged skill and the install record, and revokes the key. It keeps edited files and `.openmerit/scout.json`.
474
+ Removal deletes the MCP server, the unchanged skill and startup rule, and the install record, and revokes the key. It keeps edited files and `.openmerit/scout.json`.
451
475
 
452
476
  ### Using Scout from the agent
453
477
 
454
- The `openmerit-scout` skill loads when an application inference model is being chosen, even if you do not mention Scout. It is not loaded for unrelated edits, ordinary inference calls or the agent's own model. On first use in a project, it asks once whether the project is **existing-only** or **existing-first** and which public catalogue endpoint IDs it can reach, then saves them with `openmerit project`. Every search passes the saved `endpointIds`, including `[]`. Existing-only never broadens; existing-first asks before an unscoped search. Task descriptions go to Scout and OpenAI; do not send source files, secrets or customer records.
478
+ Claude can load the `openmerit-scout` skill when choosing an application inference model, even if you do not mention Scout. The skill-only setup in npm 0.1.7 can miss implicit choices while building an app, including selecting pretrained browser models. Availability in the skill list does not prove invocation. Unrelated edits, ordinary inference calls and the coding agent's own model are outside its scope. On first use in a project, the skill instructs Claude to ask once whether the project is **existing-only** or **existing-first** and which public catalogue endpoint IDs it can reach, then save them with `openmerit project`. Every search passes the saved `endpointIds`, including `[]`. Existing-only never broadens; existing-first asks before an unscoped search. Task descriptions go to Scout and OpenAI; do not send source files, secrets or customer records.
455
479
 
456
480
  The agent compares model and endpoint, evidence, reference cost/time and unknowns, and helps configure the chosen route. New providers, credentials, spending and deployments need separate approval. A skill is a default, not enforcement. The MCP tool, `scout_models`, is read-only and does not execute models.
457
481
 
482
+ ### Startup model-decision rule — npm 0.1.8
483
+
484
+ The 0.1.8 installer adds `rules/openmerit-scout.md` alongside the skill. Unlike an on-demand skill, an unscoped [Claude Code rule](https://code.claude.com/docs/en/memory#organize-rules-with-claude/rules/) loads at session startup. The rule requires loading the skill and consulting Scout before any application-model choice, even when the user only asks for a feature. It explicitly includes pretrained/local/browser models, weight downloads, SDK defaults, embeddings, OCR, speech, classification, detection and segmentation. An explicit user model choice must be assessed, not silently replaced. Unchanged approved models and ordinary inference calls do not require another search.
485
+
486
+ Rerunning `npx -y openmerit@latest install --client claude-code` with version 0.1.8 or later upgrades an unchanged npm 0.1.7 installation without another device approval. User scope writes the rule under `~/.claude/rules/` (or `CLAUDE_CONFIG_DIR`) and covers new projects and Desktop's no-folder sessions. Project scope writes `.claude/rules/openmerit-scout.md` and applies only in that project. The existing skill, private MCP entry and key locations are unchanged. Start a **new Claude Code session** after installation or upgrade; reconnecting `/mcp` alone does not reload startup instructions.
487
+
488
+ The installer records rule ownership, repairs missing rules, resumes interrupted updates and refuses to overwrite edited or unowned rules. `--plan` includes the rule without writing it; `status` reports its state and `doctor` treats a missing rule as incomplete. `remove` deletes only the unchanged owned rule and retains edited rules and other user instructions.
489
+
490
+ If policy is missing, Claude must ask before searching. If Scout is unavailable or has no suitable evidence/candidate for the required deployment, the instructions require disclosing the gap and pausing model selection, not choosing from memory or an SDK default. Model-independent work may continue. Catalogue coverage is not guaranteed for local/browser models. These are behavioral instructions, **not hard enforcement**; Claude can still fail to follow them. Older 0.1.7 installations remain skill-only until upgraded; website deployment alone does not update an installed client.
491
+
458
492
  ## JSON API
459
493
 
460
494
  ```sh
package/docs/security.md CHANGED
@@ -57,10 +57,14 @@ The [coding-agent setup](scout.md#connect-a-coding-agent) in npm 0.1.7 uses host
57
57
 
58
58
  **Historical reset, superseded by npm 0.1.7 and hosted device setup.** The reset temporarily removed the installer and MCP transport. The current [Claude Code setup](scout.md#connect-a-coding-agent) uses browser-approved, device-specific keys scoped to MCP recommendations, not pasted session tokens or agent OAuth. The key is stored locally in mode-600 credentials and Claude Code's plain-text MCP configuration. Setup installs an owned Scout skill; conflicts are not overwritten, and removal revokes the device key. Browser Google/GitHub sign-in remains separate.
59
59
 
60
+ **npm 0.1.8:** setup also installs an owned, always-loaded `rules/openmerit-scout.md` instruction requiring Scout before implicit or explicit application-model choices, including pretrained/local/browser models and SDK defaults. Edited or unowned rules are not overwritten or removed. Missing policy requires asking; unavailable Scout or insufficient coverage requires pausing model selection. This is model guidance, not an enforced tool gate or authorization to download weights, add providers, spend or deploy. Scope, device-key permissions and task-data handling are unchanged. New sessions are required after upgrading; older npm 0.1.7 installations remain skill-only until upgraded.
61
+
60
62
  Scout serving, main-site documentation and npm publication are separate releases. This reset does not revoke remote grants or clean up files previously installed on other machines. Earlier hosted MCP/authentication descriptions in this guide refer to past deployments, not this build. Browser Google/GitHub sign-in and its Clerk account lifecycle are separate and unchanged.
61
63
 
62
64
  ## Hosted Scout analytics
63
65
 
66
+ **Coding-agent lifecycle instrumentation — npm 0.1.9 and matching Scout deployment:** setup clicks, clipboard/deep-link attempts, device authorization states, opted-in installation/component verification and each dispatched MCP invocation gain typed milestones and fixed error codes. Linked setup uses random setup/installation UUIDs, never paths, machine names, device codes, keys or transcript content. The browser handoff explicitly carries usage/unlink choices into setup; standalone CLI analytics remains opt-in. Setup/installation IDs are omitted in unlinked mode, DNT wins, and installed MCP prompt-content capture stays off. Request-only suppression no longer overwrites the saved account-wide preference. Explicit account opt-out/deletion gates remain in force. See the [lifecycle contract](scout.md#coding-agent-lifecycle-instrumentation-npm-019) for event meanings, bounded delivery retries and diagnostic counters. Existing installations require an installer rerun; release acceptance is separate, and no historical completeness, replay, transcript collection or skipped-call detection is claimed.
67
+
64
68
  **Version-5 instrumentation:** the [serving contract](scout.md#version-5-instrumentation) adds validated native web events and explicit CTA-only `$autocapture` mapping, not general DOM capture. Acquisition/UTM/device values are categorical allowlists; public URL properties are reconstructed without raw URLs. Browser performance is sampled and support-dependent. Linked same-tab signup/search identity is separate from unlinking's request-only identifiers. Logical MCP errors count even with HTTP 200; stage/provider codes, actual attempt tokens and feedback/retry traces exclude raw messages and free text. CLI diagnostics are opt-in in npm 0.1.7, adding sanitized version/platform/runtime and bounded application basename frames, never absolute paths or environment contents. Prompt sharing, retention, deletion gates and necessary provider processing are unchanged. Replay is a separate, inactive spike; no experiment is active. Main-site publication remains blocked by documented source divergence; the Scout rollout does not publish npm or activate alert notifications.
65
69
 
66
70
  **Beta evaluation rollout, 22 October 2026:** The serving configuration enables `SCOUT_BETA_EVALUATION_LOGGING=true`, alongside `POSTHOG_CAPTURE_CONTENT=true` and complete account-sync/deletion configuration. The browser defaults content on only without a stored content choice; saved off, general opt-out, DNT/GPC and unreadable storage keep it off. API/MCP still require `X-Scout-Analytics-Content: on`; CLI content is always off. This is product evaluation/debugging, not foundation model training. Deployment is owner-approved; this is not legal certification, and beta status establishes no consent or legal basis. Main-site `/docs/` publication remains blocked by source divergence; the [policy](/scout/data-policy/) is generated from the same Markdown and published with Scout instead.
package/docs/testing.md CHANGED
@@ -1,5 +1,15 @@
1
1
  # Testing and validation
2
2
 
3
+ ## Stable 0.1.7 acceptance — 24 October 2026
4
+
5
+ Published `openmerit@0.1.7` to npm's `latest` channel after full repository verification: 757 passing tests, one existing skip, syntax/type checks, builds and documentation checks. The `preview` tag remains `0.1.7-preview.5`. A fresh registry installation on macOS ARM64 with Node.js 22.23.1 matched the tested 104-file tarball byte-for-byte. Packed core/protocol/Pi exports and the installed terminal runtime smoke checks passed, including passive collection, source removal, runtime outage/recovery and shutdown.
6
+
7
+ The live Scout button opened a prefilled Claude Desktop Code session through Chrome. That session ran `npx -y openmerit@latest install --client claude-code`, opened the real device-approval page, and completed after approval of the matching code. This was a clean setup after removing and revoking this test's earlier source installation, not reuse of an existing key. Status then reported the MCP server and skill installed and connected; credentials had mode 600 and other MCP servers were preserved.
8
+
9
+ A fresh Desktop session received an application-model request that never mentioned Scout. It loaded `openmerit-scout` and called the real `scout_models` tool with the saved empty inventory, returning zero candidates. It asked before broadening; after separate approval, its second MCP call returned five candidates and it produced a provisional recommendation. No application code, providers, candidate inference or deployments were performed by that session. Anonymous MCP access and use of the device key on the JSON API both returned 401, while authenticated tool discovery succeeded.
10
+
11
+ These are integration observations, not a general guarantee that every agent follows the skill. Both search responses reported stale/degraded source data, including incomplete Parallel research. Their approximately 228,000- and 538,000-character payloads exceeded Desktop's inline tool limit; Claude recovered through saved tool-result files. The flow works, but these checks do not establish catalogue freshness, task accuracy or small response size. A separate recovery-path check exposed an obsolete preview install command in the MCP 401 message; its stable-command regression test failed before the source correction and passed afterward. Hosted correction is a separate deployment from the npm release.
12
+
3
13
  ## Published preview.5 acceptance — 17 October 2026
4
14
 
5
15
  Published `0.1.7-preview.5` to npm's `preview` channel after all **679 repository tests**, syntax/type checks, builds and documentation checks passed, including 336 Scout tests. Registry integrity matches the reviewed tarball and stable `latest` remains `0.1.6`. All 101 packaged files are accounted for; runtime bytes, public exports and legal metadata are unchanged from preview.4. Version metadata and public guides carry this release. The frozen package contains the preparation record; this post-publication result is maintained here after the registry and VM checks.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "openmerit",
3
- "version": "0.1.7",
3
+ "version": "0.1.9",
4
4
  "description": "Evidence-driven model decisions for coding harnesses.",
5
5
  "type": "module",
6
6
  "bin": {