openmerit 0.1.10 → 0.1.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +13 -0
- package/README.md +20 -53
- package/dist/pi/scout-extension.mjs +177 -0
- package/dist/pi/src/index.d.ts +6 -32
- package/dist/pi/src/index.js +49 -787
- package/dist/terminal/scout-update.json +1 -1
- package/dist/terminal/src/install.mjs +102 -166
- package/dist/terminal/src/scout-credentials.d.ts +12 -0
- package/dist/terminal/src/scout-credentials.js +41 -0
- package/dist/terminal/src/scout-credentials.ts +44 -0
- package/dist/terminal/src/scout-instructions.d.ts +5 -0
- package/dist/terminal/src/scout-instructions.js +103 -0
- package/dist/terminal/src/scout-instructions.ts +106 -0
- package/dist/terminal/src/scout-pi.d.ts +44 -0
- package/dist/terminal/src/scout-pi.js +84 -0
- package/dist/terminal/src/scout-pi.ts +76 -0
- package/dist/terminal/src/telemetry.mjs +2 -2
- package/dist/terminal/src/updates.js +2 -1
- package/dist/terminal/src/updates.ts +2 -1
- package/docs/adapter-guide.md +2 -0
- package/docs/architecture.md +7 -5
- package/docs/automation.md +2 -0
- package/docs/budgets.md +2 -0
- package/docs/commands.md +3 -1
- package/docs/demo.md +1 -1
- package/docs/first-trial.md +2 -0
- package/docs/getting-started.md +19 -44
- package/docs/index.md +10 -6
- package/docs/inference-terminal.md +1 -1
- package/docs/lifecycle.md +2 -0
- package/docs/operations.md +2 -0
- package/docs/pi-extension.md +32 -28
- package/docs/roadmap.md +6 -5
- package/docs/scout-data-policy.md +22 -64
- package/docs/scout.md +137 -19
- package/docs/security.md +8 -2
- package/docs/testing.md +2 -0
- package/docs/troubleshooting.md +23 -28
- package/docs/ux-reference.md +1 -1
- package/package.json +11 -7
- package/dist/pi/src/scheduler.d.ts +0 -11
- package/dist/pi/src/scheduler.js +0 -137
- package/dist/pi/src/wakeup.d.ts +0 -2
- package/dist/pi/src/wakeup.js +0 -108
- package/docs/memo.md +0 -126
|
@@ -0,0 +1,5 @@
|
|
|
1
|
+
import type { ScoutClient } from './scout-credentials.ts';
|
|
2
|
+
export declare const rule = "# Application model decisions \u00B7 OpenMerit Scout\n\nBefore choosing, recommending, adding, downloading, replacing or configuring\nANY application inference model, load the `openmerit-scout` skill and consult\n`scout_models` on the `openmerit` MCP server before committing to a model.\nThis applies even when the user only asks you to build a feature, never mentions\nmodels or Scout, or names a model themselves (assess it; do not silently replace it).\n\nA pretrained model, local weights, browser inference, an SDK's default model,\nembeddings, OCR, speech, image classification, detection and segmentation all\ncount as model choices, not implementation details. Identify this decision before\nwriting model-specific code, installing a model package or downloading weights.\n\nFollow the skill's project access policy. If it is missing, ask and save it first.\nIf Scout is unavailable, fails or has no suitable evidence/candidate, disclose the\ngap and pause the model decision; do not fall back to a remembered or SDK-default\nmodel. You may continue work that does not commit to a model.\n\nDo not consult Scout for unrelated edits, normal inference calls using an unchanged\napproved model, or this coding agent's own model. Do not reselect approved models\nunless the task introduces or changes a model decision.\n\nWhen the user says \"update openmerit\", run\n`npx -y openmerit@latest update --scope user` (use `--scope project` if this\nrule is in the project's .claude/rules directory). Report the result, including\nany conflict or manual-upgrade requirement. Do not delete edited files, bypass\npermissions or rerun login automatically. A new session loads updated instructions.\n";
|
|
3
|
+
export declare const skill = "---\nname: openmerit-scout\ndescription: Consults OpenMerit Scout before choosing, adding, downloading, replacing or configuring any application inference model, including implicit choices while building features, pretrained browser/local models, SDK defaults, embeddings, OCR, speech, classification, detection, segmentation, LLMs and agents. Not for unrelated edits, ordinary inference calls or this coding agent's own model.\n---\n\n# OpenMerit \u00B7 Scout\n\nBefore introducing, recommending, replacing or configuring an application\ninference model, call the `scout_models` tool on the `openmerit` MCP server.\nThis applies even when the user never mentions models or Scout. Downloading local\nweights, using a pretrained browser model or accepting an SDK default is a model\ndecision: consult Scout before model-specific code, installs or downloads.\nPreserve the user's explicit model choice; assess it rather than silently replacing\nit. Leave unchanged, approved application selections in place.\n\n## Project policy\n\nRead `.openmerit/scout.json` before searching. If it is absent, ask once whether this\nproject uses **existing-only** (only endpoints it can already reach) or\n**existing-first** (prefer them, ask before searching the whole catalogue), and\nwhich public catalogue endpoint IDs it can access. Save the answer:\n\n```sh\nnpx -y openmerit@latest project --policy existing-only --endpoint-id <id> ...\n```\n\nNever infer access from a model name, SDK or environment-variable name. Never\nrecord credentials, private URLs, prompts or customer data. An empty inventory\nis not permission to search all models.\n\n## Searching\n\nCall `scout_models` with a non-confidential task description and\n`endpointIds` set to the saved inventory, including `[]` when empty.\nExisting-only never broadens. For existing-first, if no existing candidate fits,\nexplain the gap and ask before a separate search without `endpointIds`. Send\nonly task requirements, never source files, secrets or real customer records.\n\nCompare model AND endpoint, evidence, reference cost/time, unknowns and\nwarnings. For multiple tasks read `tasks[].slots[0].recommendations`; the\ntop-level list is empty. Prefer the cheapest usable option the quality evidence\nsupports, without claiming benchmarks prove application accuracy. Use the\nreturned provider model ID and the project's actual SDK and protocol.\n\n## Boundaries\n\nAsk before adding providers or credentials, spending, deploying or external\nwrites. Report recommendation, endpoint connectivity and application checks as\nseparate results, each only after it succeeds. Do not switch this coding agent's\nown model.\n\nIf Scout fails or has no suitable candidate/evidence for the required deployment,\nsay so and pause the model decision; do not substitute a remembered recommendation\nor SDK default. Continue only model-independent work. Local/browser models are not\nexempt when the catalogue lacks coverage. If it reports that the device key is missing,\nexpired or revoked, run `npx -y openmerit@latest login` and then\n`npx -y openmerit@latest install` again. Never ask the user to paste a key.\n";
|
|
4
|
+
export declare function scoutRule(client: ScoutClient): string;
|
|
5
|
+
export declare function scoutSkill(client: ScoutClient): string;
|
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
export const rule = `# Application model decisions · OpenMerit Scout
|
|
2
|
+
|
|
3
|
+
Before choosing, recommending, adding, downloading, replacing or configuring
|
|
4
|
+
ANY application inference model, load the \`openmerit-scout\` skill and consult
|
|
5
|
+
\`scout_models\` on the \`openmerit\` MCP server before committing to a model.
|
|
6
|
+
This applies even when the user only asks you to build a feature, never mentions
|
|
7
|
+
models or Scout, or names a model themselves (assess it; do not silently replace it).
|
|
8
|
+
|
|
9
|
+
A pretrained model, local weights, browser inference, an SDK's default model,
|
|
10
|
+
embeddings, OCR, speech, image classification, detection and segmentation all
|
|
11
|
+
count as model choices, not implementation details. Identify this decision before
|
|
12
|
+
writing model-specific code, installing a model package or downloading weights.
|
|
13
|
+
|
|
14
|
+
Follow the skill's project access policy. If it is missing, ask and save it first.
|
|
15
|
+
If Scout is unavailable, fails or has no suitable evidence/candidate, disclose the
|
|
16
|
+
gap and pause the model decision; do not fall back to a remembered or SDK-default
|
|
17
|
+
model. You may continue work that does not commit to a model.
|
|
18
|
+
|
|
19
|
+
Do not consult Scout for unrelated edits, normal inference calls using an unchanged
|
|
20
|
+
approved model, or this coding agent's own model. Do not reselect approved models
|
|
21
|
+
unless the task introduces or changes a model decision.
|
|
22
|
+
|
|
23
|
+
When the user says "update openmerit", run
|
|
24
|
+
\`npx -y openmerit@latest update --scope user\` (use \`--scope project\` if this
|
|
25
|
+
rule is in the project's .claude/rules directory). Report the result, including
|
|
26
|
+
any conflict or manual-upgrade requirement. Do not delete edited files, bypass
|
|
27
|
+
permissions or rerun login automatically. A new session loads updated instructions.
|
|
28
|
+
`;
|
|
29
|
+
export const skill = `---
|
|
30
|
+
name: openmerit-scout
|
|
31
|
+
description: Consults OpenMerit Scout before choosing, adding, downloading, replacing or configuring any application inference model, including implicit choices while building features, pretrained browser/local models, SDK defaults, embeddings, OCR, speech, classification, detection, segmentation, LLMs and agents. Not for unrelated edits, ordinary inference calls or this coding agent's own model.
|
|
32
|
+
---
|
|
33
|
+
|
|
34
|
+
# OpenMerit · Scout
|
|
35
|
+
|
|
36
|
+
Before introducing, recommending, replacing or configuring an application
|
|
37
|
+
inference model, call the \`scout_models\` tool on the \`openmerit\` MCP server.
|
|
38
|
+
This applies even when the user never mentions models or Scout. Downloading local
|
|
39
|
+
weights, using a pretrained browser model or accepting an SDK default is a model
|
|
40
|
+
decision: consult Scout before model-specific code, installs or downloads.
|
|
41
|
+
Preserve the user's explicit model choice; assess it rather than silently replacing
|
|
42
|
+
it. Leave unchanged, approved application selections in place.
|
|
43
|
+
|
|
44
|
+
## Project policy
|
|
45
|
+
|
|
46
|
+
Read \`.openmerit/scout.json\` before searching. If it is absent, ask once whether this
|
|
47
|
+
project uses **existing-only** (only endpoints it can already reach) or
|
|
48
|
+
**existing-first** (prefer them, ask before searching the whole catalogue), and
|
|
49
|
+
which public catalogue endpoint IDs it can access. Save the answer:
|
|
50
|
+
|
|
51
|
+
\`\`\`sh
|
|
52
|
+
npx -y openmerit@latest project --policy existing-only --endpoint-id <id> ...
|
|
53
|
+
\`\`\`
|
|
54
|
+
|
|
55
|
+
Never infer access from a model name, SDK or environment-variable name. Never
|
|
56
|
+
record credentials, private URLs, prompts or customer data. An empty inventory
|
|
57
|
+
is not permission to search all models.
|
|
58
|
+
|
|
59
|
+
## Searching
|
|
60
|
+
|
|
61
|
+
Call \`scout_models\` with a non-confidential task description and
|
|
62
|
+
\`endpointIds\` set to the saved inventory, including \`[]\` when empty.
|
|
63
|
+
Existing-only never broadens. For existing-first, if no existing candidate fits,
|
|
64
|
+
explain the gap and ask before a separate search without \`endpointIds\`. Send
|
|
65
|
+
only task requirements, never source files, secrets or real customer records.
|
|
66
|
+
|
|
67
|
+
Compare model AND endpoint, evidence, reference cost/time, unknowns and
|
|
68
|
+
warnings. For multiple tasks read \`tasks[].slots[0].recommendations\`; the
|
|
69
|
+
top-level list is empty. Prefer the cheapest usable option the quality evidence
|
|
70
|
+
supports, without claiming benchmarks prove application accuracy. Use the
|
|
71
|
+
returned provider model ID and the project's actual SDK and protocol.
|
|
72
|
+
|
|
73
|
+
## Boundaries
|
|
74
|
+
|
|
75
|
+
Ask before adding providers or credentials, spending, deploying or external
|
|
76
|
+
writes. Report recommendation, endpoint connectivity and application checks as
|
|
77
|
+
separate results, each only after it succeeds. Do not switch this coding agent's
|
|
78
|
+
own model.
|
|
79
|
+
|
|
80
|
+
If Scout fails or has no suitable candidate/evidence for the required deployment,
|
|
81
|
+
say so and pause the model decision; do not substitute a remembered recommendation
|
|
82
|
+
or SDK default. Continue only model-independent work. Local/browser models are not
|
|
83
|
+
exempt when the catalogue lacks coverage. If it reports that the device key is missing,
|
|
84
|
+
expired or revoked, run \`npx -y openmerit@latest login\` and then
|
|
85
|
+
\`npx -y openmerit@latest install\` again. Never ask the user to paste a key.
|
|
86
|
+
`;
|
|
87
|
+
export function scoutRule(client) {
|
|
88
|
+
if (client === 'claude-code')
|
|
89
|
+
return rule;
|
|
90
|
+
return rule.replace('load the `openmerit-scout` skill and consult', 'follow the Scout project policy below and consult')
|
|
91
|
+
.replaceAll('`scout_models`', '`mcp__openmerit__scout_models`')
|
|
92
|
+
.replace("Follow the skill's project access policy.", 'Follow the project access policy below.')
|
|
93
|
+
.replace('update --scope user', 'install --client pi --scope user')
|
|
94
|
+
.replace("rule is in the project's .claude/rules directory", "Pi installation is project-scoped");
|
|
95
|
+
}
|
|
96
|
+
export function scoutSkill(client) {
|
|
97
|
+
if (client === 'claude-code')
|
|
98
|
+
return skill;
|
|
99
|
+
return skill.replaceAll('`scout_models`', '`mcp__openmerit__scout_models`')
|
|
100
|
+
.replace('openmerit@latest login', 'openmerit@latest login --client pi')
|
|
101
|
+
.replace('openmerit@latest install`', 'openmerit@latest install --client pi`')
|
|
102
|
+
.replace('again. Never ask the user to paste a key.', 'again, using `--scope project` on both commands for a project-scoped installation. Never ask the user to paste a key.');
|
|
103
|
+
}
|
|
@@ -0,0 +1,106 @@
|
|
|
1
|
+
import type { ScoutClient } from './scout-credentials.ts';
|
|
2
|
+
|
|
3
|
+
export const rule = `# Application model decisions · OpenMerit Scout
|
|
4
|
+
|
|
5
|
+
Before choosing, recommending, adding, downloading, replacing or configuring
|
|
6
|
+
ANY application inference model, load the \`openmerit-scout\` skill and consult
|
|
7
|
+
\`scout_models\` on the \`openmerit\` MCP server before committing to a model.
|
|
8
|
+
This applies even when the user only asks you to build a feature, never mentions
|
|
9
|
+
models or Scout, or names a model themselves (assess it; do not silently replace it).
|
|
10
|
+
|
|
11
|
+
A pretrained model, local weights, browser inference, an SDK's default model,
|
|
12
|
+
embeddings, OCR, speech, image classification, detection and segmentation all
|
|
13
|
+
count as model choices, not implementation details. Identify this decision before
|
|
14
|
+
writing model-specific code, installing a model package or downloading weights.
|
|
15
|
+
|
|
16
|
+
Follow the skill's project access policy. If it is missing, ask and save it first.
|
|
17
|
+
If Scout is unavailable, fails or has no suitable evidence/candidate, disclose the
|
|
18
|
+
gap and pause the model decision; do not fall back to a remembered or SDK-default
|
|
19
|
+
model. You may continue work that does not commit to a model.
|
|
20
|
+
|
|
21
|
+
Do not consult Scout for unrelated edits, normal inference calls using an unchanged
|
|
22
|
+
approved model, or this coding agent's own model. Do not reselect approved models
|
|
23
|
+
unless the task introduces or changes a model decision.
|
|
24
|
+
|
|
25
|
+
When the user says "update openmerit", run
|
|
26
|
+
\`npx -y openmerit@latest update --scope user\` (use \`--scope project\` if this
|
|
27
|
+
rule is in the project's .claude/rules directory). Report the result, including
|
|
28
|
+
any conflict or manual-upgrade requirement. Do not delete edited files, bypass
|
|
29
|
+
permissions or rerun login automatically. A new session loads updated instructions.
|
|
30
|
+
`;
|
|
31
|
+
|
|
32
|
+
export const skill = `---
|
|
33
|
+
name: openmerit-scout
|
|
34
|
+
description: Consults OpenMerit Scout before choosing, adding, downloading, replacing or configuring any application inference model, including implicit choices while building features, pretrained browser/local models, SDK defaults, embeddings, OCR, speech, classification, detection, segmentation, LLMs and agents. Not for unrelated edits, ordinary inference calls or this coding agent's own model.
|
|
35
|
+
---
|
|
36
|
+
|
|
37
|
+
# OpenMerit · Scout
|
|
38
|
+
|
|
39
|
+
Before introducing, recommending, replacing or configuring an application
|
|
40
|
+
inference model, call the \`scout_models\` tool on the \`openmerit\` MCP server.
|
|
41
|
+
This applies even when the user never mentions models or Scout. Downloading local
|
|
42
|
+
weights, using a pretrained browser model or accepting an SDK default is a model
|
|
43
|
+
decision: consult Scout before model-specific code, installs or downloads.
|
|
44
|
+
Preserve the user's explicit model choice; assess it rather than silently replacing
|
|
45
|
+
it. Leave unchanged, approved application selections in place.
|
|
46
|
+
|
|
47
|
+
## Project policy
|
|
48
|
+
|
|
49
|
+
Read \`.openmerit/scout.json\` before searching. If it is absent, ask once whether this
|
|
50
|
+
project uses **existing-only** (only endpoints it can already reach) or
|
|
51
|
+
**existing-first** (prefer them, ask before searching the whole catalogue), and
|
|
52
|
+
which public catalogue endpoint IDs it can access. Save the answer:
|
|
53
|
+
|
|
54
|
+
\`\`\`sh
|
|
55
|
+
npx -y openmerit@latest project --policy existing-only --endpoint-id <id> ...
|
|
56
|
+
\`\`\`
|
|
57
|
+
|
|
58
|
+
Never infer access from a model name, SDK or environment-variable name. Never
|
|
59
|
+
record credentials, private URLs, prompts or customer data. An empty inventory
|
|
60
|
+
is not permission to search all models.
|
|
61
|
+
|
|
62
|
+
## Searching
|
|
63
|
+
|
|
64
|
+
Call \`scout_models\` with a non-confidential task description and
|
|
65
|
+
\`endpointIds\` set to the saved inventory, including \`[]\` when empty.
|
|
66
|
+
Existing-only never broadens. For existing-first, if no existing candidate fits,
|
|
67
|
+
explain the gap and ask before a separate search without \`endpointIds\`. Send
|
|
68
|
+
only task requirements, never source files, secrets or real customer records.
|
|
69
|
+
|
|
70
|
+
Compare model AND endpoint, evidence, reference cost/time, unknowns and
|
|
71
|
+
warnings. For multiple tasks read \`tasks[].slots[0].recommendations\`; the
|
|
72
|
+
top-level list is empty. Prefer the cheapest usable option the quality evidence
|
|
73
|
+
supports, without claiming benchmarks prove application accuracy. Use the
|
|
74
|
+
returned provider model ID and the project's actual SDK and protocol.
|
|
75
|
+
|
|
76
|
+
## Boundaries
|
|
77
|
+
|
|
78
|
+
Ask before adding providers or credentials, spending, deploying or external
|
|
79
|
+
writes. Report recommendation, endpoint connectivity and application checks as
|
|
80
|
+
separate results, each only after it succeeds. Do not switch this coding agent's
|
|
81
|
+
own model.
|
|
82
|
+
|
|
83
|
+
If Scout fails or has no suitable candidate/evidence for the required deployment,
|
|
84
|
+
say so and pause the model decision; do not substitute a remembered recommendation
|
|
85
|
+
or SDK default. Continue only model-independent work. Local/browser models are not
|
|
86
|
+
exempt when the catalogue lacks coverage. If it reports that the device key is missing,
|
|
87
|
+
expired or revoked, run \`npx -y openmerit@latest login\` and then
|
|
88
|
+
\`npx -y openmerit@latest install\` again. Never ask the user to paste a key.
|
|
89
|
+
`;
|
|
90
|
+
|
|
91
|
+
export function scoutRule(client: ScoutClient): string {
|
|
92
|
+
if (client === 'claude-code') return rule;
|
|
93
|
+
return rule.replace('load the `openmerit-scout` skill and consult', 'follow the Scout project policy below and consult')
|
|
94
|
+
.replaceAll('`scout_models`', '`mcp__openmerit__scout_models`')
|
|
95
|
+
.replace("Follow the skill's project access policy.", 'Follow the project access policy below.')
|
|
96
|
+
.replace('update --scope user', 'install --client pi --scope user')
|
|
97
|
+
.replace("rule is in the project's .claude/rules directory", "Pi installation is project-scoped");
|
|
98
|
+
}
|
|
99
|
+
|
|
100
|
+
export function scoutSkill(client: ScoutClient): string {
|
|
101
|
+
if (client === 'claude-code') return skill;
|
|
102
|
+
return skill.replaceAll('`scout_models`', '`mcp__openmerit__scout_models`')
|
|
103
|
+
.replace('openmerit@latest login', 'openmerit@latest login --client pi')
|
|
104
|
+
.replace('openmerit@latest install`', 'openmerit@latest install --client pi`')
|
|
105
|
+
.replace('again. Never ask the user to paste a key.', 'again, using `--scope project` on both commands for a project-scoped installation. Never ask the user to paste a key.');
|
|
106
|
+
}
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
import type { ScoutScope } from './scout-credentials.ts';
|
|
2
|
+
export declare const piVersion = "1.5.0";
|
|
3
|
+
type RecordValue = Record<string, unknown>;
|
|
4
|
+
export declare function piPaths(root: string, scope: ScoutScope): {
|
|
5
|
+
skill: string;
|
|
6
|
+
rule: string;
|
|
7
|
+
connection: string;
|
|
8
|
+
record: string;
|
|
9
|
+
legacyRule: string;
|
|
10
|
+
mcpFile: string;
|
|
11
|
+
mcpScope: ScoutScope;
|
|
12
|
+
agentDir: string;
|
|
13
|
+
};
|
|
14
|
+
export declare function inspectPi(root: string, scope: ScoutScope): Promise<{
|
|
15
|
+
paths: {
|
|
16
|
+
skill: string;
|
|
17
|
+
rule: string;
|
|
18
|
+
connection: string;
|
|
19
|
+
record: string;
|
|
20
|
+
legacyRule: string;
|
|
21
|
+
mcpFile: string;
|
|
22
|
+
mcpScope: ScoutScope;
|
|
23
|
+
agentDir: string;
|
|
24
|
+
};
|
|
25
|
+
record: RecordValue;
|
|
26
|
+
owned: boolean;
|
|
27
|
+
current: string | null;
|
|
28
|
+
installedSkill: string | null;
|
|
29
|
+
installedRule: string | null;
|
|
30
|
+
serverOwned: boolean;
|
|
31
|
+
skillOwned: boolean;
|
|
32
|
+
ruleOwned: boolean;
|
|
33
|
+
updates: {
|
|
34
|
+
saved: null;
|
|
35
|
+
active: boolean;
|
|
36
|
+
conflicts: string[];
|
|
37
|
+
};
|
|
38
|
+
legacyRule: string;
|
|
39
|
+
shadowed: boolean;
|
|
40
|
+
}>;
|
|
41
|
+
export declare function piConflicts(state: Awaited<ReturnType<typeof inspectPi>>): string[];
|
|
42
|
+
export declare function configurePi(root: string, scope: ScoutScope, credentialsPath: string, ids: RecordValue): Promise<void>;
|
|
43
|
+
export declare function removePi(state: Awaited<ReturnType<typeof inspectPi>>): Promise<void>;
|
|
44
|
+
export {};
|
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
import { createHash } from 'node:crypto';
|
|
2
|
+
import { readFile, mkdir, writeFile, rm } from 'node:fs/promises';
|
|
3
|
+
import { homedir } from 'node:os';
|
|
4
|
+
import { dirname, join, resolve } from 'node:path';
|
|
5
|
+
import { scoutSkill } from "./scout-instructions.js";
|
|
6
|
+
export const piVersion = '1.5.0';
|
|
7
|
+
const hash = (value) => createHash('sha256').update(value).digest('hex');
|
|
8
|
+
const optional = async (path) => {
|
|
9
|
+
try {
|
|
10
|
+
return await readFile(path, 'utf8');
|
|
11
|
+
}
|
|
12
|
+
catch (error) {
|
|
13
|
+
if (error instanceof Error && 'code' in error && error.code === 'ENOENT')
|
|
14
|
+
return null;
|
|
15
|
+
throw error;
|
|
16
|
+
}
|
|
17
|
+
};
|
|
18
|
+
async function object(path) {
|
|
19
|
+
const value = JSON.parse(await optional(path) ?? '{}');
|
|
20
|
+
if (!value || typeof value !== 'object' || Array.isArray(value))
|
|
21
|
+
throw new Error(`Invalid Scout installation metadata at ${path}; nothing changed.`);
|
|
22
|
+
return value;
|
|
23
|
+
}
|
|
24
|
+
export function piPaths(root, scope) {
|
|
25
|
+
const agentDir = resolve(process.env.PI_CODING_AGENT_DIR ?? join(homedir(), '.pi/agent'));
|
|
26
|
+
const base = scope === 'user' ? agentDir : join(root, '.pi');
|
|
27
|
+
return { skill: join(base, 'skills/openmerit-scout/SKILL.md'), rule: join(base, 'extensions/openmerit-scout.ts'),
|
|
28
|
+
connection: join(base, 'extensions/openmerit-scout.json'), record: join(base, 'openmerit-install.json'),
|
|
29
|
+
legacyRule: join(base, 'rules/openmerit.md'), mcpFile: join(base, 'mcp.json'), mcpScope: scope, agentDir };
|
|
30
|
+
}
|
|
31
|
+
export async function inspectPi(root, scope) {
|
|
32
|
+
const paths = piPaths(root, scope);
|
|
33
|
+
const otherPaths = piPaths(root, scope === 'user' ? 'project' : 'user');
|
|
34
|
+
const [record, connection, installedSkill, installedRule, mcp, otherMcp, other] = await Promise.all([
|
|
35
|
+
object(paths.record), optional(paths.connection), optional(paths.skill), optional(paths.rule), object(paths.mcpFile),
|
|
36
|
+
object(otherPaths.mcpFile), optional(otherPaths.rule),
|
|
37
|
+
]);
|
|
38
|
+
const owned = record.version === 1 && record.client === 'pi' && record.scope === scope;
|
|
39
|
+
const matches = (value, field) => value === null || owned && [record[field], record[`pending${field[0].toUpperCase()}${field.slice(1)}`]].includes(hash(value));
|
|
40
|
+
return { paths, record, owned, current: connection, installedSkill, installedRule,
|
|
41
|
+
serverOwned: matches(connection, 'serverHash'), skillOwned: matches(installedSkill, 'skillHash'), ruleOwned: matches(installedRule, 'ruleHash'),
|
|
42
|
+
updates: { saved: null, active: false, conflicts: [] }, legacyRule: 'none',
|
|
43
|
+
shadowed: other !== null || [mcp, otherMcp].some(config => config.mcpServers && typeof config.mcpServers === 'object' && 'openmerit' in config.mcpServers), };
|
|
44
|
+
}
|
|
45
|
+
export function piConflicts(state) {
|
|
46
|
+
return [
|
|
47
|
+
...(!state.serverOwned ? [`${state.paths.connection} was edited or is unowned.`] : []),
|
|
48
|
+
...(!state.skillOwned ? [`${state.paths.skill} was edited or is unowned.`] : []),
|
|
49
|
+
...(!state.ruleOwned ? [`${state.paths.rule} was edited or is unowned.`] : []),
|
|
50
|
+
...(state.shadowed ? ['Another Pi Scout installation or an openmerit MCP entry shadows this installation. Resolve the duplicate before installing.'] : []),
|
|
51
|
+
];
|
|
52
|
+
}
|
|
53
|
+
export async function configurePi(root, scope, credentialsPath, ids) {
|
|
54
|
+
const state = await inspectPi(root, scope);
|
|
55
|
+
const conflicts = piConflicts(state);
|
|
56
|
+
if (conflicts.length)
|
|
57
|
+
throw new Error(conflicts.join('\n'));
|
|
58
|
+
const extension = await readFile(new URL('../../../dist/pi/scout-extension.mjs', import.meta.url), 'utf8');
|
|
59
|
+
const connection = `${JSON.stringify({ credentialsPath }, null, 2)}\n`;
|
|
60
|
+
const skill = scoutSkill('pi');
|
|
61
|
+
const next = { version: 1, client: 'pi', scope, ...ids, serverHash: hash(connection), skillHash: hash(skill), ruleHash: hash(extension), installedAt: new Date().toISOString() };
|
|
62
|
+
const save = async (path, value) => {
|
|
63
|
+
await mkdir(dirname(path), { recursive: true, mode: 0o700 });
|
|
64
|
+
await writeFile(path, typeof value === 'string' ? value : `${JSON.stringify(value, null, 2)}\n`, { mode: 0o600 });
|
|
65
|
+
};
|
|
66
|
+
// Record intent before writing files, so interrupted installs are recoverable.
|
|
67
|
+
await save(state.paths.record, { ...state.record, version: 1, client: 'pi', scope,
|
|
68
|
+
pendingServerHash: next.serverHash, pendingSkillHash: next.skillHash, pendingRuleHash: next.ruleHash });
|
|
69
|
+
await save(state.paths.connection, connection);
|
|
70
|
+
await save(state.paths.skill, skill);
|
|
71
|
+
await save(state.paths.rule, extension);
|
|
72
|
+
await save(state.paths.record, next);
|
|
73
|
+
}
|
|
74
|
+
export async function removePi(state) {
|
|
75
|
+
if (state.current !== null && state.serverOwned)
|
|
76
|
+
await rm(state.paths.connection, { force: true });
|
|
77
|
+
if (state.installedRule !== null && state.ruleOwned)
|
|
78
|
+
await rm(state.paths.rule, { force: true });
|
|
79
|
+
if (state.installedSkill !== null && state.skillOwned)
|
|
80
|
+
await rm(state.paths.skill, { force: true });
|
|
81
|
+
// Keep ownership of edited files, so a future install does not adopt them.
|
|
82
|
+
if (state.serverOwned && state.ruleOwned && state.skillOwned)
|
|
83
|
+
await rm(state.paths.record, { force: true });
|
|
84
|
+
}
|
|
@@ -0,0 +1,76 @@
|
|
|
1
|
+
import { createHash } from 'node:crypto';
|
|
2
|
+
import { readFile, mkdir, writeFile, rm } from 'node:fs/promises';
|
|
3
|
+
import { homedir } from 'node:os';
|
|
4
|
+
import { dirname, join, resolve } from 'node:path';
|
|
5
|
+
import type { ScoutScope } from './scout-credentials.ts';
|
|
6
|
+
import { scoutSkill } from './scout-instructions.ts';
|
|
7
|
+
|
|
8
|
+
export const piVersion = '1.5.0';
|
|
9
|
+
type RecordValue = Record<string, unknown>;
|
|
10
|
+
const hash = (value: string) => createHash('sha256').update(value).digest('hex');
|
|
11
|
+
const optional = async (path: string): Promise<string | null> => {
|
|
12
|
+
try { return await readFile(path, 'utf8'); }
|
|
13
|
+
catch (error) { if (error instanceof Error && 'code' in error && error.code === 'ENOENT') return null; throw error; }
|
|
14
|
+
};
|
|
15
|
+
async function object(path: string): Promise<RecordValue> {
|
|
16
|
+
const value: unknown = JSON.parse(await optional(path) ?? '{}');
|
|
17
|
+
if (!value || typeof value !== 'object' || Array.isArray(value)) throw new Error(`Invalid Scout installation metadata at ${path}; nothing changed.`);
|
|
18
|
+
return value as RecordValue;
|
|
19
|
+
}
|
|
20
|
+
export function piPaths(root: string, scope: ScoutScope) {
|
|
21
|
+
const agentDir = resolve(process.env.PI_CODING_AGENT_DIR ?? join(homedir(), '.pi/agent'));
|
|
22
|
+
const base = scope === 'user' ? agentDir : join(root, '.pi');
|
|
23
|
+
return { skill: join(base, 'skills/openmerit-scout/SKILL.md'), rule: join(base, 'extensions/openmerit-scout.ts'),
|
|
24
|
+
connection: join(base, 'extensions/openmerit-scout.json'), record: join(base, 'openmerit-install.json'),
|
|
25
|
+
legacyRule: join(base, 'rules/openmerit.md'), mcpFile: join(base, 'mcp.json'), mcpScope: scope, agentDir };
|
|
26
|
+
}
|
|
27
|
+
export async function inspectPi(root: string, scope: ScoutScope) {
|
|
28
|
+
const paths = piPaths(root, scope);
|
|
29
|
+
const otherPaths = piPaths(root, scope === 'user' ? 'project' : 'user');
|
|
30
|
+
const [record, connection, installedSkill, installedRule, mcp, otherMcp, other] = await Promise.all([
|
|
31
|
+
object(paths.record), optional(paths.connection), optional(paths.skill), optional(paths.rule), object(paths.mcpFile),
|
|
32
|
+
object(otherPaths.mcpFile), optional(otherPaths.rule),
|
|
33
|
+
]);
|
|
34
|
+
const owned = record.version === 1 && record.client === 'pi' && record.scope === scope;
|
|
35
|
+
const matches = (value: string | null, field: string) => value === null || owned && [record[field], record[`pending${field[0]!.toUpperCase()}${field.slice(1)}`]].includes(hash(value));
|
|
36
|
+
return { paths, record, owned, current: connection, installedSkill, installedRule,
|
|
37
|
+
serverOwned: matches(connection, 'serverHash'), skillOwned: matches(installedSkill, 'skillHash'), ruleOwned: matches(installedRule, 'ruleHash'),
|
|
38
|
+
updates: { saved: null, active: false, conflicts: [] as string[] }, legacyRule: 'none',
|
|
39
|
+
shadowed: other !== null || [mcp, otherMcp].some(config => config.mcpServers && typeof config.mcpServers === 'object' && 'openmerit' in config.mcpServers),
|
|
40
|
+
};
|
|
41
|
+
}
|
|
42
|
+
export function piConflicts(state: Awaited<ReturnType<typeof inspectPi>>): string[] {
|
|
43
|
+
return [
|
|
44
|
+
...(!state.serverOwned ? [`${state.paths.connection} was edited or is unowned.`] : []),
|
|
45
|
+
...(!state.skillOwned ? [`${state.paths.skill} was edited or is unowned.`] : []),
|
|
46
|
+
...(!state.ruleOwned ? [`${state.paths.rule} was edited or is unowned.`] : []),
|
|
47
|
+
...(state.shadowed ? ['Another Pi Scout installation or an openmerit MCP entry shadows this installation. Resolve the duplicate before installing.'] : []),
|
|
48
|
+
];
|
|
49
|
+
}
|
|
50
|
+
export async function configurePi(root: string, scope: ScoutScope, credentialsPath: string, ids: RecordValue) {
|
|
51
|
+
const state = await inspectPi(root, scope);
|
|
52
|
+
const conflicts = piConflicts(state);
|
|
53
|
+
if (conflicts.length) throw new Error(conflicts.join('\n'));
|
|
54
|
+
const extension = await readFile(new URL('../../../dist/pi/scout-extension.mjs', import.meta.url), 'utf8');
|
|
55
|
+
const connection = `${JSON.stringify({ credentialsPath }, null, 2)}\n`;
|
|
56
|
+
const skill = scoutSkill('pi');
|
|
57
|
+
const next = { version: 1, client: 'pi', scope, ...ids, serverHash: hash(connection), skillHash: hash(skill), ruleHash: hash(extension), installedAt: new Date().toISOString() };
|
|
58
|
+
const save = async (path: string, value: string | RecordValue) => {
|
|
59
|
+
await mkdir(dirname(path), { recursive: true, mode: 0o700 });
|
|
60
|
+
await writeFile(path, typeof value === 'string' ? value : `${JSON.stringify(value, null, 2)}\n`, { mode: 0o600 });
|
|
61
|
+
};
|
|
62
|
+
// Record intent before writing files, so interrupted installs are recoverable.
|
|
63
|
+
await save(state.paths.record, { ...state.record, version: 1, client: 'pi', scope,
|
|
64
|
+
pendingServerHash: next.serverHash, pendingSkillHash: next.skillHash, pendingRuleHash: next.ruleHash });
|
|
65
|
+
await save(state.paths.connection, connection);
|
|
66
|
+
await save(state.paths.skill, skill);
|
|
67
|
+
await save(state.paths.rule, extension);
|
|
68
|
+
await save(state.paths.record, next);
|
|
69
|
+
}
|
|
70
|
+
export async function removePi(state: Awaited<ReturnType<typeof inspectPi>>) {
|
|
71
|
+
if (state.current !== null && state.serverOwned) await rm(state.paths.connection, { force: true });
|
|
72
|
+
if (state.installedRule !== null && state.ruleOwned) await rm(state.paths.rule, { force: true });
|
|
73
|
+
if (state.installedSkill !== null && state.skillOwned) await rm(state.paths.skill, { force: true });
|
|
74
|
+
// Keep ownership of edited files, so a future install does not adopt them.
|
|
75
|
+
if (state.serverOwned && state.ruleOwned && state.skillOwned) await rm(state.paths.record, { force: true });
|
|
76
|
+
}
|
|
@@ -38,7 +38,7 @@ export const setupUUID = value => typeof value === 'string' && /^[0-9a-f]{8}-[0-
|
|
|
38
38
|
export function setupHeaders(ids = {}, client = 'cli') {
|
|
39
39
|
return {
|
|
40
40
|
'X-Scout-Client': client,
|
|
41
|
-
'X-Scout-Agent': 'claude-code',
|
|
41
|
+
'X-Scout-Agent': ids.agent === 'pi' ? 'pi' : 'claude-code',
|
|
42
42
|
...(clientVersion ? { 'X-Scout-Client-Version': clientVersion } : {}),
|
|
43
43
|
'X-Scout-Analytics-Content': 'off',
|
|
44
44
|
...(client === 'cli' ? { 'X-Scout-Analytics': telemetryEnabled(true) ? 'on' : 'off' } : !telemetryEnabled(true) ? { 'X-Scout-Analytics': 'off' } : {}),
|
|
@@ -54,7 +54,7 @@ export function installationProperties(input) {
|
|
|
54
54
|
stage: ['started', 'preflight', 'login', 'browser_open', 'configuration', 'verification', 'status', 'remove', 'completed'],
|
|
55
55
|
outcome: ['started', 'ok', 'error', 'pending', 'denied', 'conflict', 'not_installed'],
|
|
56
56
|
command: ['install', 'status', 'remove', 'login', 'logout', 'project', 'doctor', 'unknown'],
|
|
57
|
-
agent: ['claude-code'], scope: ['user', 'project'],
|
|
57
|
+
agent: ['claude-code', 'pi'], scope: ['user', 'project'],
|
|
58
58
|
rule_status: ['matches', 'missing', 'modified_or_unowned'],
|
|
59
59
|
skill_status: ['matches', 'missing', 'modified_or_unowned'],
|
|
60
60
|
server_status: ['matches', 'missing', 'modified_or_unowned'],
|
|
@@ -2,6 +2,7 @@
|
|
|
2
2
|
// built-ins: it must survive the npx cache disappearing, without installing or
|
|
3
3
|
// executing code from the release it downloads.
|
|
4
4
|
import { createHash, randomUUID } from 'node:crypto';
|
|
5
|
+
import { realpathSync } from 'node:fs';
|
|
5
6
|
import { mkdir, open, readFile, rename, rm, stat, writeFile } from 'node:fs/promises';
|
|
6
7
|
import { dirname, join, resolve } from 'node:path';
|
|
7
8
|
import { fileURLToPath } from 'node:url';
|
|
@@ -381,7 +382,7 @@ export async function runUpdate(base, scope, options = {}) {
|
|
|
381
382
|
}
|
|
382
383
|
// Hooks deliberately never ask Claude to continue, read transcripts or emit
|
|
383
384
|
// blocking decisions. Offline and concurrent-hook failures exit successfully.
|
|
384
|
-
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
|
|
385
|
+
if (process.argv[1] && realpathSync(resolve(process.argv[1])) === fileURLToPath(import.meta.url)) {
|
|
385
386
|
const args = process.argv.slice(2), base = args[args.indexOf('--base') + 1], scope = args[args.indexOf('--scope') + 1];
|
|
386
387
|
if (base && ['user', 'project'].includes(scope ?? '')) {
|
|
387
388
|
process.stdin.resume();
|
|
@@ -2,6 +2,7 @@
|
|
|
2
2
|
// built-ins: it must survive the npx cache disappearing, without installing or
|
|
3
3
|
// executing code from the release it downloads.
|
|
4
4
|
import { createHash, randomUUID } from 'node:crypto';
|
|
5
|
+
import { realpathSync } from 'node:fs';
|
|
5
6
|
import { mkdir, open, readFile, rename, rm, stat, writeFile } from 'node:fs/promises';
|
|
6
7
|
import { dirname, join, resolve } from 'node:path';
|
|
7
8
|
import { fileURLToPath } from 'node:url';
|
|
@@ -300,7 +301,7 @@ export async function runUpdate(base: string, scope: string, options: { hook?: b
|
|
|
300
301
|
|
|
301
302
|
// Hooks deliberately never ask Claude to continue, read transcripts or emit
|
|
302
303
|
// blocking decisions. Offline and concurrent-hook failures exit successfully.
|
|
303
|
-
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
|
|
304
|
+
if (process.argv[1] && realpathSync(resolve(process.argv[1])) === fileURLToPath(import.meta.url)) {
|
|
304
305
|
const args = process.argv.slice(2), base = args[args.indexOf('--base') + 1], scope = args[args.indexOf('--scope') + 1];
|
|
305
306
|
if (base && ['user', 'project'].includes(scope ?? '')) {
|
|
306
307
|
process.stdin.resume();
|
package/docs/adapter-guide.md
CHANGED
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# Building a harness adapter
|
|
2
2
|
|
|
3
|
+
**Evaluation coordinator boundary:** this guide remains applicable to custom harnesses. Pi-specific examples refer to the historical Pi 0.87 evaluation adapter. The unreleased Pi 1.5.0 `openmerit/pi` export is instead a thin Scout adapter and does not implement `HarnessAdapter`; see [native MCP and SDK embedding](pi-extension.md#sdk-embedding).
|
|
4
|
+
|
|
3
5
|
OpenMerit adapters are transport bridges. They do not reproduce OpenMerit's lifecycle, validation, persistence, or Pareto rules, and OpenMerit does not reproduce the harness's intelligence.
|
|
4
6
|
|
|
5
7
|
## Required components
|
package/docs/architecture.md
CHANGED
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# Architecture
|
|
2
2
|
|
|
3
|
+
**Source status:** the Pi 1.5.0 native Scout replacement is unreleased and unpublished. The protocol/core evaluation coordinator described here remains available for custom harnesses; it is not invoked by the new Pi adapter. The standalone evaluation CLI and Inference Terminal are unchanged.
|
|
4
|
+
|
|
3
5
|
## Design principle
|
|
4
6
|
|
|
5
7
|
OpenMerit is an orchestrator, not an intelligence layer. The product application LLM call is the system under test; the coding harness is the evaluator and execution environment.
|
|
@@ -12,7 +14,7 @@ OpenMerit deterministic orchestration
|
|
|
12
14
|
|
|
|
13
15
|
| typed intent: outcome, constraints, evidence
|
|
14
16
|
v
|
|
15
|
-
|
|
17
|
+
Custom evaluation harness
|
|
16
18
|
|
|
|
17
19
|
| intelligent work, tools, evals, calculation
|
|
18
20
|
v
|
|
@@ -38,13 +40,13 @@ Contains deterministic behavior: choosing the next lifecycle intent according to
|
|
|
38
40
|
|
|
39
41
|
`OpenMeritCoordinator` is the reusable adapter boundary. It constructs canonical intents, negotiates capabilities, records lifecycle progress and observations, validates results, and advances durable state. `OpenMeritStore` makes persistence replaceable; `ProjectStore` is the bundled filesystem backend.
|
|
40
42
|
|
|
41
|
-
The coordinator also accepts idempotent application-target automation signals and evaluates the user-confirmed `AutomationCheckPolicy`. `buildAutomationPlan` tells an adapter which signals and wakeups are required and reports unsupported persistent scheduling honestly. Provisioning belongs to
|
|
43
|
+
The coordinator also accepts idempotent application-target automation signals and evaluates the user-confirmed `AutomationCheckPolicy`. `buildAutomationPlan` tells an adapter which signals and wakeups are required and reports unsupported persistent scheduling honestly. Provisioning belongs to a custom adapter; the coordinator itself does not create operating-system schedulers or paid cloud workers. The removed Pi 0.87 adapter's LaunchAgent is historical functionality, not part of the new Scout connection.
|
|
42
44
|
|
|
43
45
|
### `openmerit/pi`
|
|
44
46
|
|
|
45
|
-
|
|
47
|
+
Exports `createScoutPiExtension` and a default native Scout extension for Pi 1.5.0. It registers the authenticated Scout HTTP MCP server through Pi and adds shared rule/skill prompt sections. Pi owns discovery, tools, permissions, model and sessions; SDK hosts must load Pi's `createMcpExtension()` explicitly alongside it.
|
|
46
48
|
|
|
47
|
-
The
|
|
49
|
+
The package no longer implements `HarnessAdapter`, exposes lifecycle commands, or schedules evaluation work. It shares credential and instruction code with the installer instead of duplicating an MCP client or evaluation coordinator. See [Pi installation and migration](pi-extension.md).
|
|
48
50
|
|
|
49
51
|
### Inference Terminal
|
|
50
52
|
|
|
@@ -52,7 +54,7 @@ The Pi package implements `HarnessAdapter`. It does not define canonical intents
|
|
|
52
54
|
|
|
53
55
|
## Intelligence resources
|
|
54
56
|
|
|
55
|
-
Models, web research, tools, evaluators, and fast decision systems such as Jev System One are harness resources. They are configured and invoked by
|
|
57
|
+
Models, web research, tools, evaluators, and fast decision systems such as Jev System One are harness resources. They are configured and invoked by the host harness. This keeps the coordinator provider-neutral and lets custom adapters choose their own implementation.
|
|
56
58
|
|
|
57
59
|
## Trust boundary
|
|
58
60
|
|
package/docs/automation.md
CHANGED
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# Harness-neutral automation
|
|
2
2
|
|
|
3
|
+
**Core reference / historical Pi workflow:** the unreleased Pi 1.5.0 Scout adapter replaces the Pi 0.87 lifecycle adapter. It has no signals, `/openmerit` commands, scheduler, headless wakeups, or LaunchAgent. Pi-specific provisioning instructions below are historical, not available in the replacement. The coordinator's policy and signal APIs remain available to custom harnesses. See [current Pi setup and migration](pi-extension.md); the standalone [evaluation watcher](agent-evaluations.md) is unchanged.
|
|
4
|
+
|
|
3
5
|
The user defines cadence, thresholds, budget, scheduling mode, and automation permission. OpenMerit stores that policy, evaluates durable signals, and decides whether a bounded intent is due. A harness adapter arranges native wakeups and reports events. The coding harness performs the resulting intelligent work.
|
|
4
6
|
|
|
5
7
|
```text
|
package/docs/budgets.md
CHANGED
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# Budgets and permissions
|
|
2
2
|
|
|
3
|
+
**Core reference / historical Pi workflow:** these coordinator policies are available to custom evaluation harnesses. Pi-specific setup and automation references describe the removed Pi 0.87 adapter, not the unreleased Pi 1.5.0 Scout connection. The new adapter grants only Scout MCP access and shared guidance; it does not run evaluations or authorize swaps. See [Pi setup](pi-extension.md). The standalone CLI has its own [budget controls](agent-evaluations.md#budget-selection), unchanged by this replacement.
|
|
4
|
+
|
|
3
5
|
Setup asks you to confirm the limits and permissions for evaluation work. OpenMerit stores the policy and passes its constraints to the harness.
|
|
4
6
|
|
|
5
7
|
## Evaluation costs
|
package/docs/commands.md
CHANGED
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
# Commands
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
**Pi replacement — unreleased, unpublished source:** Pi 1.5.0 uses only native Scout MCP and shared guidance. Run `npx -y openmerit@latest install --client pi [--scope user|project]` after publication and approve in the browser. Shell `login`, `logout`, `status`, `remove`, and `doctor` accept the same target flags. Updates require explicit reinstall; no Pi update hooks or scheduler are installed. See [Pi setup](pi-extension.md).
|
|
4
|
+
|
|
5
|
+
The standalone `openmerit dash` and `openmerit eval` commands are unchanged. All `/openmerit` slash commands below are **historical Pi 0.87 references**, not commands in the new Scout adapter. Core lifecycle APIs remain available for custom harnesses.
|
|
4
6
|
|
|
5
7
|
Stable npm `openmerit@0.1.7` provides `openmerit eval init`, `doctor`, `plan`, `run`, `watch` and `report`. See [agent evaluations](agent-evaluations.md) for scope selection, repeated task checks and budgets. These are shell commands, separate from Pi commands. The same release includes [Scout setup commands](scout.md#commands), beginning with `npx -y openmerit@latest install --client claude-code`.
|
|
6
8
|
|
package/docs/demo.md
CHANGED
|
@@ -13,7 +13,7 @@ The separately deployed [delivery-resolution experiment](https://openmerit.ai/de
|
|
|
13
13
|
|
|
14
14
|
## Run your own supervised trial
|
|
15
15
|
|
|
16
|
-
Use [
|
|
16
|
+
Use [the standalone evaluation CLI](agent-evaluations.md) or a [custom coordinator harness](adapter-guide.md) for real application measurements. [Your first trial](first-trial.md) retains the historical Pi 0.87 lifecycle walkthrough; the unreleased [Pi 1.5.0 Scout adapter](pi-extension.md) does not provide it. The repository's separate `npm run demo` sandbox makes live provider calls; its requirements and spend boundaries are in [testing](testing.md#experimental-sandbox).
|
|
17
17
|
|
|
18
18
|
## Recorded player
|
|
19
19
|
|
package/docs/first-trial.md
CHANGED
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# Your first supervised trial
|
|
2
2
|
|
|
3
|
+
**Historical Pi 0.87 walkthrough / core evaluation reference:** the unreleased Pi 1.5.0 replacement is a native Scout adapter, not an evaluation lifecycle adapter. It has none of the `/openmerit` commands or setup flow below. Retain this walkthrough for custom harness/coordinator work; for current setup use [Scout for Pi](pi-extension.md), and for standalone comparisons use [agent evaluations](agent-evaluations.md). This page is not a receipt of a successful trial.
|
|
4
|
+
|
|
3
5
|
Use one representative product task to learn whether OpenMerit and your harness can collect useful evidence. This guide describes a trial to perform; it does not claim that a trial has already succeeded for your product.
|
|
4
6
|
|
|
5
7
|
## Choose a bounded task
|