@piagent/platform 1.2.5 → 1.2.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +20 -0
- package/README.md +3 -2
- package/catalog/capabilities.json +4 -4
- package/docs/command-reference-vietnamese.md +9 -8
- package/docs/context-engine.md +177 -0
- package/docs/context-window-policy.md +13 -8
- package/docs/distribution-standard.md +7 -7
- package/docs/operator-manual-vietnamese.md +7 -7
- package/docs/publishing-for-teams.md +4 -0
- package/docs/quality-benchmark.md +14 -1
- package/docs/release-install-policy.md +6 -6
- package/docs/runtime-harness-standard.md +16 -7
- package/docs/security-threat-model.md +1 -1
- package/docs/task-lifecycle-vietnamese.md +4 -4
- package/docs/team-onboarding.md +2 -2
- package/docs/usage-observability.md +6 -0
- package/docs-site/architecture.html +4 -3
- package/docs-site/commands.html +2 -1
- package/docs-site/content/context-engine.html +47 -0
- package/docs-site/context-engine.html +171 -0
- package/docs-site/index.html +2 -1
- package/docs-site/mcp-approval.html +2 -1
- package/docs-site/mcp-auth.html +2 -1
- package/docs-site/mcp-servers.html +2 -1
- package/docs-site/mcp.html +2 -1
- package/docs-site/permissions.html +2 -1
- package/docs-site/profiles.html +2 -1
- package/docs-site/quickstart.html +2 -1
- package/docs-site/runtime.html +4 -3
- package/docs-site/scope.html +2 -1
- package/docs-site/security.html +2 -1
- package/docs-site/team.html +2 -1
- package/docs-site/workflows.html +2 -1
- package/package.json +3 -2
- package/packages/piagent-core/README.md +8 -2
- package/packages/piagent-core/capabilities/capability-core.js +4 -0
- package/packages/piagent-core/extensions/context-engine.js +1175 -0
- package/packages/piagent-core/extensions/piagent-guard.ts +785 -17
- package/packages/piagent-core/package.json +1 -1
- package/packages/piagent-core/policies/base-policy.json +2 -0
- package/packages/piagent-core/prompts/scout.md +10 -24
- package/packages/piagent-core/prompts/task.md +25 -45
- package/scripts/build-docs-site.mjs +6 -0
- package/scripts/context-engine.mjs +115 -0
- package/scripts/install-global.sh +14 -1
- package/scripts/piagent-cli.mjs +2 -1
- package/scripts/quality-benchmark.sh +71 -1
- package/scripts/verify-local.sh +8 -0
|
@@ -11,37 +11,23 @@ $ARGUMENTS
|
|
|
11
11
|
|
|
12
12
|
Use this when the user needs evidence before deciding whether to implement. Do not edit source.
|
|
13
13
|
|
|
14
|
-
|
|
14
|
+
Lean scouting flow:
|
|
15
15
|
|
|
16
|
-
1.
|
|
17
|
-
2.
|
|
18
|
-
3.
|
|
19
|
-
4.
|
|
20
|
-
5. Call `
|
|
21
|
-
6.
|
|
22
|
-
7.
|
|
23
|
-
|
|
24
|
-
- expected output;
|
|
25
|
-
- scope / out of scope;
|
|
26
|
-
- protected paths;
|
|
27
|
-
- required context;
|
|
28
|
-
- read-only verify command.
|
|
29
|
-
- review lenses and a compact read-only workPlan.
|
|
30
|
-
8. Read required context and targeted task files. Use `piagent_context_budget` before large files.
|
|
31
|
-
9. Use read-only subagents for independent mapping when useful: `piagent-scout`, builtin `context-builder`, and `piagent-reviewer` for final evidence review. Continue single-agent if unavailable, too small, or likely to burn more context than it saves.
|
|
32
|
-
10. Record context with `piagent_context_record`.
|
|
33
|
-
11. If shell is needed beyond simple read/list/test, call `piagent_exec_policy_check`.
|
|
34
|
-
12. Run the exact verify command from the task contract and record with `piagent_verify_record`.
|
|
35
|
-
13. Record handoff with `piagent_trace_record`.
|
|
36
|
-
14. Call `piagent_task_gate_check`. If the gate fails, report partial/blocked.
|
|
37
|
-
15. Do not paste full tool/test logs into chat or final output. Summarize the signal, quote only relevant lines, and use `/piagent-logs` or a narrower command when oversized output has been compacted.
|
|
16
|
+
1. For a narrow lookup, call `piagent_context` with concise detail, read targeted files, and return cited evidence. Do not create a task contract or load memory/index/orchestration policy automatically.
|
|
17
|
+
2. For a broad, cross-module, or high-risk scout, run `piagent_context_preflight` once. Stop if it recommends a fresh session.
|
|
18
|
+
3. Create a read-only contract with `piagent_task_start` only for a broad governed scout that needs a persisted scope, verification evidence, or handoff.
|
|
19
|
+
4. For an unfamiliar or cross-module area, call `piagent_context_engine` once with `action=pack` before manual search. Use memory, Context7, or a read-only subagent only when the pack is insufficient or the work has independent lanes. Verify every advisory hit against current files.
|
|
20
|
+
5. Call `piagent_context_budget` only for large files and `piagent_exec_policy_check` only for complex or unfamiliar shell commands. Simple read/list commands do not need proactive policy calls.
|
|
21
|
+
6. Record the files that materially support the conclusion. For a governed scout, record verification, trace, and task gate; for a narrow lookup, provide the evidence directly.
|
|
22
|
+
7. A low-confidence pack may trigger one bounded finder pass. Do not loop broad search after the requested evidence is covered.
|
|
23
|
+
8. Do not edit source and do not paste full command output. Summarize relevant evidence and unknowns.
|
|
38
24
|
|
|
39
25
|
Output:
|
|
40
26
|
|
|
41
27
|
- Changed files: none.
|
|
42
28
|
- Verify command/result.
|
|
43
29
|
- Context manifest.
|
|
44
|
-
-
|
|
30
|
+
- Extra workflow tools or subagents used, if any, and why.
|
|
45
31
|
- Review lenses applied.
|
|
46
32
|
- Memory cited, if any.
|
|
47
33
|
- Evidence matrix.
|
|
@@ -3,56 +3,36 @@ description: Implement a bounded project task using piagent Pi policy
|
|
|
3
3
|
argument-hint: "<task>"
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
Implement this task:
|
|
6
|
+
Implement this bounded project task:
|
|
7
7
|
|
|
8
8
|
```text
|
|
9
9
|
$ARGUMENTS
|
|
10
10
|
```
|
|
11
11
|
|
|
12
|
-
|
|
12
|
+
Core flow:
|
|
13
13
|
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
7. Decide whether subagents are worth their extra token/tool cost. If the bundled `pi-subagents` parent skill is available, use it for delegation patterns and safety boundaries. If `pi-subagents`/`subagent(...)` is available, use bounded subagents only for independent read-heavy scout/planning/review work:
|
|
37
|
-
- use `piagent-scout` for unfamiliar module/spec mapping;
|
|
38
|
-
- use builtin `context-builder` when a large task needs a handoff context/meta-prompt before planning;
|
|
39
|
-
- use `piagent-planner` for medium/high-risk implementation planning;
|
|
40
|
-
- use `piagent-reviewer` for final diff/test/scope review;
|
|
41
|
-
- use explicit review lenses instead of spawning a broad swarm;
|
|
42
|
-
- keep implementation single-writer unless the user explicitly asks for parallel writers or worktree isolation is clearly safe;
|
|
43
|
-
- if subagents are unavailable or not useful, continue single-agent and record why.
|
|
44
|
-
8. Record context manifest with `piagent_context_record`: file + reason.
|
|
45
|
-
9. If the task requires shell commands beyond simple read/list/test, call `piagent_exec_policy_check` first.
|
|
46
|
-
10. If the task requires non-piagent MCP/app tools, call `piagent_tool_policy_check` first.
|
|
47
|
-
11. If the task requires source writes, make a short plan first.
|
|
48
|
-
12. Do not touch protected paths.
|
|
49
|
-
13. Use MCP/tools only when the profile capability allows it.
|
|
50
|
-
14. Before final answer, run the exact verify command from `task.verifyCommands` through Pi bash, then record the observed result with `piagent_verify_record`. Do not use `true`, `echo ok`, or `|| true` as passing evidence unless that exact command is part of the task verify plan.
|
|
51
|
-
15. Record handoff with `piagent_trace_record`. If the task produces durable, non-secret project knowledge and the workflow is approved, record a compact cited node with `piagent_context_index_record`; do not save raw transcript.
|
|
52
|
-
16. Call `piagent_task_gate_check`. If gate fails, final outcome is blocked/partial, not done.
|
|
53
|
-
17. If the user asks about token/context/cost usage, call `piagent_usage_snapshot`; for exact token/cost totals, tell the user to run `/session` or `piagent-usage <project-path>`.
|
|
54
|
-
18. If verify cannot run, stop and report the exact blocker. Do not call it done.
|
|
55
|
-
19. Do not paste full test/build/tool logs into chat or final output. Summarize the signal, quote only the relevant failing lines, and use `/piagent-logs` or a narrower command when oversized output has been compacted.
|
|
14
|
+
1. Run `piagent_context_preflight` once with `workflow=task`, then call `piagent_context` with concise detail. Stop if preflight recommends a fresh session or onboarding context is missing.
|
|
15
|
+
2. Classify the request as `tiny`, `normal`, or `high-risk`, then create the Task Implementation Contract with `piagent_task_start` before source edits. Keep the contract compact and include acceptance criteria, scope, verify command, review lenses, and a short work plan.
|
|
16
|
+
3. Read only files required for the task and record them with `piagent_context_record`. Call `piagent_context_budget` only before reading a large or unfamiliar file; do not call it for ordinary targeted files.
|
|
17
|
+
4. Make a short plan before source writes. Keep one writer unless the user explicitly requests otherwise.
|
|
18
|
+
5. Call `piagent_exec_policy_check` only for complex, generated, unfamiliar, destructive, or externally acting shell commands. Simple read/list and configured test commands do not need a proactive check because runtime guards still apply.
|
|
19
|
+
6. Call `piagent_tool_policy_check` only before a non-piagent MCP/app tool whose capability is not already clear from concise project context.
|
|
20
|
+
7. Run the exact configured verify command through Pi bash, record observed evidence with `piagent_verify_record`, record the final handoff with `piagent_trace_record`, then call `piagent_task_gate_check`.
|
|
21
|
+
|
|
22
|
+
Risk-adaptive enrichment:
|
|
23
|
+
|
|
24
|
+
- `tiny`: use only the core flow. Do not call orchestration, memory, context-index, tech-context, or subagent tools unless the task actually depends on them.
|
|
25
|
+
- `normal`: for unfamiliar or cross-module work, call `piagent_context_engine` once with `action=pack`; read the ranked current files and stop broad searching when confidence/evidence is sufficient. Use memory only for a known project decision. Call `piagent_orchestration_policy` only when bounded read-only delegation may save context.
|
|
26
|
+
- `high-risk`: use a bounded Context Engine pack, inspect orchestration policy, relevant memory/context-index evidence, and current vendor documentation when needed. Apply explicit security/data/release review lenses before the final gate.
|
|
27
|
+
|
|
28
|
+
Always:
|
|
29
|
+
|
|
30
|
+
- Treat memory, context-index, and generated tech snapshots as advisory; verify against current files.
|
|
31
|
+
- If a Context Engine pack reports low confidence, run at most one bounded read-only finder pass and return paths/symbols/evidence; do not start an open-ended scout loop.
|
|
32
|
+
- Do not touch protected paths or use undeclared capabilities.
|
|
33
|
+
- If verification cannot run or the task gate fails, report `blocked` or `partial`, never done.
|
|
34
|
+
- Do not paste full test/build/tool logs. Summarize the signal and quote only relevant failures.
|
|
35
|
+
- Call `piagent_usage_snapshot` only when the user asks about usage; exact totals remain available through `/usage`.
|
|
56
36
|
|
|
57
37
|
Do not ask the user to paste this mandatory flow. The platform prompt already contains it. If the user pasted the full flow, treat it as boilerplate and extract only the task request.
|
|
58
38
|
|
|
@@ -61,7 +41,7 @@ Output format:
|
|
|
61
41
|
- Changed files.
|
|
62
42
|
- Verify command/result.
|
|
63
43
|
- Context manifest.
|
|
64
|
-
-
|
|
44
|
+
- Extra workflow tools or subagents used, if any, and why.
|
|
65
45
|
- Review lenses applied.
|
|
66
46
|
- Memory cited, if any.
|
|
67
47
|
- Context index used/updated, if any.
|
|
@@ -124,6 +124,12 @@ const NAV = [
|
|
|
124
124
|
title: "Kiến trúc repo",
|
|
125
125
|
lead: "Repo này không nhét business logic vào core. Core chỉ giữ policy, prompts, guard, skills và scripts dùng lại được."
|
|
126
126
|
},
|
|
127
|
+
{
|
|
128
|
+
slug: "context-engine",
|
|
129
|
+
nav: "Context Engine",
|
|
130
|
+
title: "Ít context hơn, giữ nguyên chất lượng model",
|
|
131
|
+
lead: "Index local, retrieval có xếp hạng, tool loading theo task, semantic compaction và telemetry minh bạch cho Agent Watch."
|
|
132
|
+
},
|
|
127
133
|
{
|
|
128
134
|
slug: "runtime",
|
|
129
135
|
nav: "Runtime tools",
|
|
@@ -0,0 +1,115 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
|
|
3
|
+
import fs from "node:fs";
|
|
4
|
+
import path from "node:path";
|
|
5
|
+
|
|
6
|
+
import {
|
|
7
|
+
buildContextEfficiencyReport,
|
|
8
|
+
buildContextIndexV2,
|
|
9
|
+
buildContextPack,
|
|
10
|
+
buildTestImpact,
|
|
11
|
+
contextIndexV2Status,
|
|
12
|
+
searchContextIndexV2
|
|
13
|
+
} from "../packages/piagent-core/extensions/context-engine.js";
|
|
14
|
+
|
|
15
|
+
function usage() {
|
|
16
|
+
return [
|
|
17
|
+
"Pi Context Engine",
|
|
18
|
+
"",
|
|
19
|
+
"Usage:",
|
|
20
|
+
" piagent-context status [--project <path>] [--json]",
|
|
21
|
+
" piagent-context rebuild [--project <path>] [--json]",
|
|
22
|
+
" piagent-context search <query> [--project <path>] [--json]",
|
|
23
|
+
" piagent-context pack <task> [--tokens <n>] [--project <path>] [--json]",
|
|
24
|
+
" piagent-context impact [file ...] [--project <path>] [--json]",
|
|
25
|
+
" piagent-context efficiency [--project <path>] [--json]",
|
|
26
|
+
"",
|
|
27
|
+
"All index and telemetry data stays under .pi/piagent-state/context-engine/."
|
|
28
|
+
].join("\n");
|
|
29
|
+
}
|
|
30
|
+
|
|
31
|
+
function parseArgs(argv) {
|
|
32
|
+
const args = [...argv];
|
|
33
|
+
if (args[0] === "--help" || args[0] === "-h") return { help: true };
|
|
34
|
+
const command = args.shift() ?? "status";
|
|
35
|
+
let project = process.cwd();
|
|
36
|
+
let json = false;
|
|
37
|
+
let tokens = 6_000;
|
|
38
|
+
const values = [];
|
|
39
|
+
for (let index = 0; index < args.length; index += 1) {
|
|
40
|
+
const value = args[index];
|
|
41
|
+
if (value === "--project" && args[index + 1]) {
|
|
42
|
+
project = path.resolve(args[++index]);
|
|
43
|
+
} else if (value === "--tokens" && args[index + 1]) {
|
|
44
|
+
tokens = Number(args[++index]);
|
|
45
|
+
} else if (value === "--json") {
|
|
46
|
+
json = true;
|
|
47
|
+
} else if (value === "--help" || value === "-h") {
|
|
48
|
+
return { help: true };
|
|
49
|
+
} else {
|
|
50
|
+
values.push(value);
|
|
51
|
+
}
|
|
52
|
+
}
|
|
53
|
+
return { command, project, json, tokens, values };
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
function projectExcludes(project) {
|
|
57
|
+
const target = path.join(project, ".pi", "piagent-profile.json");
|
|
58
|
+
try {
|
|
59
|
+
const profile = JSON.parse(fs.readFileSync(target, "utf8"));
|
|
60
|
+
return [
|
|
61
|
+
...(Array.isArray(profile.protectedPaths) ? profile.protectedPaths : []),
|
|
62
|
+
...(Array.isArray(profile.shellProtectedPaths) ? profile.shellProtectedPaths : []),
|
|
63
|
+
".pi/context-index.json",
|
|
64
|
+
".pi/piagent-state/**"
|
|
65
|
+
];
|
|
66
|
+
} catch {
|
|
67
|
+
return [".pi/context-index.json", ".pi/piagent-state/**"];
|
|
68
|
+
}
|
|
69
|
+
}
|
|
70
|
+
|
|
71
|
+
function print(value, json) {
|
|
72
|
+
if (json || typeof value === "string") {
|
|
73
|
+
process.stdout.write(`${typeof value === "string" ? value : JSON.stringify(value, null, 2)}\n`);
|
|
74
|
+
return;
|
|
75
|
+
}
|
|
76
|
+
process.stdout.write(`${JSON.stringify(value, null, 2)}\n`);
|
|
77
|
+
}
|
|
78
|
+
|
|
79
|
+
const parsed = parseArgs(process.argv.slice(2));
|
|
80
|
+
if (parsed.help) {
|
|
81
|
+
process.stdout.write(`${usage()}\n`);
|
|
82
|
+
process.exit(0);
|
|
83
|
+
}
|
|
84
|
+
|
|
85
|
+
if (!fs.existsSync(parsed.project) || !fs.statSync(parsed.project).isDirectory()) {
|
|
86
|
+
process.stderr.write(`Project directory not found: ${parsed.project}\n`);
|
|
87
|
+
process.exit(1);
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
try {
|
|
91
|
+
if (["status", "show"].includes(parsed.command)) {
|
|
92
|
+
print(await contextIndexV2Status(parsed.project), parsed.json);
|
|
93
|
+
} else if (["rebuild", "build", "refresh"].includes(parsed.command)) {
|
|
94
|
+
print(await buildContextIndexV2(parsed.project, { excludePatterns: projectExcludes(parsed.project) }), parsed.json);
|
|
95
|
+
} else if (parsed.command === "search") {
|
|
96
|
+
const query = parsed.values.join(" ").trim();
|
|
97
|
+
if (!query) throw new Error("search requires a query");
|
|
98
|
+
print(await searchContextIndexV2(parsed.project, query, { limit: 15 }), parsed.json);
|
|
99
|
+
} else if (parsed.command === "pack") {
|
|
100
|
+
const query = parsed.values.join(" ").trim();
|
|
101
|
+
if (!query) throw new Error("pack requires a task or query");
|
|
102
|
+
const pack = await buildContextPack(parsed.project, query, { budgetTokens: parsed.tokens, includeCode: true });
|
|
103
|
+
print(parsed.json ? pack : pack.text, parsed.json);
|
|
104
|
+
} else if (["impact", "tests"].includes(parsed.command)) {
|
|
105
|
+
print(await buildTestImpact(parsed.project, parsed.values), parsed.json);
|
|
106
|
+
} else if (["efficiency", "stats", "waste"].includes(parsed.command)) {
|
|
107
|
+
print(buildContextEfficiencyReport(parsed.project), parsed.json);
|
|
108
|
+
} else {
|
|
109
|
+
process.stderr.write(`${usage()}\n`);
|
|
110
|
+
process.exit(1);
|
|
111
|
+
}
|
|
112
|
+
} catch (error) {
|
|
113
|
+
process.stderr.write(`FAIL: ${error instanceof Error ? error.message : String(error)}\n`);
|
|
114
|
+
process.exit(1);
|
|
115
|
+
}
|
|
@@ -203,7 +203,20 @@ is_piagent_platform_source() {
|
|
|
203
203
|
|
|
204
204
|
local resolved_source
|
|
205
205
|
resolved_source="$(resolve_user_package_path "$source" 2>/dev/null || true)"
|
|
206
|
-
[[ "$resolved_source"
|
|
206
|
+
[[ -n "$resolved_source" ]] || return 1
|
|
207
|
+
[[ "$resolved_source" == "$PLATFORM_ROOT" ]] && return 0
|
|
208
|
+
|
|
209
|
+
node --input-type=module - "$resolved_source" <<'NODE'
|
|
210
|
+
import fs from "node:fs";
|
|
211
|
+
import path from "node:path";
|
|
212
|
+
|
|
213
|
+
try {
|
|
214
|
+
const manifest = JSON.parse(fs.readFileSync(path.join(process.argv[2], "package.json"), "utf8"));
|
|
215
|
+
process.exit(manifest.name === "@piagent/platform" ? 0 : 1);
|
|
216
|
+
} catch {
|
|
217
|
+
process.exit(1);
|
|
218
|
+
}
|
|
219
|
+
NODE
|
|
207
220
|
}
|
|
208
221
|
|
|
209
222
|
remove_existing_piagent_platform_sources() {
|
package/scripts/piagent-cli.mjs
CHANGED
|
@@ -19,7 +19,8 @@ const scriptByCommand = {
|
|
|
19
19
|
"piagent-capabilities": "scripts/capability-catalog.mjs",
|
|
20
20
|
"piagent-migrate": "scripts/migrate-project-state.mjs",
|
|
21
21
|
"piagent-import-instructions": "scripts/import-agent-instructions.mjs",
|
|
22
|
-
"piagent-auto": "scripts/pi-auto.sh"
|
|
22
|
+
"piagent-auto": "scripts/pi-auto.sh",
|
|
23
|
+
"piagent-context": "scripts/context-engine.mjs"
|
|
23
24
|
};
|
|
24
25
|
|
|
25
26
|
const packageRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
|
|
@@ -11,8 +11,16 @@ Options:
|
|
|
11
11
|
--task-file <path> File containing task prompt/spec.
|
|
12
12
|
--verify <command> Verification command used for the run.
|
|
13
13
|
--tokens <number> Total tokens reported by the agent/provider.
|
|
14
|
+
--input-tokens <number> Fresh input tokens reported by the provider.
|
|
15
|
+
--output-tokens <number>
|
|
16
|
+
Output tokens reported by the provider.
|
|
17
|
+
--cache-read-tokens <number>
|
|
18
|
+
Cached input tokens reported by the provider.
|
|
14
19
|
--cost <number> Cost reported for the run.
|
|
15
20
|
--duration <seconds> Wall-clock duration.
|
|
21
|
+
--first-correct-edit <seconds>
|
|
22
|
+
Time until the first edit retained in the final fix.
|
|
23
|
+
--rework <number> Number of discarded/reworked edit attempts.
|
|
16
24
|
--notes <text> Short notes or quality observations.
|
|
17
25
|
--agent <name> Backward-compatible alias for --surface.
|
|
18
26
|
|
|
@@ -42,6 +50,7 @@ fi
|
|
|
42
50
|
shift || true
|
|
43
51
|
|
|
44
52
|
PROJECT_PATH="$(cd "$PROJECT_PATH" && pwd)"
|
|
53
|
+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
45
54
|
MODE=""
|
|
46
55
|
SCENARIO=""
|
|
47
56
|
SURFACE=""
|
|
@@ -49,8 +58,13 @@ RESULT=""
|
|
|
49
58
|
TASK_FILE=""
|
|
50
59
|
VERIFY_CMD=""
|
|
51
60
|
TOKENS=""
|
|
61
|
+
INPUT_TOKENS=""
|
|
62
|
+
OUTPUT_TOKENS=""
|
|
63
|
+
CACHE_READ_TOKENS=""
|
|
52
64
|
COST=""
|
|
53
65
|
DURATION=""
|
|
66
|
+
FIRST_CORRECT_EDIT=""
|
|
67
|
+
REWORK=""
|
|
54
68
|
NOTES=""
|
|
55
69
|
|
|
56
70
|
# A flag left without a value otherwise consumes the next flag as its value, and
|
|
@@ -104,6 +118,21 @@ while [[ $# -gt 0 ]]; do
|
|
|
104
118
|
TOKENS="$2"
|
|
105
119
|
shift 2
|
|
106
120
|
;;
|
|
121
|
+
--input-tokens)
|
|
122
|
+
require_value "$1" "${2:-}"
|
|
123
|
+
INPUT_TOKENS="$2"
|
|
124
|
+
shift 2
|
|
125
|
+
;;
|
|
126
|
+
--output-tokens)
|
|
127
|
+
require_value "$1" "${2:-}"
|
|
128
|
+
OUTPUT_TOKENS="$2"
|
|
129
|
+
shift 2
|
|
130
|
+
;;
|
|
131
|
+
--cache-read-tokens)
|
|
132
|
+
require_value "$1" "${2:-}"
|
|
133
|
+
CACHE_READ_TOKENS="$2"
|
|
134
|
+
shift 2
|
|
135
|
+
;;
|
|
107
136
|
--cost)
|
|
108
137
|
require_value "$1" "${2:-}"
|
|
109
138
|
COST="$2"
|
|
@@ -114,6 +143,16 @@ while [[ $# -gt 0 ]]; do
|
|
|
114
143
|
DURATION="$2"
|
|
115
144
|
shift 2
|
|
116
145
|
;;
|
|
146
|
+
--first-correct-edit)
|
|
147
|
+
require_value "$1" "${2:-}"
|
|
148
|
+
FIRST_CORRECT_EDIT="$2"
|
|
149
|
+
shift 2
|
|
150
|
+
;;
|
|
151
|
+
--rework)
|
|
152
|
+
require_value "$1" "${2:-}"
|
|
153
|
+
REWORK="$2"
|
|
154
|
+
shift 2
|
|
155
|
+
;;
|
|
117
156
|
--notes)
|
|
118
157
|
require_value "$1" "${2:-}"
|
|
119
158
|
NOTES="$2"
|
|
@@ -173,6 +212,13 @@ case "$RESULT" in
|
|
|
173
212
|
;;
|
|
174
213
|
esac
|
|
175
214
|
|
|
215
|
+
# Keep benchmark evidence and context-efficiency evidence in the same record.
|
|
216
|
+
# Failure here does not invalidate a task result; it leaves the optional
|
|
217
|
+
# contextEfficiency field null for older hosts or telemetry-disabled runs.
|
|
218
|
+
if [[ -f "$SCRIPT_DIR/context-engine.mjs" ]]; then
|
|
219
|
+
node "$SCRIPT_DIR/context-engine.mjs" efficiency --project "$PROJECT_PATH" --json >/dev/null 2>&1 || true
|
|
220
|
+
fi
|
|
221
|
+
|
|
176
222
|
export PI_BENCHMARK_PROJECT_PATH="$PROJECT_PATH"
|
|
177
223
|
export PI_BENCHMARK_SCENARIO="$SCENARIO"
|
|
178
224
|
export PI_BENCHMARK_SURFACE="$SURFACE"
|
|
@@ -180,8 +226,13 @@ export PI_BENCHMARK_RESULT="$RESULT"
|
|
|
180
226
|
export PI_BENCHMARK_TASK_FILE="$TASK_FILE"
|
|
181
227
|
export PI_BENCHMARK_VERIFY="$VERIFY_CMD"
|
|
182
228
|
export PI_BENCHMARK_TOKENS="$TOKENS"
|
|
229
|
+
export PI_BENCHMARK_INPUT_TOKENS="$INPUT_TOKENS"
|
|
230
|
+
export PI_BENCHMARK_OUTPUT_TOKENS="$OUTPUT_TOKENS"
|
|
231
|
+
export PI_BENCHMARK_CACHE_READ_TOKENS="$CACHE_READ_TOKENS"
|
|
183
232
|
export PI_BENCHMARK_COST="$COST"
|
|
184
233
|
export PI_BENCHMARK_DURATION="$DURATION"
|
|
234
|
+
export PI_BENCHMARK_FIRST_CORRECT_EDIT="$FIRST_CORRECT_EDIT"
|
|
235
|
+
export PI_BENCHMARK_REWORK="$REWORK"
|
|
185
236
|
export PI_BENCHMARK_NOTES="$NOTES"
|
|
186
237
|
|
|
187
238
|
node --input-type=module <<'NODE'
|
|
@@ -195,8 +246,15 @@ const numberOrNull = (value) => {
|
|
|
195
246
|
const parsed = Number(value);
|
|
196
247
|
return Number.isFinite(parsed) ? parsed : null;
|
|
197
248
|
};
|
|
249
|
+
const contextReportPath = path.join(root, ".pi", "piagent-state", "context-engine", "efficiency-report.json");
|
|
250
|
+
let contextReport = null;
|
|
251
|
+
try {
|
|
252
|
+
contextReport = JSON.parse(fs.readFileSync(contextReportPath, "utf8"));
|
|
253
|
+
} catch {
|
|
254
|
+
// Context telemetry is optional and may be disabled for a baseline run.
|
|
255
|
+
}
|
|
198
256
|
const payload = {
|
|
199
|
-
schemaVersion:
|
|
257
|
+
schemaVersion: 2,
|
|
200
258
|
recordedAt: new Date().toISOString(),
|
|
201
259
|
scenario: process.env.PI_BENCHMARK_SCENARIO,
|
|
202
260
|
surface: process.env.PI_BENCHMARK_SURFACE,
|
|
@@ -204,8 +262,20 @@ const payload = {
|
|
|
204
262
|
taskFile: process.env.PI_BENCHMARK_TASK_FILE || null,
|
|
205
263
|
verifyCommand: process.env.PI_BENCHMARK_VERIFY || null,
|
|
206
264
|
tokens: numberOrNull(process.env.PI_BENCHMARK_TOKENS),
|
|
265
|
+
inputTokens: numberOrNull(process.env.PI_BENCHMARK_INPUT_TOKENS),
|
|
266
|
+
outputTokens: numberOrNull(process.env.PI_BENCHMARK_OUTPUT_TOKENS),
|
|
267
|
+
cacheReadTokens: numberOrNull(process.env.PI_BENCHMARK_CACHE_READ_TOKENS),
|
|
207
268
|
cost: numberOrNull(process.env.PI_BENCHMARK_COST),
|
|
208
269
|
durationSeconds: numberOrNull(process.env.PI_BENCHMARK_DURATION),
|
|
270
|
+
firstCorrectEditSeconds: numberOrNull(process.env.PI_BENCHMARK_FIRST_CORRECT_EDIT),
|
|
271
|
+
reworkCount: numberOrNull(process.env.PI_BENCHMARK_REWORK),
|
|
272
|
+
contextEfficiency: contextReport
|
|
273
|
+
? {
|
|
274
|
+
generatedAt: contextReport.generatedAt ?? null,
|
|
275
|
+
sample: contextReport.sample ?? null,
|
|
276
|
+
metrics: contextReport.metrics ?? null
|
|
277
|
+
}
|
|
278
|
+
: null,
|
|
209
279
|
notes: process.env.PI_BENCHMARK_NOTES || null
|
|
210
280
|
};
|
|
211
281
|
fs.appendFileSync(target, `${JSON.stringify(payload)}\n`);
|
package/scripts/verify-local.sh
CHANGED
|
@@ -51,6 +51,7 @@ required_files=(
|
|
|
51
51
|
"$ROOT/.pi/project-context.md"
|
|
52
52
|
"$ROOT/packages/piagent-core/package.json"
|
|
53
53
|
"$ROOT/packages/piagent-core/extensions/piagent-guard.ts"
|
|
54
|
+
"$ROOT/packages/piagent-core/extensions/context-engine.js"
|
|
54
55
|
"$ROOT/packages/piagent-core/extensions/policy-core.js"
|
|
55
56
|
"$ROOT/packages/piagent-core/extensions/redaction-core.js"
|
|
56
57
|
"$ROOT/packages/piagent-core/extensions/runtime-evidence.js"
|
|
@@ -115,6 +116,7 @@ required_files=(
|
|
|
115
116
|
"$ROOT/docs/publishing-for-teams.md"
|
|
116
117
|
"$ROOT/docs/herdr-workflow.md"
|
|
117
118
|
"$ROOT/docs/runtime-harness-standard.md"
|
|
119
|
+
"$ROOT/docs/context-engine.md"
|
|
118
120
|
"$ROOT/docs/task-implementation-contract.md"
|
|
119
121
|
"$ROOT/docs/runtime-quality-baseline.md"
|
|
120
122
|
"$ROOT/docs/quality-benchmark.md"
|
|
@@ -128,9 +130,11 @@ required_files=(
|
|
|
128
130
|
"$ROOT/docs-site/assets/docs.css"
|
|
129
131
|
"$ROOT/docs-site/assets/docs.js"
|
|
130
132
|
"$ROOT/docs-site/content/index.html"
|
|
133
|
+
"$ROOT/docs-site/content/context-engine.html"
|
|
131
134
|
"$ROOT/docs-site/content/mcp.html"
|
|
132
135
|
"$ROOT/docs-site/vercel.json"
|
|
133
136
|
"$ROOT/scripts/build-docs-site.mjs"
|
|
137
|
+
"$ROOT/scripts/context-engine.mjs"
|
|
134
138
|
"$ROOT/scripts/preview-docs-site.mjs"
|
|
135
139
|
"$ROOT/schemas/project-profile.schema.json"
|
|
136
140
|
"$ROOT/schemas/task-contract.schema.json"
|
|
@@ -171,6 +175,8 @@ required_files=(
|
|
|
171
175
|
"$ROOT/scripts/import-agent-instructions.mjs"
|
|
172
176
|
"$ROOT/tests/capability-core.test.mjs"
|
|
173
177
|
"$ROOT/tests/piagent-guard-integration.test.mjs"
|
|
178
|
+
"$ROOT/tests/context-engine.test.mjs"
|
|
179
|
+
"$ROOT/tests/workflow-prompts.test.mjs"
|
|
174
180
|
"$ROOT/tests/install-global.test.mjs"
|
|
175
181
|
"$ROOT/tests/package-distribution.test.mjs"
|
|
176
182
|
"$ROOT/tests/release-identity.test.mjs"
|
|
@@ -538,6 +544,7 @@ if grep -R -E -i \
|
|
|
538
544
|
fi
|
|
539
545
|
|
|
540
546
|
node --check "$ROOT/packages/piagent-core/extensions/piagent-guard.ts" >/dev/null
|
|
547
|
+
node --check "$ROOT/packages/piagent-core/extensions/context-engine.js" >/dev/null
|
|
541
548
|
node --check "$ROOT/packages/piagent-core/extensions/policy-core.js" >/dev/null
|
|
542
549
|
node --check "$ROOT/packages/piagent-core/extensions/redaction-core.js" >/dev/null
|
|
543
550
|
node --check "$ROOT/packages/piagent-core/extensions/runtime-evidence.js" >/dev/null
|
|
@@ -546,6 +553,7 @@ node --check "$ROOT/packages/piagent-core/capabilities/capability-core.js" >/dev
|
|
|
546
553
|
node --check "$ROOT/packages/piagent-core/capabilities/capability-sources.js" >/dev/null
|
|
547
554
|
node --check "$ROOT/scripts/capability-catalog.mjs" >/dev/null
|
|
548
555
|
node --check "$ROOT/scripts/piagent-cli.mjs" >/dev/null
|
|
556
|
+
node --check "$ROOT/scripts/context-engine.mjs" >/dev/null
|
|
549
557
|
node --check "$ROOT/scripts/pi-usage-history.mjs" >/dev/null
|
|
550
558
|
node --check "$ROOT/scripts/migrate-project-state.mjs" >/dev/null
|
|
551
559
|
node --check "$ROOT/scripts/import-agent-instructions.mjs" >/dev/null
|