@cassiomc1/forgeloop 1.6.4 → 1.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENT_COMPATIBILITY.md +11 -0
- package/DOCS_INDEX.md +1 -0
- package/GUIDE_ROUTER.md +26 -1
- package/LOOP_ENGINEERING.md +48 -0
- package/ORCHESTRATOR_INTEGRATION.md +38 -0
- package/PROTOCOL_INTEGRATION.md +62 -0
- package/README.md +25 -0
- package/benchmarks/execution-profiles/README.md +44 -0
- package/benchmarks/execution-profiles/api-feature.json +18 -0
- package/benchmarks/execution-profiles/authentication-change.json +18 -0
- package/benchmarks/execution-profiles/documentation-correction.json +18 -0
- package/benchmarks/execution-profiles/infrastructure-release.json +18 -0
- package/benchmarks/execution-profiles/novatask-saas-landing-page.json +36 -0
- package/benchmarks/execution-profiles/small-bug-fix.json +18 -0
- package/benchmarks/execution-profiles/static-landing-page.json +18 -0
- package/completions/_forgeloop +6 -4
- package/completions/forgeloop.bash +8 -4
- package/completions/forgeloop.fish +24 -1
- package/docs/AGENT_PROTOCOL_SUMMARY.md +45 -1
- package/docs/ARTIFACT_REFERENCE.md +53 -7
- package/docs/CLI_REFERENCE.md +83 -1
- package/docs/EXECUTION_PROFILE_BENCHMARKS.md +208 -0
- package/docs/GETTING_STARTED.md +28 -0
- package/docs/MCP.md +6 -0
- package/docs/RELEASE_CHECKLIST.md +4 -0
- package/docs/TROUBLESHOOTING.md +6 -0
- package/docs/UNIVERSAL_INTEGRATION.md +58 -0
- package/package.json +14 -2
- package/schemas/config.schema.json +1 -0
- package/schemas/execution-profile-benchmark-aggregate.schema.json +43 -0
- package/schemas/execution-profile-benchmark-run.schema.json +106 -0
- package/schemas/execution-profile-benchmark-scenario.schema.json +66 -0
- package/schemas/routing-result.schema.json +12 -0
- package/schemas/usage.schema.json +30 -0
- package/scripts/check-efficiency-regression.mjs +99 -0
- package/scripts/generate-agent-protocol-summary.mjs +35 -0
- package/scripts/lib/execution-profile-benchmark-io.mjs +67 -0
- package/scripts/run-execution-profile-benchmarks.mjs +265 -0
- package/scripts/summarize-execution-profile-benchmarks.mjs +84 -0
- package/scripts/validate-execution-profile-benchmarks.mjs +120 -0
- package/src/cli.js +16 -4
- package/src/commands/efficiency.js +12 -0
- package/src/commands/eval.js +8 -2
- package/src/commands/metrics.js +2 -2
- package/src/commands/next.js +26 -2
- package/src/commands/route.js +20 -2
- package/src/commands/task-show.js +31 -2
- package/src/commands/usage-record.js +61 -0
- package/src/core/artifact-registry.js +12 -0
- package/src/core/cli-command-definitions.js +27 -0
- package/src/core/command-executors.js +29 -5
- package/src/core/command-input.js +34 -0
- package/src/core/config.js +9 -0
- package/src/core/efficiency.js +197 -0
- package/src/core/error-codes.js +18 -0
- package/src/core/execution-profile-benchmarks.js +674 -0
- package/src/core/execution-profile-context.js +177 -0
- package/src/core/execution-profile.js +248 -0
- package/src/core/integration-invocation-policy.js +43 -0
- package/src/core/integration-resources.js +22 -1
- package/src/core/protocol-info.js +42 -0
- package/src/core/resumability.js +27 -1
- package/src/core/router.js +23 -1
- package/src/core/runtime-context.js +11 -0
- package/src/core/schema-validation.js +4 -0
- package/src/core/task-paths.js +2 -0
- package/src/core/templates.js +4 -0
- package/src/core/trace.js +1 -0
- package/src/core/trajectory-evaluation.js +2 -2
- package/src/core/trajectory-metrics.js +18 -2
- package/src/core/usage.js +137 -0
- package/src/integration.d.ts +84 -0
- package/src/integration.js +14 -0
package/docs/GETTING_STARTED.md
CHANGED
|
@@ -26,6 +26,34 @@ Core mental model:
|
|
|
26
26
|
- `checks = verification truth`
|
|
27
27
|
- `receipt = completion/publication record`
|
|
28
28
|
|
|
29
|
+
### Adaptive execution profiles
|
|
30
|
+
|
|
31
|
+
ForgeLoop uses `executionProfile` to choose the appropriate process and context
|
|
32
|
+
depth without changing its assurance guarantees. It is separate from
|
|
33
|
+
`complianceMode`, which controls policy enforcement.
|
|
34
|
+
|
|
35
|
+
```text
|
|
36
|
+
auto → light | balanced | full
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
Small, local, reversible documentation and UI work normally resolves to
|
|
40
|
+
`light`; behavior changes and ordinary application work resolve to
|
|
41
|
+
`balanced`; authentication, publication, infrastructure, secrets, personal
|
|
42
|
+
data, critical paths, and irreversible work resolve to `full`. A CLI or project
|
|
43
|
+
request may raise the profile, but never lower the deterministic safety floor.
|
|
44
|
+
|
|
45
|
+
For compact host context, use:
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
forgeloop next --task task-contact-form-001 --compact --json
|
|
49
|
+
forgeloop task-show --task task-contact-form-001 --compact --json
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
These commands do not bypass contracts, gates, verification, provenance,
|
|
53
|
+
lifecycle phases, or validator-backed completion. Usage telemetry is optional,
|
|
54
|
+
never estimated, and never verification evidence; `efficiency --task` compares
|
|
55
|
+
only against a metadata-compatible local baseline.
|
|
56
|
+
|
|
29
57
|
---
|
|
30
58
|
|
|
31
59
|
## 2. Prerequisites
|
package/docs/MCP.md
CHANGED
|
@@ -50,6 +50,7 @@ Capability flags (process-scoped, immutable after launch):
|
|
|
50
50
|
- `forgeloop://task/{taskId}/action/{actionId}`
|
|
51
51
|
- `forgeloop://task/{taskId}/approvals`
|
|
52
52
|
- `forgeloop://task/{taskId}/metrics`
|
|
53
|
+
- `forgeloop://task/{taskId}/context` — bounded profile-aware host context
|
|
53
54
|
- `forgeloop://task/{taskId}/evaluations`
|
|
54
55
|
- `forgeloop://project/capability-policy`
|
|
55
56
|
|
|
@@ -83,6 +84,11 @@ content.
|
|
|
83
84
|
Raw recovery artifacts, transaction journals, lock files, and unbounded event
|
|
84
85
|
ledgers are intentionally not exposed.
|
|
85
86
|
|
|
87
|
+
The context resource is read-only. It lets an MCP host adapt presentation depth
|
|
88
|
+
from the canonical resolved execution profile while preserving lifecycle
|
|
89
|
+
phases, required gates, verification truth, authority, provenance, and
|
|
90
|
+
validator-backed completion.
|
|
91
|
+
|
|
86
92
|
Optional workspace binding, handoff, responsibility, verification-scope,
|
|
87
93
|
RevisionProvider, SigningProvider, and attestation operations remain governed
|
|
88
94
|
by the same canonical command executors when exposed by a compatible host.
|
|
@@ -16,6 +16,10 @@ preparation and verification checklist; it does not authorize publication.
|
|
|
16
16
|
- [ ] `npm run dependency:policy` passes without adding runtime dependencies.
|
|
17
17
|
- [ ] `npm run lint` passes.
|
|
18
18
|
- [ ] `npm test` passes.
|
|
19
|
+
- [ ] `npm run benchmark:profiles:check` passes; absent provider/host history
|
|
20
|
+
is reported as `NOT_MEASURED`, never as zero or a passing efficiency claim.
|
|
21
|
+
- [ ] `npm run benchmark:profiles:regression` reports the observed status;
|
|
22
|
+
`EFFICIENCY_REGRESSION` remains a non-blocking warning.
|
|
19
23
|
- [ ] `npm run coverage` passes the configured global and critical-module gates.
|
|
20
24
|
- [ ] `npm run docs:check`, `npm run docs:generated:check`,
|
|
21
25
|
`npm run docs:conformance`, and `npm run docs:examples:check` pass.
|
package/docs/TROUBLESHOOTING.md
CHANGED
|
@@ -986,6 +986,7 @@ forgeloop next --task <id> --json
|
|
|
986
986
|
| `E_DIAGNOSTIC_CASE_CYCLE_MISMATCH` | Diagnostic case verification cycle does not match the active work state verification cycle. | Record the diagnostic case for the current active verification cycle. |
|
|
987
987
|
| `E_DIAGNOSTIC_CASE_EVIDENCE_INVALID` | A diagnostic case evidence reference does not match any check from the active verification cycle. | Reference check IDs recorded during the active verification cycle. |
|
|
988
988
|
| `E_DIAGNOSTIC_CASE_INVALID` | Structured diagnostic case details or parameters are malformed. | Provide valid observations, contributors, hypotheses with settlement criteria, and nextSafeAction. |
|
|
989
|
+
| `E_EFFICIENCY_BASELINE_INVALID` | A ForgeLoop boundary, artifact, provider, or attestation validation condition was not satisfied. | Inspect the structured command result, correct the named boundary or artifact, then retry the canonical command. |
|
|
989
990
|
| `E_EVIDENCE_COVERAGE_INVALID` | A ForgeLoop protocol validation or lifecycle condition was not satisfied. | Inspect the structured command result, correct the named artifact or prerequisite, then run forgeloop next --json. |
|
|
990
991
|
| `E_EVIDENCE_COVERAGE_PARTIAL` | A ForgeLoop protocol validation or lifecycle condition was not satisfied. | Inspect the structured command result, correct the named artifact or prerequisite, then run forgeloop next --json. |
|
|
991
992
|
| `E_EVIDENCE_INVALID` | A ForgeLoop protocol validation or lifecycle condition was not satisfied. | Inspect the structured command result, correct the named artifact or prerequisite, then run forgeloop next --json. |
|
|
@@ -993,6 +994,9 @@ forgeloop next --task <id> --json
|
|
|
993
994
|
| `E_EVIDENCE_PARTIAL` | A ForgeLoop protocol validation or lifecycle condition was not satisfied. | Inspect the structured command result, correct the named artifact or prerequisite, then run forgeloop next --json. |
|
|
994
995
|
| `E_EVIDENCE_REQUIRED` | A ForgeLoop protocol validation or lifecycle condition was not satisfied. | Inspect the structured command result, correct the named artifact or prerequisite, then run forgeloop next --json. |
|
|
995
996
|
| `E_EVIDENCE_STALE` | A ForgeLoop protocol validation or lifecycle condition was not satisfied. | Inspect the structured command result, correct the named artifact or prerequisite, then run forgeloop next --json. |
|
|
997
|
+
| `E_EXECUTION_PROFILE_INCONSISTENT` | A ForgeLoop boundary, artifact, provider, or attestation validation condition was not satisfied. | Inspect the structured command result, correct the named boundary or artifact, then retry the canonical command. |
|
|
998
|
+
| `E_EXECUTION_PROFILE_INVALID` | A ForgeLoop boundary, artifact, provider, or attestation validation condition was not satisfied. | Inspect the structured command result, correct the named boundary or artifact, then retry the canonical command. |
|
|
999
|
+
| `E_EXECUTION_PROFILE_SAFETY_FLOOR_INVALID` | A ForgeLoop boundary, artifact, provider, or attestation validation condition was not satisfied. | Inspect the structured command result, correct the named boundary or artifact, then retry the canonical command. |
|
|
996
1000
|
| `E_EXECUTION_REF_INVALID` | Referenced execution ID does not exist. | Re-run check via forgeloop run-check. |
|
|
997
1001
|
| `E_FAILURE_SIGNATURE_INVALID` | A ForgeLoop protocol validation or lifecycle condition was not satisfied. | Inspect the structured command result, correct the named artifact or prerequisite, then run forgeloop next --json. |
|
|
998
1002
|
| `E_FUTURE_LIFECYCLE_EVIDENCE` | A ForgeLoop protocol validation or lifecycle condition was not satisfied. | Inspect the structured command result, correct the named artifact or prerequisite, then run forgeloop next --json. |
|
|
@@ -1123,6 +1127,8 @@ forgeloop next --task <id> --json
|
|
|
1123
1127
|
| `E_TRACE_SNAPSHOT_INCONSISTENT` | Task artifacts changed while the execution trace was being read. | Rerun the read-only projection to obtain a consistent view. |
|
|
1124
1128
|
| `E_TRAJECTORY_REFERENCE_REQUIRED` | Comparative efficiency requires a reference scenario with positive comparableSteps. | Provide --scenario with reference.comparableSteps, or omit efficiency from the result. |
|
|
1125
1129
|
| `E_TRAJECTORY_SCENARIO_INVALID` | Trajectory scenario file is missing required fields or schema-invalid. | Correct the scenario JSON against schemas/trajectory-scenario.schema.json. |
|
|
1130
|
+
| `E_USAGE_INVALID` | A ForgeLoop boundary, artifact, provider, or attestation validation condition was not satisfied. | Inspect the structured command result, correct the named boundary or artifact, then retry the canonical command. |
|
|
1131
|
+
| `E_USAGE_SOURCE_INVALID` | A ForgeLoop boundary, artifact, provider, or attestation validation condition was not satisfied. | Inspect the structured command result, correct the named boundary or artifact, then retry the canonical command. |
|
|
1126
1132
|
| `E_VERIFICATION_EXECUTION_INVALID` | The trusted verification execution adapter returned incomplete or invalid execution metadata. | Repair the adapter contract and rerun verification; do not promote incomplete execution evidence. |
|
|
1127
1133
|
| `E_VERIFICATION_ISOLATION_UNAVAILABLE` | Verification cannot run because the required disposable or system isolation boundary is unavailable. | Use a trusted ForgeLoop execution adapter with the required isolation mode; never run the check in the live project. |
|
|
1128
1134
|
| `E_VERIFICATION_SCOPE_INVALID` | A ForgeLoop boundary, artifact, provider, or attestation validation condition was not satisfied. | Inspect the structured command result, correct the named boundary or artifact, then retry the canonical command. |
|
|
@@ -35,6 +35,64 @@ import {
|
|
|
35
35
|
FORCE_DESTRUCTIVE) describe what an invocation would do; launch policy
|
|
36
36
|
decides what is allowed.
|
|
37
37
|
|
|
38
|
+
### Profile-aware context
|
|
39
|
+
|
|
40
|
+
The resolved execution profile is available from the persisted route and from
|
|
41
|
+
compact `next`/`task-show` projections. `complianceMode` remains the policy
|
|
42
|
+
enforcement dimension; `executionProfile` is the independent process/context
|
|
43
|
+
depth dimension.
|
|
44
|
+
|
|
45
|
+
The API's read-only `task/context` resource provides the canonical host
|
|
46
|
+
projection for this decision. It includes the objective, deliverables,
|
|
47
|
+
constraints, selected guide IDs, current phase, next action, verification
|
|
48
|
+
requirements, resolved profile, and bounded optional-context policy. The MCP
|
|
49
|
+
adapter exposes the same projection at
|
|
50
|
+
`forgeloop://task/{taskId}/context`.
|
|
51
|
+
|
|
52
|
+
For `light`, an adapter should send only the current objective, deliverables,
|
|
53
|
+
hard constraints, resolved profile, selected guide IDs or relevant guide
|
|
54
|
+
sections, current phase, exact next action, and verification requirements. It
|
|
55
|
+
should reuse unchanged state locally instead of repeatedly sending full
|
|
56
|
+
ledgers, schemas, receipts, or protocol documents. This is presentation
|
|
57
|
+
optimization only: required gates, verification truth, authority, provenance,
|
|
58
|
+
and validated completion remain unchanged. A lifecycle fast path is not part
|
|
59
|
+
of protocol v1.
|
|
60
|
+
|
|
61
|
+
Optional reflection, trajectory evaluation, handoff, responsibility,
|
|
62
|
+
attestation, benchmark, and continuity artifacts are lazy and should be
|
|
63
|
+
created only when policy, an explicit request, or recovery requires them.
|
|
64
|
+
|
|
65
|
+
The profile never authorizes a lifecycle-phase or required-gate skip. A phase
|
|
66
|
+
may be absent only when the canonical protocol marks it not applicable;
|
|
67
|
+
presentation depth cannot change evidence, verification truth, authority,
|
|
68
|
+
provenance, safety-floor, or validator-backed completion requirements.
|
|
69
|
+
|
|
70
|
+
### Usage and efficiency boundary
|
|
71
|
+
|
|
72
|
+
Create a context with an optional trusted provider:
|
|
73
|
+
|
|
74
|
+
```js
|
|
75
|
+
const context = createForgeLoopContext({
|
|
76
|
+
usageProvider: {
|
|
77
|
+
async getTaskUsage({ projectPath, taskId }) {
|
|
78
|
+
return hostUsageStore.lookup({ projectPath, taskId });
|
|
79
|
+
},
|
|
80
|
+
},
|
|
81
|
+
});
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
The provider returns `PROVIDER_REPORTED` or `HOST_REPORTED` snapshots. The CLI
|
|
85
|
+
fallback is explicitly `ACTOR_REPORTED`; missing fields remain `null` and no
|
|
86
|
+
token or cost estimate is produced. Usage never satisfies a check or
|
|
87
|
+
completion requirement. Use `efficiency --task --baseline <project-local-json>`
|
|
88
|
+
only for metadata-compatible comparisons; otherwise the result is
|
|
89
|
+
`NOT_COMPARABLE`.
|
|
90
|
+
|
|
91
|
+
For measured execution-profile comparisons, use the reproducible benchmark
|
|
92
|
+
commands in [`EXECUTION_PROFILE_BENCHMARKS.md`](./EXECUTION_PROFILE_BENCHMARKS.md).
|
|
93
|
+
The benchmark source policy accepts provider or host observations only and
|
|
94
|
+
keeps unavailable or non-comparable measurements out of efficiency claims.
|
|
95
|
+
|
|
38
96
|
## Consumers
|
|
39
97
|
|
|
40
98
|
| Surface | Entry |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@cassiomc1/forgeloop",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.7.0",
|
|
4
4
|
"description": "Portable, verifiable engineering protocol for AI coding environments and developer workflows",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
|
@@ -22,12 +22,14 @@
|
|
|
22
22
|
"src",
|
|
23
23
|
"ENG",
|
|
24
24
|
"schemas",
|
|
25
|
+
"benchmarks/execution-profiles",
|
|
25
26
|
".forgeloop/forgeloop.gitignore",
|
|
26
27
|
"AGENTS.md",
|
|
27
28
|
"CLAUDE.md",
|
|
28
29
|
"GUIDE_ROUTER.md",
|
|
29
30
|
"LOOP_ENGINEERING.md",
|
|
30
31
|
"PROTOCOL_INTEGRATION.md",
|
|
32
|
+
"AGENT_COMPATIBILITY.md",
|
|
31
33
|
"LOOP_SYSTEM_DESIGN.md",
|
|
32
34
|
"QUALITY_SCORECARD.md",
|
|
33
35
|
"TERMINOLOGY.md",
|
|
@@ -64,8 +66,14 @@
|
|
|
64
66
|
"docs/SIGNING_PROVIDERS.md",
|
|
65
67
|
"docs/PLATFORM_ADAPTERS.md",
|
|
66
68
|
"docs/AGENT_PROTOCOL_SUMMARY.md",
|
|
69
|
+
"docs/EXECUTION_PROFILE_BENCHMARKS.md",
|
|
67
70
|
"scripts/generate-shell-completions.mjs",
|
|
68
71
|
"scripts/generate-agent-protocol-summary.mjs",
|
|
72
|
+
"scripts/run-execution-profile-benchmarks.mjs",
|
|
73
|
+
"scripts/summarize-execution-profile-benchmarks.mjs",
|
|
74
|
+
"scripts/validate-execution-profile-benchmarks.mjs",
|
|
75
|
+
"scripts/check-efficiency-regression.mjs",
|
|
76
|
+
"scripts/lib/execution-profile-benchmark-io.mjs",
|
|
69
77
|
"scripts/benchmark-cli-startup.mjs",
|
|
70
78
|
"scripts/check-critical-coverage.mjs",
|
|
71
79
|
"scripts/check-changelog-freshness.mjs",
|
|
@@ -103,7 +111,11 @@
|
|
|
103
111
|
"mcp:setup": "node scripts/mcp-setup.mjs",
|
|
104
112
|
"mcp:pack:check": "node scripts/mcp-package-smoke.mjs",
|
|
105
113
|
"poc:evidence:verify": "node scripts/verify_poc_evidence.mjs",
|
|
106
|
-
"poc:evidence:test": "node --test poc/test/poc-evidence-publication.test.js"
|
|
114
|
+
"poc:evidence:test": "node --test poc/test/poc-evidence-publication.test.js",
|
|
115
|
+
"benchmark:profiles": "node scripts/run-execution-profile-benchmarks.mjs",
|
|
116
|
+
"benchmark:profiles:summary": "node scripts/summarize-execution-profile-benchmarks.mjs",
|
|
117
|
+
"benchmark:profiles:check": "node scripts/validate-execution-profile-benchmarks.mjs",
|
|
118
|
+
"benchmark:profiles:regression": "node scripts/check-efficiency-regression.mjs"
|
|
107
119
|
},
|
|
108
120
|
"devDependencies": {
|
|
109
121
|
"c8": "^12.0.0",
|
|
@@ -8,6 +8,7 @@
|
|
|
8
8
|
"schemaVersion": { "const": 1 },
|
|
9
9
|
"protocolVersion": { "const": 1 },
|
|
10
10
|
"complianceMode": { "enum": ["advisory", "standard", "strict"] },
|
|
11
|
+
"executionProfile": { "enum": ["auto", "light", "balanced", "full"] },
|
|
11
12
|
"policy": { "type": "string", "minLength": 1 },
|
|
12
13
|
"requiredGates": { "type": "array", "items": { "type": "string", "minLength": 1 } },
|
|
13
14
|
"requiredEvidence": { "type": "array", "items": { "type": "string", "minLength": 1 } },
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
3
|
+
"$id": "forgeloop://schemas/execution-profile-benchmark-aggregate.schema.json",
|
|
4
|
+
"title": "ForgeLoop execution-profile benchmark aggregate",
|
|
5
|
+
"type": "object",
|
|
6
|
+
"required": ["schemaVersion", "benchmarkVersion", "runSetId", "scenarioId", "expectedProfile", "modeAggregates", "comparisons", "lightObjectives", "sourcePolicy", "claimsAllowed", "generatedFromRunCount"],
|
|
7
|
+
"properties": {
|
|
8
|
+
"schemaVersion": { "const": 1 },
|
|
9
|
+
"benchmarkVersion": { "const": "1" },
|
|
10
|
+
"runSetId": { "type": "string", "minLength": 1 },
|
|
11
|
+
"scenarioId": { "type": "string", "pattern": "^[a-z0-9][a-z0-9-]{0,127}$" },
|
|
12
|
+
"expectedProfile": { "enum": ["light", "balanced", "full"] },
|
|
13
|
+
"modeAggregates": {
|
|
14
|
+
"type": "object",
|
|
15
|
+
"required": ["direct", "forgeloopBalanced", "forgeloopAdaptive"],
|
|
16
|
+
"additionalProperties": { "type": "object" }
|
|
17
|
+
},
|
|
18
|
+
"comparisons": {
|
|
19
|
+
"type": "object",
|
|
20
|
+
"required": ["direct", "forgeloopBalanced", "forgeloopAdaptive"],
|
|
21
|
+
"additionalProperties": true
|
|
22
|
+
},
|
|
23
|
+
"lightObjectives": { "type": ["object", "null"] },
|
|
24
|
+
"contextInflation": {
|
|
25
|
+
"type": "object",
|
|
26
|
+
"required": ["status", "comparablePairs", "inflatedPairs", "lightContextTokens", "balancedContextTokens", "blocking", "reason"],
|
|
27
|
+
"properties": {
|
|
28
|
+
"status": { "enum": ["CONTEXT_INFLATION", "NOT_DETECTED", "NOT_COMPARABLE"] },
|
|
29
|
+
"comparablePairs": { "type": "integer", "minimum": 0 },
|
|
30
|
+
"inflatedPairs": { "type": "integer", "minimum": 0 },
|
|
31
|
+
"lightContextTokens": { "type": "object" },
|
|
32
|
+
"balancedContextTokens": { "type": "object" },
|
|
33
|
+
"blocking": { "const": false },
|
|
34
|
+
"reason": { "type": "string", "minLength": 1 }
|
|
35
|
+
},
|
|
36
|
+
"additionalProperties": false
|
|
37
|
+
},
|
|
38
|
+
"sourcePolicy": { "const": "PROVIDER_REPORTED_OR_HOST_REPORTED_ONLY" },
|
|
39
|
+
"claimsAllowed": { "type": "boolean" },
|
|
40
|
+
"generatedFromRunCount": { "type": "integer", "minimum": 1 }
|
|
41
|
+
},
|
|
42
|
+
"additionalProperties": false
|
|
43
|
+
}
|
|
@@ -0,0 +1,106 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
3
|
+
"$id": "forgeloop://schemas/execution-profile-benchmark-run.schema.json",
|
|
4
|
+
"title": "ForgeLoop execution-profile benchmark run",
|
|
5
|
+
"type": "object",
|
|
6
|
+
"required": ["schemaVersion", "benchmarkVersion", "runSetId", "runId", "scenarioId", "mode", "runIndex", "recordedAt", "usage", "wallClockMs", "verification", "verificationCycles", "comparableSteps", "metadata"],
|
|
7
|
+
"properties": {
|
|
8
|
+
"schemaVersion": { "const": 1 },
|
|
9
|
+
"benchmarkVersion": { "const": "1" },
|
|
10
|
+
"runSetId": { "type": "string", "pattern": "^[A-Za-z0-9][A-Za-z0-9_-]{0,63}$" },
|
|
11
|
+
"runId": { "type": "string", "pattern": "^run-[A-Za-z0-9][A-Za-z0-9_-]{0,127}$" },
|
|
12
|
+
"scenarioId": { "type": "string", "pattern": "^[a-z0-9][a-z0-9-]{0,127}$" },
|
|
13
|
+
"mode": { "enum": ["direct", "forgeloopBalanced", "forgeloopAdaptive"] },
|
|
14
|
+
"runIndex": { "type": "integer", "minimum": 1 },
|
|
15
|
+
"recordedAt": { "type": "string", "minLength": 1 },
|
|
16
|
+
"usage": {
|
|
17
|
+
"type": "object",
|
|
18
|
+
"required": ["inputTokens", "outputTokens", "cacheReadTokens", "cacheWriteTokens", "totalTokens", "costUsd", "model", "provider", "source"],
|
|
19
|
+
"properties": {
|
|
20
|
+
"inputTokens": { "type": ["integer", "null"], "minimum": 0 },
|
|
21
|
+
"outputTokens": { "type": ["integer", "null"], "minimum": 0 },
|
|
22
|
+
"cacheReadTokens": { "type": ["integer", "null"], "minimum": 0 },
|
|
23
|
+
"cacheWriteTokens": { "type": ["integer", "null"], "minimum": 0 },
|
|
24
|
+
"totalTokens": { "type": ["integer", "null"], "minimum": 0 },
|
|
25
|
+
"costUsd": { "type": ["number", "null"], "minimum": 0 },
|
|
26
|
+
"model": { "type": ["string", "null"] },
|
|
27
|
+
"provider": { "type": ["string", "null"] },
|
|
28
|
+
"source": { "enum": ["PROVIDER_REPORTED", "HOST_REPORTED", "UNKNOWN"] }
|
|
29
|
+
},
|
|
30
|
+
"additionalProperties": false
|
|
31
|
+
},
|
|
32
|
+
"wallClockMs": { "type": ["number", "null"], "minimum": 0 },
|
|
33
|
+
"verification": { "enum": ["PASS", "FAIL", "NOT_AVAILABLE"] },
|
|
34
|
+
"verificationCycles": { "type": ["integer", "null"], "minimum": 0 },
|
|
35
|
+
"comparableSteps": { "type": ["integer", "null"], "minimum": 0 },
|
|
36
|
+
"contextUsage": { "$ref": "#/$defs/contextUsage" },
|
|
37
|
+
"quality": { "$ref": "#/$defs/quality" },
|
|
38
|
+
"metadata": {
|
|
39
|
+
"type": "object",
|
|
40
|
+
"required": ["scenarioId", "mode", "model", "provider", "promptSpecFingerprint", "projectRevision", "benchmarkVersion", "environmentClass", "requestedProfile", "resolvedProfile", "verificationCycles", "comparableSteps"],
|
|
41
|
+
"properties": {
|
|
42
|
+
"scenarioId": { "type": "string", "pattern": "^[a-z0-9][a-z0-9-]{0,127}$" },
|
|
43
|
+
"mode": { "enum": ["direct", "forgeloopBalanced", "forgeloopAdaptive"] },
|
|
44
|
+
"model": { "type": ["string", "null"] },
|
|
45
|
+
"provider": { "type": ["string", "null"] },
|
|
46
|
+
"promptSpecFingerprint": { "type": "string", "minLength": 1 },
|
|
47
|
+
"projectRevision": { "type": ["string", "null"] },
|
|
48
|
+
"benchmarkVersion": { "const": "1" },
|
|
49
|
+
"environmentClass": { "type": "string", "minLength": 1 },
|
|
50
|
+
"requestedProfile": { "type": ["string", "null"] },
|
|
51
|
+
"resolvedProfile": { "type": ["string", "null"] },
|
|
52
|
+
"verificationCycles": { "type": ["integer", "null"], "minimum": 0 },
|
|
53
|
+
"comparableSteps": { "type": ["integer", "null"], "minimum": 0 },
|
|
54
|
+
"nodeVersion": { "type": "string", "minLength": 1 },
|
|
55
|
+
"os": { "type": "string", "minLength": 1 },
|
|
56
|
+
"arch": { "type": "string", "minLength": 1 }
|
|
57
|
+
},
|
|
58
|
+
"additionalProperties": false
|
|
59
|
+
}
|
|
60
|
+
},
|
|
61
|
+
"$defs": {
|
|
62
|
+
"contextUsage": {
|
|
63
|
+
"type": "object",
|
|
64
|
+
"required": ["source", "profile", "items"],
|
|
65
|
+
"properties": {
|
|
66
|
+
"source": { "enum": ["HOST_REPORTED", "UNKNOWN"] },
|
|
67
|
+
"profile": { "enum": ["light", "balanced", "full", null] },
|
|
68
|
+
"items": {
|
|
69
|
+
"type": "object",
|
|
70
|
+
"required": ["taskContext", "guides", "history", "protocolInstructions", "repositoryContext", "other"],
|
|
71
|
+
"properties": {
|
|
72
|
+
"taskContext": { "type": ["integer", "null"], "minimum": 0 },
|
|
73
|
+
"guides": { "type": ["integer", "null"], "minimum": 0 },
|
|
74
|
+
"history": { "type": ["integer", "null"], "minimum": 0 },
|
|
75
|
+
"protocolInstructions": { "type": ["integer", "null"], "minimum": 0 },
|
|
76
|
+
"repositoryContext": { "type": ["integer", "null"], "minimum": 0 },
|
|
77
|
+
"other": { "type": ["integer", "null"], "minimum": 0 }
|
|
78
|
+
},
|
|
79
|
+
"additionalProperties": false
|
|
80
|
+
}
|
|
81
|
+
},
|
|
82
|
+
"additionalProperties": false
|
|
83
|
+
},
|
|
84
|
+
"quality": {
|
|
85
|
+
"type": "object",
|
|
86
|
+
"required": ["source", "scores"],
|
|
87
|
+
"properties": {
|
|
88
|
+
"source": { "enum": ["EXTERNAL_REPORTED", "HOST_REPORTED", "UNKNOWN"] },
|
|
89
|
+
"scores": {
|
|
90
|
+
"type": "object",
|
|
91
|
+
"required": ["visualQuality", "responsiveQuality", "accessibility", "interactionPolish", "requirementsCompleteness"],
|
|
92
|
+
"properties": {
|
|
93
|
+
"visualQuality": { "type": ["number", "null"], "minimum": 0, "maximum": 5 },
|
|
94
|
+
"responsiveQuality": { "type": ["number", "null"], "minimum": 0, "maximum": 5 },
|
|
95
|
+
"accessibility": { "type": ["number", "null"], "minimum": 0, "maximum": 5 },
|
|
96
|
+
"interactionPolish": { "type": ["number", "null"], "minimum": 0, "maximum": 5 },
|
|
97
|
+
"requirementsCompleteness": { "type": ["number", "null"], "minimum": 0, "maximum": 5 }
|
|
98
|
+
},
|
|
99
|
+
"additionalProperties": false
|
|
100
|
+
}
|
|
101
|
+
},
|
|
102
|
+
"additionalProperties": false
|
|
103
|
+
}
|
|
104
|
+
},
|
|
105
|
+
"additionalProperties": false
|
|
106
|
+
}
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
3
|
+
"$id": "forgeloop://schemas/execution-profile-benchmark-scenario.schema.json",
|
|
4
|
+
"title": "ForgeLoop execution-profile benchmark scenario",
|
|
5
|
+
"type": "object",
|
|
6
|
+
"required": ["schemaVersion", "benchmarkVersion", "scenarioId", "description", "input", "expectedProfile", "measurements"],
|
|
7
|
+
"properties": {
|
|
8
|
+
"schemaVersion": { "const": 1 },
|
|
9
|
+
"benchmarkVersion": { "type": "string", "minLength": 1 },
|
|
10
|
+
"scenarioId": { "type": "string", "pattern": "^[a-z0-9][a-z0-9-]{0,127}$" },
|
|
11
|
+
"description": { "type": "string", "minLength": 1 },
|
|
12
|
+
"input": {
|
|
13
|
+
"type": "object",
|
|
14
|
+
"required": ["workType", "surfaces", "risks", "platforms"],
|
|
15
|
+
"properties": {
|
|
16
|
+
"workType": { "type": "string", "minLength": 1 },
|
|
17
|
+
"surfaces": { "type": "array", "items": { "type": "string", "minLength": 1 } },
|
|
18
|
+
"risks": { "type": "array", "items": { "type": "string", "minLength": 1 } },
|
|
19
|
+
"platforms": { "type": "array", "items": { "type": "string", "minLength": 1 } },
|
|
20
|
+
"behaviorChange": { "type": "boolean" },
|
|
21
|
+
"executableChange": { "type": "boolean" }
|
|
22
|
+
},
|
|
23
|
+
"additionalProperties": false
|
|
24
|
+
},
|
|
25
|
+
"expectedProfile": { "enum": ["light", "balanced", "full"] },
|
|
26
|
+
"referenceTask": { "$ref": "#/$defs/referenceTask" },
|
|
27
|
+
"measurements": {
|
|
28
|
+
"type": "object",
|
|
29
|
+
"required": ["direct", "forgeloopBalanced", "forgeloopAdaptive"],
|
|
30
|
+
"properties": {
|
|
31
|
+
"direct": { "$ref": "#/$defs/measurement" },
|
|
32
|
+
"forgeloopBalanced": { "$ref": "#/$defs/measurement" },
|
|
33
|
+
"forgeloopAdaptive": { "$ref": "#/$defs/measurement" }
|
|
34
|
+
},
|
|
35
|
+
"additionalProperties": false
|
|
36
|
+
}
|
|
37
|
+
},
|
|
38
|
+
"$defs": {
|
|
39
|
+
"referenceTask": {
|
|
40
|
+
"type": "object",
|
|
41
|
+
"required": ["name", "requirements", "exclusions", "artifactMode"],
|
|
42
|
+
"properties": {
|
|
43
|
+
"name": { "type": "string", "minLength": 1 },
|
|
44
|
+
"requirements": { "type": "array", "items": { "type": "string", "minLength": 1 }, "minItems": 1 },
|
|
45
|
+
"exclusions": { "type": "array", "items": { "type": "string", "minLength": 1 }, "minItems": 1 },
|
|
46
|
+
"artifactMode": { "type": "string", "minLength": 1 }
|
|
47
|
+
},
|
|
48
|
+
"additionalProperties": false
|
|
49
|
+
},
|
|
50
|
+
"measurement": {
|
|
51
|
+
"type": "object",
|
|
52
|
+
"required": ["inputTokens", "outputTokens", "cacheReadTokens", "cacheWriteTokens", "totalTokens", "wallClockMs", "verification"],
|
|
53
|
+
"properties": {
|
|
54
|
+
"inputTokens": { "type": ["integer", "null"], "minimum": 0 },
|
|
55
|
+
"outputTokens": { "type": ["integer", "null"], "minimum": 0 },
|
|
56
|
+
"cacheReadTokens": { "type": ["integer", "null"], "minimum": 0 },
|
|
57
|
+
"cacheWriteTokens": { "type": ["integer", "null"], "minimum": 0 },
|
|
58
|
+
"totalTokens": { "type": ["integer", "null"], "minimum": 0 },
|
|
59
|
+
"wallClockMs": { "type": ["number", "null"], "minimum": 0 },
|
|
60
|
+
"verification": { "enum": ["PASS", "FAIL", "NOT_AVAILABLE"] }
|
|
61
|
+
},
|
|
62
|
+
"additionalProperties": false
|
|
63
|
+
}
|
|
64
|
+
},
|
|
65
|
+
"additionalProperties": false
|
|
66
|
+
}
|
|
@@ -8,6 +8,18 @@
|
|
|
8
8
|
"schemaVersion": { "const": 1 },
|
|
9
9
|
"protocolVersion": { "const": 1 },
|
|
10
10
|
"contractFingerprint": { "type": "string", "pattern": "^[a-f0-9]{64}$" },
|
|
11
|
+
"executionProfile": {
|
|
12
|
+
"type": "object",
|
|
13
|
+
"required": ["requested", "floor", "resolved", "reasons", "escalated"],
|
|
14
|
+
"additionalProperties": false,
|
|
15
|
+
"properties": {
|
|
16
|
+
"requested": { "enum": ["auto", "light", "balanced", "full"] },
|
|
17
|
+
"floor": { "enum": ["light", "balanced", "full"] },
|
|
18
|
+
"resolved": { "enum": ["light", "balanced", "full"] },
|
|
19
|
+
"reasons": { "type": "array", "minItems": 1, "items": { "type": "string", "minLength": 1 } },
|
|
20
|
+
"escalated": { "type": "boolean" }
|
|
21
|
+
}
|
|
22
|
+
},
|
|
11
23
|
"input": { "type": "object" },
|
|
12
24
|
"primary": { "oneOf": [{ "type": "string" }, { "type": "null" }] },
|
|
13
25
|
"guides": { "type": "array", "items": { "type": "string" } },
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
3
|
+
"$id": "forgeloop://schemas/usage.schema.json",
|
|
4
|
+
"title": "ForgeLoop task usage telemetry",
|
|
5
|
+
"type": "object",
|
|
6
|
+
"required": ["schemaVersion", "protocolVersion", "taskId", "recordedAt", "usage"],
|
|
7
|
+
"properties": {
|
|
8
|
+
"schemaVersion": { "const": 1 },
|
|
9
|
+
"protocolVersion": { "const": 1 },
|
|
10
|
+
"taskId": { "type": "string", "minLength": 1 },
|
|
11
|
+
"recordedAt": { "type": "string", "minLength": 1 },
|
|
12
|
+
"usage": {
|
|
13
|
+
"type": "object",
|
|
14
|
+
"required": ["inputTokens", "outputTokens", "cacheReadTokens", "cacheWriteTokens", "totalTokens", "costUsd", "model", "provider", "source"],
|
|
15
|
+
"additionalProperties": false,
|
|
16
|
+
"properties": {
|
|
17
|
+
"inputTokens": { "type": ["integer", "null"], "minimum": 0 },
|
|
18
|
+
"outputTokens": { "type": ["integer", "null"], "minimum": 0 },
|
|
19
|
+
"cacheReadTokens": { "type": ["integer", "null"], "minimum": 0 },
|
|
20
|
+
"cacheWriteTokens": { "type": ["integer", "null"], "minimum": 0 },
|
|
21
|
+
"totalTokens": { "type": ["integer", "null"], "minimum": 0 },
|
|
22
|
+
"costUsd": { "type": ["number", "null"], "minimum": 0 },
|
|
23
|
+
"model": { "type": ["string", "null"] },
|
|
24
|
+
"provider": { "type": ["string", "null"] },
|
|
25
|
+
"source": { "enum": ["PROVIDER_REPORTED", "HOST_REPORTED", "ACTOR_REPORTED", "UNKNOWN"] }
|
|
26
|
+
}
|
|
27
|
+
}
|
|
28
|
+
},
|
|
29
|
+
"additionalProperties": false
|
|
30
|
+
}
|
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
|
|
3
|
+
import path from "node:path";
|
|
4
|
+
import { fileURLToPath } from "node:url";
|
|
5
|
+
|
|
6
|
+
import { aggregateBenchmarkRuns } from "../src/core/execution-profile-benchmarks.js";
|
|
7
|
+
import { readBenchmarkRunSets, readBenchmarkScenarios } from "./lib/execution-profile-benchmark-io.mjs";
|
|
8
|
+
|
|
9
|
+
const repositoryRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
|
|
10
|
+
const defaultResultsDirectory = path.join(repositoryRoot, "benchmarks", "execution-profiles", "results");
|
|
11
|
+
|
|
12
|
+
function parseArgs(argv) {
|
|
13
|
+
const options = { results: defaultResultsDirectory, json: false };
|
|
14
|
+
for (let index = 0; index < argv.length; index += 1) {
|
|
15
|
+
const argument = argv[index];
|
|
16
|
+
if (argument === "--json") options.json = true;
|
|
17
|
+
else if (argument === "--results") {
|
|
18
|
+
const value = argv[++index];
|
|
19
|
+
if (!value || value.startsWith("--")) throw new Error("--results requires a value");
|
|
20
|
+
options.results = value;
|
|
21
|
+
} else if (argument === "--help" || argument === "-h") options.help = true;
|
|
22
|
+
else throw new Error(`unknown option: ${argument}`);
|
|
23
|
+
}
|
|
24
|
+
return options;
|
|
25
|
+
}
|
|
26
|
+
|
|
27
|
+
function statusForAggregate(aggregate, runs) {
|
|
28
|
+
const hasMeasuredUsage = runs.some((run) => (
|
|
29
|
+
["PROVIDER_REPORTED", "HOST_REPORTED"].includes(run.usage.source)
|
|
30
|
+
&& run.usage.totalTokens !== null
|
|
31
|
+
));
|
|
32
|
+
if (!hasMeasuredUsage) return "NOT_MEASURED";
|
|
33
|
+
if (!aggregate.claimsAllowed) return "NOT_COMPARABLE";
|
|
34
|
+
if (aggregate.lightObjectives
|
|
35
|
+
&& (aggregate.lightObjectives.p50Pass === false || aggregate.lightObjectives.p95Pass === false)) {
|
|
36
|
+
return "EFFICIENCY_REGRESSION";
|
|
37
|
+
}
|
|
38
|
+
return "OK";
|
|
39
|
+
}
|
|
40
|
+
|
|
41
|
+
function overallStatus(statuses) {
|
|
42
|
+
if (statuses.includes("EFFICIENCY_REGRESSION")) return "EFFICIENCY_REGRESSION";
|
|
43
|
+
if (statuses.includes("NOT_COMPARABLE")) return "NOT_COMPARABLE";
|
|
44
|
+
if (statuses.length === 0 || statuses.every((status) => status === "NOT_MEASURED")) return "NOT_MEASURED";
|
|
45
|
+
return "OK";
|
|
46
|
+
}
|
|
47
|
+
|
|
48
|
+
function helpText() {
|
|
49
|
+
return "Usage: npm run benchmark:profiles:regression -- [--results <directory>] [--json]";
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
async function main() {
|
|
53
|
+
const options = parseArgs(process.argv.slice(2));
|
|
54
|
+
if (options.help) {
|
|
55
|
+
console.log(helpText());
|
|
56
|
+
return;
|
|
57
|
+
}
|
|
58
|
+
const scenarios = await readBenchmarkScenarios(repositoryRoot);
|
|
59
|
+
const runSets = await readBenchmarkRunSets(path.resolve(options.results));
|
|
60
|
+
const reports = [];
|
|
61
|
+
for (const runSet of runSets) {
|
|
62
|
+
for (const scenario of scenarios) {
|
|
63
|
+
const runs = runSet.runs.filter((run) => run.scenarioId === scenario.scenarioId);
|
|
64
|
+
if (runs.length === 0) {
|
|
65
|
+
reports.push({ runSetId: runSet.runSetId, scenarioId: scenario.scenarioId, status: "NOT_COMPARABLE" });
|
|
66
|
+
continue;
|
|
67
|
+
}
|
|
68
|
+
const aggregate = aggregateBenchmarkRuns({ scenario, runs });
|
|
69
|
+
reports.push({
|
|
70
|
+
runSetId: runSet.runSetId,
|
|
71
|
+
scenarioId: scenario.scenarioId,
|
|
72
|
+
expectedProfile: scenario.expectedProfile,
|
|
73
|
+
status: statusForAggregate(aggregate, runs),
|
|
74
|
+
claimsAllowed: aggregate.claimsAllowed,
|
|
75
|
+
contextInflation: aggregate.contextInflation?.status ?? null,
|
|
76
|
+
});
|
|
77
|
+
}
|
|
78
|
+
}
|
|
79
|
+
const status = overallStatus(reports.map((report) => report.status));
|
|
80
|
+
const output = {
|
|
81
|
+
status,
|
|
82
|
+
blocking: false,
|
|
83
|
+
runSetCount: runSets.length,
|
|
84
|
+
scenarioCount: scenarios.length,
|
|
85
|
+
reports,
|
|
86
|
+
policy: "EFFICIENCY_REGRESSION is an observational release-quality warning and never a ForgeLoop lifecycle failure.",
|
|
87
|
+
};
|
|
88
|
+
if (options.json) console.log(JSON.stringify(output));
|
|
89
|
+
else {
|
|
90
|
+
console.log(`Efficiency regression status: ${status}`);
|
|
91
|
+
console.log(`Run sets: ${runSets.length}`);
|
|
92
|
+
console.log("This report is observational and does not change lifecycle completion.");
|
|
93
|
+
}
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
main().catch((error) => {
|
|
97
|
+
console.error(`ForgeLoop efficiency regression check: ${error.message}`);
|
|
98
|
+
process.exitCode = 1;
|
|
99
|
+
});
|
|
@@ -79,6 +79,41 @@ Package version: ${packageJson.version}
|
|
|
79
79
|
6. Run forgeloop complete; accept completion only when the validator returns VALID.
|
|
80
80
|
7. Run forgeloop next again and follow the returned lifecycle action to a terminal state or an explicit blocker.
|
|
81
81
|
|
|
82
|
+
## Adaptive execution profiles
|
|
83
|
+
|
|
84
|
+
\`complianceMode\` controls how strongly project policy is enforced. The
|
|
85
|
+
orthogonal \`executionProfile\` controls process and context depth: \`auto\`
|
|
86
|
+
resolves deterministically to \`light\`, \`balanced\`, or \`full\` from route,
|
|
87
|
+
contract, and task metadata. CLI requests take precedence over project
|
|
88
|
+
configuration, but a safety floor always wins. Profiles never remove required
|
|
89
|
+
contracts, gates, verification, provenance, lifecycle phases, or validated
|
|
90
|
+
completion. Protocol-v1 routes without the field project to \`balanced\` for
|
|
91
|
+
compatibility without rewriting historical artifacts.
|
|
92
|
+
|
|
93
|
+
For \`light\` tasks, hosts should use \`next --compact\` or \`task-show --compact\`,
|
|
94
|
+
load only relevant guide sections, keep plans concise, and avoid optional
|
|
95
|
+
reflection, trajectory evaluation, handoff, attestation, and continuity
|
|
96
|
+
artifacts unless requested, required, or needed for recovery. The lifecycle
|
|
97
|
+
chronology remains unchanged.
|
|
98
|
+
|
|
99
|
+
The read-only \`task/context\` integration resource provides the canonical
|
|
100
|
+
profile-aware projection: objective, deliverables, constraints, selected guide
|
|
101
|
+
IDs, phase, next action, verification requirements, and context policy. A
|
|
102
|
+
profile changes presentation and optional context only; it never permits a
|
|
103
|
+
phase or required gate to be skipped.
|
|
104
|
+
|
|
105
|
+
Usage telemetry is provider or host reported when available, actor-reported
|
|
106
|
+
only through the explicit \`usage-record\` fallback, and \`UNKNOWN\` otherwise.
|
|
107
|
+
ForgeLoop never estimates tokens or treats usage as verification evidence.
|
|
108
|
+
\`efficiency --task\` is read-only and returns \`NOT_COMPARABLE\` unless a
|
|
109
|
+
project-local baseline has matching metadata.
|
|
110
|
+
|
|
111
|
+
Measured execution-profile benchmarks are observational. The reproducible
|
|
112
|
+
runner accepts provider or host usage, records actual timing, requires PASS
|
|
113
|
+
verification and matching metadata for comparisons, and reports \`NOT_MEASURED\`
|
|
114
|
+
or \`NOT_COMPARABLE\` when evidence is absent or incompatible. See
|
|
115
|
+
\`docs/EXECUTION_PROFILE_BENCHMARKS.md\` for the runner and schemas.
|
|
116
|
+
|
|
82
117
|
## Authority boundaries
|
|
83
118
|
|
|
84
119
|
- Protocol-derived facts outrank actor-provided labels, free-form summaries, and guessed identities.
|