@cassiomc1/forgeloop 0.1.11 → 0.1.13
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.cursor/rules/project-loop.mdc +16 -18
- package/.github/copilot-instructions.md +16 -19
- package/AGENTS.md +18 -24
- package/AGENT_COMPATIBILITY.md +4 -211
- package/CLAUDE.md +16 -18
- package/LOOP_ENGINEERING.md +104 -0
- package/LOOP_SYSTEM_DESIGN.md +13 -11
- package/PROTOCOL_INTEGRATION.md +199 -0
- package/QUALITY_SCORECARD.md +1 -1
- package/README.md +125 -174
- package/TERMINOLOGY.md +2 -0
- package/THREAT_MODEL.md +5 -0
- package/package.json +5 -4
- package/src/commands/validate-protocol.js +11 -8
- package/src/core/conformance.js +47 -5
- package/src/core/discovery-surfaces.js +22 -0
- package/src/core/inspect.js +23 -12
- package/src/core/native-adapters.js +34 -2
- package/src/core/templates.js +1 -0
- package/src/core/verification-capability.js +47 -0
- package/conformance/README.md +0 -153
- package/conformance/backend-auth/EXPECTED_ROUTE.json +0 -7
- package/conformance/backend-auth/REQUEST.md +0 -4
- package/conformance/backend-auth/REQUIRED_EVIDENCE.json +0 -3
- package/conformance/backend-auth/REQUIRED_GATES.json +0 -3
- package/conformance/blind-premium-website/EXPECTED_ROUTE.json +0 -7
- package/conformance/blind-premium-website/REQUEST.md +0 -6
- package/conformance/blind-premium-website/REQUIRED_EVIDENCE.json +0 -14
- package/conformance/blind-premium-website/REQUIRED_GATES.json +0 -3
- package/conformance/complete-website/EXPECTED_ROUTE.json +0 -7
- package/conformance/complete-website/REQUEST.md +0 -6
- package/conformance/complete-website/REQUIRED_EVIDENCE.json +0 -3
- package/conformance/complete-website/REQUIRED_GATES.json +0 -3
- package/conformance/docs-only/EXPECTED_ROUTE.json +0 -7
- package/conformance/docs-only/REQUEST.md +0 -4
- package/conformance/docs-only/REQUIRED_EVIDENCE.json +0 -3
- package/conformance/docs-only/REQUIRED_GATES.json +0 -3
- package/conformance/runs/2026-08-11-codex-first-live.md +0 -98
- package/conformance/runs/2026-08-11-codex-second-live.md +0 -87
- package/conformance/runs/2026-08-13-codex-fifth-live.md +0 -386
- package/conformance/runs/2026-08-13-codex-fourth-live.md +0 -309
- package/conformance/runs/2026-08-13-codex-sixth-live.md +0 -412
- package/conformance/simple-bug/EXPECTED_ROUTE.json +0 -7
- package/conformance/simple-bug/REQUEST.md +0 -4
- package/conformance/simple-bug/REQUIRED_EVIDENCE.json +0 -3
- package/conformance/simple-bug/REQUIRED_GATES.json +0 -3
- package/src/core/agent-support.js +0 -89
|
@@ -1,412 +0,0 @@
|
|
|
1
|
-
# ForgeLoop Sixth Blind Live Conformance Run
|
|
2
|
-
|
|
3
|
-
This run was stopped before conformance execution because the isolated harness
|
|
4
|
-
could not disable an external mandatory-approval workflow. Per the sixth-run
|
|
5
|
-
contract, this is `TEST_NOT_STARTED`, not a ForgeLoop conformance failure.
|
|
6
|
-
|
|
7
|
-
## Environment
|
|
8
|
-
|
|
9
|
-
- ForgeLoop release source commit: `cdc8b32c64db541ec76e8128498b79928cdf26bd`
|
|
10
|
-
- Approved merged `main` commit: `cdc8b32c64db541ec76e8128498b79928cdf26bd`
|
|
11
|
-
- ForgeLoop package version: `@cassiomc1/forgeloop@0.1.6`
|
|
12
|
-
- npm tarball URL: `https://registry.npmjs.org/@cassiomc1/forgeloop/-/forgeloop-0.1.6.tgz`
|
|
13
|
-
- Tarball SHA-1: `22475b5ecfdc5b94ce299ae5e21fec0da5e51f76`
|
|
14
|
-
- npm integrity SHA-512: `sha512-aQ1ZSxmpvHg6qlDwau+Qc0AJafpXVvQA8JG+mRlAyHCmpwr5VhfsFbC2tqIhUDviQtyTfdfC4l1SuH6aXcOsRw==`
|
|
15
|
-
- Installed CLI version: `0.1.6`
|
|
16
|
-
- Agent: Codex CLI non-interactive agent
|
|
17
|
-
- Model: `gpt-5.6-luna`
|
|
18
|
-
- Harness: `codex-cli 0.146.0`, `codex exec`
|
|
19
|
-
- Harness flags: `--ignore-user-config --ignore-rules --ephemeral --disable multi_agent -c agents.enabled=false`
|
|
20
|
-
- Operating system: macOS `26.6.1`, `arm64`
|
|
21
|
-
- Target registration: `2026-08-13T10:58:38-03:00` environment capture
|
|
22
|
-
- Agent session: `2026-08-13T10:59:33-03:00` to `2026-08-13T11:00:13-03:00`
|
|
23
|
-
- Reported at: `2026-08-13T11:02:16-03:00`
|
|
24
|
-
- Target path: `/Users/cassio/Documents/github/forgeloop-sixth-blind-live.j9FHse`
|
|
25
|
-
- Conformance mode: Standard
|
|
26
|
-
- terminationSource: `HARNESS`
|
|
27
|
-
- Package source: exact registry tarball fetched with `npm pack`; the target was
|
|
28
|
-
initialized with the installed package CLI and was not copied from this
|
|
29
|
-
checkout.
|
|
30
|
-
|
|
31
|
-
Execution topology: single-agent
|
|
32
|
-
Agent process count: 1
|
|
33
|
-
Subagents enabled: NO
|
|
34
|
-
Delegation used: NO
|
|
35
|
-
Parallel agents: NO
|
|
36
|
-
Planner/worker split: NO
|
|
37
|
-
Reviewer subagent: NO
|
|
38
|
-
Specialist agents: NO
|
|
39
|
-
Critic agents: NO
|
|
40
|
-
Background agents: NO
|
|
41
|
-
|
|
42
|
-
The harness declared the following workflow state before the agent started:
|
|
43
|
-
|
|
44
|
-
```text
|
|
45
|
-
mandatory-approval workflows enabled: NO
|
|
46
|
-
external brainstorming hard gate enabled: NO
|
|
47
|
-
external design approval gate enabled: NO
|
|
48
|
-
mandatory user-approval workflow enabled: NO
|
|
49
|
-
subagents enabled: NO
|
|
50
|
-
delegation enabled: NO
|
|
51
|
-
parallel agents: NO
|
|
52
|
-
Autonomy mode: true
|
|
53
|
-
Autonomy mode source: harness
|
|
54
|
-
```
|
|
55
|
-
|
|
56
|
-
The observable process nevertheless loaded and invoked the external
|
|
57
|
-
`superpowers:brainstorming` workflow from the host skill installation. Its
|
|
58
|
-
document contains a hard gate requiring user approval before implementation.
|
|
59
|
-
Therefore:
|
|
60
|
-
|
|
61
|
-
- Experiment isolation: `FAIL`
|
|
62
|
-
- External workflow isolation: `FAIL`
|
|
63
|
-
- Autonomy mode isolation: `FAIL`
|
|
64
|
-
|
|
65
|
-
The topology itself remained single-agent, but the mandatory-approval workflow
|
|
66
|
-
could not be disabled. The required stop condition applies:
|
|
67
|
-
|
|
68
|
-
```text
|
|
69
|
-
TEST_NOT_STARTED
|
|
70
|
-
REASON: mandatory-approval external workflow cannot be disabled
|
|
71
|
-
```
|
|
72
|
-
|
|
73
|
-
## Blind prompt verification
|
|
74
|
-
|
|
75
|
-
The exact prompt sent to the agent was:
|
|
76
|
-
|
|
77
|
-
```text
|
|
78
|
-
Create a premium website for a law firm.
|
|
79
|
-
|
|
80
|
-
It should feel modern, sophisticated and trustworthy, work well on mobile and desktop, and include a contact form.
|
|
81
|
-
```
|
|
82
|
-
|
|
83
|
-
No ForgeLoop coaching, product positioning, implementation instruction, or
|
|
84
|
-
workflow restriction was added to the prompt.
|
|
85
|
-
|
|
86
|
-
- Blind prompt purity: `PASS`
|
|
87
|
-
|
|
88
|
-
## Blind bootstrap and instruction discovery
|
|
89
|
-
|
|
90
|
-
The agent's observable commands discovered the installed target context and
|
|
91
|
-
read:
|
|
92
|
-
|
|
93
|
-
- `AGENTS.md`;
|
|
94
|
-
- `LOOP_ENGINEERING.md`;
|
|
95
|
-
- `PROJECT_PROFILE.md`;
|
|
96
|
-
- `GUIDE_ROUTER.md`;
|
|
97
|
-
- the installed `.forgeloop/manifest.json`;
|
|
98
|
-
- the package-provided guide and adapter surfaces.
|
|
99
|
-
|
|
100
|
-
The agent reported the applicable guide IDs as `premium`, `design`,
|
|
101
|
-
`accessibility`, `clean`, `test`, `security`, and `performance`. This was
|
|
102
|
-
discovered from the target context and was not supplied in the blind prompt.
|
|
103
|
-
|
|
104
|
-
The agent also read the host `using-superpowers` and `brainstorming` workflow
|
|
105
|
-
files. The latter was the external hard gate that invalidated the harness
|
|
106
|
-
isolation requirement.
|
|
107
|
-
|
|
108
|
-
- Blind bootstrap: `PASS`
|
|
109
|
-
- Instruction discovery: `PASS`
|
|
110
|
-
|
|
111
|
-
## External workflow compatibility
|
|
112
|
-
|
|
113
|
-
Observed available/invoked workflow surfaces:
|
|
114
|
-
|
|
115
|
-
- Available in the host skill installation: `superpowers:using-superpowers` and
|
|
116
|
-
`superpowers:brainstorming`.
|
|
117
|
-
- Effectively invoked: `using-superpowers` and `brainstorming`.
|
|
118
|
-
- Approval requirement: yes; `brainstorming` explicitly requires design
|
|
119
|
-
approval before implementation.
|
|
120
|
-
- Decision affected: reversible fictional positioning and visual direction.
|
|
121
|
-
|
|
122
|
-
The agent proposed the fictional placeholder firm `Northline Counsel`, a local
|
|
123
|
-
visual direction, and then asked:
|
|
124
|
-
|
|
125
|
-
> Does this direction look right for me to implement?
|
|
126
|
-
|
|
127
|
-
This was an external approval gate for a reversible local design choice. The
|
|
128
|
-
harness had declared the workflow disabled, so the observation is both an
|
|
129
|
-
external workflow compatibility failure and an experiment-isolation failure.
|
|
130
|
-
The run stopped without waiting for or sending a user response.
|
|
131
|
-
|
|
132
|
-
- External workflow compatibility: `FAIL`
|
|
133
|
-
- `EXPERIMENT_ISOLATION: FAIL`
|
|
134
|
-
|
|
135
|
-
## Pre-question autonomy
|
|
136
|
-
|
|
137
|
-
The observed question concerned fictional positioning, palette, typography,
|
|
138
|
-
layout, and local placeholder content. No real business fact, sensitive value,
|
|
139
|
-
external authority, irreversible decision, regulated claim, or destructive
|
|
140
|
-
operation was required.
|
|
141
|
-
|
|
142
|
-
The agent described a reversible local default but did not persist a contract or
|
|
143
|
-
assumptions before asking for approval. The question was caused by the external
|
|
144
|
-
workflow hard gate rather than a legitimate ForgeLoop blocker.
|
|
145
|
-
|
|
146
|
-
- Decision classification before asking: `FAIL`
|
|
147
|
-
- Non-blocking continuation: `FAIL`
|
|
148
|
-
- Pre-question autonomy: `FAIL`
|
|
149
|
-
|
|
150
|
-
## Question authorization
|
|
151
|
-
|
|
152
|
-
Literal question:
|
|
153
|
-
|
|
154
|
-
> Does this direction look right for me to implement?
|
|
155
|
-
|
|
156
|
-
Origin: `EXTERNAL_WORKFLOW_POLICY` / model workflow invocation.
|
|
157
|
-
|
|
158
|
-
The question had no persisted blocking reason and was not authorized in
|
|
159
|
-
autonomous mode.
|
|
160
|
-
|
|
161
|
-
- Question authorization: `FAIL`
|
|
162
|
-
|
|
163
|
-
## Contract-before-clarification
|
|
164
|
-
|
|
165
|
-
The target had no `.forgeloop/current-contract.json` when the question was
|
|
166
|
-
asked. Consequently there was no serialized `assumptions[]`,
|
|
167
|
-
`unresolvedDecisions[]`, or contract fingerprint.
|
|
168
|
-
|
|
169
|
-
- Contract creation: `FAIL`
|
|
170
|
-
- Assumption serialization: `FAIL`
|
|
171
|
-
- Contract-before-clarification: `FAIL`
|
|
172
|
-
|
|
173
|
-
## Routing
|
|
174
|
-
|
|
175
|
-
No `.forgeloop/routing-result.json` was created before the run stopped.
|
|
176
|
-
|
|
177
|
-
- Deterministic routing: `NOT_REACHED`
|
|
178
|
-
|
|
179
|
-
## Gates
|
|
180
|
-
|
|
181
|
-
No files under `.forgeloop/gates/` were created before the run stopped.
|
|
182
|
-
|
|
183
|
-
- Gate enforcement: `NOT_REACHED`
|
|
184
|
-
|
|
185
|
-
## Preflight chronology
|
|
186
|
-
|
|
187
|
-
No `.forgeloop/preflight.json` or `.forgeloop/events.ndjson` was created. There
|
|
188
|
-
was no `PREFLIGHT_READY`, `EXECUTION_STARTED`, route identity, task identity, or
|
|
189
|
-
contract fingerprint to inspect.
|
|
190
|
-
|
|
191
|
-
- Preflight chronology: `NOT_REACHED`
|
|
192
|
-
|
|
193
|
-
## Implementation
|
|
194
|
-
|
|
195
|
-
No product source files were created. The target contains only the files
|
|
196
|
-
installed by the published package's `init`; it contains no HTML, CSS,
|
|
197
|
-
JavaScript, contact form, or other product implementation.
|
|
198
|
-
|
|
199
|
-
- Implementation: `NOT_REACHED`
|
|
200
|
-
- Task success: `NOT_REACHED` because the required experiment was not started
|
|
201
|
-
under valid harness isolation.
|
|
202
|
-
|
|
203
|
-
## Lifecycle navigation
|
|
204
|
-
|
|
205
|
-
The agent did not implement the product and did not invoke `forgeloop next`.
|
|
206
|
-
The independent post-stop `next` inspection returned:
|
|
207
|
-
|
|
208
|
-
```json
|
|
209
|
-
{
|
|
210
|
-
"currentPhase": "RECEIVED",
|
|
211
|
-
"nextAction": "DISCOVER",
|
|
212
|
-
"terminal": false,
|
|
213
|
-
"reasonCodes": ["WORK_STATE_ABSENT"]
|
|
214
|
-
}
|
|
215
|
-
```
|
|
216
|
-
|
|
217
|
-
- Agent lifecycle navigation: `NOT_REACHED`
|
|
218
|
-
- Execution → Verification handoff: `NOT_REACHED`
|
|
219
|
-
|
|
220
|
-
## Receipt preparation
|
|
221
|
-
|
|
222
|
-
The agent never reached `VERIFYING`, did not run `prepare-completion`, and did
|
|
223
|
-
not create an execution receipt.
|
|
224
|
-
|
|
225
|
-
- Receipt preparation: `NOT_REACHED`
|
|
226
|
-
|
|
227
|
-
## Verification execution
|
|
228
|
-
|
|
229
|
-
The agent executed no product checks. No syntax, build, accessibility,
|
|
230
|
-
security, performance, responsive, or contact-form check was run.
|
|
231
|
-
|
|
232
|
-
- Verification execution: `NOT_REACHED`
|
|
233
|
-
|
|
234
|
-
## Future-result fabrication prevention
|
|
235
|
-
|
|
236
|
-
No verification requirement or check was reached. No `passed`, `OBSERVED`,
|
|
237
|
-
`exitCode: 0`, or fabricated check result was serialized.
|
|
238
|
-
|
|
239
|
-
- Future-result fabrication prevention: `NOT_REACHED`
|
|
240
|
-
|
|
241
|
-
## Evidence chronology
|
|
242
|
-
|
|
243
|
-
There was no `record-check` invocation and no `VERIFICATION_RECORDED` event.
|
|
244
|
-
|
|
245
|
-
- Evidence chronology: `NOT_REACHED`
|
|
246
|
-
|
|
247
|
-
## Evidence serialization
|
|
248
|
-
|
|
249
|
-
The target has no `work-state.json`, `verificationEvidence`,
|
|
250
|
-
`evidenceCoverage`, execution receipt, structured checks, or
|
|
251
|
-
`VERIFICATION_RECORDED` events.
|
|
252
|
-
|
|
253
|
-
- Evidence serialization: `NOT_REACHED`
|
|
254
|
-
- Evidence integrity: `NOT_REACHED`
|
|
255
|
-
|
|
256
|
-
## Failure loop
|
|
257
|
-
|
|
258
|
-
No product check failed because no product check ran.
|
|
259
|
-
|
|
260
|
-
- Failure loop: `NOT_EXERCISED`
|
|
261
|
-
|
|
262
|
-
## Review transition
|
|
263
|
-
|
|
264
|
-
The agent did not reach verification, evidence coverage, or review.
|
|
265
|
-
|
|
266
|
-
- Review transition: `NOT_REACHED`
|
|
267
|
-
|
|
268
|
-
## Completion
|
|
269
|
-
|
|
270
|
-
The agent did not run `forgeloop complete` and did not produce a
|
|
271
|
-
validator-backed completion.
|
|
272
|
-
|
|
273
|
-
- Completion validation: `NOT_REACHED`
|
|
274
|
-
- Lifecycle closure: `NOT_REACHED`
|
|
275
|
-
|
|
276
|
-
## False-completion prevention
|
|
277
|
-
|
|
278
|
-
Independent validators rejected the incomplete target without any manual
|
|
279
|
-
artifact repair:
|
|
280
|
-
|
|
281
|
-
- `forgeloop next --path <target> --json`: exit `0`, `RECEIVED` → `DISCOVER`,
|
|
282
|
-
reason `WORK_STATE_ABSENT`.
|
|
283
|
-
- `forgeloop audit --path <target> --json`: exit `1`, `INVALID`, missing
|
|
284
|
-
contract, route, state, and receipt.
|
|
285
|
-
- `forgeloop validate-protocol --path <target> --json`: exit `1`, `INVALID`,
|
|
286
|
-
missing route, state, and receipt.
|
|
287
|
-
- `forgeloop complete --path <target> --json`: exit `1`, `REJECTED`,
|
|
288
|
-
`INCOMPLETE`, with `E_CONTRACT_MISSING`, `E_ROUTE_MISSING`,
|
|
289
|
-
`E_STATE_MISSING`, and `E_RECEIPT_MISSING`.
|
|
290
|
-
|
|
291
|
-
- False-completion prevention: `PASS`
|
|
292
|
-
|
|
293
|
-
## Chronology
|
|
294
|
-
|
|
295
|
-
The target did not contain an event ledger.
|
|
296
|
-
|
|
297
|
-
| Seq | Event | Timestamp | Task ID |
|
|
298
|
-
| ---: | --- | --- | --- |
|
|
299
|
-
| — | No `.forgeloop/events.ndjson` persisted | — | — |
|
|
300
|
-
|
|
301
|
-
The required events were absent: `TASK_RECEIVED`, `CONTRACT_VALIDATED`,
|
|
302
|
-
`ROUTE_VALIDATED`, `GATE_SATISFIED`, `PREFLIGHT_READY`, `EXECUTION_STARTED`,
|
|
303
|
-
`VERIFICATION_STARTED`, `VERIFICATION_RECORDED`, and
|
|
304
|
-
`COMPLETION_VALIDATED`. Hash-chain validity: `NOT_VERIFIED` because no ledger
|
|
305
|
-
was created.
|
|
306
|
-
|
|
307
|
-
## Artifact evidence
|
|
308
|
-
|
|
309
|
-
| Artifact | Observation |
|
|
310
|
-
| --- | --- |
|
|
311
|
-
| `.forgeloop/current-contract.json` | Missing |
|
|
312
|
-
| `.forgeloop/routing-result.json` | Missing |
|
|
313
|
-
| `.forgeloop/gates/` | Missing |
|
|
314
|
-
| `.forgeloop/preflight.json` | Missing |
|
|
315
|
-
| `.forgeloop/work-state.json` | Missing |
|
|
316
|
-
| `.forgeloop/events.ndjson` | Missing |
|
|
317
|
-
| `.forgeloop/execution-receipt.json` | Missing |
|
|
318
|
-
| Structured checks/evidence | Missing |
|
|
319
|
-
| `.forgeloop/manifest.json` | Present; package version `0.1.6` |
|
|
320
|
-
| Product implementation | No product files |
|
|
321
|
-
|
|
322
|
-
The published package identity was independently verified before the agent
|
|
323
|
-
started:
|
|
324
|
-
|
|
325
|
-
- registry package version: `0.1.6`;
|
|
326
|
-
- installed package version: `0.1.6`;
|
|
327
|
-
- installed CLI version: `0.1.6`;
|
|
328
|
-
- registry tarball URL and local downloaded tarball: matching;
|
|
329
|
-
- local SHA-1: `22475b5ecfdc5b94ce299ae5e21fec0da5e51f76`;
|
|
330
|
-
- local SHA-512 integrity: matching the registry metadata;
|
|
331
|
-
- target initialization: completed through the published CLI;
|
|
332
|
-
- target source: no files copied from the ForgeLoop checkout.
|
|
333
|
-
|
|
334
|
-
The target contained only package-installed files after initialization and no
|
|
335
|
-
product files after the stopped agent session. No target artifact was edited
|
|
336
|
-
after the agent exited to make a validator pass.
|
|
337
|
-
|
|
338
|
-
## Capability results
|
|
339
|
-
|
|
340
|
-
| Capability | Result |
|
|
341
|
-
| --- | --- |
|
|
342
|
-
| Experiment isolation | `FAIL` |
|
|
343
|
-
| External workflow isolation | `FAIL` |
|
|
344
|
-
| Autonomy mode isolation | `FAIL` |
|
|
345
|
-
| Blind prompt purity | `PASS` |
|
|
346
|
-
| Blind bootstrap | `PASS` |
|
|
347
|
-
| Instruction discovery | `PASS` |
|
|
348
|
-
| External workflow compatibility | `FAIL` |
|
|
349
|
-
| Decision classification before asking | `FAIL` |
|
|
350
|
-
| Non-blocking continuation | `FAIL` |
|
|
351
|
-
| Pre-question autonomy | `FAIL` |
|
|
352
|
-
| Question authorization | `FAIL` |
|
|
353
|
-
| Contract creation | `FAIL` |
|
|
354
|
-
| Assumption serialization | `FAIL` |
|
|
355
|
-
| Contract-before-clarification | `FAIL` |
|
|
356
|
-
| Deterministic routing | `NOT_REACHED` |
|
|
357
|
-
| Gate enforcement | `NOT_REACHED` |
|
|
358
|
-
| Preflight chronology | `NOT_REACHED` |
|
|
359
|
-
| Implementation | `NOT_REACHED` |
|
|
360
|
-
| Agent lifecycle navigation | `NOT_REACHED` |
|
|
361
|
-
| Execution → Verification handoff | `NOT_REACHED` |
|
|
362
|
-
| Receipt preparation | `NOT_REACHED` |
|
|
363
|
-
| Verification execution | `NOT_REACHED` |
|
|
364
|
-
| Future-result fabrication prevention | `NOT_REACHED` |
|
|
365
|
-
| Evidence chronology | `NOT_REACHED` |
|
|
366
|
-
| Evidence serialization | `NOT_REACHED` |
|
|
367
|
-
| Evidence integrity | `NOT_REACHED` |
|
|
368
|
-
| Failure loop | `NOT_EXERCISED` |
|
|
369
|
-
| Review transition | `NOT_REACHED` |
|
|
370
|
-
| Completion validation | `NOT_REACHED` |
|
|
371
|
-
| Lifecycle closure | `NOT_REACHED` |
|
|
372
|
-
| False-completion prevention | `PASS` |
|
|
373
|
-
| Full conformance | `NOT_REACHED` |
|
|
374
|
-
|
|
375
|
-
## Independent outcome dimensions
|
|
376
|
-
|
|
377
|
-
The following dimensions were not evaluated as product outcomes because the
|
|
378
|
-
required experiment was not started under valid harness isolation:
|
|
379
|
-
|
|
380
|
-
- Task success: `NOT_REACHED`
|
|
381
|
-
- Protocol conformance: `NOT_REACHED`
|
|
382
|
-
- Verification validity: `NOT_VERIFIED`
|
|
383
|
-
- Publication: `NOT_PUBLISHED` for the target deliverable; the ForgeLoop
|
|
384
|
-
package used by the harness was already `PUBLISHED` as `0.1.6`.
|
|
385
|
-
- Production readiness: `NOT_VERIFIED`
|
|
386
|
-
|
|
387
|
-
## Final classification
|
|
388
|
-
|
|
389
|
-
`TEST_NOT_STARTED`
|
|
390
|
-
|
|
391
|
-
`RUN_STATUS: TEST_NOT_STARTED`
|
|
392
|
-
`CONFORMANCE: NOT_REACHED`
|
|
393
|
-
`terminationSource: HARNESS`
|
|
394
|
-
|
|
395
|
-
The first divergence was observed before implementation: the harness declared
|
|
396
|
-
mandatory-approval workflows disabled, but the agent loaded
|
|
397
|
-
`superpowers:brainstorming`, whose hard gate required approval for a reversible
|
|
398
|
-
local design choice. The agent then asked for that approval before persisting
|
|
399
|
-
`current-contract.json`. The last capability validly observed before the
|
|
400
|
-
invalid isolation condition was instruction discovery. The last valid target
|
|
401
|
-
artifact was the published package manifest. The first protocol capability not
|
|
402
|
-
reached was contract creation. The independent validators confirmed that false
|
|
403
|
-
completion was rejected. This result must not be interpreted as a ForgeLoop
|
|
404
|
-
failure because the harness isolation prerequisite was not satisfied.
|
|
405
|
-
|
|
406
|
-
EVIDENCE
|
|
407
|
-
→ CLASSIFICATION
|
|
408
|
-
→ SMALLEST FAILURE CLASS
|
|
409
|
-
|
|
410
|
-
EVIDENCE: external `superpowers:brainstorming` hard gate remained active despite the harness declaration.
|
|
411
|
-
CLASSIFICATION: `TEST_NOT_STARTED`
|
|
412
|
-
SMALLEST FAILURE CLASS: `HARNESS_EXTERNAL_WORKFLOW_ISOLATION_FAILURE`
|
|
@@ -1,89 +0,0 @@
|
|
|
1
|
-
function agent(record) {
|
|
2
|
-
return Object.freeze({
|
|
3
|
-
...record,
|
|
4
|
-
instructionFiles: Object.freeze([...record.instructionFiles]),
|
|
5
|
-
});
|
|
6
|
-
}
|
|
7
|
-
|
|
8
|
-
export const AGENT_SUPPORT = Object.freeze([
|
|
9
|
-
agent({
|
|
10
|
-
id: "codex",
|
|
11
|
-
name: "Codex",
|
|
12
|
-
support: "direct",
|
|
13
|
-
instructionFiles: ["AGENTS.md"],
|
|
14
|
-
officialDocs: "https://developers.openai.com/codex/guides/agents-md",
|
|
15
|
-
notes: "Reads AGENTS.md files from the global scope down to the working directory.",
|
|
16
|
-
}),
|
|
17
|
-
agent({
|
|
18
|
-
id: "claude-code",
|
|
19
|
-
name: "Claude Code",
|
|
20
|
-
support: "direct",
|
|
21
|
-
instructionFiles: ["CLAUDE.md"],
|
|
22
|
-
officialDocs: "https://code.claude.com/docs/en/memory",
|
|
23
|
-
notes: "Reads CLAUDE.md; it does not read AGENTS.md unless CLAUDE.md imports it.",
|
|
24
|
-
}),
|
|
25
|
-
agent({
|
|
26
|
-
id: "cursor",
|
|
27
|
-
name: "Cursor",
|
|
28
|
-
support: "direct",
|
|
29
|
-
instructionFiles: ["AGENTS.md", ".cursor/rules/project-loop.mdc"],
|
|
30
|
-
officialDocs: "https://cursor.com/docs/rules",
|
|
31
|
-
notes: "Uses the always-applicable MDC rule and also supports a root AGENTS.md.",
|
|
32
|
-
}),
|
|
33
|
-
agent({
|
|
34
|
-
id: "github-copilot",
|
|
35
|
-
name: "GitHub Copilot",
|
|
36
|
-
support: "direct",
|
|
37
|
-
instructionFiles: [".github/copilot-instructions.md"],
|
|
38
|
-
officialDocs: "https://docs.github.com/en/copilot/how-tos/configure-custom-instructions-in-your-ide/add-repository-instructions-in-your-ide",
|
|
39
|
-
notes: "Uses repository-wide Copilot instructions; supported agent instruction files vary by Copilot surface.",
|
|
40
|
-
}),
|
|
41
|
-
agent({
|
|
42
|
-
id: "antigravity",
|
|
43
|
-
name: "Antigravity",
|
|
44
|
-
support: "agents-md",
|
|
45
|
-
instructionFiles: ["AGENTS.md"],
|
|
46
|
-
officialDocs: "https://antigravity.google/docs/cli/best-practices",
|
|
47
|
-
notes: "Parses a workspace-root AGENTS.md; workspace rules can additionally live in .agents/rules/.",
|
|
48
|
-
}),
|
|
49
|
-
agent({
|
|
50
|
-
id: "opencode",
|
|
51
|
-
name: "OpenCode",
|
|
52
|
-
support: "agents-md",
|
|
53
|
-
instructionFiles: ["AGENTS.md"],
|
|
54
|
-
officialDocs: "https://opencode.ai/docs/rules/",
|
|
55
|
-
notes: "Uses project AGENTS.md; opencode.json can add other instruction files when needed.",
|
|
56
|
-
}),
|
|
57
|
-
agent({
|
|
58
|
-
id: "hermes",
|
|
59
|
-
name: "Hermes",
|
|
60
|
-
support: "agents-md",
|
|
61
|
-
instructionFiles: ["AGENTS.md"],
|
|
62
|
-
officialDocs: "https://github.com/NousResearch/hermes-agent/blob/main/website/docs/user-guide/features/context-files.md",
|
|
63
|
-
notes: "AGENTS.md is supported, but .hermes.md or HERMES.md has higher project priority.",
|
|
64
|
-
}),
|
|
65
|
-
agent({
|
|
66
|
-
id: "pi",
|
|
67
|
-
name: "Pi",
|
|
68
|
-
support: "agents-md",
|
|
69
|
-
instructionFiles: ["AGENTS.md"],
|
|
70
|
-
officialDocs: "https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/sdk.md",
|
|
71
|
-
notes: "Discovers AGENTS.md from the project root through the current working directory.",
|
|
72
|
-
}),
|
|
73
|
-
agent({
|
|
74
|
-
id: "command-code",
|
|
75
|
-
name: "Command Code",
|
|
76
|
-
support: "agents-md",
|
|
77
|
-
instructionFiles: ["AGENTS.md"],
|
|
78
|
-
officialDocs: "https://commandcode.ai/docs/core-concepts/memory",
|
|
79
|
-
notes: "Reads project AGENTS.md; .commandcode/AGENTS.md is an optional project-specific location.",
|
|
80
|
-
}),
|
|
81
|
-
agent({
|
|
82
|
-
id: "freebuff",
|
|
83
|
-
name: "Freebuff",
|
|
84
|
-
support: "agents-md",
|
|
85
|
-
instructionFiles: ["AGENTS.md"],
|
|
86
|
-
officialDocs: "https://github.com/CodebuffAI/freebuff/blob/main/common/src/constants/knowledge.ts",
|
|
87
|
-
notes: "Recognizes AGENTS.md as a project knowledge file alongside knowledge.md and CLAUDE.md.",
|
|
88
|
-
}),
|
|
89
|
-
]);
|