@orkestrel/scaffold 0.0.17 → 0.0.19
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/host/AGENTS.md +4 -2
- package/dist/host/CLAUDE.md +22 -10
- package/dist/host/agents/skills/enterprise-bootstrap/SKILL.md +267 -0
- package/dist/host/agents/skills/enterprise-bootstrap/agents/openai.yaml +4 -0
- package/dist/host/agents/skills/enterprise-bootstrap/references/bootstrap-reference.md +609 -0
- package/dist/host/agents/skills/enterprise-bootstrap/references/components.md +994 -0
- package/dist/host/agents/skills/enterprise-bootstrap/references/frontend-design.md +59 -0
- package/dist/host/agents/skills/enterprise-bootstrap/references/utilities.md +312 -0
- package/dist/host/agents/skills/orkestrel-align-packages/references/integration.md +4 -13
- package/dist/host/agents/skills/orkestrel-build-application/SKILL.md +78 -50
- package/dist/host/agents/skills/orkestrel-debrief/SKILL.md +81 -0
- package/dist/host/agents/skills/orkestrel-debrief/agents/openai.yaml +4 -0
- package/dist/host/agents/skills/orkestrel-debrief/references/field-testing.md +75 -0
- package/dist/host/agents/skills/orkestrel-harden-package/SKILL.md +11 -9
- package/dist/host/agents/skills/orkestrel-harden-package/references/centralization.md +44 -59
- package/dist/host/agents/skills/orkestrel-harden-package/references/hardening.md +14 -6
- package/dist/host/agents/skills/orkestrel-polish-surface/SKILL.md +113 -0
- package/dist/host/agents/skills/orkestrel-polish-surface/agents/openai.yaml +4 -0
- package/dist/host/agents/skills/orkestrel-polish-surface/references/capture-harness.md +82 -0
- package/dist/host/claude/agents/builder.md +2 -0
- package/dist/host/claude/agents/codex.md +33 -26
- package/dist/host/claude/agents/grok.md +7 -0
- package/dist/host/claude/agents/implementer.md +2 -1
- package/dist/host/claude/agents/orkestrel.md +20 -15
- package/dist/host/claude/agents/planner.md +2 -1
- package/dist/host/claude/agents/reviewer.md +6 -0
- package/dist/host/claude/rules/documentation.md +1 -0
- package/dist/host/claude/rules/names.md +5 -7
- package/dist/host/claude/rules/quality.md +7 -5
- package/dist/host/claude/rules/styles.md +1 -0
- package/dist/host/claude/rules/tests.md +1 -0
- package/dist/host/claude/rules/typescript.md +3 -10
- package/dist/host/claude/rules/workspace.md +2 -5
- package/dist/host/claude/skills/enterprise-bootstrap/SKILL.md +12 -0
- package/dist/host/claude/skills/orkestrel-debrief/SKILL.md +12 -0
- package/dist/host/claude/skills/orkestrel-polish-surface/SKILL.md +12 -0
- package/dist/host/codex/agents/analyst.toml +6 -3
- package/dist/host/codex/agents/builder.toml +3 -2
- package/dist/host/codex/agents/checker.toml +4 -2
- package/dist/host/codex/agents/grok.toml +3 -1
- package/dist/host/codex/agents/implementer.toml +4 -2
- package/dist/host/codex/agents/opus.toml +5 -3
- package/dist/host/codex/agents/orkestrel.toml +6 -5
- package/dist/host/codex/agents/planner.toml +6 -2
- package/dist/host/codex/agents/reviewer.toml +7 -2
- package/dist/host/codex/config.toml +11 -2
- package/dist/host/dotfiles/prettierignore +3 -0
- package/dist/host/guides/src/scaffold.md +42 -12
- package/dist/host/manifest.json +80 -9
- package/dist/host/tests/setupPolicy.ts +4 -3
- package/dist/src/core/index.cjs +188 -44
- package/dist/src/core/index.cjs.map +1 -1
- package/dist/src/core/index.d.cts +17 -6
- package/dist/src/core/index.d.ts +17 -6
- package/dist/src/core/index.js +188 -45
- package/dist/src/core/index.js.map +1 -1
- package/dist/src/server/index.cjs +11 -5
- package/dist/src/server/index.cjs.map +1 -1
- package/dist/src/server/index.d.cts +2 -1
- package/dist/src/server/index.d.ts +2 -1
- package/dist/src/server/index.js +10 -4
- package/dist/src/server/index.js.map +1 -1
- package/package.json +4 -4
- package/dist/host/agents/skills/orkestrel-build-application/references/application.md +0 -129
- package/dist/host/claude/agents/application.md +0 -30
- package/dist/host/codex/agents/application.toml +0 -25
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
interface:
|
|
2
|
+
display_name: 'Debrief Closed Campaign'
|
|
3
|
+
short_description: 'Fold a closed campaign into portable truth on field evidence'
|
|
4
|
+
default_prompt: 'Use $orkestrel-debrief to debrief the closed campaign: field passes, findings ledger, fix loops, canon refinement, propagation, disposal.'
|
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
# Field testing an agent-facing surface
|
|
2
|
+
|
|
3
|
+
The method for producing the live evidence a debrief judges by. Proven across the
|
|
4
|
+
supervisor MCP campaign's five-model roster; portable to any surface a model consumes.
|
|
5
|
+
|
|
6
|
+
## The tier ladder
|
|
7
|
+
|
|
8
|
+
Test from the top down, and do not stop at the tier that passes:
|
|
9
|
+
|
|
10
|
+
1. **Frontier** (the harness's default model) — proves the surface works at all.
|
|
11
|
+
2. **Mid tier** (e.g. a codex mechanical model) — proves the surface survives a harness's
|
|
12
|
+
schema abbreviation and a model that reads less carefully.
|
|
13
|
+
3. **Small harness-native** (e.g. Haiku, a codex high-volume model, a cursor composer) —
|
|
14
|
+
the acceptance tier: these must walk the surface unaided, or the surface is not done.
|
|
15
|
+
4. **Local floor** (a quantized 2B-class model through a real tool-calling client) — not
|
|
16
|
+
an acceptance gate; a stochastic probe that exposes teaching gaps nothing else hits.
|
|
17
|
+
Its residual failures must be provably consumer-floor (malformed emission, attention
|
|
18
|
+
loss), never surface darkness — every refusal it received must have named the fix.
|
|
19
|
+
|
|
20
|
+
## The pass discipline
|
|
21
|
+
|
|
22
|
+
- **Goal-only prompts.** State the outcome, never the shape: no field names, no schema
|
|
23
|
+
hints, no tool names beyond the surface's own. The teaching surface must carry
|
|
24
|
+
everything else. Identical wording across models; only identifiers vary.
|
|
25
|
+
- **Fresh state per round.** New workflow/resource ids each round; a fresh server on the
|
|
26
|
+
exact build under test. Never let a model inherit a sibling's residue.
|
|
27
|
+
- **No coaching, no retries by hand.** The transcript ends when the model finishes or its
|
|
28
|
+
cap fires. A human nudge invalidates the pass.
|
|
29
|
+
- **Caps and journals.** Every pass runs as a tracked background command under a hard
|
|
30
|
+
time cap with its transcript journaled; the journal is the evidence of record.
|
|
31
|
+
|
|
32
|
+
## Capture the reasoning, not just the calls
|
|
33
|
+
|
|
34
|
+
Where the runtime exposes thinking (local runtimes expose it directly; harness stream
|
|
35
|
+
formats carry interstitial text), record it. The call log shows WHAT failed; the trace
|
|
36
|
+
shows WHY — and the why is what the fix targets.
|
|
37
|
+
|
|
38
|
+
## The confusion-signature catalog
|
|
39
|
+
|
|
40
|
+
Read every trace against these; each maps to a class of surface fix:
|
|
41
|
+
|
|
42
|
+
| Signature | What it looks like | The fix class |
|
|
43
|
+
| ----------------------- | ---------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
|
|
44
|
+
| Rationalized silence | The model meets an anomaly, invents a plausible cause, reports success | A silent path must refuse loudly; false affordances removed |
|
|
45
|
+
| Guess loop | Repeated invented shapes with no reading between attempts | The critical shape belongs in the description text, stated early |
|
|
46
|
+
| Abbreviation blindness | The model names what it cannot see ("the abbreviated schema") | Teaching moved to the surface every harness shows intact |
|
|
47
|
+
| One-correction-per-step | A refusal names two defects; the model fixes one and keeps the other | Refusals name ALL missing and unexpected items every time |
|
|
48
|
+
| Opaque-wall regression | A correct structure abandoned after an unnamed refusal | Every refusal names the subject, path, expectation, and candidates |
|
|
49
|
+
| Repetition fixpoint | The identical failing call repeated verbatim | The refusal must change the caller's information state; if it already does, the residual is consumer-floor |
|
|
50
|
+
|
|
51
|
+
## The teaching-surface doctrine
|
|
52
|
+
|
|
53
|
+
What the field passes repeatedly prove:
|
|
54
|
+
|
|
55
|
+
- **Descriptions are the surviving surface.** Harnesses abbreviate nested schemas;
|
|
56
|
+
the tool description reaches every model intact. The critical path — a worked example,
|
|
57
|
+
the key nesting, the reply shape — lives there, importance-first, so a model that
|
|
58
|
+
stops reading early still gets it.
|
|
59
|
+
- **Every refusal orients, names, or corrects.** Subject, exact key path, expected
|
|
60
|
+
shape, valid candidates, missing versus unexpected — a refusal that names nothing ends
|
|
61
|
+
runs; a refusal that names everything converges even a 2B model.
|
|
62
|
+
- **Strictness with teaching beats tolerance.** Never coerce, never silently drop;
|
|
63
|
+
refuse with the correction in the message. Encode untrusted identifiers (JSON-encoded,
|
|
64
|
+
length-capped) so hostile input cannot forge or balloon a diagnostic.
|
|
65
|
+
- **Acknowledge or state, never both.** Mutations return acknowledgements; reads return
|
|
66
|
+
state; the split is stated in both tools' descriptions so neither is polled for the
|
|
67
|
+
other's job.
|
|
68
|
+
|
|
69
|
+
## The scoreboard
|
|
70
|
+
|
|
71
|
+
Report each round as a table: model, harness, calls, failed calls, outcome, and the
|
|
72
|
+
delta from the prior round. A tier passes when its models complete the goal with zero
|
|
73
|
+
failed calls and no out-of-band reading (no source excavation, no filesystem search —
|
|
74
|
+
watch for it in the transcript; it means the surface leaked its teaching job to the
|
|
75
|
+
repository).
|
|
@@ -42,21 +42,23 @@ Load [hardening.md](references/hardening.md) for the hardening lane and for any
|
|
|
42
42
|
1. **Bound the campaign.** Record requested outcomes, in-scope environments/domains, explicit exclusions, supported hosts, dirty files, and evidence needed for acceptance.
|
|
43
43
|
2. **Map before editing.** Trace public types, implementations, callers, tests, guides, exports, runtime boundaries, installed Orkestrel dependencies, and applicable legacy/upstream references.
|
|
44
44
|
3. **Establish the intended contract.** Build a capability/defect matrix. Separate verified fact from inference. Mark each row implement, repair, retain, or exclude with a reason.
|
|
45
|
-
4. **Design types first.** Update guide/spec intent and `*/types.ts` before implementation
|
|
46
|
-
5. **Implement completely.** Finish every in-scope branch
|
|
47
|
-
6. **
|
|
48
|
-
7. **
|
|
49
|
-
8. **
|
|
50
|
-
9. **
|
|
51
|
-
10. **
|
|
52
|
-
11. **
|
|
53
|
-
12. **
|
|
45
|
+
4. **Design types first.** Update guide/spec intent and `*/types.ts` before implementation, under the root design laws. A contract that needs a compatibility shim is the wrong contract.
|
|
46
|
+
5. **Implement completely.** Finish every in-scope branch and reuse the exact installed Orkestrel primitives whose semantics match. The root completion law decides what may not be left behind.
|
|
47
|
+
6. **Prove each defect before repairing it.** A repair begins with a test that fails for that defect: record the exact command and its failing count before the fix and the same command's passing count after. A repair with no red-then-green record is unproven.
|
|
48
|
+
7. **Consolidate.** Run the complete centralization and wrapper sweep. Update all call sites to the real symbol rather than leaving aliases or 1:1 delegates.
|
|
49
|
+
8. **Challenge seams.** Add deterministic tests for invariants, boundaries, failures, lifecycle, cleanup, cancellation, concurrency, hostile input, and resource pressure as applicable, under the test rules' real-implementation law.
|
|
50
|
+
9. **Use live services deliberately.** Put real external services/models in their dedicated project, require readiness, and make each request minimally sufficient, robust, and behaviorally meaningful. When the claim is that a foreign client can use this package, drive one representative real client end to end.
|
|
51
|
+
10. **Document the final behavior.** Update the governing guide, examples, method tables, limitations, and parity coverage. Document architectural limits honestly.
|
|
52
|
+
11. **Audit completion.** Inspect test discovery, `.todo`/`.skip`/conditional skip use, source/test helper duplication, exports, environment isolation, unexpected text corruption, and the entire diff.
|
|
53
|
+
12. **Verify.** Run the repository-prescribed gates in order and inspect the generated outputs relevant to the request.
|
|
54
|
+
13. **Review independently, and never by the author.** When orchestration is available, require a design-fit reviewer, an objective correctness/constraints analyst, and a mechanical checker; add an adversarial pass for security, concurrency, destructive paths, or external input. A unit's auditor is the engine that did not write it — same-engine re-review returns the author's own blind spot. Resolve every required finding, then rerun affected verification.
|
|
54
55
|
|
|
55
56
|
## Accept the result
|
|
56
57
|
|
|
57
58
|
Do not accept “mostly complete,” “tests pass” without adequacy review, or a builder's self-report. Completion requires:
|
|
58
59
|
|
|
59
60
|
- all requested capabilities implemented or explicitly proven out of scope;
|
|
61
|
+
- red-then-green evidence for every repaired defect;
|
|
60
62
|
- public contract, implementation, tests, guides, and generated outputs aligned;
|
|
61
63
|
- no unresolved centralization, wrapper, dependency-reuse, test-discovery, lifecycle, security, or text-integrity finding;
|
|
62
64
|
- exact command evidence for every final gate.
|
|
@@ -1,85 +1,70 @@
|
|
|
1
1
|
# Centralization and simplification
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Placement, kind purity, the wrapper test, the no-nested-function law, and barrel law live
|
|
4
|
+
in `.claude/rules/architecture.md`; shared test infrastructure and helper placement live in
|
|
5
|
+
`.claude/rules/tests.md`. This reference adds only the sweep those laws assume: what to
|
|
6
|
+
inventory, how to classify, and what must be proven before acceptance.
|
|
4
7
|
|
|
5
|
-
|
|
8
|
+
## Inventory what you touched
|
|
6
9
|
|
|
7
|
-
|
|
10
|
+
Inspect every touched implementation and centralized file, then sweep the full affected
|
|
11
|
+
environments.
|
|
8
12
|
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
- imports or exports left behind after moves.
|
|
13
|
+
| Where | Inventory |
|
|
14
|
+
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
15
|
+
| Implementation file | Interfaces and type aliases, module constants and data, free functions/guards/parsers/factories/schemas, function declarations or assignments inside bodies, extra classes, imports and exports stranded by a move |
|
|
16
|
+
| Centralized file | Every module declaration: does it match that file's kind, is it exported, is it reachable from the sole public barrel when public, does it carry direct behavioral coverage when it carries logic |
|
|
17
|
+
| Environment root | Every declaration promoted there: at least two consuming modules, or it belongs to the owning module's centralized file |
|
|
15
18
|
|
|
16
|
-
|
|
19
|
+
The rare runtime-self-contained entrypoint exception must be literally required for
|
|
20
|
+
execution and must explain why sibling imports cannot work.
|
|
17
21
|
|
|
18
|
-
|
|
22
|
+
## Classify every function
|
|
19
23
|
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
-
|
|
24
|
+
| Signal | Home |
|
|
25
|
+
| ------------------------------------------ | ---------------------------------------------------- |
|
|
26
|
+
| Reaches instance state or a sibling method | Class method |
|
|
27
|
+
| Pure self-contained computation | Exported centralized helper, parser, compiler, guard |
|
|
28
|
+
| Defining recursive or compositional spine | Class method, after extracting its pure leaves |
|
|
29
|
+
| Trivial and genuinely one-use | Inline it into the caller |
|
|
24
30
|
|
|
25
|
-
|
|
31
|
+
Never move logic into a nested function to evade centralization. An anonymous callback
|
|
32
|
+
passed directly to another operation stays a callback, not a hidden helper declaration.
|
|
26
33
|
|
|
27
|
-
##
|
|
34
|
+
## Hunt the wrapper
|
|
28
35
|
|
|
29
|
-
|
|
36
|
+
Search callers and callees for one-line delegates, pass-through factories, getters that
|
|
37
|
+
rename another public getter, duplicate guards or parsers a declared dependency already
|
|
38
|
+
supplies, compatibility aliases and re-exports, and functions whose only purpose is
|
|
39
|
+
avoiding a downstream rename. The architecture rules' wrapper test decides each one; a
|
|
40
|
+
survivor owns a real boundary, invariant, composition, translation, lifecycle, or
|
|
41
|
+
materially narrower contract. Otherwise use or rename the real symbol and update every
|
|
42
|
+
consumer.
|
|
30
43
|
|
|
31
|
-
|
|
32
|
-
2. Pure, self-contained computation: exported centralized helper/parser/compiler/etc.
|
|
33
|
-
3. Defining recursive/compositional engine spine: class method after extracting its pure leaves.
|
|
34
|
-
4. Trivial, genuinely one-use expression: inline it.
|
|
35
|
-
|
|
36
|
-
Never move logic into a nested function to evade centralization. Anonymous callbacks passed directly to another operation remain callbacks, not hidden helper declarations.
|
|
37
|
-
|
|
38
|
-
## Remove superfluous wrappers
|
|
39
|
-
|
|
40
|
-
Search callers and callees for:
|
|
41
|
-
|
|
42
|
-
- one-line helper delegates;
|
|
43
|
-
- pass-through factories;
|
|
44
|
-
- getters that merely rename another public getter;
|
|
45
|
-
- duplicate guards/parsers already supplied by a declared dependency;
|
|
46
|
-
- compatibility aliases and re-exports;
|
|
47
|
-
- functions whose only purpose is avoiding a downstream rename.
|
|
48
|
-
|
|
49
|
-
A wrapper survives only if it owns a boundary, invariant, composition, translation, lifecycle, or materially narrower contract. Otherwise use or rename the real symbol and update every consumer.
|
|
50
|
-
|
|
51
|
-
Do not hollow a class into public methods that each forward to one helper. Keep meaningful orchestration on the entity and export only pure leaves.
|
|
44
|
+
Do not hollow a class into public methods that each forward to one helper.
|
|
52
45
|
|
|
53
46
|
## Consolidate tests
|
|
54
47
|
|
|
55
|
-
Sweep test files for repeated or reusable
|
|
56
|
-
|
|
57
|
-
-
|
|
58
|
-
|
|
59
|
-
- recorders and event capture;
|
|
60
|
-
- wait/readiness helpers;
|
|
61
|
-
- temporary workspace or fixture-server setup;
|
|
62
|
-
- browser/DOM builders and event factories;
|
|
63
|
-
- service request builders and response assertions.
|
|
64
|
-
|
|
65
|
-
Move host-independent helpers to `tests/setup.ts`; Node helpers to `tests/setupServer.ts`; browser helpers to `tests/setupBrowser.ts`; styles helpers to `tests/setupStyles.ts`; and live-service helpers to that project's dedicated setup.
|
|
66
|
-
|
|
67
|
-
Use customizable factories and inert stubs for data shapes. A scripted boundary stub may implement the real interface/protocol minimally to drive the system under test, but must not reproduce project-owned behavior or replace the integration being claimed. Otherwise use the real implementation, a temporary resource, a protocol-faithful fixture server, or the real external service.
|
|
68
|
-
|
|
69
|
-
Prefer recorders over spies. Do not use mocks or fakes.
|
|
48
|
+
Sweep test files for repeated or reusable input and result records, builders and
|
|
49
|
+
factories, recorders and event capture, wait and readiness helpers, temporary workspace or
|
|
50
|
+
fixture-server setup, browser and DOM builders, event factories, and service request
|
|
51
|
+
builders or response assertions. Move each into the setup file its environment owns.
|
|
70
52
|
|
|
71
|
-
Add focused tests for every exported function extracted from production code. Do not create
|
|
53
|
+
Add focused tests for every exported function extracted from production code. Do not create
|
|
54
|
+
isolated tests for declaration-only types, constants, barrels, or error definitions.
|
|
72
55
|
|
|
73
|
-
##
|
|
56
|
+
## Prove the sweep
|
|
74
57
|
|
|
75
58
|
Before acceptance, prove:
|
|
76
59
|
|
|
77
60
|
- no stray declarations remain in implementation files;
|
|
78
61
|
- no non-exported or wrong-kind declaration remains in centralized files;
|
|
79
|
-
- no prohibited nested function declaration
|
|
62
|
+
- no prohibited nested function declaration or assignment remains;
|
|
80
63
|
- no duplicate or rename-only wrapper remains;
|
|
81
64
|
- every move updated imports, barrels, guides, and tests;
|
|
82
65
|
- test helpers are consolidated without over-generalizing one-off setup;
|
|
83
|
-
- files are valid UTF-8
|
|
66
|
+
- files are valid UTF-8 with no replacement characters, mojibake, unintended control
|
|
67
|
+
characters, or accidental trailing debris.
|
|
84
68
|
|
|
85
|
-
Review the complete diff after formatting. Formatting cannot substitute for the structural
|
|
69
|
+
Review the complete diff after formatting. Formatting cannot substitute for the structural
|
|
70
|
+
sweep.
|
|
@@ -21,12 +21,20 @@ For each applicable seam, state the invariant, failure mode, observable result,
|
|
|
21
21
|
|
|
22
22
|
Test through public behavior with:
|
|
23
23
|
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
-
|
|
29
|
-
|
|
24
|
+
| Seam | The real thing it must be driven with |
|
|
25
|
+
| --------------------------------------- | ------------------------------------------------------------------------ |
|
|
26
|
+
| Package behavior | Real package classes and composed managers |
|
|
27
|
+
| Filesystem | Temporary directories and files |
|
|
28
|
+
| Network protocol | Actual local sockets and protocol-faithful fixture servers |
|
|
29
|
+
| Browser | An actual Chromium-family browser when the browser is the system |
|
|
30
|
+
| Declared dependency | The actual installed dependency |
|
|
31
|
+
| External service or model | The real service in its dedicated project when its behavior is the claim |
|
|
32
|
+
| Third-party client or protocol consumer | One representative real foreign client, driven end to end |
|
|
33
|
+
|
|
34
|
+
A third-party seam is the one case where the package's own tests cannot close the claim:
|
|
35
|
+
protocol tests prove the protocol, and only a real client of that class proves the
|
|
36
|
+
integration. Drive it end to end, record the exact commands, and record what that client
|
|
37
|
+
could NOT reach as an honest limit rather than an untested assumption.
|
|
30
38
|
|
|
31
39
|
Never use mocks, behavioral fakes, module replacement, or framework spies. Use recorders for callbacks and customizable data factories/stubs for inert shapes.
|
|
32
40
|
|
|
@@ -0,0 +1,113 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: orkestrel-polish-surface
|
|
3
|
+
description: Drive a rendered or externally driven surface to shipped quality through capture-evidence verdict rounds. Use when asked to polish an interface, bring a rendered surface to enterprise grade, judge what actually renders rather than what the source claims, reconcile design, state-truth, and inventory findings into fix units, or converge repeated review rounds on captured proof. Run one round for a narrow request; run the full campaign for a polish or production-readiness request.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Polish a rendered surface
|
|
7
|
+
|
|
8
|
+
## Load authority
|
|
9
|
+
|
|
10
|
+
Read the current files in this order:
|
|
11
|
+
|
|
12
|
+
1. `AGENTS.md`.
|
|
13
|
+
2. Every applicable `.claude/rules/*.md`. The style, browser, test, and documentation
|
|
14
|
+
laws bind every fix unit; this skill adds only the campaign.
|
|
15
|
+
3. [capture-harness.md](references/capture-harness.md) before building, eyeballing, or
|
|
16
|
+
trusting a portfolio.
|
|
17
|
+
4. `guides/README.md`, the governing guide for the surface, and `ROADMAP.md` when present.
|
|
18
|
+
5. The authoritative `*/types.ts` for the surface, its components and partials, and the
|
|
19
|
+
shipped resolved cascade of every stylesheet the surface actually loads.
|
|
20
|
+
|
|
21
|
+
Treat the current user instruction as authoritative. Treat repository rules as the coding
|
|
22
|
+
contract and this skill as the workflow. Preserve dirty and user-owned work.
|
|
23
|
+
|
|
24
|
+
## Judge the rendering, not the source
|
|
25
|
+
|
|
26
|
+
A claim about a rendered surface is proven by capture, never by reading the code that was
|
|
27
|
+
supposed to produce it. Source-reading review passes a component that renders nothing.
|
|
28
|
+
|
|
29
|
+
- The portfolio IS the review input: captures at both viewports and both themes, an
|
|
30
|
+
accessibility snapshot, and an interaction log.
|
|
31
|
+
- Source is corroboration for a mechanism, never the proof that the surface shows it.
|
|
32
|
+
- A claim the portfolio cannot show is unproven, not passed. Say so.
|
|
33
|
+
- The same law governs an externally driven surface: a compatibility claim is proven by
|
|
34
|
+
driving it with a representative real client, not by its own tests.
|
|
35
|
+
|
|
36
|
+
## Select the scope
|
|
37
|
+
|
|
38
|
+
Choose the smallest scope that satisfies the request:
|
|
39
|
+
|
|
40
|
+
- **Verdict:** one portfolio, one lane, no fixes.
|
|
41
|
+
- **Round:** portfolio → three independent verdicts → reconciliation → fix units →
|
|
42
|
+
recapture.
|
|
43
|
+
- **Campaign:** rounds repeated until one terminal converged line, then independent gates.
|
|
44
|
+
|
|
45
|
+
A narrow verdict request does not authorize fixes. A polish request authorizes fixing every
|
|
46
|
+
confirmed finding in scope and rebuilding the harness gaps the verdicts expose.
|
|
47
|
+
|
|
48
|
+
## Execute the campaign
|
|
49
|
+
|
|
50
|
+
1. **Build the portfolio.** Produce the full evidence set with the harness reference, then
|
|
51
|
+
eyeball every artifact yourself before spending a verdict round on it. An unexamined
|
|
52
|
+
portfolio buys harness bugs at verdict prices.
|
|
53
|
+
2. **Seed candidates.** Turn your own mid-integration observations into numbered
|
|
54
|
+
confirm-or-refute candidates inside the verdict brief. Observations that stay in your
|
|
55
|
+
head are neither evidence nor findings.
|
|
56
|
+
3. **Take three independent verdicts** on the SAME portfolio, in the fixed shape below, no
|
|
57
|
+
lane seeing another's answer before all have returned. The three work classes are
|
|
58
|
+
subjective design fit; objective state truth; and mechanical inventory of copy, classes,
|
|
59
|
+
icons, and accessibility attributes.
|
|
60
|
+
4. **Reconcile.** Confirmed findings get carriers; refuted findings die on the record with
|
|
61
|
+
the evidence that killed them; conflicts get a dated ruling. Then walk the reconciled
|
|
62
|
+
list once and name the fix-brief item carrying each finding — a finding with no carrier
|
|
63
|
+
is a dropped finding, and it will cost the next round.
|
|
64
|
+
5. **Dispatch serialized fix units,** objective lane first because it changes the ground
|
|
65
|
+
truth the subjective lane reads. Each unit records the failing proof before the
|
|
66
|
+
repair — the exact command and its red count — and the same command's green count
|
|
67
|
+
after. Commit each accepted unit before dispatching the next.
|
|
68
|
+
6. **Cross-audit.** A fix unit's auditor is the engine that did not write it. Same-engine
|
|
69
|
+
re-review returns the author's own blind spot.
|
|
70
|
+
7. **Recapture,** repairing every harness gap the verdicts exposed, and re-verdict against
|
|
71
|
+
the new portfolio, not the old one.
|
|
72
|
+
8. **Iterate to the terminal line.** Repeat rounds until every lane returns converged.
|
|
73
|
+
9. **Verify independently.** An independent runner executes the repository-prescribed
|
|
74
|
+
gates; a fixer's own report never establishes green.
|
|
75
|
+
|
|
76
|
+
## Fix the treatment, not the symptom
|
|
77
|
+
|
|
78
|
+
- Resolve every visual treatment in the shipped cascade before writing it; a dependency's
|
|
79
|
+
own default rules decide what a bare element renders as.
|
|
80
|
+
- Prove a defect with a test that fails for that defect before the repair exists.
|
|
81
|
+
- Do not narrow a selector, add a local exception, or restyle one instance to make a
|
|
82
|
+
capture pass while the underlying rule stays wrong.
|
|
83
|
+
- When a law of the repository genuinely conflicts with the surface's required anatomy,
|
|
84
|
+
stop and report both horns with evidence. An improvised local exception poisons an
|
|
85
|
+
enforced invariant.
|
|
86
|
+
|
|
87
|
+
## Return the fixed verdict shape
|
|
88
|
+
|
|
89
|
+
Free-form verdicts cost one reconciliation pass each. Every verdict returns:
|
|
90
|
+
|
|
91
|
+
- one line per item, in the brief's numbering:
|
|
92
|
+
- `RENDERED-PROVEN(<capture>)` — the artifact shows it;
|
|
93
|
+
- `REGRESSED(<capture>, <what regressed>)` — the artifact shows it is now worse;
|
|
94
|
+
- `NOT-EVIDENCED(<what the portfolio lacks>)` — the portfolio cannot decide it;
|
|
95
|
+
- referrals to another lane, each with the specific evidence and no verdict attached;
|
|
96
|
+
- exactly one final line: `CONVERGED` or `ANOTHER ROUND(<item list>)`.
|
|
97
|
+
|
|
98
|
+
Treat every `NOT-EVIDENCED` line as a harness question first and a product defect second.
|
|
99
|
+
Triage it against the capture reference before it becomes a finding.
|
|
100
|
+
|
|
101
|
+
## Accept the result
|
|
102
|
+
|
|
103
|
+
Do not accept a source-only verdict, a self-audited fix unit, a round without a terminal
|
|
104
|
+
line, or a green claim from the engine that wrote the code. Completion requires:
|
|
105
|
+
|
|
106
|
+
- every confirmed finding fixed or explicitly ruled out of scope on the record;
|
|
107
|
+
- a final portfolio whose captures show the accepted state;
|
|
108
|
+
- red-then-green evidence for every defect unit;
|
|
109
|
+
- every lane converged in one round against that final portfolio;
|
|
110
|
+
- independent gate evidence.
|
|
111
|
+
|
|
112
|
+
Report what changed, which capture proves each change, the exact test and gate results, and
|
|
113
|
+
any residual risk the portfolio genuinely cannot settle.
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
# The capture harness
|
|
2
|
+
|
|
3
|
+
The harness produces the only evidence the verdict lanes are allowed to judge. It is owned
|
|
4
|
+
by the campaign owner, not by a review lane, and it is a throwaway instrument: written for
|
|
5
|
+
this surface, kept honest, deleted or rebuilt when the surface changes.
|
|
6
|
+
|
|
7
|
+
## One call, one lifecycle
|
|
8
|
+
|
|
9
|
+
Background processes started inside one tool call die with that call's process group, and a
|
|
10
|
+
verdict round spent on a half-dead harness is a wasted round.
|
|
11
|
+
|
|
12
|
+
- Write the harness as one self-contained script that spawns its own children, waits for
|
|
13
|
+
readiness, does the capture, and kills them before returning.
|
|
14
|
+
- Never leave a child running across calls or expect one to survive its parent.
|
|
15
|
+
- Give every child a pinned working directory: a process that resolves assets, config, or
|
|
16
|
+
fixtures relative to the current directory dies silently when launched from elsewhere.
|
|
17
|
+
- Pipe child standard error somewhere readable and print it on failure. A discarded stream
|
|
18
|
+
turns a one-line configuration refusal into a debugging round.
|
|
19
|
+
- Wait on an observable readiness signal — a served response, a printed line, a health
|
|
20
|
+
probe — never on a fixed sleep.
|
|
21
|
+
- Tear down on every exit path, including assertion and setup failure, so a failed capture
|
|
22
|
+
leaves no orphaned server, browser, or port.
|
|
23
|
+
|
|
24
|
+
## Validate the seed before capturing
|
|
25
|
+
|
|
26
|
+
Most "the surface is broken" verdicts trace back to a seed the surface legitimately
|
|
27
|
+
refused.
|
|
28
|
+
|
|
29
|
+
- Build seed payloads from the surface's own published contract, not from memory of it: a
|
|
30
|
+
near-miss field name produces an empty screen that looks exactly like a product defect.
|
|
31
|
+
- Assert the seeded state is present before shooting: the row exists, the prompt is parked,
|
|
32
|
+
the list is non-empty.
|
|
33
|
+
- Drive the surface through its real entry path so the captured state is one a user can
|
|
34
|
+
actually reach.
|
|
35
|
+
- Reset to a known state between scenarios; a capture that inherits the previous scenario's
|
|
36
|
+
selection, focus, or scroll proves nothing about either.
|
|
37
|
+
|
|
38
|
+
## Capture the full portfolio
|
|
39
|
+
|
|
40
|
+
Every round produces all of it, for every scenario in scope:
|
|
41
|
+
|
|
42
|
+
| Artifact | Requirement |
|
|
43
|
+
| ---------------------- | -------------------------------------------------------------------------------------- |
|
|
44
|
+
| Viewport captures | The narrow and wide breakpoints the surface actually declares, not one convenient size |
|
|
45
|
+
| Theme captures | Every theme the surface ships, each at both viewports |
|
|
46
|
+
| Accessibility snapshot | The rendered accessible tree: roles, names, states, and focus order |
|
|
47
|
+
| Interaction log | Each scripted interaction, its trigger, and the observed result |
|
|
48
|
+
| Console and error log | Anything the page or process emitted during the run |
|
|
49
|
+
|
|
50
|
+
- Shoot the whole surface before selecting or focusing anything inside it; a capture taken
|
|
51
|
+
after a selection reports a duplicate or highlighted artifact that does not exist.
|
|
52
|
+
- Start a keyboard walk from a neutral state, never from an already-focused control, or the
|
|
53
|
+
log will "prove" a broken order the user never sees.
|
|
54
|
+
- Name artifacts so a verdict can cite one exactly: scenario, viewport, theme, step.
|
|
55
|
+
- Keep the artifacts of each round beside its verdicts; a round judged against the previous
|
|
56
|
+
round's captures is not a round.
|
|
57
|
+
|
|
58
|
+
## Preflight before spending a round
|
|
59
|
+
|
|
60
|
+
The campaign owner opens every artifact before dispatching a verdict lane:
|
|
61
|
+
|
|
62
|
+
- each capture shows the scenario it claims, in the theme and viewport it claims;
|
|
63
|
+
- the seeded state is visible;
|
|
64
|
+
- the accessibility snapshot is non-empty and matches the captured screen;
|
|
65
|
+
- the interaction log records the interactions the brief asked for;
|
|
66
|
+
- nothing in the console log indicates the harness, rather than the surface, failed.
|
|
67
|
+
|
|
68
|
+
A portfolio that fails preflight is repaired before dispatch. A verdict round is the
|
|
69
|
+
most expensive way to discover a harness bug.
|
|
70
|
+
|
|
71
|
+
## Triage missing evidence to the harness first
|
|
72
|
+
|
|
73
|
+
When a verdict returns a not-evidenced item, the harness is the first suspect and the
|
|
74
|
+
surface is the second. In order:
|
|
75
|
+
|
|
76
|
+
1. Confirm the artifact that should decide the item exists and is named as the brief said.
|
|
77
|
+
2. Confirm the scenario reached the state the item is about.
|
|
78
|
+
3. Confirm the seed and the entry path match the surface's real contract.
|
|
79
|
+
4. Only then treat it as a product finding.
|
|
80
|
+
|
|
81
|
+
Every harness gap a round exposes is repaired before the recapture, and the repair is
|
|
82
|
+
recorded with the round so the next portfolio is strictly better than the last.
|
|
@@ -16,6 +16,8 @@ You are an Executor: do the work yourself, spawn nothing.
|
|
|
16
16
|
|
|
17
17
|
- Before writing, read **AGENTS.md**, every applicable `.claude/rules/*.md`, the
|
|
18
18
|
dispatch-named skill and required references, and the governing guide/spec. All bind you.
|
|
19
|
+
An app-layer unit additionally binds `.claude/rules/application.md` and
|
|
20
|
+
`.claude/rules/workspace.md`.
|
|
19
21
|
- Write ONLY the owned files named in your dispatch. Shared or off-limits files are
|
|
20
22
|
report-only: if one needs a change, RETURN the exact patch — never edit it.
|
|
21
23
|
- NO tree-wide or mutating commands: never `format`, lint `--fix`, or `build`.
|
|
@@ -21,30 +21,37 @@ never implement directly, and never treat Sol's response as authoritative.
|
|
|
21
21
|
`tmp/codex/<unit>.session` — an interrupted MCP call whose id was never written
|
|
22
22
|
to disk is unrecoverable, and that whole exchange is then treated as failed.
|
|
23
23
|
- **Long-running work** (audits, implementation units, anything multi-minute):
|
|
24
|
-
the journaled CLI is MANDATORY
|
|
25
|
-
is one interruption away from losing the session
|
|
24
|
+
the journaled CLI is MANDATORY, the MCP tools are forbidden, and YOU DO NOT
|
|
25
|
+
LAUNCH IT. A long MCP call is one interruption away from losing the session
|
|
26
|
+
invisibly; a backgrounded exec you start and walk away from has no owner, no
|
|
27
|
+
completion signal, and no death notice. Prepare it and hand it back.
|
|
26
28
|
|
|
27
|
-
##
|
|
29
|
+
## Prepare the journaled CLI launch
|
|
28
30
|
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
`codex exec --json --sandbox <route-sandbox> --model gpt-5.6-sol -c "model_reasoning_effort=\"high\"" --output-last-message tmp/codex/<unit>-last.md "Read and execute the brief at tmp/codex/<unit>-brief.md exactly. Your final message must be the report it specifies." > tmp/codex/<unit>.jsonl`
|
|
31
|
+
Your two jobs are drafting the brief and short MCP exchanges. For long work you
|
|
32
|
+
prepare the launch and return it; the Orchestrator runs it as a harness-tracked
|
|
33
|
+
background command under a hard cap.
|
|
33
34
|
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
-
|
|
39
|
-
|
|
35
|
+
Create `tmp/codex/` first. Write the full brief to `tmp/codex/<unit>-brief.md` —
|
|
36
|
+
briefs never travel as shell arguments — then return the exact resolved command
|
|
37
|
+
with a pointer prompt:
|
|
38
|
+
|
|
39
|
+
`timeout <cap> codex exec --json -C <working-directory> --sandbox <route-sandbox> --model gpt-5.6-sol -c "model_reasoning_effort=\"high\"" --output-last-message tmp/codex/<unit>-last.md "Read and execute the brief at tmp/codex/<unit>-brief.md exactly. Your final message must be the report it specifies." < /dev/null > tmp/codex/<unit>.jsonl`
|
|
40
|
+
|
|
41
|
+
- Return four things: the brief path, that resolved command, the journal path,
|
|
42
|
+
and a cap recommendation with its basis — the observed duration high-mark for
|
|
43
|
+
this work class, plus an independently budgeted gate allowance, plus explicit
|
|
44
|
+
slack. Never launch, background, poll, sleep-loop, restart, or kill an exec.
|
|
45
|
+
- Keep `< /dev/null`: a background-launched exec that inherits an open stdin pipe
|
|
46
|
+
wedges before its first event and only the cap ever surfaces it. Add
|
|
47
|
+
`--skip-git-repo-check` when the working directory is outside a trusted git
|
|
48
|
+
repository, and `--output-schema <file>` when the Orchestrator supplies one.
|
|
40
49
|
- The journal at `tmp/codex/<unit>.jsonl` is the live progress record (the user
|
|
41
50
|
tails it) and its mtime is the liveness signal the Orchestrator watches. Never
|
|
42
51
|
re-print the stream into your report.
|
|
43
|
-
-
|
|
44
|
-
-
|
|
45
|
-
report.
|
|
46
|
-
- When the Orchestrator supplies a JSON Schema for the return shape, pass it with
|
|
47
|
-
`--output-schema <file>`.
|
|
52
|
+
- When the Orchestrator hands back a finished exec, read Sol's answer from the
|
|
53
|
+
`--output-last-message` file, not from stdout, and record the session id
|
|
54
|
+
(`thread_id` in the journal's opening events) in every report.
|
|
48
55
|
|
|
49
56
|
## Recovery ladder
|
|
50
57
|
|
|
@@ -53,8 +60,8 @@ On any interruption or missing result, in order:
|
|
|
53
60
|
1. Interrupted MCP call WITH a persisted thread id → `mcp__codex__codex-reply`
|
|
54
61
|
asking Sol to re-emit the complete final report (the reasoning may have
|
|
55
62
|
finished server-side).
|
|
56
|
-
2. No persisted id, or the reply fails → fresh journaled CLI
|
|
57
|
-
same brief file.
|
|
63
|
+
2. No persisted id, or the reply fails → prepare a fresh journaled CLI launch
|
|
64
|
+
with the same brief file and return it.
|
|
58
65
|
3. Interrupted CLI exec → the journal survives; report the thread id and the last
|
|
59
66
|
journal events as a deviation and let the Orchestrator choose resume or fresh.
|
|
60
67
|
|
|
@@ -74,12 +81,12 @@ unsupported claims as dropped.
|
|
|
74
81
|
|
|
75
82
|
Sandbox `workspace-write`, main checkout, sole serial writer from a clean
|
|
76
83
|
committed baseline with owned files, off-limits files, and a deviation contract.
|
|
77
|
-
When the exec
|
|
78
|
-
scoped validation) and report once, completely. The
|
|
79
|
-
installation, commits, pushes, publishing, credentials,
|
|
80
|
-
shared-file edits, and tree-wide mutating gates. Return the
|
|
81
|
-
diffstat, scoped validation, and deviation state for independent
|
|
82
|
-
review.
|
|
84
|
+
When the Orchestrator hands the finished exec back, verify the result with direct
|
|
85
|
+
evidence (git status, diff, scoped validation) and report once, completely. The
|
|
86
|
+
brief forbids dependency installation, commits, pushes, publishing, credentials,
|
|
87
|
+
destructive commands, shared-file edits, and tree-wide mutating gates. Return the
|
|
88
|
+
touched files, diffstat, scoped validation, and deviation state for independent
|
|
89
|
+
integration and review.
|
|
83
90
|
|
|
84
91
|
## Routing exclusion — defensive negative-test units
|
|
85
92
|
|
|
@@ -30,6 +30,13 @@ on disk:
|
|
|
30
30
|
|
|
31
31
|
`<agent-cli> -p --trust --mode=ask --model "$CURSOR_GROK_MODEL" "<brief or pointer>" | tee tmp/cursor/<unit>.log`
|
|
32
32
|
|
|
33
|
+
Run that yourself only for a short bounded ask that finishes in about two minutes.
|
|
34
|
+
For anything longer your job ends at drafting: return the brief path, the exact
|
|
35
|
+
resolved command, the journal path, and a cap recommendation — the observed
|
|
36
|
+
duration high-mark plus explicit slack — and let the Orchestrator launch it as a
|
|
37
|
+
harness-tracked background command. Never detach a run and end your turn: an
|
|
38
|
+
unowned run has no completion signal and no death notice.
|
|
39
|
+
|
|
33
40
|
The brief must say read-only, name the evidence sought, require file:line
|
|
34
41
|
pointers, and forbid raw file dumps, decisions, design, and edits. Never use
|
|
35
42
|
`--force`, expose `CURSOR_API_KEY`, inspect unrelated environment values, or read
|
|
@@ -22,7 +22,8 @@ dispatched unit. You are an Executor: do the work yourself, spawn nothing.
|
|
|
22
22
|
criteria, and a deviation contract. Write ONLY owned files; shared or off-limits
|
|
23
23
|
files are report-only — return exact patches, never edit them.
|
|
24
24
|
- TTTDD: types first, then a failing test reproducing each finding, then the fix,
|
|
25
|
-
then green.
|
|
25
|
+
then green. For a defect unit, report the exact command and its failing count
|
|
26
|
+
before the fix and the same command's passing count after.
|
|
26
27
|
- Never add dependencies, suppress diagnostics, use mocks, leave current-scope
|
|
27
28
|
deferrals, commit, push, publish, install, read secrets, or run destructive
|
|
28
29
|
commands or tree-wide mutating gates. Validate read-only and scoped to owned
|