solmate-skills 2.0.12 → 2.0.13

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. package/AGENTS.md +9 -0
  2. package/CLAUDE.md +13 -3
  3. package/README.md +91 -5
  4. package/USAGE.md +118 -12
  5. package/bin/cli.js +171 -1
  6. package/bin/harness-artifact.js +621 -0
  7. package/bin/harness-artifact.test.js +520 -0
  8. package/bin/harness-check.js +373 -0
  9. package/bin/harness-check.test.js +159 -0
  10. package/bin/test.js +2 -0
  11. package/docs/01_Concept_Design/01_AGENT_HARNESS_REQUIREMENTS_ANALYSIS.md +162 -0
  12. package/docs/03_Technical_Specs/01_AGENT_HARNESS_ARCHITECTURE.md +376 -0
  13. package/docs/03_Technical_Specs/assets/01_agent_harness_data_flow.svg +94 -0
  14. package/docs/04_Logic_Progress/00_BACKLOG.md +339 -0
  15. package/docs/04_Logic_Progress/01_EXECUTION_PLAN.md +160 -0
  16. package/docs/04_Logic_Progress/03_DECISION_LOG.md +98 -0
  17. package/docs/05_QA_Validation/01_TEST_SCENARIOS.md +313 -0
  18. package/docs/05_QA_Validation/02_AGENT_HARNESS_DESIGN_REVIEW.md +100 -0
  19. package/docs/05_QA_Validation/03_AGENT_HARNESS_CONTRACT_QA.md +96 -0
  20. package/manage-collaboration/SKILL.md +2 -0
  21. package/package.json +6 -4
  22. package/rules-docs/SKILL.md +15 -1
  23. package/rules-product/SKILL.md +30 -6
  24. package/rules-product/agents/openai.yaml +2 -2
  25. package/rules-workflow/SKILL.md +32 -7
  26. package/rules-workflow/adapters/claude/solmate-context-reader.md +17 -0
  27. package/rules-workflow/adapters/claude/solmate-implementer.md +20 -0
  28. package/rules-workflow/adapters/claude/solmate-verifier.md +21 -0
  29. package/rules-workflow/agents/openai.yaml +2 -2
  30. package/rules-workflow/resources/agent-harness-contract.md +187 -0
  31. package/rules-workflow/resources/agent-harness-v1.schema.json +432 -0
  32. package/verify-docs/SKILL.md +33 -6
  33. package/verify-docs/agents/openai.yaml +2 -2
  34. package/verify-implementation/SKILL.md +14 -2
  35. package/verify-implementation/agents/openai.yaml +2 -2
  36. package/verify-skills/SKILL.md +27 -0
@@ -0,0 +1,313 @@
1
+ # Agent Harness Test Scenarios
2
+ > Created: 2026-07-17 01:04
3
+ > Last Updated: 2026-07-17 01:53
4
+
5
+ ## 1. Purpose
6
+
7
+ This document defines design-stage acceptance and future implementation scenarios for the enhanced Solmate agent harness. A scenario is not considered passing until independent evidence identifies the command, fixture, result, and relevant requirement.
8
+
9
+ ## 2. Global Rubric Validation
10
+
11
+ | Criterion | Status | Evidence / Design Response |
12
+ |---|---|---|
13
+ | Functionality | Pass at design stage | Requirements, architecture, state transitions, messages, errors, execution plan, and tests are defined; implementation remains pending |
14
+ | Potential Impact | Pass at design stage | The design targets the recurring failures of unread context, improvised workflows, conflicting agents, and unsupported completion claims |
15
+ | Novelty | Pass at design stage | Solmate combines role/topology design with machine-enforced document and independent-verification Receipts rather than relying on personas alone |
16
+ | UX | Pass at design stage | Minimal topology is the default; optional specialists and warning rollout limit procedural and latency overhead |
17
+ | Open-source | Pass at design stage | Runtime-neutral contracts, standard JSON/JSONL, and thin adapters allow reuse without a proprietary orchestration service |
18
+ | Business Plan | Pass at design stage | Reliable Claude/Codex workflow behavior strengthens the package's value and reduces support cost from inconsistent agent execution |
19
+
20
+ Implementation-stage status must replace these design-stage statements with executable evidence.
21
+
22
+ ## 3. Requirement Traceability Matrix
23
+
24
+ | Requirement | Architecture Coverage | Primary Scenario | Design Status |
25
+ |---|---|---|---|
26
+ | FR-001 | Artifact Model, Receipt Integration | QA-REQ-001 | Covered |
27
+ | FR-002 | Role Catalog, Activation Rules | QA-ROLE-001 | Covered |
28
+ | FR-003 | Persona Contract, Permissions | QA-ROLE-002 | Covered |
29
+ | FR-004 | Receipt Integration, Compatibility | QA-COMPAT-001 | Covered |
30
+ | FR-005 | Optional Specialists, State Machine | QA-DESIGN-001 | Covered |
31
+ | FR-006 | Message Protocol | QA-SCHEMA-001, QA-MSG-001, QA-MSG-002 | Covered |
32
+ | FR-007 | State Machine, Error Handling | QA-STATE-001, QA-STATE-002 | Covered |
33
+ | FR-008 | Permissions, QA Role | QA-QA-001, QA-QA-002 | Covered |
34
+ | FR-009 | Error Handling | QA-ERR-001 through QA-ERR-006 | Covered |
35
+ | FR-010 | Runtime Adapters | QA-ADAPTER-001 | Covered |
36
+ | FR-011 | Execution Workspace | QA-RECOVERY-001, QA-RECOVERY-002 | Covered |
37
+ | FR-012 | Requirements Traceability | QA-TRACE-001 | Covered |
38
+ | FR-013 | Topology Selection | QA-TOPO-001 through QA-TOPO-003 | Covered |
39
+ | FR-014 | Compatibility And Maintenance | QA-DRIFT-001 | Covered |
40
+ | FR-015 | Warning Pilot | QA-PILOT-001 | Covered |
41
+
42
+ ### 3.1 Non-Functional Requirement Traceability
43
+
44
+ | Requirement | Primary Scenario | Design Status |
45
+ |---|---|---|
46
+ | NFR-001 | QA-SEC-001, QA-ERR-006 | Covered |
47
+ | NFR-002 | QA-STATE-001, QA-RECOVERY-002 | Covered |
48
+ | NFR-003 | QA-ADAPTER-001 | Covered |
49
+ | NFR-004 | QA-ROLE-001, QA-TOPO-001 | Covered |
50
+ | NFR-005 | QA-COST-001, QA-PILOT-001 | Covered |
51
+ | NFR-006 | QA-SCHEMA-001, QA-MSG-001, QA-STATE-002 | Covered |
52
+ | NFR-007 | QA-COMPAT-001 | Covered |
53
+ | NFR-008 | QA-QA-001, QA-QA-002 | Covered |
54
+ | NFR-009 | QA-RECOVERY-001, QA-RECOVERY-002 | Covered |
55
+ | NFR-010 | QA-LICENSE-001 | Covered |
56
+
57
+ ## 4. Requirements And Context Scenarios
58
+
59
+ ### QA-REQ-001 - Requirements Artifact Completeness
60
+
61
+ - **Given**: A new feature request with product behavior and implementation expectations.
62
+ - **When**: Requirements Analyst prepares the canonical analysis.
63
+ - **Then**: The document contains user need, FR, NFR, included/excluded scope, acceptance criteria, assumptions, unresolved questions, risks, and related-document links.
64
+ - **Failure**: Implementation begins from chat text without the required artifact or explicit small-task skip decision.
65
+ - **Evidence**: Parsed document fields and Coordinator acceptance event.
66
+
67
+ ### QA-CONTEXT-001 - Linked Document Read Proof
68
+
69
+ - **Given**: A code task links Concept, Technical, and QA documents.
70
+ - **When**: Context Reader returns a Context Receipt.
71
+ - **Then**: Every linked path exists and appears in `Required References Read`; constraints and conflicts are present.
72
+ - **Failure**: Any required document is missing, unread, or silently omitted.
73
+ - **Evidence**: Existing strict `preflight` command plus manifest references.
74
+
75
+ ### QA-CONTEXT-002 - Conflicting Documents
76
+
77
+ - **Given**: Two linked documents define incompatible behavior.
78
+ - **When**: Context Reader compares them.
79
+ - **Then**: State becomes `BLOCKED_CONTEXT`; both sources are preserved and Coordinator routes a decision.
80
+ - **Failure**: Context Reader chooses one silently or Implementer starts work.
81
+
82
+ ## 5. Role And Design Scenarios
83
+
84
+ ### QA-ROLE-001 - Minimal Task Avoids Optional Specialists
85
+
86
+ - **Given**: A small, single-module change with complete requirements and no public contract or state-machine change.
87
+ - **When**: Coordinator selects topology.
88
+ - **Then**: Core linear path is used; Requirements Analyst, Architect, Logic Builder, Security Inspector, and Release Guardian are recorded as `Skipped - reason` where considered.
89
+ - **Failure**: Full specialist team is spawned only because multiple roles are available.
90
+
91
+ ### QA-ROLE-002 - Persona Contract Completeness
92
+
93
+ - **Given**: A runtime agent definition.
94
+ - **When**: Persona validation runs.
95
+ - **Then**: All persona fields are present, permissions match role responsibility, and denied tools are explicit for read-only roles.
96
+ - **Failure**: A narrative persona has no non-goals, output contract, escalation, completion evidence, or write scope.
97
+
98
+ ### QA-DESIGN-001 - Architect And Logic Builder Activation
99
+
100
+ - **Given**: A feature changes an API contract and introduces three business states with failure transitions.
101
+ - **When**: Coordinator evaluates specialists.
102
+ - **Then**: Architect and Logic Builder are both activated, with non-overlapping outputs and linked Design Receipt.
103
+ - **Failure**: Implementer invents the API or state behavior during coding.
104
+
105
+ ## 6. Message And State Scenarios
106
+
107
+ ### QA-SCHEMA-001 - Versioned Artifact Schema Selection
108
+
109
+ - **Given**: A v1 manifest, message, or individual event object.
110
+ - **When**: A standard Draft 2020-12 validator or the Solmate CLI selects the artifact contract.
111
+ - **Then**: Top-level `oneOf` selects exactly one artifact shape, `schema_version: 1` is required, and every non-empty JSONL line validates as an event before ordered-log checks run.
112
+ - **Failure**: The top-level schema accepts an unconstrained object, an unsupported version passes, or malformed JSONL is treated as a contract warning instead of an operational error.
113
+
114
+ ### QA-MSG-001 - Valid Structured Handoff
115
+
116
+ - **Given**: Architect completes an approved Architecture Decision Packet.
117
+ - **When**: A `HANDOFF` message is emitted.
118
+ - **Then**: Schema version, message/task IDs, attempt, timestamp, sender, recipient, status, summary, requirement/artifact/decision refs, blockers, next action, and retry count validate.
119
+ - **Failure**: A free-form “done” message is accepted without an artifact.
120
+
121
+ ### QA-MSG-002 - Direct Peer Decision Is Rejected
122
+
123
+ - **Given**: Frontend Implementer asks Backend Implementer to change an accepted API response.
124
+ - **When**: Backend replies directly with a scope-changing decision.
125
+ - **Then**: Direct peer messages are limited to `STATUS` or `QUESTION` with `INFO` or `PENDING`; PASS, FAIL, BLOCKED, decision, rework, and completion claims route through Coordinator.
126
+ - **Failure**: The API and UI drift without an updated requirement or decision.
127
+
128
+ ### QA-STATE-001 - Legal Transition Sequence
129
+
130
+ - **Given**: All required artifacts pass.
131
+ - **When**: Coordinator advances the task.
132
+ - **Then**: Every state follows a legal transition and records reason, actor, timestamp, and artifacts.
133
+ - **Evidence**: Ordered `events.jsonl` fixture.
134
+
135
+ ### QA-STATE-002 - Completion Bypass Is Rejected
136
+
137
+ - **Given**: A Change Receipt exists but independent verification has not passed.
138
+ - **When**: A role requests `CHANGE_READY -> COMPLETE`.
139
+ - **Then**: The transition is rejected as a contract error.
140
+ - **Failure**: PR, merge, publish, or deploy is reported ready.
141
+
142
+ ## 7. Error And Recovery Scenarios
143
+
144
+ ### QA-ERR-001 - Context Error Does Not Retry
145
+
146
+ - **Given**: A required technical document is missing.
147
+ - **Then**: State becomes `BLOCKED_CONTEXT`; no automatic retry or implementation occurs.
148
+
149
+ ### QA-ERR-002 - Transient Tool Error Retries Once
150
+
151
+ - **Given**: An otherwise valid non-destructive command fails from a temporary runtime interruption.
152
+ - **Then**: One retry occurs in a new recorded attempt with unchanged scope.
153
+ - **Failure**: Retry changes requirements, deletes prior evidence, or loops indefinitely.
154
+
155
+ ### QA-ERR-003 - Repeated Transient Failure Blocks Dependency
156
+
157
+ - **Given**: The allowed retry fails again.
158
+ - **Then**: State becomes `BLOCKED_DEPENDENCY` and reports attempts, evidence, and recovery condition.
159
+
160
+ ### QA-ERR-004 - Verification Failure Routes Rework
161
+
162
+ - **Given**: QA finds a failed acceptance criterion.
163
+ - **Then**: QA emits a finding; Coordinator sets `REWORK`; Implementer creates a new attempt and Change Receipt.
164
+ - **Failure**: QA modifies source files or marks its own fix as verified.
165
+
166
+ ### QA-ERR-005 - Rework Loop Limit
167
+
168
+ - **Given**: Two implementation rework attempts fail the same criterion.
169
+ - **Then**: Coordinator pauses further changes and requests a user decision with the repeated failure pattern.
170
+ - **Failure**: A third automatic loop starts.
171
+
172
+ ### QA-ERR-006 - Critical Artifact Failure Cannot Degrade Silently
173
+
174
+ - **Given**: Architect, Implementer, or QA fails without producing a required critical artifact.
175
+ - **Then**: Code/deploy task remains blocked; final output explicitly names the missing artifact.
176
+ - **Failure**: Remaining partial results are used to report completion.
177
+
178
+ ### QA-RECOVERY-001 - Interrupted Task Resume
179
+
180
+ - **Given**: The active agent stops after valid requirements and context events.
181
+ - **When**: The task resumes.
182
+ - **Then**: Runtime loads the latest valid manifest and events, creates or resumes the legal attempt, and does not repeat approved work unnecessarily.
183
+
184
+ ### QA-RECOVERY-002 - Attempt Preservation
185
+
186
+ - **Given**: Attempt 1 fails and attempt 2 begins.
187
+ - **Then**: Attempt 1 artifacts and failure evidence remain immutable and manifest points to attempt 2 as active.
188
+
189
+ ## 8. Independent QA And Boundary Scenarios
190
+
191
+ ### QA-QA-001 - QA Write Attempt Is Denied
192
+
193
+ - **Given**: QA Inspector identifies a code defect.
194
+ - **When**: It attempts to use Edit or Write on implementation files.
195
+ - **Then**: Runtime permission or contract validation rejects the operation and records the violation.
196
+
197
+ ### QA-QA-002 - Implementer Cannot Self-Verify
198
+
199
+ - **Given**: Implementer reports basic checks as passing.
200
+ - **Then**: Change Receipt is accepted as a handoff but cannot satisfy Verification Receipt or completion.
201
+
202
+ ### QA-BOUNDARY-001 - Producer And Consumer Contract Comparison
203
+
204
+ - **Given**: A producer returns a structured response and a consumer declares its expected type.
205
+ - **Then**: QA reads both sides, compares shape, optionality, naming, and failure behavior, and links evidence.
206
+ - **Failure**: QA checks only that both files exist or only that compilation passes.
207
+
208
+ ### QA-BOUNDARY-002 - State Definition And Mutation Comparison
209
+
210
+ - **Given**: A Logic Contract defines legal transitions.
211
+ - **Then**: QA compares every implementation mutation and UI branch against reachable states and identifies unauthorized or dead transitions.
212
+
213
+ ### QA-TRACE-001 - End-To-End Requirement Evidence
214
+
215
+ - **Given**: A P0 functional requirement is marked complete.
216
+ - **When**: Traceability validation runs.
217
+ - **Then**: The requirement links to its approved Requirements Analysis, required Architecture or Logic artifact, Change Receipt or diff, and independent QA evidence.
218
+ - **Failure**: A requirement is marked `Pass` from implementation claims alone, or a required design artifact is absent without `Skipped - reason`.
219
+
220
+ ## 9. Topology And Adapter Scenarios
221
+
222
+ ### QA-TOPO-001 - Read-Only Fan-Out/Fan-In
223
+
224
+ - **Given**: Four independent document groups require analysis.
225
+ - **Then**: Read-only agents run in parallel, write no shared source, and Coordinator integrates artifact references and conflicts.
226
+
227
+ ### QA-TOPO-002 - Parallel Write Ownership
228
+
229
+ - **Given**: Frontend and Backend work can proceed in parallel.
230
+ - **Then**: Exclusive paths are canonical project-relative paths; aliases and overlapping literal or recursive scopes are rejected, while shared contracts are frozen or owned serially.
231
+ - **Failure**: Both agents modify the same shared file without coordination.
232
+
233
+ ### QA-TOPO-003 - Hierarchy Depth Limit
234
+
235
+ - **Given**: A proposed topology delegates more than two levels deep.
236
+ - **Then**: Coordinator flattens or rejects the topology unless the user approves a documented exception.
237
+
238
+ ### QA-ADAPTER-001 - Claude And Codex Contract Parity
239
+
240
+ - **Given**: The same fixture task runs through both adapters.
241
+ - **Then**: Role activation, canonical states, required artifact refs, failure classification, and Receipt fields are semantically equivalent.
242
+ - **Allowed Difference**: Native tool names, peer-message support, concurrency, and resume capability.
243
+
244
+ ### QA-ADAPTER-002 - Degraded Independent Agent Capability
245
+
246
+ - **Given**: A runtime cannot provide an independent QA execution context.
247
+ - **Then**: State becomes `DEGRADED`; user approval is required before any completion claim and the limitation remains visible.
248
+
249
+ ## 10. Compatibility, Drift, And Security Scenarios
250
+
251
+ ### QA-COMPAT-001 - Existing Receipt Fixtures Remain Valid
252
+
253
+ - **Given**: Current valid Context and Verification Receipt fixtures.
254
+ - **When**: The enhanced validator runs in compatibility mode.
255
+ - **Then**: Existing commands and exit codes remain valid; new fields are warning-only until migration.
256
+
257
+ ### QA-DRIFT-001 - Role And Adapter Drift Detection
258
+
259
+ - **Given**: A role is renamed or a runtime adapter omits a mandatory persona field.
260
+ - **Then**: Package verification identifies the stale reference, duplicate role, or missing adapter mapping.
261
+
262
+ ### QA-COST-001 - Agent And Model Cost Is Proportional
263
+
264
+ - **Given**: Equivalent small, medium, and complex task fixtures.
265
+ - **Then**: The small fixture uses the minimal role path; optional specialists and stronger model tiers appear only with a recorded task need.
266
+ - **Failure**: Every task uses the full team or globally forces the most expensive model tier.
267
+
268
+ ### QA-LICENSE-001 - External Reference Attribution And Originality
269
+
270
+ - **Given**: A new contract or persona is informed by an external open-source project.
271
+ - **Then**: Source references and applicable license obligations are reviewed, while Solmate text and schemas are independently authored unless copied material is explicitly attributed and licensed.
272
+ - **Failure**: Substantial external text is copied without notice, source, or license review.
273
+
274
+ ### QA-SEC-001 - Sensitive Workspace Content Rejected
275
+
276
+ - **Given**: A message or evidence artifact contains a credential, secret token, or prohibited raw personal data.
277
+ - **Then**: Persistence is rejected or redacted according to explicit policy and the task reports the blocked evidence.
278
+ - **Failure**: Sensitive content is written to a commit-ready workspace.
279
+
280
+ ## 11. Pilot And Release Scenarios
281
+
282
+ ### QA-PILOT-001 - Five-Task Warning Calibration
283
+
284
+ - **Given**: Five representative real code/deploy tasks.
285
+ - **Then**: Each records selected topology, activated/skipped roles, missing fields, false positives, retries, rework, user overrides, runtime gaps, and escaped findings.
286
+ - **Exit**: User reviews the pilot and explicitly approves or rejects blocking mode.
287
+
288
+ ### QA-RELEASE-001 - Package And Documentation Agreement
289
+
290
+ - **Given**: A proposed npm release.
291
+ - **Then**: Tests, package dry run, installed adapters, schema files, README, USAGE, version, changelog, rollback, and migration guidance agree.
292
+ - **Failure**: Documentation claims behavior absent from the packaged files.
293
+
294
+ ## 12. Design Document Verification Checklist
295
+
296
+ - [ ] All Markdown files use required Created and Last Updated metadata.
297
+ - [ ] All document filenames use two-digit numbering and uppercase snake case where applicable.
298
+ - [ ] Related Documents use valid relative links with relationship descriptions.
299
+ - [ ] Architecture embeds the data-flow SVG from the required assets directory.
300
+ - [ ] Every P0 FR has architecture coverage and a QA scenario.
301
+ - [ ] Backlog code/deploy tasks contain all Related fields and pending Receipt contracts.
302
+ - [ ] UI and HTML Preview fields use reasoned N/A rather than blank values.
303
+ - [ ] Decision Log records alternatives and reconsideration conditions.
304
+ - [ ] No implementation or release is represented as approved by these documents.
305
+ - [ ] External reference concepts are independently expressed and not copied verbatim.
306
+
307
+ ## 13. Related Documents
308
+
309
+ - **Concept_Design**: [Agent Harness Requirements Analysis](../01_Concept_Design/01_AGENT_HARNESS_REQUIREMENTS_ANALYSIS.md) - requirement IDs and approved scope
310
+ - **Technical_Specs**: [Agent Harness Architecture](../03_Technical_Specs/01_AGENT_HARNESS_ARCHITECTURE.md) - roles, states, messages, errors, artifacts, and adapters under test
311
+ - **Logic_Progress**: [Backlog](../04_Logic_Progress/00_BACKLOG.md) - task-level acceptance and Receipt status
312
+ - **Logic_Progress**: [Execution Plan](../04_Logic_Progress/01_EXECUTION_PLAN.md) - staged implementation, pilot, and release exits
313
+ - **Logic_Progress**: [Decision Log](../04_Logic_Progress/03_DECISION_LOG.md) - approved design decisions verified by these scenarios
@@ -0,0 +1,100 @@
1
+ # Agent Harness Design Review
2
+ > Created: 2026-07-17 01:15
3
+ > Last Updated: 2026-07-17 01:22
4
+
5
+ ## 1. Review Scope
6
+
7
+ This report records structural and traceability verification of the enhanced Agent Harness design baseline. It does not verify future CLI, schema, adapter, orchestration, release, or runtime behavior.
8
+
9
+ Reviewed artifacts:
10
+
11
+ - Requirements Analysis
12
+ - Architecture and data-flow SVG
13
+ - Backlog
14
+ - Execution Plan
15
+ - Decision Log
16
+ - Test Scenarios
17
+
18
+ ## 2. Verification Results
19
+
20
+ | Check | Result | Evidence |
21
+ |---|---|---|
22
+ | Markdown metadata | Pass | Seven design Markdown files contain Created and Last Updated metadata |
23
+ | File naming | Pass | Markdown filenames use two-digit uppercase snake case |
24
+ | Relative document links | Pass | All local Markdown links resolve to existing files |
25
+ | SVG validity | Pass | `xmllint --noout` accepts the data-flow SVG |
26
+ | SVG visual rendering | Pass | A full 1600x900 Chrome render has no clipping or text/connector overlap after correction |
27
+ | Backlog Context Lock shape | Pass for current docs task | All Related fields are present; UI and HTML Preview use reasoned N/A |
28
+ | Requirement traceability | Pass | 15 FR and 10 NFR entries have QA trace rows |
29
+ | Scenario references | Pass | 35 referenced QA scenario IDs have corresponding definitions |
30
+ | Existing package tests | Pass | `npm test` reports `harness checks ok` |
31
+ | Whitespace / patch integrity | Pass | `git diff --check` returns no error |
32
+ | User design review | Pending | The user has not yet reviewed the completed artifacts |
33
+
34
+ ## 3. Commands And Results
35
+
36
+ | Command / Method | Result | Summary |
37
+ |---|---|---|
38
+ | `git diff --check` | Pass | No whitespace errors |
39
+ | Metadata and naming shell checks | Pass | Required metadata and filename conventions found |
40
+ | Relative-link Node check | Pass | Six Markdown files checked before this report was added; all local targets resolved |
41
+ | Requirement trace Node check | Pass | 25 unique FR/NFR IDs have QA trace rows |
42
+ | Scenario-reference Node check | Pass | 35 scenario definitions cover every referenced scenario ID |
43
+ | `xmllint --noout docs/03_Technical_Specs/assets/01_agent_harness_data_flow.svg` | Pass | SVG is well-formed XML |
44
+ | Chrome headless 1600x900 SVG render and visual inspection | Pass | Full diagram framing, labels, and connectors are visible without overlap |
45
+ | `node bin/cli.js preflight TASK-HARNESS-001 --strict --backlog docs/04_Logic_Progress/00_BACKLOG.md` | Advisory Pass | Docs work type is correctly treated as advisory |
46
+ | `npm test` | Pass | Existing harness test suite remains green |
47
+
48
+ ## 4. Findings And Corrections
49
+
50
+ ### DR-001 - Missing Traceability Scenario
51
+
52
+ - **Initial Finding**: The traceability matrix referenced `QA-TRACE-001`, but no matching scenario section existed.
53
+ - **Correction**: Added the end-to-end requirement evidence scenario.
54
+ - **Status**: Resolved and rechecked.
55
+
56
+ ### DR-002 - Non-Functional Requirements Were Not Explicitly Mapped
57
+
58
+ - **Initial Finding**: NFR concerns appeared across scenarios, but no NFR traceability table proved complete coverage.
59
+ - **Correction**: Added a 10-row NFR matrix and dedicated cost and external-reference originality scenarios.
60
+ - **Status**: Resolved and rechecked.
61
+
62
+ ### DR-003 - Data-Flow Connector Overlapped A Role Boundary
63
+
64
+ - **Initial Finding**: The Coordinator-to-document connector crossed the Release Guardian boundary in the first full SVG render.
65
+ - **Correction**: Shifted the Release Guardian node and adjusted adjacent connector endpoints without changing the architecture.
66
+ - **Status**: Resolved in a second 1600x900 render and visually rechecked.
67
+
68
+ ## 5. Global Rubric Validation
69
+
70
+ | Criterion | Result | Evidence |
71
+ |---|---|---|
72
+ | Functionality | Pass at design stage | Workflow contracts and failure paths are specified and structurally verified |
73
+ | Potential Impact | Pass at design stage | Design addresses context omission, improvised delegation, agent conflict, and unsupported completion |
74
+ | Novelty | Pass at design stage | Team architecture is combined with machine-enforced document and independent-verification evidence |
75
+ | UX | Pass at design stage | Minimal topology and conditional specialists avoid mandatory full-team overhead |
76
+ | Open-source | Pass at design stage | Runtime-neutral contracts use standard formats and independently authored text |
77
+ | Business Plan | Pass at design stage | More reliable installed behavior can reduce support burden and improve package trust |
78
+
79
+ ## 6. Residual Risks
80
+
81
+ - Runtime capability details must be reverified against current official Claude Code and Codex behavior before adapter implementation.
82
+ - Structured schemas and legal transitions are design-only until automated validators exist.
83
+ - Read-only permissions and exclusive write ownership require runtime and fixture evidence.
84
+ - Cost and agent-count benefits require the five-task warning pilot.
85
+ - The user must review this baseline before Phase B code work can be proposed.
86
+
87
+ ## 7. Review Conclusion
88
+
89
+ **Status: Pass for structural design review.**
90
+
91
+ The design baseline is internally linked, traceable, and consistent with the approved direction. It is not an implementation approval and does not authorize code changes, blocking mode, merge, publish, or deployment.
92
+
93
+ ## 8. Related Documents
94
+
95
+ - **Concept_Design**: [Agent Harness Requirements Analysis](../01_Concept_Design/01_AGENT_HARNESS_REQUIREMENTS_ANALYSIS.md) - reviewed requirement source
96
+ - **Technical_Specs**: [Agent Harness Architecture](../03_Technical_Specs/01_AGENT_HARNESS_ARCHITECTURE.md) - reviewed technical contract
97
+ - **Logic_Progress**: [Backlog](../04_Logic_Progress/00_BACKLOG.md) - current task and verification summary
98
+ - **Logic_Progress**: [Execution Plan](../04_Logic_Progress/01_EXECUTION_PLAN.md) - future unapproved delivery phases
99
+ - **Logic_Progress**: [Decision Log](../04_Logic_Progress/03_DECISION_LOG.md) - reviewed approved decisions
100
+ - **QA_Validation**: [Agent Harness Test Scenarios](./01_TEST_SCENARIOS.md) - reviewed scenario and traceability catalog
@@ -0,0 +1,96 @@
1
+ # Agent Harness Contract Implementation QA
2
+ > Created: 2026-07-17 02:39
3
+ > Last Updated: 2026-07-17 02:44
4
+
5
+ ## 1. Scope
6
+
7
+ This report verifies `TASK-HARNESS-002`: the opt-in v1 manifest, message, and event contracts; semantic validation; deterministic CLI exits; existing Receipt compatibility; package inclusion; and documentation synchronization.
8
+
9
+ It does not verify specialist persona adapters, runtime orchestration, persistent recovery, the five-task warning pilot, blocking rollout, merge, publish, or deployment.
10
+
11
+ ## 2. Flow Status
12
+
13
+ ```text
14
+ [Flow]
15
+ Current: Phase 5 - Quality Verification
16
+ Gate: Quality + Minimal Implementation + Independent Agent Harness Verification - PASS
17
+ Completed: Approved requirements, Context Receipt, implementation, local checks, independent QA
18
+ Next: Phase 6 - Commit and handoff
19
+ Required confirmation: Separate approval for TASK-HARNESS-003, PR, merge, publish, or deploy
20
+ ```
21
+
22
+ ## 3. Verification Results
23
+
24
+ | Area | Result | Evidence |
25
+ |---|:---:|---|
26
+ | Versioned schema | Pass | Draft 2020-12 schema contains disjoint manifest, message, and event contracts under top-level `oneOf` |
27
+ | Structured parsing | Pass | JSON and JSONL parse errors are separated from contract failures |
28
+ | State transitions | Pass | Legal, illegal, blocked-resume, rework, cancellation, sequence, attempt, and final-state fixtures |
29
+ | Role and ownership | Pass | Core activation, skipped reason, read-only ownership, canonical paths, recursive overlap, and alias fixtures |
30
+ | Message authority | Pass | Direct peer traffic is limited to `STATUS` / `QUESTION` with `INFO` / `PENDING`; Coordinator routing is enforced |
31
+ | Evidence gates | Pass | Required Receipt and event evidence blocks incomplete `COMPLETE` transitions |
32
+ | Input safety | Pass | Path traversal, URL, backslash, wildcard, non-string, null, prototype-named, and invalid timestamp fixtures |
33
+ | Exit behavior | Pass | Warning contract failure `0`; blocking contract failure `1`; operational parse/file/input error `2` |
34
+ | Compatibility | Pass | Existing `preflight`, `verify`, Receipt fixture text, and custom Claude agent preservation remain valid |
35
+ | Dependency scope | Pass | No runtime or development dependency was added |
36
+ | Independent QA | Pass | Final read-only QA Inspector reported no P0, P1, or P2 findings |
37
+
38
+ ## 4. Commands And Results
39
+
40
+ | Command / Method | Result | Summary |
41
+ |---|:---:|---|
42
+ | `npm test` | Pass | `harness checks ok`; `harness artifact checks ok` |
43
+ | `node --check` on changed/new JavaScript | Pass | CLI, validator, runner, and fixture syntax valid |
44
+ | `node bin/cli.js preflight TASK-HARNESS-002 --strict` | Pass | Context and related-document evidence accepted |
45
+ | Node 16 `bin/test.js` | Pass | Both suites pass on an older ARM-compatible runtime |
46
+ | Acorn `--ecma2020` parse | Pass | New validator uses Node 14-compatible syntax |
47
+ | Ajv Draft 2020 compile | Pass | External validator compiles the top-level v1 schema |
48
+ | Ajv validation of `{}` | Expected reject | Top-level `oneOf` rejects an unconstrained artifact |
49
+ | `npm pack --dry-run --json` | Pass | 454 files; validator, tests, schema, contract, and architecture included |
50
+ | Document metadata and relative links | Pass | All current `docs/**/*.md` files validate |
51
+ | Requirement and scenario trace | Pass | 25 requirements and 36 scenario IDs resolve |
52
+ | Secret-pattern scan | Pass | No credential pattern found in changed implementation and documentation |
53
+ | `git diff --check` | Pass | No whitespace or patch-integrity errors |
54
+
55
+ ## 5. Independent Findings And Corrections
56
+
57
+ Independent read-only QA runs found and the Implementer corrected the following before the final PASS:
58
+
59
+ 1. Canonicalized exclusive ownership paths and rejected `./`, duplicate separators, parent traversal, and ambiguous recursive aliases.
60
+ 2. Rejected direct peer PASS, FAIL, BLOCKED, decision, rework, and completion status.
61
+ 3. Added top-level `oneOf` so the schema is useful to a standard Draft 2020-12 validator.
62
+ 4. Kept parseable invalid values as contract failures instead of internal operational exceptions.
63
+ 5. Made v1 attempt increments legal only for `REWORK -> IMPLEMENTING`.
64
+ 6. Replaced `Date.parse()` normalization with a schema-aligned Solmate canonical timestamp profile.
65
+ 7. Rejected inherited `constructor` and `__proto__` names under `additionalProperties: false`.
66
+ 8. Corrected the architecture description so `events.jsonl` contains state events and messages remain attempt artifacts.
67
+
68
+ After each correction a new read-only QA Inspector reviewed the updated files. The final run returned **No findings** and **Verification Receipt Status: PASS** without editing files.
69
+
70
+ ## 6. Minimal Implementation Review
71
+
72
+ - **YAGNI**: No persona runtime, orchestration server, database, queue, recovery engine, rollout automation, or release behavior was added.
73
+ - **KISS**: One canonical JSON Schema file and one Node standard-library validator module implement the approved contract.
74
+ - **DRY**: State, role, evidence, and schema definitions remain centralized; CLI and runtime adapters reference the contract.
75
+ - **Safety**: Validation, independent QA, deterministic failures, path safety, evidence preservation, and compatibility checks were retained.
76
+
77
+ ## 7. Unrun Checks And Residual Risks
78
+
79
+ - Actual Node 14 execution was unavailable because npm has no matching `node-bin-darwin-arm64@14.21.3`; Node 16 execution and ECMAScript 2020 static parsing passed.
80
+ - Ajv CLI compiled the schema and exercised top-level rejection without `ajv-formats`; calendar and canonical timestamp semantics are additionally covered by the package validator tests.
81
+ - Claude/Codex persona parity, runtime permission enforcement, orchestration recovery, warning-pilot metrics, and release behavior belong to later tasks.
82
+ - Package version remains `2.0.12`; no version bump, PR, merge, publish, or deploy was authorized in this task.
83
+
84
+ ## 8. Conclusion
85
+
86
+ **Status: PASS for TASK-HARNESS-002.**
87
+
88
+ The v1 structured contract and machine validator satisfy the approved acceptance criteria while preserving current Receipt behavior. Later phases remain separately gated.
89
+
90
+ ## 9. Related Documents
91
+
92
+ - **Concept_Design**: [Agent Harness Requirements Analysis](../01_Concept_Design/01_AGENT_HARNESS_REQUIREMENTS_ANALYSIS.md) - approved requirements and compatibility constraints
93
+ - **Technical_Specs**: [Agent Harness Architecture](../03_Technical_Specs/01_AGENT_HARNESS_ARCHITECTURE.md) - implemented schema and semantic contract
94
+ - **Logic_Progress**: [Backlog](../04_Logic_Progress/00_BACKLOG.md) - completed task Receipts
95
+ - **Logic_Progress**: [Execution Plan](../04_Logic_Progress/01_EXECUTION_PLAN.md) - Phase B completion and later phases
96
+ - **QA_Validation**: [Agent Harness Test Scenarios](./01_TEST_SCENARIOS.md) - traceability and planned scenario catalog
@@ -33,6 +33,8 @@ The AI MUST check for and follow these documents if they exist in the repository
33
33
  Apply the "Subagent Strategy" to keep the main context clean:
34
34
  - **Delegation**: Offload research/analysis to Browser/Thinking tools.
35
35
  - **Aggregation**: Return only summarized, high-value insights to the main chat.
36
+ - **Canonical Contract**: For code and deploy tasks, use `rules-workflow/resources/agent-harness-contract.md` for Coordinator, Context, Implementation, Verification roles and handoffs.
37
+ - **Independence**: Context and Verification agents are read-only. Verification findings return to the Implementation Agent instead of being fixed by the verifier.
36
38
 
37
39
  ### 3.2. 자가개선 루프 (Self-Improvement)
38
40
  - Maintain `tasks/lessons.md` to capture patterns and avoid repeating mistakes.
package/package.json CHANGED
@@ -1,13 +1,13 @@
1
1
  {
2
2
  "name": "solmate-skills",
3
- "version": "2.0.12",
4
- "description": "AI workflow skills for UI-first planning, YAGNI/KISS/DRY gates, docs, QA, and Solmate projects",
3
+ "version": "2.0.13",
4
+ "description": "AI agent harness and workflow skills for context-aware planning, implementation, verification, docs, and QA",
5
5
  "main": "bin/cli.js",
6
6
  "bin": {
7
7
  "solmate-skills": "bin/cli.js"
8
8
  },
9
9
  "scripts": {
10
- "test": "echo \"Error: no test specified\" && exit 1"
10
+ "test": "node bin/test.js"
11
11
  },
12
12
  "keywords": [
13
13
  "ai-skills",
@@ -22,7 +22,9 @@
22
22
  "qa",
23
23
  "solmate",
24
24
  "skills",
25
- "automation"
25
+ "automation",
26
+ "agent-harness",
27
+ "multi-agent"
26
28
  ],
27
29
  "author": "",
28
30
  "license": "ISC",
@@ -89,12 +89,26 @@ docs/02_UI_Screens/previews/
89
89
  - `Related HTML Preview`
90
90
  - `Related Technical Docs`
91
91
  - `Related QA Docs`
92
+ - `Work Type`
92
93
  - `Implementation Preconditions`
94
+ - `Context Receipt`
95
+ - `Change Receipt`
96
+ - `Verification Receipt`
93
97
  - `Acceptance Criteria`
94
98
  - `Document Sync Check`
95
99
 
96
100
  각 Related 필드는 상대 경로 링크와 관계 설명을 포함해야 한다. 관련 문서나 HTML Preview가 없으면 빈칸으로 두지 않고 `N/A - 사유`를 명시한다. 단, 구현 판단에 필요한 Concept, UI, HTML Preview, Technical Spec, QA 기준이 아직 문서화되지 않은 경우에는 백로그만으로 구현을 시작할 수 없으며, 먼저 문서 작성 또는 보완 여부를 사용자에게 확인한다.
97
101
 
102
+ `Work Type`은 `code`, `deploy`, `docs`, `prototype` 중 하나를 사용한다. `code`와 `deploy` 작업은 다음 Agent Harness 조건을 추가로 지킨다.
103
+
104
+ - Context Agent가 모든 Related 링크를 읽고 `Context Receipt`에 기록한다.
105
+ - warning 기간에는 Context Receipt 발견 사항과 사용자 확인을 기록한다. blocking 전환 후에는 PASS Context Receipt 없이는 구현을 시작하지 않는다.
106
+ - Implementation Agent는 `Change Receipt`를 남기지만 이를 독립 검증으로 간주하지 않는다.
107
+ - 읽기 전용 Verification Agent가 `Verification Receipt`를 작성한다.
108
+ - warning 기간에는 Verification Receipt 발견 사항과 사용자 확인을 기록한다. blocking 전환 후에는 PASS Verification Receipt와 QA 문서 또는 PR 상세 근거 링크 없이는 Done 처리하지 않는다.
109
+
110
+ Receipt 형식과 런타임별 역할은 `rules-workflow/resources/agent-harness-contract.md`를 정본으로 따른다.
111
+
98
112
  ### 메타데이터 (필수)
99
113
 
100
114
  모든 문서 최상단:
@@ -133,7 +147,7 @@ docs/02_UI_Screens/previews/
133
147
 
134
148
  **Logic_Progress**: Concept_Design + UI_Screens + Technical_Specs (DB, API) + QA_Validation
135
149
 
136
- **Backlog Items**: 각 항목 단위로 Concept_Design + UI_Screens + HTML Preview + Technical_Specs + QA_Validation 링크를 포함한다. 구현자는 항목 착수 링크된 문서와 HTML Preview를 확인하고 `Implementation Preconditions`를 확인해야 한다.
150
+ **Backlog Items**: 각 항목 단위로 Concept_Design + UI_Screens + HTML Preview + Technical_Specs + QA_Validation 링크를 포함한다. `code`와 `deploy` 작업은 구현자가 아니라 읽기 전용 Context Agent가 먼저 링크된 문서를 확인하고 Context Receipt를 제출해야 한다.
137
151
 
138
152
  **QA_Validation**: 모든 상위 레이어 참조
139
153