gemstack-ai 1.2.0 → 1.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.gemstack/state.json +11 -10
- package/CHANGELOG.md +97 -0
- package/README.md +89 -9
- package/RELEASE_NOTES.md +77 -0
- package/{gemstack-ai-1.2.0.tgz → gemstack-ai-1.4.0.tgz} +0 -0
- package/handoff.md +14 -12
- package/package.json +2 -2
- package/specs/008-cost-provider-safety-gates/.gemstack.json +5 -0
- package/specs/008-cost-provider-safety-gates/closure.json +59 -0
- package/specs/008-cost-provider-safety-gates/plan.md +456 -0
- package/specs/008-cost-provider-safety-gates/spec.md +633 -0
- package/specs/008-cost-provider-safety-gates/tasks.md +635 -0
- package/specs/009-context-capsule/closure.json +59 -0
- package/specs/009-context-capsule/context-capsule.json +428 -0
- package/specs/009-context-capsule/plan.md +663 -0
- package/specs/009-context-capsule/spec.md +913 -0
- package/specs/009-context-capsule/tasks.md +720 -0
- package/specs/010-agent-swarm-visual-qa/.gemstack.json +5 -0
- package/specs/010-agent-swarm-visual-qa/closure.json +59 -0
- package/specs/010-agent-swarm-visual-qa/plan.md +759 -0
- package/specs/010-agent-swarm-visual-qa/spec.md +842 -0
- package/specs/010-agent-swarm-visual-qa/swarm.json +49 -0
- package/specs/010-agent-swarm-visual-qa/tasks.md +873 -0
- package/specs/010-agent-swarm-visual-qa/visual-qa.json +41 -0
- package/src/cli.js +10 -0
- package/src/commands/context.js +95 -0
- package/src/commands/swarm.js +111 -0
- package/src/commands/verify.js +92 -0
- package/src/commands/visual.js +82 -0
- package/src/lib/closure-context.js +18 -1
- package/src/lib/context-capsule.js +594 -0
- package/src/lib/cost-ledger.js +355 -0
- package/src/lib/provider-boundary.js +186 -0
- package/src/lib/provider-registry.js +265 -0
- package/src/lib/safety-gates.js +277 -0
- package/src/lib/swarm.js +639 -0
- package/src/lib/visual-qa.js +499 -0
|
@@ -0,0 +1,842 @@
|
|
|
1
|
+
# Especificación de Funcionalidad: Agent Swarm & Visual QA (Upgrade E)
|
|
2
|
+
|
|
3
|
+
**Feature Branch**: `010-agent-swarm-visual-qa`
|
|
4
|
+
**Feature Directory**: `specs/010-agent-swarm-visual-qa/`
|
|
5
|
+
**Lifecycle Status**: `SPEC_COMPLETE`
|
|
6
|
+
**Stop Reason**: `SPEC_COMPLETE_AWAITING_REVIEW`
|
|
7
|
+
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
## 1. Problem Statement
|
|
11
|
+
|
|
12
|
+
Gemstack provides a rigorous, offline-first, deterministic foundation for specification-driven software engineering. Through closed Upgrades A, B, C, and D, the repository guarantees:
|
|
13
|
+
- Frozen contract integrity and feature state transitions (Upgrade A),
|
|
14
|
+
- Verification purity, offline enforcement, and clean CLI ergonomics (Upgrade B),
|
|
15
|
+
- Deterministic capability and cost safety gating for external and billable actions (Upgrade C),
|
|
16
|
+
- Bounded, lossless continuation context compression with cryptographic provenance (Upgrade D).
|
|
17
|
+
|
|
18
|
+
However, as software systems grow in complexity, modern engineering workflows increasingly demand:
|
|
19
|
+
1. **Parallel Execution via Multi-Agent Swarms**: Decomposing feature implementation into concurrent subtasks executed by bounded specialized agents (workers, researchers, security auditors).
|
|
20
|
+
2. **Visual Verification of User Interfaces**: Validating that frontend screens, responsive layouts, design token compliance, and visual user journeys match specifications across target viewports without manual human spot-checking.
|
|
21
|
+
|
|
22
|
+
Without explicit architectural controls, these two capabilities introduce catastrophic risks of system degradation:
|
|
23
|
+
- **Authority Inversion**: Autonomous agents or browser screenshot comparison tools attempt to declare themselves architectural authorities, overwriting specifications, bypassing frozen contracts, or claiming task completion based on subjective visual appeal.
|
|
24
|
+
- **Concurrent State Corruption & Write Collisions**: Multiple parallel workers modifying the same files concurrently without strict partition ownership, leading to race conditions, overwritten work, and untracked mutations.
|
|
25
|
+
- **Role Contamination & Self-Review**: The agent that authors code reviewing and approving its own implementation, bypassing the fundamental separation of duties (`AUTHOR = REVIEWER`).
|
|
26
|
+
- **Cost Runaway & Rate-Limit Exhaustion**: Swarms launching unbounded parallel API queries or billable tool actions, bypassing Upgrade C cost ledgers and spending limits.
|
|
27
|
+
- **Flaky Visual Diffing & Viewport Nondeterminism**: Pixel-diffing across dynamic animations, anti-aliased font rendering across OS platforms, or variable viewport sizes producing nondeterministic false positives in CI.
|
|
28
|
+
- **Runtime Bloat**: Introducing gigabyte-sized headless browser frameworks (e.g., full Chromium/Playwright binaries) into Gemstack's core runtime package, violating the permanent zero-runtime-dependency invariant.
|
|
29
|
+
|
|
30
|
+
Gemstack requires a deterministic, auditable, offline-verifiable architecture that coordinates multi-agent swarms and validates visual QA evidence while preserving the absolute authority of specifications, plans, tasks, and frozen contracts.
|
|
31
|
+
|
|
32
|
+
---
|
|
33
|
+
|
|
34
|
+
## 2. Goals & Non-Goals
|
|
35
|
+
|
|
36
|
+
### Goals
|
|
37
|
+
- **Deterministic Swarm Planning & Coordination**: Define formal data structures and rules for decomposing `tasks.md` into parallel, non-overlapping work packages assigned to bounded worker roles with explicit write-set partitions.
|
|
38
|
+
- **Strict Separation of Duties (`AUTHOR != REVIEWER`)**: Enforce that an agent or worker role assigned to implement code or visual artifacts cannot act as the final reviewer, security auditor, or closure verifier.
|
|
39
|
+
- **Provider & Cost Safety Integration**: Mandate that every billable or model invocation executed within a swarm passes through Upgrade C's `ProviderCapabilityGate` and `BillableActionGate`.
|
|
40
|
+
- **Context Projection Integration**: Require every swarm worker to receive its context strictly as a deterministic projection of Upgrade D's `context-capsule.json`, preventing conversational context pollution.
|
|
41
|
+
- **Structured Visual QA Evidence & Scenario Contracts**: Define a canonical schema (`visual-qa.json`) for declaring UI visual acceptance criteria, deterministic viewports, element selectors, and baseline evidence hashes.
|
|
42
|
+
- **Zero-Runtime-Dependency Core**: Ensure that Gemstack's core CLI, verification engine, and swarm/visual validators require zero external npm packages and zero bundled browser binaries.
|
|
43
|
+
- **Deterministic Baseline & Diffing Rules**: Establish strict criteria for visual evidence collection (fixed viewports, masked dynamic regions, explicit baseline update semantics) to prevent flaky nondeterministic verification.
|
|
44
|
+
- **Verification Purity (`VERIFY = VALIDATE`)**: Guarantee that `gemstack verify` inspects swarm schedules, write ownership, review attestations, and visual evidence manifests in pure read-only mode with zero file mutations and zero live browser invocations.
|
|
45
|
+
- **Fail-Closed Conflict Detection**: Mechanically reject any swarm plan or evidence submission where overlapping write permissions, unreviewed changes, gate bypasses, or visual regressions occur.
|
|
46
|
+
- **Legacy Compatibility**: Ensure existing projects without swarms or visual UI components verify cleanly with zero false blockers.
|
|
47
|
+
|
|
48
|
+
### Non-Goals
|
|
49
|
+
- Gemstack does **NOT** build a live autonomous multi-process daemon server or cluster orchestrator in its core runtime.
|
|
50
|
+
- Gemstack does **NOT** bundle Puppeteer, Playwright, or Chromium into its runtime npm dependencies.
|
|
51
|
+
- Gemstack does **NOT** perform fuzzy, subjective, or perceptual AI aesthetic grading as authoritative visual verification.
|
|
52
|
+
- Gemstack does **NOT** allow visual screenshots or worker logs to supersede or modify `spec.md`, `plan.md`, `tasks.md`, or `closure.json`.
|
|
53
|
+
- Gemstack does **NOT** allow automated visual diff tools to silently update visual baselines during verification.
|
|
54
|
+
- Gemstack does **NOT** manage real-time agent networking, socket protocols, or distributed consensus.
|
|
55
|
+
|
|
56
|
+
---
|
|
57
|
+
|
|
58
|
+
## 3. Core Architectural Principles & Invariants
|
|
59
|
+
|
|
60
|
+
```text
|
|
61
|
+
================================================================================
|
|
62
|
+
UPGRADE E CANONICAL SAFETY INVARIANTS
|
|
63
|
+
================================================================================
|
|
64
|
+
|
|
65
|
+
1. SWARM_AUTHORITY_SUBORDINATE
|
|
66
|
+
SPEC / PLAN / TASKS / FROZEN CONTRACTS = SOLE AUTHORITY.
|
|
67
|
+
Worker agents produce PROPOSED WORK. Orchestrators produce PROPOSED SCHEDULES.
|
|
68
|
+
Neither worker outputs nor orchestrator decisions can alter frozen contracts,
|
|
69
|
+
bypass tasks, or declare closure autonomously.
|
|
70
|
+
|
|
71
|
+
2. AUTHOR_NOT_REVIEWER
|
|
72
|
+
The entity, agent role, or worker ID that authors an implementation task
|
|
73
|
+
MUST NOT serve as the reviewer, security auditor, or closure verifier for
|
|
74
|
+
that task. Independent verification is non-waivable.
|
|
75
|
+
|
|
76
|
+
3. EXCLUSIVE_TASK_WRITE_OWNERSHIP
|
|
77
|
+
Every active task in a swarm must have exactly one designated write owner.
|
|
78
|
+
No two concurrently executing tasks may declare overlapping write scopes
|
|
79
|
+
(file paths or directory trees). Conflicting writes produce fail-closed rejection.
|
|
80
|
+
|
|
81
|
+
4. SWARM_PROVIDER_SAFETY_GATED
|
|
82
|
+
All model queries, tool executions, and external operations performed by
|
|
83
|
+
swarm workers are subject to Upgrade C ProviderCapabilityGate and
|
|
84
|
+
BillableActionGate policies. Offline and unknown-cost invariants apply.
|
|
85
|
+
|
|
86
|
+
5. SWARM_CONTEXT_PROJECTED
|
|
87
|
+
All swarm worker context payloads MUST be deterministic projections of the
|
|
88
|
+
authoritative Upgrade D Context Capsule. Workers must not receive raw chat
|
|
89
|
+
histories or undeclared environment state.
|
|
90
|
+
|
|
91
|
+
6. VISUAL_EVIDENCE_SUBORDINATE
|
|
92
|
+
Visual QA screenshots, DOM snapshots, and diff metrics represent EVIDENCE,
|
|
93
|
+
never ARCHITECTURAL AUTHORITY. Visual evidence can prove failure of a visual
|
|
94
|
+
acceptance criterion, but cannot waive functional, cost, or security gates.
|
|
95
|
+
|
|
96
|
+
7. DETERMINISTIC_VIEWPORTS_AND_ENVIRONMENT
|
|
97
|
+
Visual verification criteria MUST explicitly declare deterministic viewport
|
|
98
|
+
dimensions, device scale factors, and color scheme modes. Unspecified or
|
|
99
|
+
ambient display properties are strictly forbidden.
|
|
100
|
+
|
|
101
|
+
8. BASELINE_EXPLICIT_UPDATE_ONLY
|
|
102
|
+
Visual regression baselines MUST NOT be updated automatically or implicitly
|
|
103
|
+
during verification. Baseline updates require an explicit, intentional command
|
|
104
|
+
invoked by a human engineer or authenticated closure workflow.
|
|
105
|
+
|
|
106
|
+
9. VERIFY_PURITY_READ_ONLY
|
|
107
|
+
`gemstack verify` operates strictly in read-only mode. It validates swarm
|
|
108
|
+
manifests, write boundaries, review attestations, and visual QA evidence
|
|
109
|
+
without spawning live workers, mutating files, or launching browsers.
|
|
110
|
+
|
|
111
|
+
10. ZERO_RUNTIME_DEPENDENCY_CORE
|
|
112
|
+
Gemstack core maintains runtime dependency delta = 0. Swarm coordination
|
|
113
|
+
and visual QA verification logic rely exclusively on Node.js built-ins.
|
|
114
|
+
Browser automation occurs via external development adapters or environment tools.
|
|
115
|
+
|
|
116
|
+
================================================================================
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
---
|
|
120
|
+
|
|
121
|
+
## 4. Upgrade E Bootstrap Contracts
|
|
122
|
+
|
|
123
|
+
```gemstack-contracts
|
|
124
|
+
[
|
|
125
|
+
{
|
|
126
|
+
"id": "swarm-authority-subordinate",
|
|
127
|
+
"type": "BOOLEAN_INVARIANT",
|
|
128
|
+
"value": true,
|
|
129
|
+
"description": "Agent swarm orchestrators and workers are strictly subordinate to repository specifications, plans, tasks, and frozen contracts."
|
|
130
|
+
},
|
|
131
|
+
{
|
|
132
|
+
"id": "author-not-reviewer",
|
|
133
|
+
"type": "BOOLEAN_INVARIANT",
|
|
134
|
+
"value": true,
|
|
135
|
+
"description": "Separation of duties requires that the author of a code or task change cannot review or approve their own work."
|
|
136
|
+
},
|
|
137
|
+
{
|
|
138
|
+
"id": "exclusive-task-write-ownership",
|
|
139
|
+
"type": "BOOLEAN_INVARIANT",
|
|
140
|
+
"value": true,
|
|
141
|
+
"description": "Concurrent swarm tasks must declare disjoint, non-overlapping file write sets to prevent write collisions."
|
|
142
|
+
},
|
|
143
|
+
{
|
|
144
|
+
"id": "swarm-provider-safety-gated",
|
|
145
|
+
"type": "BOOLEAN_INVARIANT",
|
|
146
|
+
"value": true,
|
|
147
|
+
"description": "All worker operations must pass Upgrade C capability and billable action gates."
|
|
148
|
+
},
|
|
149
|
+
{
|
|
150
|
+
"id": "swarm-context-projected",
|
|
151
|
+
"type": "BOOLEAN_INVARIANT",
|
|
152
|
+
"value": true,
|
|
153
|
+
"description": "Worker contexts must be deterministically projected from Upgrade D context capsule without chat history leakage."
|
|
154
|
+
},
|
|
155
|
+
{
|
|
156
|
+
"id": "visual-evidence-subordinate",
|
|
157
|
+
"type": "BOOLEAN_INVARIANT",
|
|
158
|
+
"value": true,
|
|
159
|
+
"description": "Visual artifacts and screenshots are verification evidence only and cannot alter specifications or override gates."
|
|
160
|
+
},
|
|
161
|
+
{
|
|
162
|
+
"id": "deterministic-viewports",
|
|
163
|
+
"type": "BOOLEAN_INVARIANT",
|
|
164
|
+
"value": true,
|
|
165
|
+
"description": "All visual QA scenarios must specify explicit width, height, and device scale factor."
|
|
166
|
+
},
|
|
167
|
+
{
|
|
168
|
+
"id": "baseline-explicit-update-only",
|
|
169
|
+
"type": "BOOLEAN_INVARIANT",
|
|
170
|
+
"value": true,
|
|
171
|
+
"description": "Visual reference baselines may only be modified through explicit, recorded update operations."
|
|
172
|
+
},
|
|
173
|
+
{
|
|
174
|
+
"id": "verify-swarm-visual-offline",
|
|
175
|
+
"type": "BOOLEAN_INVARIANT",
|
|
176
|
+
"value": true,
|
|
177
|
+
"description": "gemstack verify validates swarm manifests and visual evidence offline with zero network and zero browser spawns."
|
|
178
|
+
},
|
|
179
|
+
{
|
|
180
|
+
"id": "legacy-swarm-visual-compatibility",
|
|
181
|
+
"type": "BOOLEAN_INVARIANT",
|
|
182
|
+
"value": true,
|
|
183
|
+
"description": "Repositories lacking swarm schedules or visual QA manifests operate cleanly without error."
|
|
184
|
+
}
|
|
185
|
+
]
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
---
|
|
189
|
+
|
|
190
|
+
## 5. Canonical Terminology
|
|
191
|
+
|
|
192
|
+
| Term | Canonical Meaning |
|
|
193
|
+
| :--- | :--- |
|
|
194
|
+
| **`SwarmPlan`** | The deterministic schedule and partition graph assigning parallel tasks (`[P]`) to bounded worker roles with explicit write sets. |
|
|
195
|
+
| **`SwarmManifest`** | The machine-readable JSON artifact (`swarm.json`) tracking active workers, task assignments, write leases, and execution states. |
|
|
196
|
+
| **`WorkerRole`** | A bounded behavioral persona (e.g., `implementer`, `reviewer`, `researcher`, `security-auditor`) with constrained tool permissions. |
|
|
197
|
+
| **`WriteSetPartition`** | The strictly enumerated set of relative file paths that a specific worker or task is authorized to create or modify. |
|
|
198
|
+
| **`IndependentReview`** | A mandatory attestation by a distinct worker role proving verification of correctness, security, and contract compliance. |
|
|
199
|
+
| **`VisualQAScenario`** | A discrete UI test case declaring a target route/URL, viewport dimensions, user interaction steps, and expected visual outcomes. |
|
|
200
|
+
| **`VisualQAManifest`** | The canonical JSON artifact (`visual-qa.json`) defining all visual test scenarios, selectors, masks, and baseline hashes. |
|
|
201
|
+
| **`VisualEvidence`** | Structured output (screenshot files, DOM snapshots, bounding box hashes) generated during scenario execution. |
|
|
202
|
+
| **`VisualBaseline`** | An approved, canonical reference image or digest representing the authoritative visual acceptance standard for a scenario. |
|
|
203
|
+
| **`VisualDiffReport`** | The deterministic comparison result between live visual evidence and the recorded baseline, identifying pixel or layout deviations. |
|
|
204
|
+
|
|
205
|
+
---
|
|
206
|
+
|
|
207
|
+
## 6. Architecture & System Overview
|
|
208
|
+
|
|
209
|
+
```
|
|
210
|
+
+---------------------------------------------------------------------------------------------------+
|
|
211
|
+
| GEMSTACK ARCHITECTURAL CORE |
|
|
212
|
+
| |
|
|
213
|
+
| +-------------------------------------------------------------------------------------------+ |
|
|
214
|
+
| | AUTHORITATIVE SPECIFICATION & CONTRACT LAYER | |
|
|
215
|
+
| | spec.md | plan.md | tasks.md | frozen-contracts.json | context-capsule.json | |
|
|
216
|
+
| +---------------------------------------------+---------------------------------------------+ |
|
|
217
|
+
| | |
|
|
218
|
+
| v |
|
|
219
|
+
| +-------------------------------------------------------------------------------------------+ |
|
|
220
|
+
| | SWARM ORCHESTRATION SUBSYSTEM | |
|
|
221
|
+
| | | |
|
|
222
|
+
| | [tasks.md [P] Tasks] ----> [Swarm Partition Planner] ----> [swarm.json (Manifest)] | |
|
|
223
|
+
| | | | |
|
|
224
|
+
| | v | |
|
|
225
|
+
| | +-----------------------------------------------+ | |
|
|
226
|
+
| | | Concurrency & Write Set Conflict Engine | | |
|
|
227
|
+
| | | - Exclusive File Leases | | |
|
|
228
|
+
| | | - Author != Reviewer Enforcement | | |
|
|
229
|
+
| | | - Capsule Context Projection Provider | | |
|
|
230
|
+
| | | - Upgrade C Safety Gate Interceptor | | |
|
|
231
|
+
| | +-----------------------------------------------+ | |
|
|
232
|
+
| +---------------------------------------------+---------------------------------------------+ |
|
|
233
|
+
| | |
|
|
234
|
+
| v |
|
|
235
|
+
| +-------------------------------------------------------------------------------------------+ |
|
|
236
|
+
| | VISUAL QA SUBSYSTEM | |
|
|
237
|
+
| | | |
|
|
238
|
+
| | [UI Acceptance Criteria] -> [visual-qa.json Manifest] | |
|
|
239
|
+
| | | | |
|
|
240
|
+
| | v | |
|
|
241
|
+
| | +-----------------------------------------------+ | |
|
|
242
|
+
| | | Scenario & Viewport Matrix Validator | | |
|
|
243
|
+
| | | - Deterministic Resolution (WxH, Scale) | | |
|
|
244
|
+
| | | - Dynamic Region Masking (Timestamps, Avatars)| | |
|
|
245
|
+
| | | - Offline Evidence Hash & Digest Comparator | | |
|
|
246
|
+
| | | - Explicit Baseline Lifecycle Enforcer | | |
|
|
247
|
+
| | +-----------------------------------------------+ | |
|
|
248
|
+
| +---------------------------------------------+---------------------------------------------+ |
|
|
249
|
+
| | |
|
|
250
|
+
| v |
|
|
251
|
+
| +-------------------------------------------------------------------------------------------+ |
|
|
252
|
+
| | UNIFIED OFFLINE VERIFICATION ENGINE | |
|
|
253
|
+
| | | |
|
|
254
|
+
| | gemstack verify (Read-Only) | |
|
|
255
|
+
| | - Validates swarm task integrity & review separation | |
|
|
256
|
+
| | - Validates visual-qa.json schema & evidence completeness | |
|
|
257
|
+
| | - Validates baseline cryptographic hashes | |
|
|
258
|
+
| | - Emits fail-closed findings on regressions, collisions, or unauthorized mutations | |
|
|
259
|
+
| +-------------------------------------------------------------------------------------------+ |
|
|
260
|
+
+---------------------------------------------------------------------------------------------------+
|
|
261
|
+
```
|
|
262
|
+
|
|
263
|
+
---
|
|
264
|
+
|
|
265
|
+
## 7. Major Decision 1: Swarm Architecture — Deterministic Planning & Evidence Contracts
|
|
266
|
+
|
|
267
|
+
### Evaluation of Alternatives
|
|
268
|
+
- **Alternative 1: Live Process Daemon / Long-Running Cluster**. Gemstack embeds an active background process manager that launches child processes, manages worker heartbeats, and orchestrates live IPC.
|
|
269
|
+
- *Drawbacks*: Severe violation of zero-dependency invariant; platform inconsistencies between Windows, Linux, and macOS; process lifecycle flakiness in headless CI; high cognitive and computational overhead.
|
|
270
|
+
- **Alternative 2: Deterministic Swarm Planning, Partitioning, and Evidence Contracts (Selected)**. Gemstack acts as the architectural coordination, write-set guard, dependency enforcer, and evidence validator. The core engine compiles tasks into deterministic swarm execution graphs (`swarm.json`), enforces exclusive file partitions, validates context projections, intercepts billable actions, and requires independent review attestations. Worker execution is performed by external agent runners (IDE subagents, CI scripts, or human engineers) who submit structured evidence.
|
|
271
|
+
- *Advantages*: 100% deterministic, offline-verifiable, cross-platform, zero runtime dependencies, robust against process crashes, and preserves strict separation of authority.
|
|
272
|
+
|
|
273
|
+
### Resolution
|
|
274
|
+
Gemstack adopts **Alternative 2: Deterministic Planning, Partitioning, and Evidence Contracts**. Gemstack core governs the *rules of engagement*, *partition boundaries*, and *acceptance validation*, while remaining agnostic to the specific subagent runner implementation.
|
|
275
|
+
|
|
276
|
+
---
|
|
277
|
+
|
|
278
|
+
## 8. Major Decision 2: Visual QA Architecture — Structured Evidence & Deterministic Offline Validation
|
|
279
|
+
|
|
280
|
+
### Evaluation of Alternatives
|
|
281
|
+
- **Alternative 1: Bundled Heavy Browser Runtime**. Include Playwright/Puppeteer and Chromium binaries directly inside Gemstack's production npm package dependencies.
|
|
282
|
+
- *Drawbacks*: Adds hundreds of megabytes to package size; violates the zero-runtime-dependency invariant; causes brittle native binary installation failures across varied developer environments; fails on offline/air-gapped machines.
|
|
283
|
+
- **Alternative 2: Structured Visual Evidence & Deterministic Offline Validation with Adapter-Based Capture (Selected)**. Gemstack defines canonical visual scenario manifests (`visual-qa.json`) and validates visual evidence (screenshots, DOM trees, layout geometry hashes) offline using pure Node.js standard libraries. Capture execution is decoupled into optional dev-adapters or lightweight user-space test scripts that execute in environments where browser runtimes exist.
|
|
284
|
+
- *Advantages*: Preserves zero runtime dependencies in core; guarantees offline verification purity in CI; allows flexible capture mechanisms (Puppeteer, Playwright, MCP browser tools, or synthetic mock harnesses); provides deterministic regression detection via cryptographic baseline hashing.
|
|
285
|
+
|
|
286
|
+
### Resolution
|
|
287
|
+
Gemstack adopts **Alternative 2: Structured Visual Evidence & Deterministic Offline Validation with Adapter-Based Capture**. Core Gemstack inspects, validates, diffs, and gates visual evidence offline without requiring a bundled browser binary.
|
|
288
|
+
|
|
289
|
+
---
|
|
290
|
+
|
|
291
|
+
## 9. Authority & Subordination Hierarchy
|
|
292
|
+
|
|
293
|
+
To eliminate any ambiguity regarding who commands the system:
|
|
294
|
+
1. **Primary Authority**: `spec.md`, `plan.md`, `tasks.md`, and frozen architectural contracts.
|
|
295
|
+
2. **Derived Invariants**: Upgrade D `context-capsule.json`.
|
|
296
|
+
3. **Execution Manifests**: `swarm.json` and `visual-qa.json` (compiled from and validated against primary authority).
|
|
297
|
+
4. **Execution Evidence**: Worker diffs, agent messages, visual screenshots, and DOM captures.
|
|
298
|
+
5. **Non-Authoritative Status**: Neither an agent's reasoning trace, nor an LLM's opinion, nor a screenshot comparison tool has the authority to alter specifications, declare a task complete without passing tests, or waive a frozen contract.
|
|
299
|
+
|
|
300
|
+
---
|
|
301
|
+
|
|
302
|
+
## 10. Swarm Subsystem: Lifecycle & Execution Model
|
|
303
|
+
|
|
304
|
+
The swarm coordination lifecycle consists of five discrete states:
|
|
305
|
+
|
|
306
|
+
```
|
|
307
|
+
+--------------+ +----------------+ +-------------------+
|
|
308
|
+
| DISCOVER | ----> | SCHEDULE | ----> | EXECUTE |
|
|
309
|
+
| Read [P] in | | Compile write | | Workers perform |
|
|
310
|
+
| tasks.md | | partitions | | bounded changes |
|
|
311
|
+
+--------------+ +----------------+ +-------------------+
|
|
312
|
+
|
|
|
313
|
+
v
|
|
314
|
+
+----------------+ +-------------------+
|
|
315
|
+
| CLOSURE | <---- | VALIDATE |
|
|
316
|
+
| Merge evidence | | Independent review|
|
|
317
|
+
| and reconcile | | & gate check |
|
|
318
|
+
+----------------+ +-------------------+
|
|
319
|
+
```
|
|
320
|
+
|
|
321
|
+
1. **Discover**: The swarm planner scans the active feature's `tasks.md` for tasks annotated with the parallel marker `[P]`.
|
|
322
|
+
2. **Schedule**: The planner generates a deterministic `swarm.json` execution plan, grouping independent tasks into concurrent waves and calculating the explicit file write-set for each task.
|
|
323
|
+
3. **Execute**: Worker agents execute assigned tasks within their exclusive write-set boundary, referencing only their projected context capsule.
|
|
324
|
+
4. **Validate**: Each completed task is submitted for independent review. The reviewer role validates test passage, contract compliance, and write-set boundaries.
|
|
325
|
+
5. **Closure**: Once all wave tasks and reviews pass, task checkmarks are reconciled into `tasks.md`.
|
|
326
|
+
|
|
327
|
+
---
|
|
328
|
+
|
|
329
|
+
## 11. Swarm Worker Roles & Separation of Duties (`AUTHOR != REVIEWER`)
|
|
330
|
+
|
|
331
|
+
Upgrade E defines four canonical, bounded worker roles:
|
|
332
|
+
|
|
333
|
+
| Role Name | Permitted Capabilities | Forbidden Actions |
|
|
334
|
+
| :--- | :--- | :--- |
|
|
335
|
+
| **`implementer`** | Create and edit source files within assigned write-set partition; run local tests. | Cannot approve own PR/task; cannot review own work; cannot modify spec or plan. |
|
|
336
|
+
| **`reviewer`** | Read code diffs; execute verification tests; inspect style and contract compliance; sign review attestation. | Cannot author production code for the task under review; cannot bypass failing tests. |
|
|
337
|
+
| **`security-auditor`** | Inspect code and dependencies for secret leakage, injection flaws, and gate compliance. | Cannot author implementation code; cannot alter security contracts. |
|
|
338
|
+
| **`coordinator`** | Parse `tasks.md`; compile `swarm.json`; monitor wave completion; reconcile task checkmarks. | Cannot implement code; cannot approve reviews; cannot bypass task dependencies. |
|
|
339
|
+
|
|
340
|
+
### Separation of Duties Invariant
|
|
341
|
+
For any task ( T ):
|
|
342
|
+
$$\text{Author}(T) \neq \text{Reviewer}(T)$$
|
|
343
|
+
If (\text{Author}(T) == \text{Reviewer}(T)\), the verification engine emits `SWARM_SELF_REVIEW_DETECTED` and halts verification fail-closed.
|
|
344
|
+
|
|
345
|
+
---
|
|
346
|
+
|
|
347
|
+
## 12. Write-Set Partitioning & Concurrency Safety
|
|
348
|
+
|
|
349
|
+
To prevent write collisions and merge conflicts during parallel execution:
|
|
350
|
+
1. **Explicit Declaration**: Every parallel task must declare a `write_set` containing exact POSIX-relative file paths or directory globs.
|
|
351
|
+
2. **Disjoint Intersection**: For any two tasks ( T_A ) and ( T_B ) scheduled in the same concurrent wave:
|
|
352
|
+
$$\text{WriteSet}(T_A) \cap \text{WriteSet}(T_B) = \emptyset$$
|
|
353
|
+
3. **Shared Read Access**: Multiple workers may concurrently read common files, but write access is strictly exclusive.
|
|
354
|
+
4. **Collision Detection**: If two concurrent tasks attempt to declare or modify overlapping paths, the planner refuses to schedule them in parallel, serializing them into sequential waves and emitting `SWARM_WRITE_COLLISION_PREVENTED`.
|
|
355
|
+
|
|
356
|
+
---
|
|
357
|
+
|
|
358
|
+
## 13. Swarm Context Projection (Upgrade D Integration)
|
|
359
|
+
|
|
360
|
+
Swarm workers must never be initialized with unbounded chat history or raw conversational logs. Gemstack enforces deterministic context projection:
|
|
361
|
+
1. **Capsule Extraction**: The swarm coordinator reads the active `context-capsule.json`.
|
|
362
|
+
2. **Role-Specific Projection**: A lightweight, role-tailored projection is compiled:
|
|
363
|
+
- **Implementer Projection**: Target task description, acceptance criteria, explicit write-set boundary, relevant interface contracts.
|
|
364
|
+
- **Reviewer Projection**: Task description, acceptance criteria, author identity, diff of modified files, test matrix criteria.
|
|
365
|
+
3. **Immutability & Provenance**: The projected context carries the `source_set_hash` of the parent capsule, ensuring workers operate against an identical, verified version of project reality.
|
|
366
|
+
|
|
367
|
+
---
|
|
368
|
+
|
|
369
|
+
## 14. Swarm Cost & Safety Gating (Upgrade C Integration)
|
|
370
|
+
|
|
371
|
+
All worker activities that interact with external services or AI model providers are governed by Upgrade C gates:
|
|
372
|
+
1. **Model Invocation Gate**: When a worker invokes an LLM API, the call is intercepted by `ProviderCapabilityGate`.
|
|
373
|
+
2. **Billable Action Gate**: The estimated token consumption and provider tier are evaluated by `BillableActionGate`.
|
|
374
|
+
3. **Ledger Recording**: Successful actions are logged into `cost-ledger.json`.
|
|
375
|
+
4. **Fail-Closed Budget Cap**: If a swarm wave exceeds the allocated feature token budget or encounters an `UNKNOWN` provider cost classification, execution halts immediately fail-closed with `SWARM_COST_LIMIT_EXCEEDED`.
|
|
376
|
+
|
|
377
|
+
---
|
|
378
|
+
|
|
379
|
+
## 15. Swarm Manifest Schema (`swarm.json`)
|
|
380
|
+
|
|
381
|
+
The canonical swarm plan and evidence tracking artifact is located at:
|
|
382
|
+
`specs/<feature-id>/swarm.json`
|
|
383
|
+
|
|
384
|
+
```json
|
|
385
|
+
{
|
|
386
|
+
"$schema": "https://gemstack.dev/schemas/v1/swarm.json",
|
|
387
|
+
"version": "1.0.0",
|
|
388
|
+
"feature_id": "010-agent-swarm-visual-qa",
|
|
389
|
+
"source_capsule_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
|
|
390
|
+
"waves": [
|
|
391
|
+
{
|
|
392
|
+
"wave_index": 1,
|
|
393
|
+
"status": "COMPLETED",
|
|
394
|
+
"tasks": [
|
|
395
|
+
{
|
|
396
|
+
"task_id": "TASK-SWARM-01",
|
|
397
|
+
"description": "Implement swarm partition validator",
|
|
398
|
+
"assigned_role": "implementer",
|
|
399
|
+
"worker_id": "worker-impl-alpha",
|
|
400
|
+
"write_set": [
|
|
401
|
+
"src/lib/swarm-planner.js"
|
|
402
|
+
],
|
|
403
|
+
"status": "COMPLETED",
|
|
404
|
+
"review": {
|
|
405
|
+
"reviewer_role": "reviewer",
|
|
406
|
+
"reviewer_id": "worker-rev-beta",
|
|
407
|
+
"status": "APPROVED",
|
|
408
|
+
"reviewed_commit_hash": "a1b2c3d4e5f6...",
|
|
409
|
+
"review_timestamp": "2026-09-11T19:00:00Z",
|
|
410
|
+
"attestation": "All acceptance criteria verified; write-set strictly adhered to."
|
|
411
|
+
}
|
|
412
|
+
}
|
|
413
|
+
]
|
|
414
|
+
}
|
|
415
|
+
]
|
|
416
|
+
}
|
|
417
|
+
```
|
|
418
|
+
|
|
419
|
+
---
|
|
420
|
+
|
|
421
|
+
## 16. Visual QA Subsystem: Architecture & Principles
|
|
422
|
+
|
|
423
|
+
Visual QA verifies the visual integrity and layout conformance of user interfaces. It is founded on three principles:
|
|
424
|
+
1. **Specification Subordination**: Visual criteria originate from `spec.md` acceptance criteria. A visual test cannot test arbitrary layout properties not grounded in specifications.
|
|
425
|
+
2. **Deterministic Capture Parameters**: Screen resolution, color depth, browser rendering flags, and device scaling must be explicitly locked.
|
|
426
|
+
3. **Cryptographic Baseline Hashing**: Visual state is tracked via canonical SHA-256 hashes of standardized image data and DOM structure trees, enabling fast, offline, deterministic verification.
|
|
427
|
+
|
|
428
|
+
---
|
|
429
|
+
|
|
430
|
+
## 17. Visual QA Manifest Schema (`visual-qa.json`)
|
|
431
|
+
|
|
432
|
+
The canonical visual QA configuration and evidence artifact is located at:
|
|
433
|
+
`specs/<feature-id>/visual-qa.json`
|
|
434
|
+
|
|
435
|
+
```json
|
|
436
|
+
{
|
|
437
|
+
"$schema": "https://gemstack.dev/schemas/v1/visual-qa.json",
|
|
438
|
+
"version": "1.0.0",
|
|
439
|
+
"feature_id": "010-agent-swarm-visual-qa",
|
|
440
|
+
"target_base_url": "http://localhost:3000",
|
|
441
|
+
"scenarios": [
|
|
442
|
+
{
|
|
443
|
+
"scenario_id": "VQA-LOGIN-001",
|
|
444
|
+
"description": "Login screen renders correctly on mobile viewport",
|
|
445
|
+
"route": "/login",
|
|
446
|
+
"viewport": {
|
|
447
|
+
"name": "mobile-portrait",
|
|
448
|
+
"width": 375,
|
|
449
|
+
"height": 667,
|
|
450
|
+
"device_scale_factor": 2,
|
|
451
|
+
"color_scheme": "light"
|
|
452
|
+
},
|
|
453
|
+
"selectors": {
|
|
454
|
+
"root": "#login-card",
|
|
455
|
+
"mask": [
|
|
456
|
+
".dynamic-timestamp",
|
|
457
|
+
".live-avatar"
|
|
458
|
+
]
|
|
459
|
+
},
|
|
460
|
+
"tolerances": {
|
|
461
|
+
"max_diff_percentage": 0.00,
|
|
462
|
+
"anti_aliasing_threshold": 0.1
|
|
463
|
+
},
|
|
464
|
+
"baseline": {
|
|
465
|
+
"image_path": "specs/010-agent-swarm-visual-qa/baselines/vqa-login-001-mobile.png",
|
|
466
|
+
"image_sha256": "8f4e2b...",
|
|
467
|
+
"dom_hash": "c5d6e7...",
|
|
468
|
+
"approved_by": "human-engineer-lead",
|
|
469
|
+
"approved_at": "2026-09-11T18:00:00Z"
|
|
470
|
+
}
|
|
471
|
+
}
|
|
472
|
+
]
|
|
473
|
+
}
|
|
474
|
+
```
|
|
475
|
+
|
|
476
|
+
---
|
|
477
|
+
|
|
478
|
+
## 18. Viewport Resolution & Determinism Matrix
|
|
479
|
+
|
|
480
|
+
Visual testing requires standardized, repeatable viewport configurations. Gemstack recognizes four canonical viewport profiles:
|
|
481
|
+
|
|
482
|
+
| Viewport Profile | Width (px) | Height (px) | Scale Factor | Intended Form Factor |
|
|
483
|
+
| :--- | :--- | :--- | :--- | :--- |
|
|
484
|
+
| **`desktop-standard`** | 1920 | 1080 | 1 | Standard 1080p desktop / laptop |
|
|
485
|
+
| **`desktop-compact`** | 1280 | 800 | 1 | Small laptop / compact window |
|
|
486
|
+
| **`tablet-portrait`** | 768 | 1024 | 2 | Medium tablet display |
|
|
487
|
+
| **`mobile-portrait`** | 375 | 667 | 2 | Standard mobile smartphone display |
|
|
488
|
+
|
|
489
|
+
Any scenario omitting viewport parameters or specifying zero/negative dimensions fails closed with `VQA_INVALID_VIEWPORT_SPECIFICATION`.
|
|
490
|
+
|
|
491
|
+
---
|
|
492
|
+
|
|
493
|
+
## 19. Dynamic Region Masking & Anti-Flake Protections
|
|
494
|
+
|
|
495
|
+
Visual tests often suffer from false-positive diffs caused by non-deterministic content. Gemstack provides deterministic masking rules:
|
|
496
|
+
1. **Mask Selectors**: Elements matching selectors declared in `selectors.mask` are deterministically masked with a solid neutral fill (`#808080`) prior to hash generation.
|
|
497
|
+
2. **Animation Freezing**: Scenarios mandate CSS animation disabling (`prefers-reduced-motion: reduce` and zero-duration CSS transitions).
|
|
498
|
+
3. **Caret & Font Smoothing**: Text cursors/carets are explicitly hidden; web fonts must complete layout before snapshot capture.
|
|
499
|
+
4. **Anti-Aliasing Threshold**: Diffs within configurable color-distance tolerances (`anti_aliasing_threshold`) are filtered to prevent cross-OS font smoothing noise.
|
|
500
|
+
|
|
501
|
+
---
|
|
502
|
+
|
|
503
|
+
## 20. Baseline Lifecycle Management & Update Semantics
|
|
504
|
+
|
|
505
|
+
Visual baselines represent authoritative visual standards:
|
|
506
|
+
1. **Explicit Initial Approval**: New scenarios lack baselines until captured evidence is reviewed and explicitly promoted by an engineer.
|
|
507
|
+
2. **Explicit Promotion Command**: Baselines are NEVER auto-updated during test execution or verification. They can only be updated via an explicit command (e.g., `gemstack vqa promote <scenario-id>`).
|
|
508
|
+
3. **Audit Trail**: Every baseline entry in `visual-qa.json` records `image_sha256`, `approved_by`, and `approved_at`.
|
|
509
|
+
4. **Tampering Detection**: If a baseline image on disk does not match its recorded `image_sha256`, verification halts with `VQA_BASELINE_TAMPERED`.
|
|
510
|
+
|
|
511
|
+
---
|
|
512
|
+
|
|
513
|
+
## 21. Visual Diffing & Regression Detection Engine
|
|
514
|
+
|
|
515
|
+
The offline diffing engine inspects live evidence against recorded baselines:
|
|
516
|
+
1. **Byte-for-Byte Digest Check**: Fast path: if live screenshot SHA-256 matches baseline `image_sha256`, the scenario immediately passes.
|
|
517
|
+
2. **Structural DOM Hash Check**: Compares normalized DOM tree hash; flags structural DOM shifts even when pixel diffs appear subtle.
|
|
518
|
+
3. **Offline Pixel Comparison**: If digests differ, the engine calculates the percentage of mismatched non-masked pixels.
|
|
519
|
+
4. **Tolerance Evaluation**: If `diff_percentage > max_diff_percentage`, the test fails closed with `VQA_VISUAL_REGRESSION_DETECTED`.
|
|
520
|
+
|
|
521
|
+
---
|
|
522
|
+
|
|
523
|
+
## 22. Interaction Between Swarm & Visual QA
|
|
524
|
+
|
|
525
|
+
Swarm workers and Visual QA interact under strict boundaries:
|
|
526
|
+
- **Worker Execution of Visual Tests**: An `implementer` worker can invoke the visual test adapter to check UI changes locally.
|
|
527
|
+
- **Evidence Submission**: The worker commits generated visual evidence to the feature's evidence directory.
|
|
528
|
+
- **Reviewer Inspection**: The independent `reviewer` worker inspects the generated visual diff report alongside the code changes.
|
|
529
|
+
- **No Baseline Promotion by Workers**: Swarm worker agents are STRICTLY FORBIDDEN from promoting baselines. Baseline approval is reserved for human developers or designated lead roles.
|
|
530
|
+
|
|
531
|
+
---
|
|
532
|
+
|
|
533
|
+
## 23. Verification Engine Integration (`gemstack verify`)
|
|
534
|
+
|
|
535
|
+
`gemstack verify` enforces Upgrade E validation in pure read-only mode:
|
|
536
|
+
1. **Swarm Verification**:
|
|
537
|
+
- Validates `swarm.json` schema and syntax.
|
|
538
|
+
- Verifies all parallel tasks possess disjoint write-sets.
|
|
539
|
+
- Verifies `AUTHOR != REVIEWER` separation on all approved tasks.
|
|
540
|
+
- Proves no closed task lacks an independent review attestation.
|
|
541
|
+
2. **Visual QA Verification**:
|
|
542
|
+
- Validates `visual-qa.json` schema and scenario definitions.
|
|
543
|
+
- Validates that every baseline image exists and matches its recorded SHA-256 hash.
|
|
544
|
+
- Verifies visual evidence completeness for all declared scenarios.
|
|
545
|
+
- Ensures no baseline images were mutated during the verify run.
|
|
546
|
+
3. **Offline Purity**: The entire verification process executes offline with zero socket calls, zero child process spawns, and zero browser launches.
|
|
547
|
+
|
|
548
|
+
---
|
|
549
|
+
|
|
550
|
+
## 24. Determinism & Cryptographic Provenance
|
|
551
|
+
|
|
552
|
+
Determinism is guaranteed across all Upgrade E artifacts:
|
|
553
|
+
- **Canonical Object Key Ordering**: All JSON serialization strictly orders keys using UTF-16 code units.
|
|
554
|
+
- **POSIX Path Normalization**: File paths in write sets and image locations are stored as POSIX-relative strings without drive letters or trailing slashes.
|
|
555
|
+
- **Image Hash Standards**: Screenshot files are hashed using SHA-256 over raw image bytes.
|
|
556
|
+
- **DOM Normalization**: DOM snapshots strip ephemeral session IDs, formatting attributes deterministically before hashing.
|
|
557
|
+
|
|
558
|
+
---
|
|
559
|
+
|
|
560
|
+
## 25. Security & Isolation Architecture
|
|
561
|
+
|
|
562
|
+
Upgrade E protects repository security through:
|
|
563
|
+
- **File System Boundary Enforcement**: Swarm write-sets are restricted to repository-relative paths. Attempts to write to system directories, parent directories (`../`), or git internals (`.git/`) are strictly blocked.
|
|
564
|
+
- **Secret Masking & Exclusion**: Visual QA captures are forbidden from screenshotting or logging sensitive input fields (password inputs, authorization tokens). Elements with `type="password"` or `data-sensitive="true"` are automatically masked.
|
|
565
|
+
- **No Untrusted Code Execution in Verify**: Verification reads data manifests and hashes; it never evaluates dynamic JavaScript or arbitrary plugins.
|
|
566
|
+
|
|
567
|
+
---
|
|
568
|
+
|
|
569
|
+
## 26. Performance & Scalability Targets
|
|
570
|
+
|
|
571
|
+
- **Swarm Schedule Validation**: Validating a 50-task swarm partition graph takes under 100ms.
|
|
572
|
+
- **Visual Manifest Verification**: Verifying 100 visual scenarios and baseline hashes takes under 500ms offline.
|
|
573
|
+
- **Memory Footprint**: Validation runs within standard Node.js process limits (<128MB RAM).
|
|
574
|
+
- **Core Package Size**: Zero bytes added for browser engines; zero external runtime dependencies added to `package.json`.
|
|
575
|
+
|
|
576
|
+
---
|
|
577
|
+
|
|
578
|
+
## 27. Error Codes & Diagnostic Catalog
|
|
579
|
+
|
|
580
|
+
| Code | Severity | Trigger Condition |
|
|
581
|
+
| :--- | :--- | :--- |
|
|
582
|
+
| **`SWARM_WRITE_COLLISION`** | FATAL | Two concurrent tasks declare overlapping write-set file paths. |
|
|
583
|
+
| **`SWARM_SELF_REVIEW_DETECTED`** | FATAL | The author of a task is recorded as its reviewer. |
|
|
584
|
+
| **`SWARM_MISSING_REVIEW`** | ERROR | A completed task lacks an approved independent review attestation. |
|
|
585
|
+
| **`SWARM_WRITE_SET_VIOLATION`** | FATAL | A worker modified files outside its declared write-set partition. |
|
|
586
|
+
| **`SWARM_CONTEXT_UNPROJECTED`** | ERROR | A worker context payload was generated without capsule provenance. |
|
|
587
|
+
| **`VQA_INVALID_VIEWPORT`** | ERROR | Scenario viewport specification is missing or invalid. |
|
|
588
|
+
| **`VQA_BASELINE_TAMPERED`** | FATAL | Baseline image file hash does not match recorded `image_sha256`. |
|
|
589
|
+
| **`VQA_VISUAL_REGRESSION`** | ERROR | Live visual evidence deviates from baseline beyond allowed tolerance. |
|
|
590
|
+
| **`VQA_UNAUTHORIZED_PROMOTION`** | FATAL | An unauthorized worker role attempted to promote a visual baseline. |
|
|
591
|
+
| **`VQA_MASK_FAILURE`** | WARNING | Declared mask selector did not match any element in captured DOM. |
|
|
592
|
+
|
|
593
|
+
---
|
|
594
|
+
|
|
595
|
+
## 28. Comprehensive Threat Model & Adversarial Vectors
|
|
596
|
+
|
|
597
|
+
| Threat Vector | Vulnerability Description | Mitigation Architecture |
|
|
598
|
+
| :--- | :--- | :--- |
|
|
599
|
+
| **1. Rogue Swarm Worker** | Worker attempts to edit `package.json` or security gates. | Write-set partitioning strictly limits worker write lease; verify detects out-of-bounds diffs. |
|
|
600
|
+
| **2. Self-Approving Agent** | Agent implements a flaw and signs off its own review. | Fail-closed `AUTHOR != REVIEWER` check rejects identical author/reviewer IDs. |
|
|
601
|
+
| **3. Flaky Font CI Diff** | Cross-platform font rendering causes 0.05% pixel shift. | Configurable anti-aliasing threshold and neutral mask filters eliminate subpixel noise. |
|
|
602
|
+
| **4. Silent Baseline Overwrite**| Automated tool overwrites baseline images during test run. | Verify enforces read-only purity; baseline files are cryptographically pinned in manifest. |
|
|
603
|
+
| **5. Password Leak in Screenshot** | Login visual QA captures user credentials in screenshot. | Sensitive inputs (`type=password`) are automatically masked with solid fill before capture. |
|
|
604
|
+
| **6. Token Runaway in Swarm** | Parallel workers enter infinite retry loop consuming tokens. | Upgrade C BillableActionGate intercepts all calls and halts execution on budget breach. |
|
|
605
|
+
| **7. Context Smuggling** | Worker ingests private developer prompts or uncommitted keys. | Swarm context is strictly projected from validated, secret-scanned Context Capsule. |
|
|
606
|
+
| **8. Parallel Race Condition** | Two workers modify the same module simultaneously. | Concurrency planner enforces mutually exclusive disjoint write sets per wave. |
|
|
607
|
+
| **9. Verify Browser Hang** | `gemstack verify` hangs in CI waiting for headless Chrome. | Verify does not launch browsers; it validates evidence, manifests, and hashes offline. |
|
|
608
|
+
| **10. Spec Overwrite by Visual QA** | Agent attempts to alter spec criteria because screenshot failed. | Spec authority is supreme; visual evidence is subordinate; spec edits require human review. |
|
|
609
|
+
|
|
610
|
+
---
|
|
611
|
+
|
|
612
|
+
## 29. Canonical Acceptance Matrix (TEST-SWARM-A01 .. E02 & TEST-VISUAL-A01 .. E02)
|
|
613
|
+
|
|
614
|
+
Upgrade E establishes exactly 20 canonical acceptance test requirements:
|
|
615
|
+
|
|
616
|
+
```gemstack-test-matrix
|
|
617
|
+
[
|
|
618
|
+
{
|
|
619
|
+
"id": "TEST-SWARM-A01",
|
|
620
|
+
"category": "SWARM_PARTITIONING",
|
|
621
|
+
"layer": "UNIT",
|
|
622
|
+
"description": "Validates that parallel tasks with disjoint write sets schedule cleanly in concurrent waves.",
|
|
623
|
+
"pass_criteria": "Planner assigns tasks to same wave when write sets share zero common files.",
|
|
624
|
+
"gate": "REQUIRED"
|
|
625
|
+
},
|
|
626
|
+
{
|
|
627
|
+
"id": "TEST-SWARM-A02",
|
|
628
|
+
"category": "SWARM_PARTITIONING",
|
|
629
|
+
"layer": "UNIT",
|
|
630
|
+
"description": "Detects overlapping write sets between parallel tasks and serializes them into sequential waves.",
|
|
631
|
+
"pass_criteria": "Emits SWARM_WRITE_COLLISION_PREVENTED and separates conflicting tasks into wave N and N+1.",
|
|
632
|
+
"gate": "REQUIRED"
|
|
633
|
+
},
|
|
634
|
+
{
|
|
635
|
+
"id": "TEST-SWARM-B01",
|
|
636
|
+
"category": "SEPARATION_OF_DUTIES",
|
|
637
|
+
"layer": "UNIT",
|
|
638
|
+
"description": "Rejects task review attestation when author ID matches reviewer ID.",
|
|
639
|
+
"pass_criteria": "Emits SWARM_SELF_REVIEW_DETECTED and fails validation closed.",
|
|
640
|
+
"gate": "REQUIRED"
|
|
641
|
+
},
|
|
642
|
+
{
|
|
643
|
+
"id": "TEST-SWARM-B02",
|
|
644
|
+
"category": "SEPARATION_OF_DUTIES",
|
|
645
|
+
"layer": "UNIT",
|
|
646
|
+
"description": "Accepts task review when performed by a distinct, authorized reviewer role.",
|
|
647
|
+
"pass_criteria": "Review status validates as APPROVED with verified reviewer signature.",
|
|
648
|
+
"gate": "REQUIRED"
|
|
649
|
+
},
|
|
650
|
+
{
|
|
651
|
+
"id": "TEST-SWARM-C01",
|
|
652
|
+
"category": "GATE_INTEGRATION",
|
|
653
|
+
"layer": "INTEGRATION",
|
|
654
|
+
"description": "Proves swarm worker model invocation is intercepted by Upgrade C ProviderCapabilityGate.",
|
|
655
|
+
"pass_criteria": "Invocation fails closed when provider capability is undeclared or missing.",
|
|
656
|
+
"gate": "REQUIRED"
|
|
657
|
+
},
|
|
658
|
+
{
|
|
659
|
+
"id": "TEST-SWARM-C02",
|
|
660
|
+
"category": "GATE_INTEGRATION",
|
|
661
|
+
"layer": "INTEGRATION",
|
|
662
|
+
"description": "Enforces budget limits across concurrent swarm workers via BillableActionGate.",
|
|
663
|
+
"pass_criteria": "Halted with SWARM_COST_LIMIT_EXCEEDED when cumulative wave tokens exceed budget.",
|
|
664
|
+
"gate": "REQUIRED"
|
|
665
|
+
},
|
|
666
|
+
{
|
|
667
|
+
"id": "TEST-SWARM-D01",
|
|
668
|
+
"category": "CONTEXT_PROJECTION",
|
|
669
|
+
"layer": "UNIT",
|
|
670
|
+
"description": "Validates that worker context projection matches authoritative context-capsule hash.",
|
|
671
|
+
"pass_criteria": "Projection source_capsule_hash matches active capsule digest 100%.",
|
|
672
|
+
"gate": "REQUIRED"
|
|
673
|
+
},
|
|
674
|
+
{
|
|
675
|
+
"id": "TEST-SWARM-D02",
|
|
676
|
+
"category": "CONTEXT_PROJECTION",
|
|
677
|
+
"layer": "UNIT",
|
|
678
|
+
"description": "Excludes historical chat transcripts and conversational narrative from worker payloads.",
|
|
679
|
+
"pass_criteria": "Projected payload contains only structured task, spec, and contract data.",
|
|
680
|
+
"gate": "REQUIRED"
|
|
681
|
+
},
|
|
682
|
+
{
|
|
683
|
+
"id": "TEST-SWARM-E01",
|
|
684
|
+
"category": "SWARM_VERIFY_PURITY",
|
|
685
|
+
"layer": "INTEGRATION",
|
|
686
|
+
"description": "Proves gemstack verify checks swarm.json manifest in pure read-only mode.",
|
|
687
|
+
"pass_criteria": "Zero file mutations and zero process spawns during verification.",
|
|
688
|
+
"gate": "REQUIRED"
|
|
689
|
+
},
|
|
690
|
+
{
|
|
691
|
+
"id": "TEST-SWARM-E02",
|
|
692
|
+
"category": "SWARM_LEGACY",
|
|
693
|
+
"layer": "UNIT",
|
|
694
|
+
"description": "Preserves backward compatibility for repositories lacking swarm schedules.",
|
|
695
|
+
"pass_criteria": "Verification passes with exit code 0 when swarm.json is absent.",
|
|
696
|
+
"gate": "REQUIRED"
|
|
697
|
+
},
|
|
698
|
+
{
|
|
699
|
+
"id": "TEST-VISUAL-A01",
|
|
700
|
+
"category": "VISUAL_MANIFEST",
|
|
701
|
+
"layer": "UNIT",
|
|
702
|
+
"description": "Validates visual-qa.json schema conformance and required scenario properties.",
|
|
703
|
+
"pass_criteria": "Rejects manifests missing route, viewport, or baseline references.",
|
|
704
|
+
"gate": "REQUIRED"
|
|
705
|
+
},
|
|
706
|
+
{
|
|
707
|
+
"id": "TEST-VISUAL-A02",
|
|
708
|
+
"category": "VISUAL_MANIFEST",
|
|
709
|
+
"layer": "UNIT",
|
|
710
|
+
"description": "Validates deterministic viewport specifications across standard profiles.",
|
|
711
|
+
"pass_criteria": "Passes valid profiles; fails invalid or negative width/height dimensions.",
|
|
712
|
+
"gate": "REQUIRED"
|
|
713
|
+
},
|
|
714
|
+
{
|
|
715
|
+
"id": "TEST-VISUAL-B01",
|
|
716
|
+
"category": "BASELINE_SAFETY",
|
|
717
|
+
"layer": "UNIT",
|
|
718
|
+
"description": "Verifies that baseline image files match their recorded SHA-256 hashes.",
|
|
719
|
+
"pass_criteria": "Emits VQA_BASELINE_TAMPERED if an image file is modified on disk.",
|
|
720
|
+
"gate": "REQUIRED"
|
|
721
|
+
},
|
|
722
|
+
{
|
|
723
|
+
"id": "TEST-VISUAL-B02",
|
|
724
|
+
"category": "BASELINE_SAFETY",
|
|
725
|
+
"layer": "INTEGRATION",
|
|
726
|
+
"description": "Proves baseline images are never silently modified during test execution or verification.",
|
|
727
|
+
"pass_criteria": "Baseline image file timestamps and hashes remain unchanged after test run.",
|
|
728
|
+
"gate": "REQUIRED"
|
|
729
|
+
},
|
|
730
|
+
{
|
|
731
|
+
"id": "TEST-VISUAL-C01",
|
|
732
|
+
"category": "REGRESSION_DETECTION",
|
|
733
|
+
"layer": "UNIT",
|
|
734
|
+
"description": "Detects visual regression when live screenshot deviates from baseline beyond threshold.",
|
|
735
|
+
"pass_criteria": "Fails validation with VQA_VISUAL_REGRESSION_DETECTED and diff metrics.",
|
|
736
|
+
"gate": "REQUIRED"
|
|
737
|
+
},
|
|
738
|
+
{
|
|
739
|
+
"id": "TEST-VISUAL-C02",
|
|
740
|
+
"category": "REGRESSION_DETECTION",
|
|
741
|
+
"layer": "UNIT",
|
|
742
|
+
"description": "Passes comparison cleanly when image hashes match 100%.",
|
|
743
|
+
"pass_criteria": "Immediate pass without pixel diffing overhead when SHA-256 matches.",
|
|
744
|
+
"gate": "REQUIRED"
|
|
745
|
+
},
|
|
746
|
+
{
|
|
747
|
+
"id": "TEST-VISUAL-D01",
|
|
748
|
+
"category": "MASKING_PROTECTION",
|
|
749
|
+
"layer": "UNIT",
|
|
750
|
+
"description": "Applies deterministic neutral masking to dynamic selectors prior to diffing.",
|
|
751
|
+
"pass_criteria": "Dynamic text variations within masked bounding boxes do not trigger diff failure.",
|
|
752
|
+
"gate": "REQUIRED"
|
|
753
|
+
},
|
|
754
|
+
{
|
|
755
|
+
"id": "TEST-VISUAL-D02",
|
|
756
|
+
"category": "MASKING_PROTECTION",
|
|
757
|
+
"layer": "UNIT",
|
|
758
|
+
"description": "Enforces mandatory automatic masking of password and sensitive credential fields.",
|
|
759
|
+
"pass_criteria": "Password inputs are replaced with neutral mask fill in all evidence captures.",
|
|
760
|
+
"gate": "REQUIRED"
|
|
761
|
+
},
|
|
762
|
+
{
|
|
763
|
+
"id": "TEST-VISUAL-E01",
|
|
764
|
+
"category": "VISUAL_VERIFY_PURITY",
|
|
765
|
+
"layer": "INTEGRATION",
|
|
766
|
+
"description": "Proves gemstack verify validates visual evidence offline with zero browser spawns.",
|
|
767
|
+
"pass_criteria": "Verification passes completely offline with network sockets and child processes mocked to throw.",
|
|
768
|
+
"gate": "REQUIRED"
|
|
769
|
+
},
|
|
770
|
+
{
|
|
771
|
+
"id": "TEST-VISUAL-E02",
|
|
772
|
+
"category": "VISUAL_LEGACY",
|
|
773
|
+
"layer": "UNIT",
|
|
774
|
+
"description": "Preserves clean verification for backend or headless projects lacking visual QA manifests.",
|
|
775
|
+
"pass_criteria": "Verification passes with exit code 0 when visual-qa.json is absent.",
|
|
776
|
+
"gate": "REQUIRED"
|
|
777
|
+
}
|
|
778
|
+
]
|
|
779
|
+
```
|
|
780
|
+
|
|
781
|
+
---
|
|
782
|
+
|
|
783
|
+
## 30. CLI & Developer Ergonomics (Design Only)
|
|
784
|
+
|
|
785
|
+
Upgrade E introduces dedicated subcommands under the `gemstack` CLI:
|
|
786
|
+
|
|
787
|
+
```text
|
|
788
|
+
gemstack swarm plan Compile tasks.md into deterministic swarm.json waves
|
|
789
|
+
gemstack swarm validate Validate write-set partitions and review attestations
|
|
790
|
+
gemstack vqa validate Validate visual-qa.json schema, baselines, and evidence
|
|
791
|
+
gemstack vqa promote <scenario> Explicitly promote live evidence to approved baseline
|
|
792
|
+
```
|
|
793
|
+
|
|
794
|
+
All subcommands maintain strict argument validation, JSON output formatting with `--json`, and zero file modifications during validation operations.
|
|
795
|
+
|
|
796
|
+
---
|
|
797
|
+
|
|
798
|
+
## 31. Future Test Strategy (Design Only)
|
|
799
|
+
|
|
800
|
+
*Note: Tests are NOT implemented during this specification phase. This strategy defines the mechanical requirements for future implementation.*
|
|
801
|
+
|
|
802
|
+
1. **Swarm Partitioning Suite (`tests/swarm-partitioning.test.js`)**:
|
|
803
|
+
- Asserts non-overlapping write sets schedule concurrently.
|
|
804
|
+
- Injects colliding paths; verifies sequential wave placement and warning emission.
|
|
805
|
+
2. **Separation of Duties Suite (`tests/swarm-review.test.js`)**:
|
|
806
|
+
- Asserts `AUTHOR == REVIEWER` triggers fail-closed error `SWARM_SELF_REVIEW_DETECTED`.
|
|
807
|
+
- Validates valid multi-agent review attestation structure.
|
|
808
|
+
3. **Visual QA Manifest Suite (`tests/visual-manifest.test.js`)**:
|
|
809
|
+
- Tests viewport validation (positive integers, standard aspect ratios).
|
|
810
|
+
- Validates baseline cryptographic hash integrity checks.
|
|
811
|
+
4. **Visual Regression & Masking Suite (`tests/visual-diff.test.js`)**:
|
|
812
|
+
- Injects modified pixels outside mask; asserts `VQA_VISUAL_REGRESSION_DETECTED`.
|
|
813
|
+
- Injects dynamic text inside mask; asserts clean pass.
|
|
814
|
+
5. **Verification Purity Suite (`tests/swarm-visual-purity.test.js`)**:
|
|
815
|
+
- Runs `gemstack verify` with network sockets disabled and child process spawns mocked to throw.
|
|
816
|
+
- Proves zero file mutations (hash tree comparison before and after verify).
|
|
817
|
+
|
|
818
|
+
---
|
|
819
|
+
|
|
820
|
+
## 32. Migration & Backward Compatibility Considerations
|
|
821
|
+
|
|
822
|
+
- **No Breaking Changes**: Existing repositories built with Gemstack v1.0, v1.1, v1.2, or v1.3 operate without modification.
|
|
823
|
+
- **Optional Adoption**: Projects that do not use multi-agent swarms or visual UI components omit `swarm.json` and `visual-qa.json`; verification treats their absence as valid legacy state.
|
|
824
|
+
- **Sidecar Non-Interference**: Swarm and visual metadata do not interfere with feature state sidecars (`.gemstack/state.json`) or context capsules.
|
|
825
|
+
|
|
826
|
+
---
|
|
827
|
+
|
|
828
|
+
## 33. Explicit Deferred Items
|
|
829
|
+
|
|
830
|
+
The following concepts are explicitly deferred and out of scope for Upgrade E:
|
|
831
|
+
1. **Live Process Daemon / Cluster Management**: Embedded background process supervisors or socket servers (non-goal).
|
|
832
|
+
2. **Bundled Browser Binaries**: Heavy Chromium/Playwright binaries inside npm package dependencies (non-goal).
|
|
833
|
+
3. **Fuzzy AI Aesthetic Grading**: Subjective LLM opinions on UI beauty or styling quality (non-goal).
|
|
834
|
+
4. **Distributed Swarm Consensus**: Multi-node consensus protocols or cross-network swarm federation (out of scope).
|
|
835
|
+
5. **Package Version Bump / Release**: Version bumping and npm publishing (strictly deferred to release phase).
|
|
836
|
+
|
|
837
|
+
---
|
|
838
|
+
|
|
839
|
+
## 34. Open Questions
|
|
840
|
+
|
|
841
|
+
**NONE.**
|
|
842
|
+
Both architectural decisions (Swarm planning & evidence contracts, decoupled offline visual evidence validation) are fully resolved. Invariants, schemas, review boundaries, and acceptance matrices are completely defined with zero ambiguity.
|