gsdd-cli 0.20.0 → 0.22.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/agents/executor.md +12 -1
- package/agents/planner.md +2 -0
- package/agents/verifier.md +1 -1
- package/bin/lib/health.mjs +2 -2
- package/bin/lib/init-runtime.mjs +5 -2
- package/bin/lib/lifecycle-preflight.mjs +5 -0
- package/bin/lib/rendering.mjs +3 -1
- package/bin/lib/session-fingerprint.mjs +42 -2
- package/bin/lib/ui-proof.mjs +521 -16
- package/distilled/DESIGN.md +24 -7
- package/distilled/EVIDENCE-INDEX.md +13 -0
- package/distilled/templates/delegates/plan-checker.md +1 -0
- package/distilled/templates/ui-proof.md +33 -7
- package/distilled/workflows/execute.md +2 -1
- package/distilled/workflows/plan-milestone-gaps.md +21 -0
- package/distilled/workflows/plan.md +2 -1
- package/distilled/workflows/quick.md +5 -2
- package/distilled/workflows/verify.md +2 -1
- package/package.json +1 -1
package/agents/executor.md
CHANGED
|
@@ -211,7 +211,18 @@ Before reporting a task complete:
|
|
|
211
211
|
- A task is not complete because code was written. It is complete when the intended verification path actually passes.
|
|
212
212
|
|
|
213
213
|
### UI Proof Execution
|
|
214
|
-
If the plan defines UI proof slots, record observed proof against the exact claim, route/state, observation, evidence kind, artifact path or manual step, privacy metadata, result, and claim limit before claiming task completion.
|
|
214
|
+
If the plan defines UI proof slots, record observed proof against the exact claim, route/state, observation, evidence kind, artifact path or manual step, privacy metadata, result, and claim limit before claiming task completion.
|
|
215
|
+
|
|
216
|
+
Use `agent-browser` as the default live UI proof path:
|
|
217
|
+
- open the planned route/state
|
|
218
|
+
- capture interactive snapshots/refs when relevant
|
|
219
|
+
- exercise the changed flow
|
|
220
|
+
- capture screenshots for the planned viewport(s)
|
|
221
|
+
- record relevant console/network observations
|
|
222
|
+
|
|
223
|
+
If `agent-browser` is unavailable, record the availability constraint and closest project-native interactive browser fallback in the proof bundle instead of silently treating the fallback as the default path. Existing Playwright/package-script browser tests remain canonical repeatable regression evidence when present; use Playwright scripting only for checks `agent-browser` cannot cover cleanly, such as JS-disabled, structured console, or multi-context verification.
|
|
224
|
+
|
|
225
|
+
Artifact metadata must include `visibility`, `retention`, `sensitivity`, and `safe_to_publish`; raw screenshots, traces, videos, DOM snapshots, and reports are local-only/unsafe by default and cannot back public, tracked, delivery, release, or publication proof claims. Use `gsdd ui-proof validate <path>` or `gsdd health` when a bundle exists. Artifact count, source comments, AST/cAST findings, semantic search, and Semble-like retrieval are not proof. Missing or weakly linked evidence must be recorded as proof debt, waiver, deferment, or reduced claim language rather than satisfied proof.
|
|
215
226
|
</execution_loop>
|
|
216
227
|
|
|
217
228
|
<checkpoint_protocol>
|
package/agents/planner.md
CHANGED
|
@@ -158,6 +158,8 @@ For UI-sensitive work, plan proof slots that can later be matched exactly to cla
|
|
|
158
158
|
|
|
159
159
|
Require observed artifacts to carry `visibility`, `retention`, `sensitivity`, and `safe_to_publish`; when a planned slot is meant to support public, publication, tracked, delivery, or release proof, say to validate the observed bundle with `gsdd ui-proof validate <path> --claim <...>`. `gsdd ui-proof validate`/`gsdd health` must catch invalid bundle metadata when present.
|
|
160
160
|
|
|
161
|
+
For live rendered UI proof, plan `agent-browser` as the default runtime evidence path: route open, interactive snapshot/refs when relevant, changed-flow interaction, screenshots for the planned viewport(s), and relevant console/network observations. If `agent-browser` is unavailable in the runtime, require an explicit availability constraint and the closest project-native interactive browser fallback before narrowing the claim. Existing Playwright/package-script browser tests remain the canonical repeatable regression path when present; do not scaffold new browser infrastructure by default. The planner chooses viewport coverage, but must explain why the viewport set is sufficient for the claim or narrow the claim limit; responsive claims need desktop/mobile or equivalent state coverage.
|
|
162
|
+
|
|
161
163
|
Do not let source annotations, AST/cAST findings, semantic search, comments, or Semble-like retrieval satisfy proof slots; they are discovery hints only. Human acceptance can narrow or waive a claim and record proof debt, but it must not turn missing or mismatched non-human evidence into `satisfied` proof.
|
|
162
164
|
</ui_proof_planning>
|
|
163
165
|
|
package/agents/verifier.md
CHANGED
|
@@ -123,7 +123,7 @@ Do not return a flat symptom list when the same underlying breakage explains mul
|
|
|
123
123
|
|
|
124
124
|
Visual correctness, live interaction quality, and some external integrations still need explicit human checks.
|
|
125
125
|
|
|
126
|
-
For UI proof slots, fail closed unless observed proof is matched to the exact claim, route/state, observation, evidence kind, artifact path or manual step, privacy metadata, result, and claim limit. Artifact metadata must include `visibility`, `retention`, `sensitivity`, and `safe_to_publish`; local-only or unsafe artifacts cannot back public, tracked, delivery, release, or publication proof claims, and `gsdd ui-proof validate`/`gsdd health` metadata failures block the stronger proof claim. Screenshots, traces, reports, Gherkin, a11y scans, E2E outputs, manual notes, source annotations, AST/cAST findings, semantic search, comments, and Semble-like retrieval do not satisfy proof by existence alone. Human acceptance records risk, waiver, deferment, proof debt, or a narrowed claim; it does not upgrade missing or mismatched non-human proof to `satisfied`.
|
|
126
|
+
For UI proof slots, fail closed unless observed proof is matched to the exact claim, route/state, observation, evidence kind, artifact path or manual step, privacy metadata, result, and claim limit. For live UI runtime proof, expect `agent-browser` as the default captured tool unless the observed bundle explains a project-native equivalent or an availability constraint; do not fail solely because another browser tool was used, but downgrade vague proof that lacks exact route/state, viewport coverage or rationale, interactive steps/refs where relevant, screenshot/report artifacts, or relevant console/network observations. Existing Playwright/package-script browser tests count as canonical repeatable regression evidence, not as a replacement for scoped runtime proof when the slot requires `runtime`. Artifact metadata must include `visibility`, `retention`, `sensitivity`, and `safe_to_publish`; local-only or unsafe artifacts cannot back public, tracked, delivery, release, or publication proof claims, and `gsdd ui-proof validate`/`gsdd health` metadata failures block the stronger proof claim. Screenshots, traces, reports, Gherkin, a11y scans, E2E outputs, manual notes, source annotations, AST/cAST findings, semantic search, comments, and Semble-like retrieval do not satisfy proof by existence alone. Human acceptance records risk, waiver, deferment, proof debt, or a narrowed claim; it does not upgrade missing or mismatched non-human proof to `satisfied`.
|
|
127
127
|
|
|
128
128
|
## Step 9: Determine overall status
|
|
129
129
|
|
package/bin/lib/health.mjs
CHANGED
|
@@ -147,13 +147,13 @@ export function createCmdHealth(ctx) {
|
|
|
147
147
|
const parsed = readUiProofBundleFile(bundlePath);
|
|
148
148
|
const validation = parsed.errors.length > 0
|
|
149
149
|
? { valid: false, errors: parsed.errors }
|
|
150
|
-
: validateUiProofBundle(parsed.bundle);
|
|
150
|
+
: validateUiProofBundle(parsed.bundle, { requireLocalArtifactExists: true, workspaceRoot: cwd });
|
|
151
151
|
if (!validation.valid) {
|
|
152
152
|
errors.push({
|
|
153
153
|
id: 'E10',
|
|
154
154
|
severity: 'ERROR',
|
|
155
155
|
message: `${relativePath} has invalid UI proof metadata (${validation.errors.map((entry) => entry.code).join(', ')})`,
|
|
156
|
-
fix: 'Run `gsdd ui-proof validate <path>` and add required privacy metadata, claim limits, fixed evidence kinds, observation artifact references, and safe-to-publish handling.',
|
|
156
|
+
fix: 'Run `gsdd ui-proof validate <path>` and add required privacy metadata, claim limits, fixed evidence kinds, concise tool provenance, failure classification when failed or partial, observation artifact references, existing local artifact paths, and safe-to-publish handling.',
|
|
157
157
|
});
|
|
158
158
|
}
|
|
159
159
|
}
|
package/bin/lib/init-runtime.mjs
CHANGED
|
@@ -186,9 +186,12 @@ Commands:
|
|
|
186
186
|
phase-status <N> <status> Update ROADMAP.md phase status ([ ] / [-] / [x])
|
|
187
187
|
lifecycle-preflight <surface> [phase]
|
|
188
188
|
Inspect deterministic lifecycle gate results for a workflow surface
|
|
189
|
-
session-fingerprint write
|
|
189
|
+
session-fingerprint write [--allow-changed <ROADMAP.md,SPEC.md,config.json>]
|
|
190
|
+
Rebaseline planning-state drift after reviewing changed planning files
|
|
190
191
|
ui-proof validate <path> [--claim <public|publication|tracked|delivery|release>]
|
|
191
192
|
Validate UI proof metadata; use --claim for stronger proof uses
|
|
193
|
+
ui-proof compare <planned-slots-json> [observed-bundle-json ...]
|
|
194
|
+
Compare planned UI proof slots against observed bundles
|
|
192
195
|
help Show this summary
|
|
193
196
|
|
|
194
197
|
Platforms (for --tools):
|
|
@@ -259,7 +262,7 @@ Advanced/internal helpers (kept available, but not the primary first-run user st
|
|
|
259
262
|
lifecycle-preflight Inspect deterministic lifecycle gate results for a workflow surface
|
|
260
263
|
session-fingerprint Rebaseline the local planning-state fingerprint after review
|
|
261
264
|
phase-status Update ROADMAP.md phase status through the local helper surface
|
|
262
|
-
ui-proof Validate UI proof metadata
|
|
265
|
+
ui-proof Validate UI proof metadata and compare planned slots to observed bundles
|
|
263
266
|
file-op Deterministic workspace-confined file copy/delete/text mutation
|
|
264
267
|
`;
|
|
265
268
|
}
|
|
@@ -52,6 +52,11 @@ const SURFACE_POLICIES = {
|
|
|
52
52
|
ownedWrites: ['spec', 'roadmap', 'phase-directories'],
|
|
53
53
|
explicitLifecycleMutation: 'none',
|
|
54
54
|
},
|
|
55
|
+
'plan-milestone-gaps': {
|
|
56
|
+
classification: 'owned_write',
|
|
57
|
+
ownedWrites: ['roadmap', 'phase-directories'],
|
|
58
|
+
explicitLifecycleMutation: 'none',
|
|
59
|
+
},
|
|
55
60
|
resume: {
|
|
56
61
|
classification: 'owned_write',
|
|
57
62
|
ownedWrites: ['checkpoint-cleanup'],
|
package/bin/lib/rendering.mjs
CHANGED
|
@@ -72,10 +72,12 @@ function printHelp() {
|
|
|
72
72
|
' lifecycle-preflight <surface> [phase]',
|
|
73
73
|
' Inspect lifecycle gate results for a workflow surface',
|
|
74
74
|
' Example: node .planning/bin/gsdd.mjs lifecycle-preflight verify 1 --expects-mutation phase-status',
|
|
75
|
-
' session-fingerprint write',
|
|
75
|
+
' session-fingerprint write [--allow-changed <ROADMAP.md,SPEC.md,config.json>]',
|
|
76
76
|
' Rebaseline planning-state drift after reviewing changed planning files',
|
|
77
77
|
' ui-proof validate <path> [--claim <public|publication|tracked|delivery|release>]',
|
|
78
78
|
' Validate UI proof metadata; use --claim for stronger proof uses',
|
|
79
|
+
' ui-proof compare <planned-slots-json> [observed-bundle-json ...]',
|
|
80
|
+
' Compare planned UI proof slots against observed bundles',
|
|
79
81
|
'',
|
|
80
82
|
'Advanced option:',
|
|
81
83
|
' --workspace-root <path> Override workspace root discovery before or after the subcommand',
|
|
@@ -63,16 +63,56 @@ export function cmdSessionFingerprint(...args) {
|
|
|
63
63
|
return;
|
|
64
64
|
}
|
|
65
65
|
|
|
66
|
-
const [action] = normalizedArgs;
|
|
66
|
+
const [action, ...flags] = normalizedArgs;
|
|
67
67
|
if (action !== 'write') {
|
|
68
|
-
console.error('Usage: node .planning/bin/gsdd.mjs session-fingerprint write');
|
|
68
|
+
console.error('Usage: node .planning/bin/gsdd.mjs session-fingerprint write [--allow-changed <ROADMAP.md,SPEC.md,config.json>]');
|
|
69
69
|
process.exitCode = 1;
|
|
70
70
|
return;
|
|
71
71
|
}
|
|
72
72
|
|
|
73
|
+
const allowChanged = parseAllowChanged(flags);
|
|
74
|
+
if (allowChanged.invalid) {
|
|
75
|
+
console.error('Usage: node .planning/bin/gsdd.mjs session-fingerprint write [--allow-changed <ROADMAP.md,SPEC.md,config.json>]');
|
|
76
|
+
process.exitCode = 1;
|
|
77
|
+
return;
|
|
78
|
+
}
|
|
79
|
+
|
|
80
|
+
if (allowChanged.files.length > 0) {
|
|
81
|
+
const drift = checkDrift(planningDir);
|
|
82
|
+
const changedFiles = drift.files.filter((file) => file.status !== 'unchanged').map((file) => file.file);
|
|
83
|
+
const unexpected = changedFiles.filter((file) => !allowChanged.files.includes(file));
|
|
84
|
+
if (unexpected.length > 0) {
|
|
85
|
+
output({
|
|
86
|
+
operation: 'session-fingerprint write',
|
|
87
|
+
changedFiles,
|
|
88
|
+
allowedChanged: allowChanged.files,
|
|
89
|
+
written: false,
|
|
90
|
+
reason: 'unexpected_planning_drift',
|
|
91
|
+
unexpected,
|
|
92
|
+
});
|
|
93
|
+
process.exitCode = 1;
|
|
94
|
+
return;
|
|
95
|
+
}
|
|
96
|
+
}
|
|
97
|
+
|
|
73
98
|
output({ operation: 'session-fingerprint write', fingerprint: writeFingerprint(planningDir) });
|
|
74
99
|
}
|
|
75
100
|
|
|
101
|
+
function parseAllowChanged(flags) {
|
|
102
|
+
const files = [];
|
|
103
|
+
for (let index = 0; index < flags.length; index += 1) {
|
|
104
|
+
if (flags[index] !== '--allow-changed') return { invalid: true, files: [] };
|
|
105
|
+
const value = flags[index + 1];
|
|
106
|
+
if (!value || value.startsWith('--')) return { invalid: true, files: [] };
|
|
107
|
+
files.push(...value.split(',').map((entry) => entry.trim()).filter(Boolean));
|
|
108
|
+
index += 1;
|
|
109
|
+
}
|
|
110
|
+
for (const file of files) {
|
|
111
|
+
if (!FINGERPRINT_SOURCES.includes(file)) return { invalid: true, files: [] };
|
|
112
|
+
}
|
|
113
|
+
return { invalid: false, files: [...new Set(files)] };
|
|
114
|
+
}
|
|
115
|
+
|
|
76
116
|
/**
|
|
77
117
|
* Read the stored fingerprint from .planning/.state-fingerprint.json.
|
|
78
118
|
* Returns null if the file does not exist or is unparseable.
|
package/bin/lib/ui-proof.mjs
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import { existsSync, readFileSync, readdirSync, statSync } from 'fs';
|
|
2
|
-
import { isAbsolute, join, relative, resolve } from 'path';
|
|
2
|
+
import { dirname, isAbsolute, join, relative, resolve } from 'path';
|
|
3
3
|
import { output } from './cli-utils.mjs';
|
|
4
4
|
import { resolveWorkspaceContext } from './workspace-root.mjs';
|
|
5
5
|
|
|
@@ -10,6 +10,8 @@ const ARTIFACT_VISIBILITIES = Object.freeze(['local_only', 'repo_tracked', 'publ
|
|
|
10
10
|
const RAW_ARTIFACT_TYPES = Object.freeze(['screenshot', 'trace', 'video', 'dom_snapshot', 'dom-snapshot', 'dom', 'report']);
|
|
11
11
|
const PUBLIC_CLAIM_USES = Object.freeze(['public', 'publication', 'tracked', 'delivery', 'release']);
|
|
12
12
|
const CLAIM_USES = Object.freeze([...PUBLIC_CLAIM_USES, 'local', 'local_only']);
|
|
13
|
+
const FAILURE_CLASSIFICATIONS = Object.freeze(['product_bug', 'missing_infra', 'flaky_harness', 'ambiguous_spec']);
|
|
14
|
+
const TOOL_ID_PATTERN = /^[a-z0-9][a-z0-9_.:-]*$/;
|
|
13
15
|
const REQUIRED_BUNDLE_FIELDS = Object.freeze([
|
|
14
16
|
'proof_bundle_version',
|
|
15
17
|
'scope',
|
|
@@ -25,6 +27,19 @@ const REQUIRED_BUNDLE_FIELDS = Object.freeze([
|
|
|
25
27
|
'claim_limits',
|
|
26
28
|
]);
|
|
27
29
|
const REQUIRED_SCOPE_FIELDS = Object.freeze(['work_item', 'claim', 'requirement_ids', 'slot_ids']);
|
|
30
|
+
const REQUIRED_SLOT_FIELDS = Object.freeze([
|
|
31
|
+
'slot_id',
|
|
32
|
+
'claim',
|
|
33
|
+
'route_state',
|
|
34
|
+
'required_evidence_kinds',
|
|
35
|
+
'minimum_observations',
|
|
36
|
+
'environment',
|
|
37
|
+
'viewport',
|
|
38
|
+
'expected_artifact_types',
|
|
39
|
+
'validation_command',
|
|
40
|
+
'manual_acceptance_required',
|
|
41
|
+
'claim_limit',
|
|
42
|
+
]);
|
|
28
43
|
const REQUIRED_ARTIFACT_FIELDS = Object.freeze(['visibility', 'retention', 'sensitivity', 'safe_to_publish']);
|
|
29
44
|
const REQUIRED_OBSERVATION_FIELDS = Object.freeze(['observation', 'claim', 'route_state', 'evidence_kind', 'artifact_refs', 'privacy', 'result', 'claim_limit']);
|
|
30
45
|
const REQUIRED_PRIVACY_FIELDS = Object.freeze(['data_classification', 'raw_artifacts_safe_to_publish', 'retention']);
|
|
@@ -72,12 +87,12 @@ function normalizeArray(value) {
|
|
|
72
87
|
|
|
73
88
|
function artifactType(artifact) {
|
|
74
89
|
const explicit = typeof artifact.type === 'string' ? artifact.type.toLowerCase() : '';
|
|
75
|
-
const
|
|
76
|
-
if (/screenshot|\.png$|\.jpe?g$|\.webp$/.test(
|
|
77
|
-
if (/trace|\.zip$/.test(
|
|
78
|
-
if (/video|\.mp4$|\.webm$|\.mov$/.test(
|
|
79
|
-
if (/dom|\.html?$/.test(
|
|
80
|
-
if (/report/.test(
|
|
90
|
+
const artifactRef = artifactReference(artifact)?.toLowerCase() || '';
|
|
91
|
+
if (/screenshot|\.png$|\.jpe?g$|\.webp$/.test(artifactRef)) return 'screenshot';
|
|
92
|
+
if (/trace|\.zip$/.test(artifactRef)) return 'trace';
|
|
93
|
+
if (/video|\.mp4$|\.webm$|\.mov$/.test(artifactRef)) return 'video';
|
|
94
|
+
if (/dom|\.html?$/.test(artifactRef)) return 'dom_snapshot';
|
|
95
|
+
if (/report/.test(artifactRef)) return 'report';
|
|
81
96
|
return explicit;
|
|
82
97
|
}
|
|
83
98
|
|
|
@@ -172,6 +187,19 @@ function validateEvidenceKinds(bundle, errors) {
|
|
|
172
187
|
}
|
|
173
188
|
}
|
|
174
189
|
|
|
190
|
+
function validateToolsUsed(bundle, errors) {
|
|
191
|
+
const tools = normalizeArray(bundle?.evidence_inputs?.tools_used);
|
|
192
|
+
if (tools.length === 0) {
|
|
193
|
+
addError(errors, 'missing_tools_used', 'evidence_inputs.tools_used', 'Missing UI proof tool provenance.', 'Record concise tool IDs such as browser, playwright, manual, or project-specific command IDs.');
|
|
194
|
+
return;
|
|
195
|
+
}
|
|
196
|
+
for (const [index, tool] of tools.entries()) {
|
|
197
|
+
if (!TOOL_ID_PATTERN.test(tool)) {
|
|
198
|
+
addError(errors, 'invalid_tool_id', `evidence_inputs.tools_used[${index}]`, `Invalid UI proof tool identifier: ${tool}`, 'Use a concise lowercase identifier without spaces, for example browser, playwright, manual, or gsdd-ui-proof-validate.');
|
|
199
|
+
}
|
|
200
|
+
}
|
|
201
|
+
}
|
|
202
|
+
|
|
175
203
|
function validateResult(bundle, errors) {
|
|
176
204
|
if (!isPlainObject(bundle?.result)) return;
|
|
177
205
|
if (!hasValue(bundle.result.claim_status)) {
|
|
@@ -181,6 +209,28 @@ function validateResult(bundle, errors) {
|
|
|
181
209
|
}
|
|
182
210
|
}
|
|
183
211
|
|
|
212
|
+
function validateFailureClassification(bundle, errors) {
|
|
213
|
+
const statuses = [
|
|
214
|
+
bundle?.result?.claim_status,
|
|
215
|
+
...Object.values(isPlainObject(bundle?.result?.comparison_status_by_slot) ? bundle.result.comparison_status_by_slot : {}),
|
|
216
|
+
...normalizeArray(bundle?.observations).map((observation) => isPlainObject(observation) ? observation.result : null),
|
|
217
|
+
...normalizeArray(bundle?.commands_or_manual_steps).map((step) => isPlainObject(step) ? step.result : null),
|
|
218
|
+
].filter(Boolean);
|
|
219
|
+
const failedOrPartial = statuses.some((status) => status === 'failed' || status === 'partial');
|
|
220
|
+
const classifications = normalizeArray(bundle?.result?.failure_classification || bundle?.result?.failure_classifications);
|
|
221
|
+
|
|
222
|
+
if (failedOrPartial && classifications.length === 0) {
|
|
223
|
+
addError(errors, 'missing_failure_classification', 'result.failure_classification', 'Failed or partial UI proof must classify why it failed.', `Use one of: ${FAILURE_CLASSIFICATIONS.join(', ')}.`);
|
|
224
|
+
return;
|
|
225
|
+
}
|
|
226
|
+
|
|
227
|
+
for (const [index, classification] of classifications.entries()) {
|
|
228
|
+
if (!FAILURE_CLASSIFICATIONS.includes(classification)) {
|
|
229
|
+
addError(errors, 'invalid_failure_classification', `result.failure_classification[${index}]`, `Invalid UI proof failure classification: ${classification}`, `Use only: ${FAILURE_CLASSIFICATIONS.join(', ')}.`);
|
|
230
|
+
}
|
|
231
|
+
}
|
|
232
|
+
}
|
|
233
|
+
|
|
184
234
|
function validateComparisonStatuses(bundle, errors) {
|
|
185
235
|
const statuses = bundle?.result?.comparison_status_by_slot;
|
|
186
236
|
if (!isPlainObject(statuses)) {
|
|
@@ -202,6 +252,10 @@ function validateComparisonStatuses(bundle, errors) {
|
|
|
202
252
|
addError(errors, 'invalid_comparison_status', `result.comparison_status_by_slot.${slot}`, `Invalid UI proof comparison status: ${status}`, `Use only: ${COMPARISON_STATUSES.join(', ')}.`);
|
|
203
253
|
}
|
|
204
254
|
}
|
|
255
|
+
const unsatisfiedStatuses = Object.values(statuses).filter((status) => !['satisfied', 'not_applicable'].includes(status));
|
|
256
|
+
if (bundle?.result?.claim_status === 'passed' && unsatisfiedStatuses.length > 0) {
|
|
257
|
+
addError(errors, 'inconsistent_claim_status', 'result.claim_status', 'UI proof claim_status cannot be passed when comparison statuses are unsatisfied.', 'Use partial, failed, waived, deferred, or not_applicable when any slot comparison is not satisfied.');
|
|
258
|
+
}
|
|
205
259
|
}
|
|
206
260
|
|
|
207
261
|
function validateClaimLimits(bundle, errors) {
|
|
@@ -234,7 +288,7 @@ function isSanitizedSensitivity(value) {
|
|
|
234
288
|
return typeof value === 'string' && /(^|[_\s-])(sanitized|public_safe|public-safe)($|[_\s-])/.test(value.toLowerCase());
|
|
235
289
|
}
|
|
236
290
|
|
|
237
|
-
function validateArtifacts(bundle, errors, publicClaim) {
|
|
291
|
+
function validateArtifacts(bundle, errors, publicClaim, options = {}) {
|
|
238
292
|
const artifacts = normalizeArray(bundle?.artifacts);
|
|
239
293
|
if (artifacts.length === 0) {
|
|
240
294
|
addError(errors, 'missing_artifacts', 'artifacts', 'Missing UI proof artifacts list.', 'Record artifact metadata for each referenced proof artifact.');
|
|
@@ -254,6 +308,12 @@ function validateArtifacts(bundle, errors, publicClaim) {
|
|
|
254
308
|
} else {
|
|
255
309
|
validateArtifactReferenceSafety(ref, artifactPath, errors);
|
|
256
310
|
artifactRefs.add(ref);
|
|
311
|
+
if (options.requireLocalArtifactExists && !/^https?:\/\//i.test(ref) && hasValue(options.workspaceRoot)) {
|
|
312
|
+
const artifactFile = resolve(options.workspaceRoot, ref);
|
|
313
|
+
if (!existsSync(artifactFile) || statSync(artifactFile).isDirectory()) {
|
|
314
|
+
addError(errors, 'missing_local_artifact', artifactPath, `UI proof artifact file does not exist: ${ref}`, 'Create the referenced artifact, correct the path, or narrow the proof claim.');
|
|
315
|
+
}
|
|
316
|
+
}
|
|
257
317
|
}
|
|
258
318
|
for (const field of REQUIRED_ARTIFACT_FIELDS) {
|
|
259
319
|
requireField(artifact, field, artifactPath, errors);
|
|
@@ -277,6 +337,40 @@ function validateArtifacts(bundle, errors, publicClaim) {
|
|
|
277
337
|
return artifactRefs;
|
|
278
338
|
}
|
|
279
339
|
|
|
340
|
+
export function validateUiProofSlots(slots) {
|
|
341
|
+
const errors = [];
|
|
342
|
+
const normalizedSlots = normalizeArray(slots);
|
|
343
|
+
if (normalizedSlots.length === 0) {
|
|
344
|
+
addError(errors, 'missing_planned_slots', 'ui_proof_slots', 'Planned UI proof input must include at least one slot.', 'Provide ui_proof_slots or no_ui_proof_rationale for non-UI work.');
|
|
345
|
+
return { valid: false, errors, warnings: [] };
|
|
346
|
+
}
|
|
347
|
+
|
|
348
|
+
for (const [index, slot] of normalizedSlots.entries()) {
|
|
349
|
+
const slotPath = `ui_proof_slots[${index}]`;
|
|
350
|
+
if (!isPlainObject(slot)) {
|
|
351
|
+
addError(errors, 'invalid_planned_slot', slotPath, 'Planned UI proof slot must be an object.', 'Record a scoped slot with claim, route_state, viewport, evidence, artifacts, validation, and claim limit.');
|
|
352
|
+
continue;
|
|
353
|
+
}
|
|
354
|
+
for (const field of REQUIRED_SLOT_FIELDS) requireField(slot, field, slotPath, errors);
|
|
355
|
+
for (const [kindIndex, kind] of normalizeArray(slot.required_evidence_kinds || slot.requiredEvidenceKinds).entries()) {
|
|
356
|
+
if (!EVIDENCE_KINDS.includes(kind)) {
|
|
357
|
+
addError(errors, 'unsupported_planned_evidence_kind', `${slotPath}.required_evidence_kinds[${kindIndex}]`, `Unsupported planned UI proof evidence kind: ${kind}`, `Use only: ${EVIDENCE_KINDS.join(', ')}.`);
|
|
358
|
+
}
|
|
359
|
+
}
|
|
360
|
+
if (hasValue(slot.manual_acceptance_required) && typeof slot.manual_acceptance_required !== 'boolean') {
|
|
361
|
+
addError(errors, 'invalid_manual_acceptance_required', `${slotPath}.manual_acceptance_required`, 'manual_acceptance_required must be a boolean.', 'Use true only when human judgment is required for this slot; otherwise use false.');
|
|
362
|
+
}
|
|
363
|
+
if (normalizeArray(slot.minimum_observations || slot.minimumObservations).length === 0) {
|
|
364
|
+
addError(errors, 'missing_minimum_observations', `${slotPath}.minimum_observations`, 'Planned UI proof slot must include minimum observations.', 'List the observations execution must prove for this slot.');
|
|
365
|
+
}
|
|
366
|
+
if (normalizeArray(slot.expected_artifact_types || slot.expectedArtifactTypes).length === 0) {
|
|
367
|
+
addError(errors, 'missing_expected_artifact_types', `${slotPath}.expected_artifact_types`, 'Planned UI proof slot must include expected artifact types.', 'List expected artifact types such as screenshot, trace, report, or dom_snapshot.');
|
|
368
|
+
}
|
|
369
|
+
}
|
|
370
|
+
|
|
371
|
+
return { valid: errors.length === 0, errors, warnings: [] };
|
|
372
|
+
}
|
|
373
|
+
|
|
280
374
|
function validatePrivacy(bundle, errors, publicClaim) {
|
|
281
375
|
validateObservationPrivacy(bundle.privacy, 'privacy', errors);
|
|
282
376
|
if (publicClaim && bundle.privacy?.raw_artifacts_safe_to_publish !== true) {
|
|
@@ -305,6 +399,316 @@ function validateObservationArtifactRefs(bundle, artifactRefs, errors) {
|
|
|
305
399
|
}
|
|
306
400
|
}
|
|
307
401
|
|
|
402
|
+
function stableString(value) {
|
|
403
|
+
return JSON.stringify(canonicalize(value));
|
|
404
|
+
}
|
|
405
|
+
|
|
406
|
+
function canonicalize(value) {
|
|
407
|
+
if (Array.isArray(value)) return value.map(canonicalize);
|
|
408
|
+
if (!isPlainObject(value)) return value;
|
|
409
|
+
return Object.fromEntries(Object.keys(value).sort().map((key) => [key, canonicalize(value[key])]));
|
|
410
|
+
}
|
|
411
|
+
|
|
412
|
+
function valuesMatch(planned, observed) {
|
|
413
|
+
if (!hasValue(planned)) return true;
|
|
414
|
+
if (!hasValue(observed)) return false;
|
|
415
|
+
return stableString(planned) === stableString(observed);
|
|
416
|
+
}
|
|
417
|
+
|
|
418
|
+
function slotId(slot, index) {
|
|
419
|
+
return slot?.slot_id || slot?.slotId || slot?.id || `ui-proof-slot-${index + 1}`;
|
|
420
|
+
}
|
|
421
|
+
|
|
422
|
+
function observationText(observation) {
|
|
423
|
+
if (typeof observation === 'string') return observation;
|
|
424
|
+
if (isPlainObject(observation) && typeof observation.observation === 'string') return observation.observation;
|
|
425
|
+
return '';
|
|
426
|
+
}
|
|
427
|
+
|
|
428
|
+
function includesObservation(observations, expected) {
|
|
429
|
+
const expectedText = typeof expected === 'string' ? expected.trim() : observationText(expected).trim();
|
|
430
|
+
if (!expectedText) return true;
|
|
431
|
+
return observations.some((observation) => observationText(observation).includes(expectedText));
|
|
432
|
+
}
|
|
433
|
+
|
|
434
|
+
function normalizeObservedBundle(entry) {
|
|
435
|
+
if (entry?.bundle) {
|
|
436
|
+
return {
|
|
437
|
+
bundle: entry.bundle,
|
|
438
|
+
validation: entry.validation || validateUiProofBundle(entry.bundle, entry.options || {}),
|
|
439
|
+
source: entry.source || entry.filePath || 'observed bundle',
|
|
440
|
+
};
|
|
441
|
+
}
|
|
442
|
+
return {
|
|
443
|
+
bundle: entry,
|
|
444
|
+
validation: validateUiProofBundle(entry),
|
|
445
|
+
source: 'observed bundle',
|
|
446
|
+
};
|
|
447
|
+
}
|
|
448
|
+
|
|
449
|
+
function compareSlotToBundle(slot, slotIdValue, observed) {
|
|
450
|
+
const issues = [];
|
|
451
|
+
const bundle = observed.bundle;
|
|
452
|
+
const observations = normalizeArray(bundle?.observations);
|
|
453
|
+
if (!observed.validation.valid) {
|
|
454
|
+
issues.push({
|
|
455
|
+
code: 'invalid_observed_bundle',
|
|
456
|
+
path: observed.source,
|
|
457
|
+
message: `Observed UI proof bundle for slot ${slotIdValue} failed metadata validation.`,
|
|
458
|
+
details: observed.validation.errors,
|
|
459
|
+
});
|
|
460
|
+
}
|
|
461
|
+
|
|
462
|
+
const bundleStatus = bundle?.result?.comparison_status_by_slot?.[slotIdValue];
|
|
463
|
+
if (bundle?.result?.claim_status !== 'passed') {
|
|
464
|
+
issues.push({
|
|
465
|
+
code: 'unsatisfied_observed_claim_status',
|
|
466
|
+
path: 'result.claim_status',
|
|
467
|
+
message: `Observed UI proof bundle claim status is ${bundle?.result?.claim_status || 'missing'} for slot ${slotIdValue}.`,
|
|
468
|
+
});
|
|
469
|
+
}
|
|
470
|
+
if (bundleStatus !== 'satisfied') {
|
|
471
|
+
issues.push({
|
|
472
|
+
code: 'unsatisfied_observed_comparison_status',
|
|
473
|
+
path: `result.comparison_status_by_slot.${slotIdValue}`,
|
|
474
|
+
message: `Observed UI proof bundle reports ${bundleStatus || 'missing'} for slot ${slotIdValue}.`,
|
|
475
|
+
});
|
|
476
|
+
}
|
|
477
|
+
|
|
478
|
+
const requiredKinds = normalizeArray(slot?.required_evidence_kinds || slot?.requiredEvidenceKinds);
|
|
479
|
+
const observedKinds = normalizeArray(bundle?.evidence_inputs?.kinds);
|
|
480
|
+
const missingKinds = requiredKinds.filter((kind) => !observedKinds.includes(kind));
|
|
481
|
+
if (missingKinds.length > 0) {
|
|
482
|
+
issues.push({
|
|
483
|
+
code: 'missing_required_evidence_kind',
|
|
484
|
+
path: 'evidence_inputs.kinds',
|
|
485
|
+
message: `Observed UI proof for slot ${slotIdValue} is missing required evidence kind(s): ${missingKinds.join(', ')}.`,
|
|
486
|
+
});
|
|
487
|
+
}
|
|
488
|
+
const missingNonHuman = missingKinds.filter((kind) => kind !== 'human');
|
|
489
|
+
if (missingNonHuman.length > 0 && observedKinds.includes('human')) {
|
|
490
|
+
issues.push({
|
|
491
|
+
code: 'human_evidence_cannot_bypass_required_non_human_evidence',
|
|
492
|
+
path: 'evidence_inputs.kinds',
|
|
493
|
+
message: `Human evidence cannot satisfy missing non-human UI proof evidence for slot ${slotIdValue}: ${missingNonHuman.join(', ')}.`,
|
|
494
|
+
});
|
|
495
|
+
}
|
|
496
|
+
|
|
497
|
+
if (!valuesMatch(slot?.route_state || slot?.routeState, bundle?.route_state)) {
|
|
498
|
+
issues.push({
|
|
499
|
+
code: 'route_state_mismatch',
|
|
500
|
+
path: 'route_state',
|
|
501
|
+
message: `Observed UI proof route/state does not match planned slot ${slotIdValue}.`,
|
|
502
|
+
});
|
|
503
|
+
}
|
|
504
|
+
|
|
505
|
+
if (!valuesMatch(slot?.environment, bundle?.environment)) {
|
|
506
|
+
issues.push({
|
|
507
|
+
code: 'environment_mismatch',
|
|
508
|
+
path: 'environment',
|
|
509
|
+
message: `Observed UI proof environment does not match planned slot ${slotIdValue}.`,
|
|
510
|
+
});
|
|
511
|
+
}
|
|
512
|
+
|
|
513
|
+
if (!valuesMatch(slot?.viewport, bundle?.viewport)) {
|
|
514
|
+
issues.push({
|
|
515
|
+
code: 'viewport_mismatch',
|
|
516
|
+
path: 'viewport',
|
|
517
|
+
message: `Observed UI proof viewport does not match planned slot ${slotIdValue}.`,
|
|
518
|
+
});
|
|
519
|
+
}
|
|
520
|
+
|
|
521
|
+
const requirementId = slot?.requirement_id || slot?.requirementId;
|
|
522
|
+
if (hasValue(requirementId) && !normalizeArray(bundle?.scope?.requirement_ids).includes(requirementId)) {
|
|
523
|
+
issues.push({
|
|
524
|
+
code: 'requirement_mismatch',
|
|
525
|
+
path: 'scope.requirement_ids',
|
|
526
|
+
message: `Observed UI proof bundle does not declare planned requirement ${requirementId} for slot ${slotIdValue}.`,
|
|
527
|
+
});
|
|
528
|
+
}
|
|
529
|
+
|
|
530
|
+
if (hasValue(slot?.claim) && bundle?.scope?.claim !== slot.claim) {
|
|
531
|
+
issues.push({
|
|
532
|
+
code: 'claim_mismatch',
|
|
533
|
+
path: 'scope.claim',
|
|
534
|
+
message: `Observed UI proof bundle claim does not match planned slot ${slotIdValue}.`,
|
|
535
|
+
});
|
|
536
|
+
}
|
|
537
|
+
|
|
538
|
+
if (hasValue(slot?.claim) && !observations.some((observation) => observation?.claim === slot.claim)) {
|
|
539
|
+
issues.push({
|
|
540
|
+
code: 'observation_claim_mismatch',
|
|
541
|
+
path: 'observations[].claim',
|
|
542
|
+
message: `Observed UI proof observations do not support the exact planned claim for slot ${slotIdValue}.`,
|
|
543
|
+
});
|
|
544
|
+
}
|
|
545
|
+
|
|
546
|
+
const supportingObservations = observations
|
|
547
|
+
.map((observation, index) => ({ observation, index }))
|
|
548
|
+
.filter(({ observation }) => !hasValue(slot?.claim) || observation?.claim === slot.claim);
|
|
549
|
+
|
|
550
|
+
if (hasValue(slot?.route_state || slot?.routeState)) {
|
|
551
|
+
for (const { observation, index } of supportingObservations) {
|
|
552
|
+
if (!valuesMatch(slot?.route_state || slot?.routeState, observation?.route_state)) {
|
|
553
|
+
issues.push({
|
|
554
|
+
code: 'observation_route_state_mismatch',
|
|
555
|
+
path: `observations[${index}].route_state`,
|
|
556
|
+
message: `Observed UI proof observation route/state does not match planned slot ${slotIdValue}.`,
|
|
557
|
+
});
|
|
558
|
+
}
|
|
559
|
+
}
|
|
560
|
+
}
|
|
561
|
+
|
|
562
|
+
const passedSupportingKinds = new Set(
|
|
563
|
+
supportingObservations
|
|
564
|
+
.filter(({ observation }) => observation?.result === 'passed')
|
|
565
|
+
.map(({ observation }) => observation?.evidence_kind)
|
|
566
|
+
.filter(Boolean)
|
|
567
|
+
);
|
|
568
|
+
const missingSupportingKinds = requiredKinds.filter((kind) => !passedSupportingKinds.has(kind));
|
|
569
|
+
if (missingSupportingKinds.length > 0) {
|
|
570
|
+
issues.push({
|
|
571
|
+
code: 'missing_supporting_observation_evidence_kind',
|
|
572
|
+
path: 'observations[].evidence_kind',
|
|
573
|
+
message: `Observed UI proof for slot ${slotIdValue} lacks passed supporting observation(s) for required evidence kind(s): ${missingSupportingKinds.join(', ')}.`,
|
|
574
|
+
});
|
|
575
|
+
}
|
|
576
|
+
|
|
577
|
+
for (const [index, step] of normalizeArray(bundle?.commands_or_manual_steps).entries()) {
|
|
578
|
+
if (step?.result !== 'passed') {
|
|
579
|
+
issues.push({
|
|
580
|
+
code: 'unsatisfied_proof_step',
|
|
581
|
+
path: `commands_or_manual_steps[${index}].result`,
|
|
582
|
+
message: `Observed UI proof command/manual step is ${step?.result || 'missing'} for slot ${slotIdValue}.`,
|
|
583
|
+
});
|
|
584
|
+
}
|
|
585
|
+
}
|
|
586
|
+
|
|
587
|
+
const manualAcceptanceRequired = slot?.manual_acceptance_required === true || slot?.manualAcceptanceRequired === true;
|
|
588
|
+
if (manualAcceptanceRequired) {
|
|
589
|
+
if (!observedKinds.includes('human')) {
|
|
590
|
+
issues.push({
|
|
591
|
+
code: 'missing_manual_acceptance_evidence',
|
|
592
|
+
path: 'evidence_inputs.kinds',
|
|
593
|
+
message: `Observed UI proof for slot ${slotIdValue} is missing required human evidence for manual acceptance.`,
|
|
594
|
+
});
|
|
595
|
+
}
|
|
596
|
+
if (!passedSupportingKinds.has('human')) {
|
|
597
|
+
issues.push({
|
|
598
|
+
code: 'missing_manual_acceptance_observation',
|
|
599
|
+
path: 'observations[].evidence_kind',
|
|
600
|
+
message: `Observed UI proof for slot ${slotIdValue} lacks a passed human observation for manual acceptance.`,
|
|
601
|
+
});
|
|
602
|
+
}
|
|
603
|
+
}
|
|
604
|
+
|
|
605
|
+
for (const { observation, index } of supportingObservations) {
|
|
606
|
+
if (observation?.result !== 'passed') {
|
|
607
|
+
issues.push({
|
|
608
|
+
code: 'unsatisfied_observation_result',
|
|
609
|
+
path: `observations[${index}].result`,
|
|
610
|
+
message: `Observed UI proof observation is ${observation?.result || 'missing'} for slot ${slotIdValue}.`,
|
|
611
|
+
});
|
|
612
|
+
}
|
|
613
|
+
}
|
|
614
|
+
|
|
615
|
+
for (const expected of normalizeArray(slot?.minimum_observations || slot?.minimumObservations)) {
|
|
616
|
+
if (!includesObservation(supportingObservations.map(({ observation }) => observation), expected)) {
|
|
617
|
+
issues.push({
|
|
618
|
+
code: 'missing_minimum_observation',
|
|
619
|
+
path: 'observations',
|
|
620
|
+
message: `Observed UI proof for slot ${slotIdValue} is missing a planned minimum observation.`,
|
|
621
|
+
});
|
|
622
|
+
}
|
|
623
|
+
}
|
|
624
|
+
|
|
625
|
+
if (hasValue(slot?.claim_limit || slot?.claimLimit)) {
|
|
626
|
+
const claimLimit = slot.claim_limit || slot.claimLimit;
|
|
627
|
+
if (!normalizeArray(bundle?.claim_limits).includes(claimLimit)) {
|
|
628
|
+
issues.push({
|
|
629
|
+
code: 'missing_claim_limit',
|
|
630
|
+
path: 'claim_limits',
|
|
631
|
+
message: `Observed UI proof for slot ${slotIdValue} does not preserve the planned claim limit.`,
|
|
632
|
+
});
|
|
633
|
+
}
|
|
634
|
+
}
|
|
635
|
+
|
|
636
|
+
const expectedArtifactTypes = normalizeArray(slot?.expected_artifact_types || slot?.expectedArtifactTypes);
|
|
637
|
+
const observedArtifactTypes = new Set(normalizeArray(bundle?.artifacts).filter(isPlainObject).flatMap((artifact) => [
|
|
638
|
+
typeof artifact.type === 'string' ? artifact.type.toLowerCase() : '',
|
|
639
|
+
artifactType(artifact),
|
|
640
|
+
]).filter(Boolean));
|
|
641
|
+
const missingArtifactTypes = expectedArtifactTypes.filter((type) => !observedArtifactTypes.has(type));
|
|
642
|
+
if (missingArtifactTypes.length > 0) {
|
|
643
|
+
issues.push({
|
|
644
|
+
code: 'missing_expected_artifact_type',
|
|
645
|
+
path: 'artifacts[].type',
|
|
646
|
+
message: `Observed UI proof for slot ${slotIdValue} is missing expected artifact type(s): ${missingArtifactTypes.join(', ')}.`,
|
|
647
|
+
});
|
|
648
|
+
}
|
|
649
|
+
|
|
650
|
+
const status = issues.length === 0 ? 'satisfied' : (bundleStatus === 'missing' ? 'missing' : 'partial');
|
|
651
|
+
return { status, issues, source: observed.source };
|
|
652
|
+
}
|
|
653
|
+
|
|
654
|
+
export function compareUiProofSlots(plannedSlots, observedBundles) {
|
|
655
|
+
const slots = normalizeArray(plannedSlots);
|
|
656
|
+
const slotValidation = validateUiProofSlots(slots);
|
|
657
|
+
const bundles = normalizeArray(observedBundles).map(normalizeObservedBundle);
|
|
658
|
+
const results = [];
|
|
659
|
+
const errors = [...slotValidation.errors];
|
|
660
|
+
|
|
661
|
+
for (const observed of bundles) {
|
|
662
|
+
if (!observed.validation.valid) {
|
|
663
|
+
errors.push({
|
|
664
|
+
code: 'invalid_observed_bundle',
|
|
665
|
+
path: observed.source,
|
|
666
|
+
message: `Observed UI proof bundle ${observed.source} failed metadata validation.`,
|
|
667
|
+
details: observed.validation.errors,
|
|
668
|
+
});
|
|
669
|
+
}
|
|
670
|
+
}
|
|
671
|
+
|
|
672
|
+
for (const [index, slot] of slots.entries()) {
|
|
673
|
+
const slotIdValue = slotId(slot, index);
|
|
674
|
+
const matchingBundles = bundles.filter((observed) => normalizeArray(observed.bundle?.scope?.slot_ids).includes(slotIdValue));
|
|
675
|
+
if (matchingBundles.length === 0) {
|
|
676
|
+
results.push({
|
|
677
|
+
slot_id: slotIdValue,
|
|
678
|
+
status: 'missing',
|
|
679
|
+
issues: [{
|
|
680
|
+
code: 'missing_observed_bundle',
|
|
681
|
+
path: 'scope.slot_ids',
|
|
682
|
+
message: `No observed UI proof bundle declares planned slot ${slotIdValue}.`,
|
|
683
|
+
}],
|
|
684
|
+
});
|
|
685
|
+
continue;
|
|
686
|
+
}
|
|
687
|
+
|
|
688
|
+
const candidates = matchingBundles.map((observed) => compareSlotToBundle(slot, slotIdValue, observed));
|
|
689
|
+
const satisfied = candidates.find((candidate) => candidate.status === 'satisfied');
|
|
690
|
+
if (satisfied) {
|
|
691
|
+
results.push({ slot_id: slotIdValue, status: 'satisfied', issues: [], source: satisfied.source });
|
|
692
|
+
continue;
|
|
693
|
+
}
|
|
694
|
+
const partial = candidates.find((candidate) => candidate.status === 'partial') || candidates[0];
|
|
695
|
+
results.push({ slot_id: slotIdValue, status: partial.status, issues: partial.issues, source: partial.source });
|
|
696
|
+
}
|
|
697
|
+
|
|
698
|
+
const statuses = results.map((result) => result.status);
|
|
699
|
+
const status = errors.length > 0
|
|
700
|
+
? 'partial'
|
|
701
|
+
: statuses.length === 0
|
|
702
|
+
? 'not_applicable'
|
|
703
|
+
: statuses.every((value) => value === 'satisfied')
|
|
704
|
+
? 'satisfied'
|
|
705
|
+
: statuses.every((value) => value === 'missing')
|
|
706
|
+
? 'missing'
|
|
707
|
+
: 'partial';
|
|
708
|
+
|
|
709
|
+
return { status, slots: results, errors };
|
|
710
|
+
}
|
|
711
|
+
|
|
308
712
|
export function validateUiProofBundle(bundle, options = {}) {
|
|
309
713
|
const errors = [];
|
|
310
714
|
const warnings = [];
|
|
@@ -319,23 +723,25 @@ export function validateUiProofBundle(bundle, options = {}) {
|
|
|
319
723
|
const publicClaim = hasPublicClaim(bundle, options);
|
|
320
724
|
validateClaimUses(bundle, options, errors);
|
|
321
725
|
validateEvidenceKinds(bundle, errors);
|
|
726
|
+
validateToolsUsed(bundle, errors);
|
|
322
727
|
validateCommandsOrManualSteps(bundle, errors);
|
|
323
728
|
validateObservations(bundle, errors);
|
|
324
729
|
validateResult(bundle, errors);
|
|
730
|
+
validateFailureClassification(bundle, errors);
|
|
325
731
|
validateComparisonStatuses(bundle, errors);
|
|
326
732
|
validateClaimLimits(bundle, errors);
|
|
327
733
|
validatePrivacy(bundle, errors, publicClaim);
|
|
328
734
|
validatePublicObservationPrivacy(bundle, errors, publicClaim);
|
|
329
|
-
const artifactRefs = validateArtifacts(bundle, errors, publicClaim);
|
|
735
|
+
const artifactRefs = validateArtifacts(bundle, errors, publicClaim, options);
|
|
330
736
|
validateObservationArtifactRefs(bundle, artifactRefs, errors);
|
|
331
737
|
|
|
332
738
|
return { valid: errors.length === 0, errors, warnings };
|
|
333
739
|
}
|
|
334
740
|
|
|
335
|
-
|
|
741
|
+
function parseJsonOrFencedContent(content, filePath, label) {
|
|
336
742
|
const trimmed = content.trim();
|
|
337
743
|
if (!trimmed) {
|
|
338
|
-
return {
|
|
744
|
+
return { value: null, errors: [{ code: 'empty_file', path: filePath, message: `${label} file is empty.`, fix: 'Write JSON metadata before validating.' }] };
|
|
339
745
|
}
|
|
340
746
|
|
|
341
747
|
const jsonCandidates = [trimmed];
|
|
@@ -344,15 +750,62 @@ export function parseUiProofBundleContent(content, filePath = 'UI proof bundle')
|
|
|
344
750
|
|
|
345
751
|
for (const candidate of jsonCandidates) {
|
|
346
752
|
try {
|
|
347
|
-
return {
|
|
753
|
+
return { value: JSON.parse(candidate), errors: [] };
|
|
348
754
|
} catch {
|
|
349
755
|
// Try next candidate; final error is reported below.
|
|
350
756
|
}
|
|
351
757
|
}
|
|
352
758
|
|
|
353
759
|
return {
|
|
354
|
-
|
|
355
|
-
errors: [{ code: '
|
|
760
|
+
value: null,
|
|
761
|
+
errors: [{ code: 'unparseable_json', path: filePath, message: `${label} metadata is not valid JSON.`, fix: 'Use a .json file or a markdown fenced JSON block; no YAML parser dependency is installed.' }],
|
|
762
|
+
};
|
|
763
|
+
}
|
|
764
|
+
|
|
765
|
+
export function parseUiProofBundleContent(content, filePath = 'UI proof bundle') {
|
|
766
|
+
const parsed = parseJsonOrFencedContent(content, filePath, 'UI proof bundle');
|
|
767
|
+
return { bundle: parsed.value, errors: parsed.errors.map((error) => ({
|
|
768
|
+
...error,
|
|
769
|
+
code: error.code === 'empty_file' ? 'empty_bundle_file' : error.code === 'unparseable_json' ? 'unparseable_bundle' : error.code,
|
|
770
|
+
message: error.code === 'empty_file'
|
|
771
|
+
? 'UI proof bundle file is empty.'
|
|
772
|
+
: error.code === 'unparseable_json'
|
|
773
|
+
? 'UI proof bundle metadata is not valid JSON.'
|
|
774
|
+
: error.message,
|
|
775
|
+
fix: error.code === 'empty_file'
|
|
776
|
+
? 'Write JSON UI proof metadata before validating.'
|
|
777
|
+
: error.fix,
|
|
778
|
+
})) };
|
|
779
|
+
}
|
|
780
|
+
|
|
781
|
+
export function parseUiProofSlotsContent(content, filePath = 'UI proof slots') {
|
|
782
|
+
const parsed = parseJsonOrFencedContent(content, filePath, 'UI proof slots');
|
|
783
|
+
if (parsed.errors.length > 0) return { slots: [], errors: parsed.errors };
|
|
784
|
+
|
|
785
|
+
const value = parsed.value;
|
|
786
|
+
const slots = Array.isArray(value)
|
|
787
|
+
? value
|
|
788
|
+
: normalizeArray(value?.ui_proof_slots || value?.uiProofSlots || value?.planned_slots || value?.plannedSlots);
|
|
789
|
+
|
|
790
|
+
if (slots.length === 0) {
|
|
791
|
+
return {
|
|
792
|
+
slots: [],
|
|
793
|
+
errors: [{
|
|
794
|
+
code: 'missing_planned_slots',
|
|
795
|
+
path: filePath,
|
|
796
|
+
message: 'Planned UI proof input must be an array or contain ui_proof_slots.',
|
|
797
|
+
fix: 'Provide JSON with an array of planned slots or an object with ui_proof_slots.',
|
|
798
|
+
}],
|
|
799
|
+
};
|
|
800
|
+
}
|
|
801
|
+
|
|
802
|
+
const validation = validateUiProofSlots(slots);
|
|
803
|
+
return {
|
|
804
|
+
slots,
|
|
805
|
+
errors: validation.errors.map((error) => ({
|
|
806
|
+
...error,
|
|
807
|
+
path: error.path === 'ui_proof_slots' ? filePath : `${filePath}.${error.path}`,
|
|
808
|
+
})),
|
|
356
809
|
};
|
|
357
810
|
}
|
|
358
811
|
|
|
@@ -428,12 +881,61 @@ function cmdValidate(cwd, args) {
|
|
|
428
881
|
const parsed = readUiProofBundleFile(target);
|
|
429
882
|
const validation = parsed.errors.length > 0
|
|
430
883
|
? { valid: false, errors: parsed.errors, warnings: [] }
|
|
431
|
-
: validateUiProofBundle(parsed.bundle, {
|
|
884
|
+
: validateUiProofBundle(parsed.bundle, {
|
|
885
|
+
claimUses: parseClaimUse(flags),
|
|
886
|
+
requireLocalArtifactExists: true,
|
|
887
|
+
workspaceRoot: cwd,
|
|
888
|
+
bundleDir: dirname(target),
|
|
889
|
+
});
|
|
432
890
|
|
|
433
891
|
output({ operation: 'ui-proof validate', target: targetArg, valid: validation.valid, errors: validation.errors, warnings: validation.warnings });
|
|
434
892
|
if (!validation.valid) process.exitCode = 1;
|
|
435
893
|
}
|
|
436
894
|
|
|
895
|
+
function cmdCompare(cwd, args) {
|
|
896
|
+
const [plannedArg, ...observedArgs] = args;
|
|
897
|
+
if (!plannedArg) fail('Usage: gsdd ui-proof compare <planned-slots-json> [observed-bundle-json ...]');
|
|
898
|
+
|
|
899
|
+
const plannedPath = resolveWorkspacePath(cwd, plannedArg);
|
|
900
|
+
if (!existsSync(plannedPath) || statSync(plannedPath).isDirectory()) fail(`Planned UI proof slots file does not exist: ${plannedArg}`);
|
|
901
|
+
|
|
902
|
+
const planned = parseUiProofSlotsContent(readFileSync(plannedPath, 'utf-8'), plannedArg);
|
|
903
|
+
const observedBundles = [];
|
|
904
|
+
const observedTargets = [];
|
|
905
|
+
const observedErrors = [];
|
|
906
|
+
|
|
907
|
+
for (const observedArg of observedArgs) {
|
|
908
|
+
const observedPath = resolveWorkspacePath(cwd, observedArg);
|
|
909
|
+
if (!existsSync(observedPath) || statSync(observedPath).isDirectory()) fail(`Observed UI proof bundle file does not exist: ${observedArg}`);
|
|
910
|
+
const parsed = readUiProofBundleFile(observedPath);
|
|
911
|
+
if (parsed.errors.length > 0) {
|
|
912
|
+
observedErrors.push(...parsed.errors.map((error) => ({ ...error, path: observedArg })));
|
|
913
|
+
observedBundles.push({
|
|
914
|
+
bundle: {},
|
|
915
|
+
validation: { valid: false, errors: parsed.errors, warnings: [] },
|
|
916
|
+
source: observedArg,
|
|
917
|
+
});
|
|
918
|
+
} else {
|
|
919
|
+
observedBundles.push({ bundle: parsed.bundle, source: observedArg });
|
|
920
|
+
}
|
|
921
|
+
observedTargets.push(observedArg);
|
|
922
|
+
}
|
|
923
|
+
|
|
924
|
+
const comparison = planned.errors.length > 0
|
|
925
|
+
? { status: 'partial', slots: [], errors: planned.errors }
|
|
926
|
+
: compareUiProofSlots(planned.slots, observedBundles);
|
|
927
|
+
|
|
928
|
+
output({
|
|
929
|
+
operation: 'ui-proof compare',
|
|
930
|
+
planned: plannedArg,
|
|
931
|
+
observed: observedTargets,
|
|
932
|
+
status: comparison.status,
|
|
933
|
+
slots: comparison.slots,
|
|
934
|
+
errors: [...(comparison.errors || []), ...observedErrors],
|
|
935
|
+
});
|
|
936
|
+
if (!['satisfied', 'not_applicable'].includes(comparison.status)) process.exitCode = 1;
|
|
937
|
+
}
|
|
938
|
+
|
|
437
939
|
export function cmdUiProof(...args) {
|
|
438
940
|
const { args: normalizedArgs, workspaceRoot, invalid, error } = resolveWorkspaceContext(args);
|
|
439
941
|
if (invalid) {
|
|
@@ -447,8 +949,11 @@ export function cmdUiProof(...args) {
|
|
|
447
949
|
case 'validate':
|
|
448
950
|
cmdValidate(workspaceRoot, rest);
|
|
449
951
|
return;
|
|
952
|
+
case 'compare':
|
|
953
|
+
cmdCompare(workspaceRoot, rest);
|
|
954
|
+
return;
|
|
450
955
|
default:
|
|
451
|
-
fail('Usage: gsdd ui-proof validate
|
|
956
|
+
fail('Usage: gsdd ui-proof <validate|compare> ...');
|
|
452
957
|
}
|
|
453
958
|
} catch (error) {
|
|
454
959
|
if (error instanceof UiProofError) {
|
package/distilled/DESIGN.md
CHANGED
|
@@ -2817,30 +2817,36 @@ Posture compatibility is part of that closeout contract: `repo_closeout` and `ru
|
|
|
2817
2817
|
|
|
2818
2818
|
## D62 - Repo-Native UI Proof Contract
|
|
2819
2819
|
|
|
2820
|
-
**Decision (2026-04-28):** UI-sensitive work should carry a compact planned proof-slot contract and, when executed, an observed UI proof bundle that references artifacts by path or link while preserving the existing closure evidence kinds: `code`, `test`, `runtime`, `delivery`, and `human`.
|
|
2820
|
+
**Decision (2026-04-28; revised 2026-05-08):** UI-sensitive work should carry a compact planned proof-slot contract and, when executed, an observed UI proof bundle that references artifacts by path or link while preserving the existing closure evidence kinds: `code`, `test`, `runtime`, `delivery`, and `human`. For live rendered UI proof, `agent-browser` is the default runtime evidence path for consumers, while existing Playwright tests remain the canonical repeatable browser-regression path when present. The deterministic `ui-proof` validator remains provider-agnostic structural validation, but it now validates planned slot specificity, concise tool provenance, local artifact path existence when validating from files, raw-artifact safety for paths and URLs, and failed/partial proof classification so the workflow cannot degrade back into unstructured "looks good" review.
|
|
2821
2821
|
|
|
2822
2822
|
**Context:**
|
|
2823
2823
|
- UI proof targets the recurring failure mode where agents claim a UI works or looks good without rendered proof, matched observations, or explicit human judgment.
|
|
2824
2824
|
- The contract defines proof slots, proof bundles, comparison statuses, fail-closed agent guardrails, deterministic metadata validation, privacy metadata, and health visibility without adding a browser-provider framework.
|
|
2825
2825
|
- GSD's archived planner, executor, and verifier roles preserve strong lifecycle discipline, but they do not provide this UI-specific planned-vs-observed proof model. GSDD keeps the lifecycle leverage and adds a repo-native UI proof substrate without adding a browser-provider framework.
|
|
2826
|
+
- OneShot's QC guidance and Vercel's `agent-browser` skill converge on an interactive browser loop for snapshots, ref-based interaction, screenshots, and network/console-adjacent inspection. GSDD adapts that as a default workflow instruction, not as a hard validator dependency.
|
|
2826
2827
|
|
|
2827
2828
|
**Decision:**
|
|
2828
2829
|
- Planning must classify UI-sensitive work and require either `ui_proof_slots` or an explicit `no_ui_proof_rationale`.
|
|
2829
|
-
- Planned slots record claim, route/state, required evidence kinds, minimum observations, environment/viewport, manual-acceptance requirement, claim limit, and requirement IDs.
|
|
2830
|
+
- Planned slots record claim, route/state, required evidence kinds, minimum observations, expected artifact types, runnable validation command, environment/viewport, manual-acceptance requirement, claim limit, and requirement IDs.
|
|
2830
2831
|
- Observed proof bundles record claim, requirement/slot IDs, route/state, environment, viewport, evidence inputs, commands/manual steps, observations, artifacts, privacy metadata, result, and claim limits.
|
|
2832
|
+
- Planned slots must be tight enough for the plan checker to reject vague proof: specific route/state, viewport rationale or narrowed claim limit, minimum observations, expected artifact types, runnable validation, and matchability back to the exact UI claim.
|
|
2833
|
+
- The planner chooses viewport coverage, but responsive or layout-sensitive claims require desktop/mobile or equivalent state coverage unless the claim is explicitly narrowed.
|
|
2834
|
+
- Execution defaults to `agent-browser` for live UI runtime proof: open the route/state, capture interactive snapshots/refs where relevant, exercise the changed flow, capture screenshots for planned viewport(s), and record relevant console/network observations.
|
|
2835
|
+
- Existing Playwright tests or package scripts remain the canonical repeatable browser-regression evidence when present. Playwright scripting is reserved for checks `agent-browser` cannot cover cleanly, such as JS-disabled behavior, structured console listeners, or multi-context testing.
|
|
2831
2836
|
- Verification compares planned slots to observed bundles using `satisfied`, `partial`, `missing`, `waived`, `deferred`, and `not_applicable`; waiver and deferral are not proof.
|
|
2832
2837
|
- UI correctness claims fail closed unless rendered proof is matched exactly to claim, route/state, observation, evidence kind, artifact path or manual step, privacy metadata, result, and claim limit, or an explicit waiver/deferment narrows the claim.
|
|
2833
2838
|
- Human acceptance may close a narrowed claim and record proof debt, but it must not convert missing or mismatched non-human evidence into `satisfied` proof.
|
|
2834
2839
|
- Screenshots, traces, videos, reports, accessibility scans, Gherkin, and visual diffs are artifact types or activities mapped onto the five existing evidence kinds, not new evidence kinds.
|
|
2835
2840
|
- Source annotations, AST/cAST findings, semantic search hits, comments, and Semble-like retrieval may discover proof obligations, but they are discovery hints only and do not satisfy proof slots.
|
|
2836
2841
|
- Visual taste, accessibility judgment, baseline acceptance, subjective polish/layout quality, and privacy publication require human evidence or explicit waiver, and human approval does not replace required `code`, `test`, `runtime`, or `delivery` evidence.
|
|
2837
|
-
- Deterministic
|
|
2838
|
-
- `gsdd health` reports invalid known UI proof bundles as E10 using the same validator, staying read-only and
|
|
2842
|
+
- Deterministic validation keeps the evidence and comparison-status vocabularies unchanged: planned slots require specific claim, route/state, evidence, expected artifacts, validation, viewport, and claim-limit fields; artifact entries require `visibility`, `retention`, `sensitivity`, and `safe_to_publish`; raw screenshots, traces, videos, DOM snapshots, and reports default to `local_only` plus `safe_to_publish: false`; `bin/lib/ui-proof.mjs` validates required bundle/observation fields, structured command/manual-step entries, fixed evidence kinds, concise `tools_used` IDs, claim/result statuses, comparison statuses, failure classification for failed/partial proof, claim limits, privacy metadata, safe artifact references, local artifact path existence when validating file-backed bundles, and public/tracked/delivery proof claims backed by local-only, unsafe, unsanitized, or privacy-contradictory artifacts.
|
|
2843
|
+
- `gsdd health` reports invalid known UI proof bundles as E10 using the same validator, staying read-only and avoiding raw artifact content inspection.
|
|
2844
|
+
- Failed UI proof is reported through existing GSDD gap/proof-debt language. Product behavior defects, missing or blocked infrastructure, flaky harnesses, and ambiguous specs explain causes, but they do not add new evidence kinds, result statuses, or comparison statuses.
|
|
2839
2845
|
|
|
2840
2846
|
**Leverage:**
|
|
2841
|
-
- Lost: UI-sensitive work now carries a small proof-contract burden, and
|
|
2842
|
-
- Kept: repo-native markdown artifacts, optional project tooling, fixed closure evidence kinds, generated-surface freshness,
|
|
2843
|
-
- Gained: exact claim-to-proof traceability, strict comparison statuses, privacy and claim-limit metadata, fail-closed overclaim guardrails, deterministic metadata validation, and health-visible protection against unsafe public proof claims.
|
|
2847
|
+
- Lost: UI-sensitive work now carries a small proof-contract burden, and default live proof guidance adds slightly more specificity for planners/checkers to enforce.
|
|
2848
|
+
- Kept: repo-native markdown artifacts, optional project tooling, fixed closure evidence kinds, generated-surface freshness, the plan/execute/verify separation, and provider-agnostic deterministic metadata validation.
|
|
2849
|
+
- Gained: exact claim-to-proof traceability, strict comparison statuses, privacy and claim-limit metadata, fail-closed overclaim guardrails, deterministic metadata validation, a concrete live browser evidence path, and health-visible protection against unsafe public proof claims.
|
|
2844
2850
|
|
|
2845
2851
|
**Evidence:**
|
|
2846
2852
|
- `distilled/templates/ui-proof.md`
|
|
@@ -2849,11 +2855,22 @@ Posture compatibility is part of that closeout contract: `repo_closeout` and `ru
|
|
|
2849
2855
|
- `bin/lib/templates.mjs`, `bin/lib/ui-proof.mjs`, `bin/lib/health.mjs`, `bin/lib/rendering.mjs`
|
|
2850
2856
|
- `tests/phase.test.cjs`, `tests/gsdd.guards.test.cjs`, `tests/gsdd.health.test.cjs`, `tests/gsdd.init.test.cjs`
|
|
2851
2857
|
- GSD comparison: the upstream planner, executor, and verifier role patterns preserve lifecycle rigor, but they do not define UI proof slots or planned-vs-observed UI proof bundles.
|
|
2858
|
+
- OneShot QC source: `https://github.com/oneshot-repo/OneShot/tree/main/skills`
|
|
2859
|
+
- Vercel `agent-browser` docs: `https://github.com/vercel-labs/agent-browser/blob/main/skill-data/core/SKILL.md` and `https://agent-browser.dev/snapshots`
|
|
2860
|
+
- Playwright docs: `https://playwright.dev/docs/trace-viewer`, `https://playwright.dev/docs/next/screenshots`, and `https://playwright.dev/mcp/tools/tracing`
|
|
2861
|
+
- Chrome DevTools for agents/MCP docs: `https://developer.chrome.com/docs/devtools/agents` and `https://developer.chrome.com/blog/chrome-devtools-mcp?hl=en`
|
|
2862
|
+
- Harness and UI-agent pitfall sources: `https://www.anthropic.com/engineering/harness-design-long-running-apps`, `https://www.huuhka.net/browser-verification-for-coding-agents-chrome-devtools-mcp-vs-agent-browser/`, `https://www.developersdigest.tech/blog/long-running-agents-need-harnesses`, `https://codemyspec.com/blog/agentic-qa-verification`, `https://dev.to/ratikkoka/your-ui-is-invisible-to-ai-agents-heres-how-to-fix-it-1ib3`, `https://dev.to/louaiboumediene/the-ai-harness-why-your-ai-coding-agent-is-only-as-smart-as-the-repo-you-put-it-in-cml`, and `https://tessl.io/blog/webmcp-making-web-apps-faster-and-cheaper-for-ai-agents/`
|
|
2863
|
+
- OpenSpec docs: `https://openspec.dev/`
|
|
2864
|
+
- LeanSpec docs: `https://www.lean-spec.dev/docs/guide/first-principles`
|
|
2865
|
+
- OpenAI Codex docs: `https://help.openai.com/en/articles/11369540-codex-in-chatgpt`
|
|
2866
|
+
- Anthropic Agent Skills docs: `https://docs.claude.com/en/docs/agents-and-tools/agent-skills`
|
|
2867
|
+
- GitHub Copilot customization docs: `https://docs.github.com/en/copilot/concepts/prompting/response-customization`
|
|
2852
2868
|
|
|
2853
2869
|
**Consequences:**
|
|
2854
2870
|
- Future UI-related phases must not add new evidence kinds by treating artifact types as proof categories.
|
|
2855
2871
|
- Future dogfood or runtime validation must not upgrade artifact counts or human waivers into proof.
|
|
2856
2872
|
- Generated runtime surfaces and local templates must stay freshness-checkable through `gsdd update --templates` and health diagnostics.
|
|
2873
|
+
- Future provider/tooling work must not make `agent-browser` a required validator field without a separate product decision; the current contract makes it the default workflow path, not a schema lock.
|
|
2857
2874
|
|
|
2858
2875
|
---
|
|
2859
2876
|
|
|
@@ -491,6 +491,19 @@
|
|
|
491
491
|
- `agents/planner.md`, `agents/executor.md`, `agents/verifier.md`, `distilled/templates/delegates/plan-checker.md`
|
|
492
492
|
- `bin/lib/templates.mjs`, `bin/lib/ui-proof.mjs`, `bin/lib/health.mjs`, `bin/lib/rendering.mjs`
|
|
493
493
|
- `tests/phase.test.cjs`, `tests/gsdd.guards.test.cjs`, `tests/gsdd.health.test.cjs`, `tests/gsdd.init.test.cjs`
|
|
494
|
+
- OneShot QC/browser policy: https://github.com/oneshot-repo/OneShot/tree/main/skills
|
|
495
|
+
- Vercel agent-browser docs: https://github.com/vercel-labs/agent-browser/blob/main/skill-data/core/SKILL.md, https://agent-browser.dev/snapshots
|
|
496
|
+
- Playwright browser proof docs: https://playwright.dev/docs/trace-viewer, https://playwright.dev/docs/next/screenshots, https://playwright.dev/mcp/tools/tracing
|
|
497
|
+
- Chrome DevTools agent/browser docs: https://developer.chrome.com/docs/devtools/agents, https://developer.chrome.com/blog/chrome-devtools-mcp?hl=en
|
|
498
|
+
- Harness engineering source: https://www.anthropic.com/engineering/harness-design-long-running-apps
|
|
499
|
+
- UI-agent browser verification source: https://www.huuhka.net/browser-verification-for-coding-agents-chrome-devtools-mcp-vs-agent-browser/
|
|
500
|
+
- Agent harness discipline source: https://www.developersdigest.tech/blog/long-running-agents-need-harnesses
|
|
501
|
+
- Agentic QA verification-gap source: https://codemyspec.com/blog/agentic-qa-verification
|
|
502
|
+
- Agent-hostile UI semantics source: https://dev.to/ratikkoka/your-ui-is-invisible-to-ai-agents-heres-how-to-fix-it-1ib3
|
|
503
|
+
- Browser-harness efficiency source: https://dev.to/louaiboumediene/the-ai-harness-why-your-ai-coding-agent-is-only-as-smart-as-the-repo-you-put-it-in-cml
|
|
504
|
+
- Dynamic-UI harness brittleness source: https://tessl.io/blog/webmcp-making-web-apps-faster-and-cheaper-for-ai-agents/
|
|
505
|
+
- Long-term pitfalls carried forward: do not accept screenshot-free "looks good" claims, weak planned slots, unverified artifact paths, stale interactive refs after page mutation, partial/failed proof without failure classification, raw artifact publication without privacy metadata, browser contention in parallel checks, or semantic/selector-poor UI that forces fragile coordinate inspection.
|
|
506
|
+
- Supporting spec/runtime docs: https://openspec.dev/, https://www.lean-spec.dev/docs/guide/first-principles, https://help.openai.com/en/articles/11369540-codex-in-chatgpt, https://docs.claude.com/en/docs/agents-and-tools/agent-skills, https://docs.github.com/en/copilot/concepts/prompting/response-customization
|
|
494
507
|
|
|
495
508
|
---
|
|
496
509
|
|
|
@@ -35,6 +35,7 @@ Verify these dimensions:
|
|
|
35
35
|
- `escalation_integrity`: tasks include checkpoints or escalation when evidence, permissions, user decisions, or risky ambiguity are required.
|
|
36
36
|
- `closure_honesty`: the plan's done criteria and evidence limits support only claims that execution can actually prove.
|
|
37
37
|
- `closure_honesty`: for UI proof, reject agent-only `looks good` closure, artifact-count proof, unsupported evidence kinds, and human acceptance that converts missing/mismatched non-human evidence into `satisfied` proof. Waiver, deferment, proof debt, or narrowed-claim language is acceptable only when the stronger UI claim is not treated as proven.
|
|
38
|
+
- `closure_honesty`: for UI proof planning, reject weak slots that omit a specific route/state, viewport rationale or narrowed viewport claim limit, minimum observations, expected artifact types, runnable validation, or a way to compare observed proof back to the planned claim. Treat under-specified viewport coverage as a blocker for responsive or layout-sensitive claims. `agent-browser` is the default live runtime evidence path; do not block a slot solely for using another project-native browser path, but require the plan to explain the `agent-browser` availability constraint and fallback choice.
|
|
38
39
|
- `closure_honesty`: for UI proof privacy, require artifact `visibility`, `retention`, `sensitivity`, and `safe_to_publish`, require `gsdd ui-proof validate` or `gsdd health` when bundle metadata exists, and reject public/tracked/delivery/publication proof claims backed by local-only or `safe_to_publish: false` artifacts.
|
|
39
40
|
- `high_leverage_review`: high-leverage surfaces have a second-pass review or equivalent contradiction/staleness check before completion.
|
|
40
41
|
- `approach_alignment`: when APPROACH.md is provided, verify that plan tasks implement the chosen approaches from the user's decisions. Check:
|
|
@@ -4,6 +4,10 @@ Use this template when work affects rendered UI or when a plan defines `ui_proof
|
|
|
4
4
|
|
|
5
5
|
UI proof uses the existing closure evidence kinds only: `code`, `test`, `runtime`, `delivery`, and `human`. Screenshots, traces, videos, reports, accessibility scans, Gherkin, visual diffs, and manual notes are artifact types or activities that map onto those evidence kinds. They are not new evidence kinds.
|
|
6
6
|
|
|
7
|
+
For live rendered UI evidence, default to `agent-browser`: open the route, capture an interactive snapshot/refs when interaction is part of the claim, exercise the changed flow, capture screenshots for the planned viewport(s), and record console/network observations when they affect the claim. If the repo already has Playwright tests or a package script wrapping them, those remain the canonical repeatable regression path; use them as `test` evidence and use `agent-browser` for complementary live runtime proof. Do not introduce new Playwright, Cypress, Storybook, CI, browser MCP, or visual-regression infrastructure just to satisfy this template. Use Playwright scripting only for checks `agent-browser` cannot cover cleanly, such as JS-disabled behavior, structured console listeners, or multi-context testing.
|
|
8
|
+
|
|
9
|
+
Tool availability is part of the proof record. In runtimes where `agent-browser` is not available, first state that availability constraint, then use the closest project-native interactive browser path and record the fallback in `evidence_inputs.tools_used`, `commands_or_manual_steps`, and `claim_limits`. A fallback can support a narrowed local runtime claim, but it must not silently pretend that the default `agent-browser` path ran.
|
|
10
|
+
|
|
7
11
|
## Planned Proof Slots
|
|
8
12
|
|
|
9
13
|
Every UI-sensitive plan needs either at least one slot under `ui_proof_slots` or an explicit `no_ui_proof_rationale` explaining why no rendered UI proof is required.
|
|
@@ -19,13 +23,15 @@ ui_proof_slots:
|
|
|
19
23
|
minimum_observations:
|
|
20
24
|
- "Changed control is visible and usable in the stated state."
|
|
21
25
|
- "Expected interaction completes without console/runtime error."
|
|
26
|
+
expected_artifact_types: [screenshot, report]
|
|
27
|
+
validation_command: "gsdd ui-proof compare {work_item_dir}/ui-proof-slots.json {work_item_dir}/UI-PROOF.md"
|
|
22
28
|
environment:
|
|
23
29
|
app_url: "http://localhost:3000"
|
|
24
30
|
data_state: "synthetic or seeded data"
|
|
25
31
|
viewport:
|
|
26
32
|
width: 1280
|
|
27
33
|
height: 720
|
|
28
|
-
notes: "
|
|
34
|
+
notes: "State why this viewport is enough for the claim, or add separate slots/observations for mobile, desktop, or responsive states."
|
|
29
35
|
manual_acceptance_required: false
|
|
30
36
|
claim_limit: "Does not prove cross-browser layout, full accessibility conformance, production delivery, or unrelated UI states."
|
|
31
37
|
no_ui_proof_rationale: null
|
|
@@ -34,6 +40,8 @@ no_ui_proof_rationale: null
|
|
|
34
40
|
Slot rules:
|
|
35
41
|
- Keep each slot tied to one exact UI claim.
|
|
36
42
|
- Use the lightest proof that can catch a botched rendered experience for that claim.
|
|
43
|
+
- Specify the route/state, viewport choice, minimum observations, expected artifact types, and runnable validation path tightly enough that a checker can reject vague proof before execution.
|
|
44
|
+
- The planner chooses the viewport set, but the slot must explain the choice. Include desktop and mobile proof when the claim covers responsive layout or when the changed surface is likely to behave differently across those sizes; otherwise narrow the claim limit.
|
|
37
45
|
- Source annotations, AST/cAST findings, semantic search hits, comments, and Semble-like retrieval may help discover proof obligations. They are discovery hints only; they do not satisfy proof slots.
|
|
38
46
|
- Do not add Playwright, Cypress, Storybook, Cucumber, CI, browser MCP, or visual-regression tooling by default.
|
|
39
47
|
- Human approval is required for visual taste, accessibility judgment, baseline acceptance, subjective polish/layout quality, and privacy publication decisions.
|
|
@@ -43,6 +51,8 @@ Slot rules:
|
|
|
43
51
|
|
|
44
52
|
Create or update this bundle during execution or verification when planned UI proof slots exist. JSON is the canonical machine-readable proof bundle format. Markdown proof files must include fenced JSON for deterministic validation.
|
|
45
53
|
|
|
54
|
+
Replace placeholders such as `{work_item_dir}` with the current phase, quick-task, or brownfield-change directory before running commands or validating the bundle.
|
|
55
|
+
|
|
46
56
|
```json
|
|
47
57
|
{
|
|
48
58
|
"proof_bundle_version": 1,
|
|
@@ -58,7 +68,7 @@ Create or update this bundle during execution or verification when planned UI pr
|
|
|
58
68
|
},
|
|
59
69
|
"environment": {
|
|
60
70
|
"app_url": "http://localhost:3000",
|
|
61
|
-
"browser": "
|
|
71
|
+
"browser": "agent-browser default; record fallback when unavailable",
|
|
62
72
|
"browser_version": "record if known",
|
|
63
73
|
"os": "record if relevant",
|
|
64
74
|
"data_state": "synthetic or seeded data"
|
|
@@ -70,7 +80,7 @@ Create or update this bundle during execution or verification when planned UI pr
|
|
|
70
80
|
},
|
|
71
81
|
"evidence_inputs": {
|
|
72
82
|
"kinds": ["test", "runtime"],
|
|
73
|
-
"tools_used": ["
|
|
83
|
+
"tools_used": ["playwright", "agent-browser"]
|
|
74
84
|
},
|
|
75
85
|
"commands_or_manual_steps": [
|
|
76
86
|
{
|
|
@@ -80,7 +90,12 @@ Create or update this bundle during execution or verification when planned UI pr
|
|
|
80
90
|
"attempts": 1
|
|
81
91
|
},
|
|
82
92
|
{
|
|
83
|
-
"
|
|
93
|
+
"command": "agent-browser open http://localhost:3000/example && agent-browser snapshot -i && agent-browser screenshot {work_item_dir}/artifacts/example-1280.png --full",
|
|
94
|
+
"result": "passed",
|
|
95
|
+
"attempts": 1
|
|
96
|
+
},
|
|
97
|
+
{
|
|
98
|
+
"manual_step": "Using agent-browser refs, complete the changed interaction as a synthetic user and check for visible breakage or relevant console/network failures.",
|
|
84
99
|
"result": "passed"
|
|
85
100
|
}
|
|
86
101
|
],
|
|
@@ -93,7 +108,7 @@ Create or update this bundle during execution or verification when planned UI pr
|
|
|
93
108
|
"state": "role, data state, feature flag, loading/error/empty state, or component story"
|
|
94
109
|
},
|
|
95
110
|
"evidence_kind": "runtime",
|
|
96
|
-
"artifact_refs": ["test-results/changed-flow-report/index.html"],
|
|
111
|
+
"artifact_refs": ["test-results/changed-flow-report/index.html", "{work_item_dir}/artifacts/example-1280.png"],
|
|
97
112
|
"privacy": {
|
|
98
113
|
"data_classification": "synthetic",
|
|
99
114
|
"raw_artifacts_safe_to_publish": false,
|
|
@@ -112,6 +127,15 @@ Create or update this bundle during execution or verification when planned UI pr
|
|
|
112
127
|
"sensitivity": "possible",
|
|
113
128
|
"safe_to_publish": false,
|
|
114
129
|
"notes": "Local report only; not public proof."
|
|
130
|
+
},
|
|
131
|
+
{
|
|
132
|
+
"path": "{work_item_dir}/artifacts/example-1280.png",
|
|
133
|
+
"type": "screenshot",
|
|
134
|
+
"visibility": "local_only",
|
|
135
|
+
"retention": "temporary_review",
|
|
136
|
+
"sensitivity": "possible",
|
|
137
|
+
"safe_to_publish": false,
|
|
138
|
+
"notes": "Local screenshot only; not public proof unless sanitized and reclassified."
|
|
115
139
|
}
|
|
116
140
|
],
|
|
117
141
|
"privacy": {
|
|
@@ -129,7 +153,8 @@ Create or update this bundle during execution or verification when planned UI pr
|
|
|
129
153
|
"claim_status": "passed",
|
|
130
154
|
"comparison_status_by_slot": {
|
|
131
155
|
"ui-01": "satisfied"
|
|
132
|
-
}
|
|
156
|
+
},
|
|
157
|
+
"failure_classification": null
|
|
133
158
|
},
|
|
134
159
|
"claim_limits": [
|
|
135
160
|
"Does not prove Safari/WebKit behavior.",
|
|
@@ -149,10 +174,11 @@ Bundle rules:
|
|
|
149
174
|
- Local-only or `safe_to_publish: false` artifacts can support local review only; they must not back tracked, public, delivery, release, or publication proof claims.
|
|
150
175
|
- Human acceptance may close a narrowed claim only by recording waiver, deferment, or proof debt; it must not upgrade missing or mismatched non-human proof to `satisfied`.
|
|
151
176
|
- Quick-mode UI proof should use deterministic synthetic IDs such as `quick-001` and `quick-001-ui-01` when roadmap requirement IDs do not exist.
|
|
177
|
+
- Classify failed UI proof using existing GSDD gap/proof-debt language: `product_bug`, `missing_infra`, `flaky_harness`, or `ambiguous_spec`. Do not add new result statuses or evidence kinds for those causes.
|
|
152
178
|
|
|
153
179
|
## Deterministic Validation
|
|
154
180
|
|
|
155
|
-
Use `gsdd ui-proof validate <path>` on JSON proof-bundle metadata or markdown fenced JSON before relying on a bundle for closure; add `--claim <public|publication|tracked|delivery|release>` only when validating that stronger proof use. Required observed-bundle top-level fields are `proof_bundle_version`, `scope`, `route_state`, `environment`, `viewport`, `evidence_inputs`, `commands_or_manual_steps`, `observations`, `artifacts`, `privacy`, `result`, and `claim_limits`. The validator checks required bundle and observation fields, structured command/manual-step entries, fixed evidence kinds, `result.claim_status`, observation `result`, comparison statuses, non-empty claim limits, locked artifact and observation privacy fields, observation-to-artifact references, workspace-relative/http(s) artifact references, and explicit public/tracked/delivery proof claims that rely on local-only, unsafe, unsanitized, or privacy-contradictory artifacts. `claim_status`, observation `result`, and command/manual-step `result` use `passed`, `failed`, `partial`, `waived`, `deferred`, or `not_applicable
|
|
181
|
+
Use `gsdd ui-proof validate <path>` on JSON proof-bundle metadata or markdown fenced JSON before relying on a bundle for closure; add `--claim <public|publication|tracked|delivery|release>` only when validating that stronger proof use. Use `gsdd ui-proof compare <planned-slots-json> [observed-bundle-json ...]` when verifying planned proof slots against observed bundles through the deterministic product-facing path. Required planned-slot fields are `slot_id`, `claim`, `route_state`, `required_evidence_kinds`, `minimum_observations`, `expected_artifact_types`, `validation_command`, `environment`, `viewport`, `manual_acceptance_required`, and `claim_limit`. Required observed-bundle top-level fields are `proof_bundle_version`, `scope`, `route_state`, `environment`, `viewport`, `evidence_inputs`, `commands_or_manual_steps`, `observations`, `artifacts`, `privacy`, `result`, and `claim_limits`. The validator checks planned-slot specificity, required bundle and observation fields, structured command/manual-step entries, fixed evidence kinds, concise `tools_used` IDs, `result.claim_status`, observation `result`, comparison statuses, failure classification for failed/partial proof, non-empty claim limits, locked artifact and observation privacy fields, observation-to-artifact references, workspace-relative/http(s) artifact references, existing local artifact paths when validating from files, and explicit public/tracked/delivery proof claims that rely on local-only, unsafe, unsanitized, or privacy-contradictory artifacts. `claim_status`, observation `result`, and command/manual-step `result` use `passed`, `failed`, `partial`, `waived`, `deferred`, or `not_applicable`; failed/partial proof uses `product_bug`, `missing_infra`, `flaky_harness`, or `ambiguous_spec`. It does not inspect raw screenshot, trace, video, DOM, or report contents and does not require any specific browser provider such as `agent-browser`.
|
|
156
182
|
|
|
157
183
|
## Comparison Statuses
|
|
158
184
|
|
|
@@ -166,8 +166,9 @@ Before reporting a task complete:
|
|
|
166
166
|
|
|
167
167
|
### UI Proof Execution
|
|
168
168
|
If the plan defines non-empty `ui_proof_slots`, create or update the observed UI proof bundle before claiming completion; required top-level fields are `proof_bundle_version`, `scope`, `route_state`, `environment`, `viewport`, `evidence_inputs`, `commands_or_manual_steps`, `observations`, `artifacts`, `privacy`, `result`, and `claim_limits`.
|
|
169
|
-
Use
|
|
169
|
+
Use `agent-browser` as the default live UI proof path. Record the planned route/state open, interactive snapshots/refs when interaction is part of the claim, changed-flow interaction, screenshots for planned viewport(s), and relevant console/network observations. If `agent-browser` is unavailable, record the availability constraint and the closest project-native interactive browser fallback in the proof bundle instead of silently treating the fallback as the default path. If the repo already has Playwright tests or a package script wrapping them, run the relevant targeted test as canonical repeatable regression evidence; keep `agent-browser` as complementary runtime proof. Use Playwright scripting only for checks `agent-browser` cannot cover cleanly, such as JS-disabled, structured console, or multi-context verification. Do not install Playwright, Cypress, Cucumber, Storybook, browser MCP, CI, or visual-regression tooling by default. Screenshots, traces, videos, reports, accessibility scans, Gherkin, visual diffs, and manual notes map onto existing evidence kinds, not new evidence kinds; reference raw artifacts by path/link instead of storing them inline.
|
|
170
170
|
Each artifact entry must include `visibility`, `retention`, `sensitivity`, and `safe_to_publish`; raw screenshots, traces, videos, DOM snapshots, and reports default to `local_only` and `safe_to_publish: false` unless explicitly sanitized. Use `gsdd ui-proof validate <path>` when bundle metadata exists, adding `--claim <...>` only when relying on the bundle for public, tracked, delivery, release, or publication proof. Visual taste, accessibility judgment, baseline acceptance, subjective polish/layout quality, and privacy publication decisions require human evidence or explicit waiver; artifact count, source comments, AST/cAST findings, semantic search, and Semble-like retrieval are not proof. If evidence does not match the slot claim, route/state, observation, artifact path/manual step, privacy metadata, result, and claim limit, record proof debt, waiver, deferment, or reduced claim language rather than `satisfied` proof.
|
|
171
|
+
Classify failed UI proof using existing gap/proof-debt language: `product_bug`, `missing_infra`, `flaky_harness`, or `ambiguous_spec`. Do not add new evidence kinds or result statuses for those causes.
|
|
171
172
|
|
|
172
173
|
### Git Guidance
|
|
173
174
|
|
|
@@ -15,6 +15,18 @@ If no audit file exists: stop and direct the user to run `/gsdd-audit-milestone`
|
|
|
15
15
|
If audit status is `passed`: stop and direct the user to run `/gsdd-complete-milestone` instead.
|
|
16
16
|
</prerequisites>
|
|
17
17
|
|
|
18
|
+
<repo_root_helper_contract>
|
|
19
|
+
All `node .planning/bin/gsdd.mjs ...` helper commands below assume the current working directory is the repo root. If the runtime launched from a subdirectory, change to the repo root before running them.
|
|
20
|
+
</repo_root_helper_contract>
|
|
21
|
+
|
|
22
|
+
<lifecycle_preflight>
|
|
23
|
+
Before writing ROADMAP gap-closure phases or phase directories, run:
|
|
24
|
+
|
|
25
|
+
- `node .planning/bin/gsdd.mjs lifecycle-preflight plan-milestone-gaps`
|
|
26
|
+
|
|
27
|
+
If the preflight result is `blocked`, STOP and report the blocker. This workflow intentionally mutates planning truth, so it must not proceed through pre-existing planning-state drift.
|
|
28
|
+
</lifecycle_preflight>
|
|
29
|
+
|
|
18
30
|
<load_context>
|
|
19
31
|
Before starting, read these files:
|
|
20
32
|
|
|
@@ -144,6 +156,14 @@ Create a directory for each gap closure phase:
|
|
|
144
156
|
|
|
145
157
|
No files inside — `/gsdd-plan` populates them.
|
|
146
158
|
|
|
159
|
+
## 8. Rebaseline Planning Fingerprint
|
|
160
|
+
|
|
161
|
+
After confirming the ROADMAP update and phase directories exist, run:
|
|
162
|
+
|
|
163
|
+
- `node .planning/bin/gsdd.mjs session-fingerprint write --allow-changed ROADMAP.md`
|
|
164
|
+
|
|
165
|
+
This records the user-confirmed ROADMAP mutation so the recommended `/gsdd-plan [N]` handoff does not immediately block on expected `planning_state_drift`. The `--allow-changed ROADMAP.md` guard must fail if `SPEC.md` or `config.json` also drifted after preflight; stop and reconcile that unexpected drift instead of rebaselining it. Do not run this if the ROADMAP write failed or the phase directories are missing.
|
|
166
|
+
|
|
147
167
|
</process>
|
|
148
168
|
|
|
149
169
|
<success_criteria>
|
|
@@ -155,6 +175,7 @@ No files inside — `/gsdd-plan` populates them.
|
|
|
155
175
|
- [ ] User confirmed gap closure plan before ROADMAP.md was updated
|
|
156
176
|
- [ ] ROADMAP.md updated with new gap closure phases
|
|
157
177
|
- [ ] Phase directories created
|
|
178
|
+
- [ ] `session-fingerprint write` ran after the reviewed ROADMAP update so `/gsdd-plan [N]` is not stranded by expected planning drift
|
|
158
179
|
</success_criteria>
|
|
159
180
|
|
|
160
181
|
**MANDATORY: `.planning/ROADMAP.md` must be updated on disk before this workflow is complete. If the write fails, STOP and report the failure. Without the updated ROADMAP, the phase cycle cannot begin.**
|
|
@@ -136,8 +136,9 @@ If any of these are missing or contradictory, STOP. Report the exact missing con
|
|
|
136
136
|
</phase_contract_gate>
|
|
137
137
|
|
|
138
138
|
<ui_proof_planning>
|
|
139
|
-
For UI-sensitive work, include compact `ui_proof_slots` with `slot_id`, optional `requirement_id`, `claim`, `route_state`, fixed evidence kinds (`code`, `test`, `runtime`, `delivery`, `human`), `minimum_observations`, `environment`, `viewport`, `manual_acceptance_required`, and `claim_limit`; otherwise set `no_ui_proof_rationale`.
|
|
139
|
+
For UI-sensitive work, include compact `ui_proof_slots` with `slot_id`, optional `requirement_id`, `claim`, `route_state`, fixed evidence kinds (`code`, `test`, `runtime`, `delivery`, `human`), `minimum_observations`, `expected_artifact_types`, `validation_command`, `environment`, `viewport`, `manual_acceptance_required`, and `claim_limit`; otherwise set `no_ui_proof_rationale`.
|
|
140
140
|
Do not create slots for backend-only, CLI-only, docs-only, or refactor-only work unless the plan claims a visible UI outcome. Evidence must later match claim, route/state, observation, artifact path, evidence kind, privacy metadata, result, and claim limit; local-only or unsafe artifacts cannot support public, publication, tracked, delivery, or release proof claims. Human approval does not replace required `code`, `test`, `runtime`, or `delivery` evidence.
|
|
141
|
+
For live rendered UI proof, plan `agent-browser` as the default runtime evidence path and existing Playwright/package-script browser tests as the repeatable regression path when the repo already has them. If the runtime does not provide `agent-browser`, require the plan to state that availability constraint and name the closest project-native interactive browser fallback before narrowing the claim. The planner chooses the viewport set, but each slot must explain why the chosen viewport(s) are enough for the claim or narrow the claim limit; responsive claims need desktop/mobile or equivalent state coverage. Do not plan new browser infrastructure by default, and use Playwright scripting only for checks `agent-browser` cannot cover cleanly, such as JS-disabled, structured console, or multi-context verification.
|
|
141
142
|
</ui_proof_planning>
|
|
142
143
|
<goal_backward_planning>
|
|
143
144
|
Plan backward from success criteria.
|
|
@@ -118,9 +118,10 @@ Delegate to the planner role in quick mode.
|
|
|
118
118
|
- No research phase, no ROADMAP requirements
|
|
119
119
|
- Do NOT extract phase requirement IDs — there is no active phase
|
|
120
120
|
- Derive must-haves directly from the task description
|
|
121
|
-
- If the quick task is UI-sensitive, include proportional `ui_proof_slots` with slot_id, claim, route_state, required_evidence_kinds, minimum_observations, environment, viewport, manual_acceptance_required, and claim_limit; otherwise include a short `no_ui_proof_rationale`
|
|
121
|
+
- If the quick task is UI-sensitive, include proportional `ui_proof_slots` with slot_id, claim, route_state, required_evidence_kinds, minimum_observations, expected_artifact_types, validation_command, environment, viewport, manual_acceptance_required, and claim_limit; otherwise include a short `no_ui_proof_rationale`
|
|
122
122
|
- UI proof slots must be matchable to exact observed evidence later: claim, route/state, observation, evidence kind, artifact path or manual step, privacy metadata, result, and claim limit. Discovery hints from source comments, AST/cAST, semantic search, or Semble-like retrieval do not satisfy proof.
|
|
123
123
|
- Observed artifact metadata must include `visibility`, `retention`, `sensitivity`, and `safe_to_publish`; raw screenshots, traces, videos, DOM snapshots, and reports are local-only/unsafe by default. Use `gsdd ui-proof validate <path>` or `gsdd health` when a bundle exists; add `--claim <...>` only for public, publication, tracked, delivery, or release proof use.
|
|
124
|
+
- For live rendered UI proof, default to `agent-browser` snapshots/refs, interactions, screenshots, and relevant console/network observations. If unavailable, state the availability constraint and closest project-native interactive browser fallback before narrowing the claim. Existing Playwright/package-script browser tests remain the canonical repeatable regression path when present. The viewport set is plan-owned, but under-specified viewport coverage is weak proof; explain the chosen viewport(s) or narrow the claim limit.
|
|
124
125
|
- Keep UI proof proportional: do not scaffold Playwright, Cypress, Cucumber, Storybook, CI, browser MCP, or visual-regression tooling by default
|
|
125
126
|
- Ignore <planning_process> Step 1 requirement extraction; use inline goal-backward planning only
|
|
126
127
|
- Target minimal context usage
|
|
@@ -160,7 +161,8 @@ Read `.planning/config.json`.
|
|
|
160
161
|
- Mode: quick
|
|
161
162
|
|
|
162
163
|
**Constraints:**
|
|
163
|
-
- Check 5 dimensions
|
|
164
|
+
- Check 5 dimensions by default: `requirement_coverage`, `task_completeness`, `dependency_correctness`, `scope_sanity`, `must_have_quality`
|
|
165
|
+
- If the quick plan contains `ui_proof_slots` or a rendered UI claim, also check `closure_honesty` so weak UI proof slots block execution
|
|
164
166
|
- Skip: `key_link_completeness`, `context_compliance`, `goal_achievement`, `approach_alignment`
|
|
165
167
|
- Maximum 1 revision cycle (if blockers found, send back to planner once, then accept result)
|
|
166
168
|
- Blocker threshold: only block on `task_completeness` or `scope_sanity` violations
|
|
@@ -268,6 +270,7 @@ Delegate to the executor role.
|
|
|
268
270
|
- Do NOT update ROADMAP.md phase status or SPEC.md current state
|
|
269
271
|
- Create summary at: `.planning/quick/$NEXT_NUM-$SLUG/$NEXT_NUM-SUMMARY.md`
|
|
270
272
|
- If the quick plan defines `ui_proof_slots`, create or update `.planning/quick/$NEXT_NUM-$SLUG/UI-PROOF.md` with fenced JSON containing required top-level fields: `proof_bundle_version`, `scope`, `route_state`, `environment`, `viewport`, `evidence_inputs`, `commands_or_manual_steps`, `observations`, `artifacts`, `privacy`, `result`, and `claim_limits`
|
|
273
|
+
- For live UI proof, record `agent-browser` in `evidence_inputs.tools_used` when used, the exact commands or manual ref-based steps, screenshot/report artifact paths, and any relevant console/network observations. If `agent-browser` was unavailable, record that availability constraint and fallback tool explicitly. If existing Playwright tests supplied regression evidence, record the package command and result separately from the `agent-browser` runtime observation.
|
|
271
274
|
- Human approval for visual taste, accessibility judgment, baseline acceptance, subjective polish/layout quality, or privacy publication does not replace required `code`, `test`, `runtime`, or `delivery` evidence
|
|
272
275
|
|
|
273
276
|
**Output:** `.planning/quick/$NEXT_NUM-$SLUG/$NEXT_NUM-SUMMARY.md`
|
|
@@ -129,8 +129,9 @@ Note: this step does NOT replace levels 1–3. An artifact can satisfy the evide
|
|
|
129
129
|
</evidence_contract>
|
|
130
130
|
|
|
131
131
|
<ui_proof_comparison>
|
|
132
|
-
If the plan defines non-empty `ui_proof_slots`, compare planned UI proof against observed bundles before closure. If the plan records only `no_ui_proof_rationale`, verify the rationale instead of requiring a bundle. Each observed bundle must include top-level `proof_bundle_version`, `scope`, `route_state`, `environment`, `viewport`, `evidence_inputs`, `commands_or_manual_steps`, `observations`, `artifacts`, `privacy`, `result`, and `claim_limits`.
|
|
132
|
+
If the plan defines non-empty `ui_proof_slots`, compare planned UI proof against observed bundles before closure. Prefer `gsdd ui-proof compare <planned-slots-json> [observed-bundle-json ...]` when planned slots are available as JSON or fenced JSON; otherwise perform the same field-by-field comparison and record reduced assurance if no deterministic command could run. If the plan records only `no_ui_proof_rationale`, verify the rationale instead of requiring a bundle. Each observed bundle must include top-level `proof_bundle_version`, `scope`, `route_state`, `environment`, `viewport`, `evidence_inputs`, `commands_or_manual_steps`, `observations`, `artifacts`, `privacy`, `result`, and `claim_limits`.
|
|
133
133
|
Classify each slot as exactly one of: `satisfied`, `partial`, `missing`, `waived`, `deferred`, or `not_applicable`. Waiver/deferment narrows the claim; it is not proof. Screenshots, traces, videos, reports, accessibility scans, Gherkin, visual diffs, and manual notes are artifact types or activities mapped onto existing evidence kinds, not new evidence kinds. Artifact count is never proof; each artifact must tie to the slot claim, route/state, observation, artifact path/link, privacy metadata, and claim limit.
|
|
134
|
+
For live UI runtime proof, expect `agent-browser` as the default captured tool unless the observed bundle explains a project-native equivalent or an availability constraint. Do not fail solely because another browser tool was used, but downgrade vague proof that lacks exact route/state, planned viewport coverage or rationale, interactive steps/refs where relevant, screenshot/report artifacts, or relevant console/network observations. Existing Playwright tests count as canonical repeatable regression evidence, not a replacement for scoped runtime evidence when the slot requires `runtime`.
|
|
134
135
|
Artifact privacy metadata must include `visibility`, `retention`, `sensitivity`, and `safe_to_publish`; raw screenshots, traces, videos, DOM snapshots, and reports default to local-only and unsafe unless sanitized. Run `gsdd ui-proof validate <path>` or treat `gsdd health` E10 as blocking; add `--claim <...>` when relying on the bundle for public, tracked, delivery, release, or publication proof. Visual taste, accessibility judgment, baseline acceptance, subjective polish/layout quality, and privacy publication require human evidence or explicit waiver; human approval does not replace required `code`, `test`, `runtime`, or `delivery` evidence. Source annotations, AST/cAST findings, semantic search, comments, and Semble-like retrieval are discovery hints only.
|
|
135
136
|
</ui_proof_comparison>
|
|
136
137
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "gsdd-cli",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.22.0",
|
|
4
4
|
"description": "Workspine — a repo-native delivery spine for long-horizon AI-assisted work, with directly validated support for Claude Code, Codex CLI, and OpenCode, published as gsdd-cli.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|