forgedock 1.0.16 → 1.0.18
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +54 -3
- package/bin/cross-publish.mjs +344 -0
- package/bin/forgedock.mjs +2205 -541
- package/bin/github-app-manifest.json.license +2 -0
- package/bin/hooks/session-start.mjs +4 -5
- package/bin/registry.mjs +20 -0
- package/bin/tui.mjs +172 -96
- package/commands/adopt.md +433 -0
- package/commands/analytics.md +2 -0
- package/commands/audit-agents.md +2 -0
- package/commands/audit.md +2 -0
- package/commands/autopilot.md +77 -0
- package/commands/changelog.md +354 -0
- package/commands/cleanup.md +2 -0
- package/commands/compat-audit.md +330 -0
- package/commands/deploy-info.md +6 -5
- package/commands/design-bench.md +176 -0
- package/commands/design-render-critique-loop.md +103 -0
- package/commands/design.md +259 -0
- package/commands/diagnose.md +483 -0
- package/commands/explain.md +349 -0
- package/commands/forge-stats.md +2 -0
- package/commands/forgedock-init.md +2 -0
- package/commands/geo-audit.md +2 -0
- package/commands/incident-response.md +2 -0
- package/commands/issue.md +9 -3
- package/commands/milestone.md +2 -0
- package/commands/optimize.md +411 -0
- package/commands/orchestrate.md +239 -0
- package/commands/pipeline-health.md +60 -1
- package/commands/qa-sweep.md +39 -2
- package/commands/quality-gate.md +67 -0
- package/commands/replay.md +263 -0
- package/commands/resume.md +264 -0
- package/commands/review-pr-agents.md +5 -1
- package/commands/review-pr-staging.md +160 -2
- package/commands/review-pr.md +85 -3
- package/commands/rollback.md +2 -0
- package/commands/scope.md +280 -0
- package/commands/security-audit.md +2 -0
- package/commands/status.md +262 -0
- package/commands/sync-ecosystem.md +2 -0
- package/commands/test-gate.md +929 -0
- package/commands/validate.md +2 -0
- package/commands/work-on/build/architect.md +128 -18
- package/commands/work-on/build/context.md +130 -17
- package/commands/work-on/build/implement.md +7 -3
- package/commands/work-on/build/validate.md +2 -0
- package/commands/work-on/build.md +32 -7
- package/commands/work-on/close.md +85 -0
- package/commands/work-on/decompose.md +2 -0
- package/commands/work-on/investigate.md +30 -3
- package/commands/work-on/review.md +13 -1
- package/commands/work-on-monolithic.md +2 -0
- package/commands/work-on.md +676 -38
- package/docs/BRAND.md +51 -0
- package/docs/CI.md +177 -0
- package/docs/CONFIG.md +94 -0
- package/docs/FORGE-PROTOCOL.md +324 -0
- package/docs/articles/b-vs-c-baseline.md +210 -0
- package/docs/articles/full-abc-benchmark.md +205 -0
- package/docs/articles/per-repo-adaptive-scripts-token-savings.md +174 -0
- package/docs/articles/selective-spec-loading-token-savings.md +140 -0
- package/docs/claude-breakpoints.json +47 -0
- package/docs/design/abc-benchmark.md +195 -0
- package/docs/design/design-architect-rationale.md +123 -0
- package/docs/design/design-memory.md +61 -0
- package/docs/design/design-spec-schema.md +248 -0
- package/docs/design/design-system-lint.md +150 -0
- package/docs/design/divergent-generation.md +106 -0
- package/docs/design/effects-appropriateness.md +133 -0
- package/docs/design/fixtures/briefs/README.md +47 -0
- package/docs/design/fixtures/briefs/cadence.md +39 -0
- package/docs/design/fixtures/briefs/plume.md +39 -0
- package/docs/design/fixtures/briefs/slipstream.md +39 -0
- package/docs/design/fixtures/briefs/tender.md +39 -0
- package/docs/design/fixtures/briefs/voltage.md +39 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/cadence/arm-b/run-1.html +364 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/cadence/arm-b/run-2.html +420 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/cadence/arm-b/run-3.html +506 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/plume/arm-b/run-1.html +611 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/plume/arm-b/run-2.html +594 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/plume/arm-b/run-3.html +817 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/runs.json +175 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/scorecard.json +138 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/slipstream/arm-b/run-1.html +373 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/slipstream/arm-b/run-2.html +398 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/slipstream/arm-b/run-3.html +595 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/tender/arm-b/run-1.html +531 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/tender/arm-b/run-2.html +565 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/tender/arm-b/run-3.html +674 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/voltage/arm-b/run-1.html +388 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/voltage/arm-b/run-2.html +463 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/voltage/arm-b/run-3.html +568 -0
- package/docs/design/fixtures/runs/full-abc/cadence/arm-a/run-1.html +562 -0
- package/docs/design/fixtures/runs/full-abc/cadence/arm-a/run-2.html +618 -0
- package/docs/design/fixtures/runs/full-abc/cadence/arm-a/run-3.html +618 -0
- package/docs/design/fixtures/runs/full-abc/cadence/arm-b/run-1.html +364 -0
- package/docs/design/fixtures/runs/full-abc/cadence/arm-b/run-2.html +420 -0
- package/docs/design/fixtures/runs/full-abc/cadence/arm-b/run-3.html +506 -0
- package/docs/design/fixtures/runs/full-abc/plume/arm-a/run-1.html +527 -0
- package/docs/design/fixtures/runs/full-abc/plume/arm-a/run-2.html +445 -0
- package/docs/design/fixtures/runs/full-abc/plume/arm-a/run-3.html +483 -0
- package/docs/design/fixtures/runs/full-abc/plume/arm-b/run-1.html +611 -0
- package/docs/design/fixtures/runs/full-abc/plume/arm-b/run-2.html +594 -0
- package/docs/design/fixtures/runs/full-abc/plume/arm-b/run-3.html +817 -0
- package/docs/design/fixtures/runs/full-abc/runs.json +190 -0
- package/docs/design/fixtures/runs/full-abc/scorecard.json +144 -0
- package/docs/design/fixtures/runs/full-abc/slipstream/arm-a/run-1.html +496 -0
- package/docs/design/fixtures/runs/full-abc/slipstream/arm-a/run-2.html +447 -0
- package/docs/design/fixtures/runs/full-abc/slipstream/arm-a/run-3.html +564 -0
- package/docs/design/fixtures/runs/full-abc/slipstream/arm-b/run-1.html +373 -0
- package/docs/design/fixtures/runs/full-abc/slipstream/arm-b/run-2.html +398 -0
- package/docs/design/fixtures/runs/full-abc/slipstream/arm-b/run-3.html +595 -0
- package/docs/design/fixtures/runs/full-abc/tender/arm-a/run-1.html +622 -0
- package/docs/design/fixtures/runs/full-abc/tender/arm-a/run-2.html +475 -0
- package/docs/design/fixtures/runs/full-abc/tender/arm-a/run-3.html +660 -0
- package/docs/design/fixtures/runs/full-abc/tender/arm-b/run-1.html +531 -0
- package/docs/design/fixtures/runs/full-abc/tender/arm-b/run-2.html +565 -0
- package/docs/design/fixtures/runs/full-abc/tender/arm-b/run-3.html +674 -0
- package/docs/design/fixtures/runs/full-abc/voltage/arm-a/run-1.html +525 -0
- package/docs/design/fixtures/runs/full-abc/voltage/arm-a/run-2.html +446 -0
- package/docs/design/fixtures/runs/full-abc/voltage/arm-a/run-3.html +463 -0
- package/docs/design/fixtures/runs/full-abc/voltage/arm-b/run-1.html +388 -0
- package/docs/design/fixtures/runs/full-abc/voltage/arm-b/run-2.html +463 -0
- package/docs/design/fixtures/runs/full-abc/voltage/arm-b/run-3.html +568 -0
- package/docs/design/fixtures/runs/full-abc-opus/cadence/arm-a/run-1.html +1068 -0
- package/docs/design/fixtures/runs/full-abc-opus/cadence/arm-a/run-2.html +1423 -0
- package/docs/design/fixtures/runs/full-abc-opus/cadence/arm-a/run-3.html +1245 -0
- package/docs/design/fixtures/runs/full-abc-opus/cadence/arm-b/run-1.html +305 -0
- package/docs/design/fixtures/runs/full-abc-opus/cadence/arm-b/run-2.html +327 -0
- package/docs/design/fixtures/runs/full-abc-opus/cadence/arm-b/run-3.html +428 -0
- package/docs/design/fixtures/runs/full-abc-opus/plume/arm-a/run-1.html +1270 -0
- package/docs/design/fixtures/runs/full-abc-opus/plume/arm-a/run-2.html +1172 -0
- package/docs/design/fixtures/runs/full-abc-opus/plume/arm-a/run-3.html +1532 -0
- package/docs/design/fixtures/runs/full-abc-opus/plume/arm-b/run-1.html +315 -0
- package/docs/design/fixtures/runs/full-abc-opus/plume/arm-b/run-2.html +342 -0
- package/docs/design/fixtures/runs/full-abc-opus/plume/arm-b/run-3.html +353 -0
- package/docs/design/fixtures/runs/full-abc-opus/runs.json +190 -0
- package/docs/design/fixtures/runs/full-abc-opus/scorecard.json +144 -0
- package/docs/design/fixtures/runs/full-abc-opus/slipstream/arm-a/run-1.html +345 -0
- package/docs/design/fixtures/runs/full-abc-opus/slipstream/arm-a/run-2.html +396 -0
- package/docs/design/fixtures/runs/full-abc-opus/slipstream/arm-a/run-3.html +502 -0
- package/docs/design/fixtures/runs/full-abc-opus/slipstream/arm-b/run-1.html +408 -0
- package/docs/design/fixtures/runs/full-abc-opus/slipstream/arm-b/run-2.html +446 -0
- package/docs/design/fixtures/runs/full-abc-opus/slipstream/arm-b/run-3.html +496 -0
- package/docs/design/fixtures/runs/full-abc-opus/tender/arm-a/run-1.html +813 -0
- package/docs/design/fixtures/runs/full-abc-opus/tender/arm-a/run-2.html +711 -0
- package/docs/design/fixtures/runs/full-abc-opus/tender/arm-a/run-3.html +721 -0
- package/docs/design/fixtures/runs/full-abc-opus/tender/arm-b/run-1.html +280 -0
- package/docs/design/fixtures/runs/full-abc-opus/tender/arm-b/run-2.html +291 -0
- package/docs/design/fixtures/runs/full-abc-opus/tender/arm-b/run-3.html +366 -0
- package/docs/design/fixtures/runs/full-abc-opus/voltage/arm-a/run-1.html +1010 -0
- package/docs/design/fixtures/runs/full-abc-opus/voltage/arm-a/run-2.html +836 -0
- package/docs/design/fixtures/runs/full-abc-opus/voltage/arm-a/run-3.html +930 -0
- package/docs/design/fixtures/runs/full-abc-opus/voltage/arm-b/run-1.html +407 -0
- package/docs/design/fixtures/runs/full-abc-opus/voltage/arm-b/run-2.html +396 -0
- package/docs/design/fixtures/runs/full-abc-opus/voltage/arm-b/run-3.html +529 -0
- package/docs/design/foundations/brutalist.css +294 -0
- package/docs/design/foundations/editorial.css +300 -0
- package/docs/design/foundations/minimal-luxury.css +295 -0
- package/docs/design/foundations/technical.css +293 -0
- package/docs/design/foundations/warm-photo.css +299 -0
- package/docs/design/reference-corpus.md +1726 -0
- package/docs/design/render-critique-loop.md +57 -0
- package/docs/site/getting-started.md +21 -0
- package/docs/spec-graph-schema.md +397 -0
- package/package.json +3 -3
- package/scripts/bench-scorecard.mjs +239 -0
- package/scripts/build-spec-graph.mjs +509 -0
- package/scripts/classify-lane.sh +138 -0
- package/scripts/design-system-lint.mjs +993 -0
- package/scripts/graph-query.sh +486 -0
- package/scripts/transition-label.sh +169 -0
- package/scripts/validate-pr-target.sh +30 -0
- package/scripts/validate-spec-graph.sh +423 -0
- package/templates/devdocs/README.md +72 -0
- package/templates/devdocs/agent/using-forgedock.md +3 -0
- package/templates/devdocs/agent/using-github.md +3 -0
- package/templates/devdocs/index.yaml +162 -0
- package/templates/devdocs/project/architecture.md +3 -0
- package/templates/devdocs/project/conventions.md +3 -0
- package/templates/devdocs/project/custom-instructions.md +3 -0
- package/templates/devdocs/project/glossary.md +3 -0
- package/templates/devdocs/project/stack.md +3 -0
- package/templates/devdocs/templates/commit-convention.md +3 -0
- package/templates/devdocs/templates/issue.md +3 -0
- package/templates/devdocs/templates/pull-request.md +3 -0
- package/templates/devdocs/templates/review-finding.md +3 -0
- package/templates/workflows/forgedock-review.yml +96 -0
package/commands/deploy-info.md
CHANGED
|
@@ -36,12 +36,13 @@ TARGET=${TARGET_BRANCH:-main}
|
|
|
36
36
|
|
|
37
37
|
# Step 1: Find all PR numbers in the bundle (same logic as review-pr-staging Phase 0A)
|
|
38
38
|
BUNDLE_PRS=$(git log origin/$TARGET..origin/$SOURCE --oneline \
|
|
39
|
-
| grep -
|
|
39
|
+
| grep -oE '#[0-9]+' \
|
|
40
40
|
| sort -u \
|
|
41
41
|
| tr -d '#')
|
|
42
42
|
|
|
43
43
|
MERGE_PRS=$(git log origin/$TARGET..origin/$SOURCE --merges --oneline \
|
|
44
|
-
| grep -
|
|
44
|
+
| grep -oE 'pull request #[0-9]+' \
|
|
45
|
+
| grep -oE '[0-9]+' \
|
|
45
46
|
| sort -u)
|
|
46
47
|
|
|
47
48
|
ALL_PR_NUMBERS=$(echo "$BUNDLE_PRS $MERGE_PRS" | tr ' ' '\n' | sort -u | grep -E '^[0-9]+$')
|
|
@@ -127,7 +128,7 @@ git log --oneline origin/$TARGET..origin/$SOURCE --format="%h %s" | head -50
|
|
|
127
128
|
|
|
128
129
|
```bash
|
|
129
130
|
# Extract PR numbers from merge commits
|
|
130
|
-
PR_NUMBERS=$(git log origin/$TARGET..origin/$SOURCE --merges --format="%s" | grep -
|
|
131
|
+
PR_NUMBERS=$(git log origin/$TARGET..origin/$SOURCE --merges --format="%s" | grep -oE '#[0-9]+' | sort -u)
|
|
131
132
|
|
|
132
133
|
# Get PR details
|
|
133
134
|
for PR in $PR_NUMBERS; do
|
|
@@ -142,7 +143,7 @@ done
|
|
|
142
143
|
# Extract issue references from PR bodies
|
|
143
144
|
for PR in $PR_NUMBERS; do
|
|
144
145
|
NUM=${PR#\#}
|
|
145
|
-
CLOSES=$(gh pr view $NUM --json body --jq '.body' 2>/dev/null | grep -
|
|
146
|
+
CLOSES=$(gh pr view $NUM --json body --jq '.body' 2>/dev/null | grep -oE 'Closes #[0-9]+' | grep -oE '[0-9]+')
|
|
146
147
|
for ISSUE in $CLOSES; do
|
|
147
148
|
gh issue view $ISSUE --json number,title,labels --jq '"\(.number) | \(.title) | \([.labels[].name] | join(","))"' 2>/dev/null
|
|
148
149
|
done
|
|
@@ -225,7 +226,7 @@ if [ -n "$NEW_ENV" ]; then
|
|
|
225
226
|
# Cross-check with SOPS chain (only when deploy.secrets_backend == "sops")
|
|
226
227
|
# If your project uses a different backend, skip this block and verify manually.
|
|
227
228
|
if [ "{DEPLOY_SECRETS_BACKEND}" = "sops" ]; then
|
|
228
|
-
for var in $(echo "$NEW_ENV" | grep -
|
|
229
|
+
for var in $(echo "$NEW_ENV" | grep -oE '^\+[A-Z_]+' | sed 's/^\+//'); do
|
|
229
230
|
if ! grep -q "$var" scripts/decrypt-secrets.sh 2>/dev/null; then
|
|
230
231
|
echo " ❌ $var NOT in decrypt-secrets.sh ENV_MAPPING"
|
|
231
232
|
fi
|
|
@@ -0,0 +1,176 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: Run the ABC benchmark — reference URL → design-blind brief → render arms A/B/C → blind judge → scorecard with win-rates over n runs
|
|
3
|
+
argument-hint: [<brief-name>|all] [--n <runs>] [--corpus-version <ver>]
|
|
4
|
+
---
|
|
5
|
+
<!-- SPDX-FileCopyrightText: Copyright (c) RapierCraft Studios -->
|
|
6
|
+
<!-- SPDX-License-Identifier: AGPL-3.0-or-later -->
|
|
7
|
+
|
|
8
|
+
# /design-bench — The ABC Benchmark Rig
|
|
9
|
+
|
|
10
|
+
**Input**: $ARGUMENTS — a seed brief name (`cadence` | `tender` | `slipstream` | `voltage` | `plume`) or `all`, with optional `--n <runs>` (default 3, min 3) and `--corpus-version <ver>`.
|
|
11
|
+
|
|
12
|
+
This is the **fitness function** for the UI Taste Harness (milestone #13, issue #878). It is built *before*
|
|
13
|
+
the harness and is the only thing that tells us whether each lever the harness adds actually moves design
|
|
14
|
+
quality. Running this command answers one question: **did arm A (the harness) score higher against the real
|
|
15
|
+
reference (arm C) than it did last time?**
|
|
16
|
+
|
|
17
|
+
Read [`docs/design/abc-benchmark.md`](../docs/design/abc-benchmark.md) (methodology) before running. The three
|
|
18
|
+
arms, the same-model rule, the C-as-calibration check, the three-layer judging protocol, and the n>=3 rule all
|
|
19
|
+
live there — this command is the executable workflow over that methodology.
|
|
20
|
+
|
|
21
|
+
**No application runtime.** ForgeDock is a set of command specs. This command is a step-by-step procedure an
|
|
22
|
+
agent executes; the only executable helper is the deterministic aggregator `scripts/bench-scorecard.mjs`.
|
|
23
|
+
|
|
24
|
+
---
|
|
25
|
+
|
|
26
|
+
## The three arms (one variable)
|
|
27
|
+
|
|
28
|
+
| Arm | What it is | How this command produces it |
|
|
29
|
+
|-----|------------|------------------------------|
|
|
30
|
+
| **A** | ForgeDock harness output | Run the brief through `/design` (#888). **Pluggable** — see [Arm A is pluggable](#arm-a-is-pluggable). |
|
|
31
|
+
| **B** | Raw frontier model, one shot | Hand the brief to the model once, no tools, no iteration. |
|
|
32
|
+
| **C** | The real reference page | Capture the brief's real reference URL through the same render pipeline. |
|
|
33
|
+
|
|
34
|
+
**Hard rule — same generation model for A and B.** The harness is the ONLY difference between A and B; that is
|
|
35
|
+
what makes the A-vs-B delta attributable to the harness. Different models benchmark models, not the harness.
|
|
36
|
+
|
|
37
|
+
**Validated generation model: `claude-opus-4-6`.** Benchmark results (#878) show a tier-level quality difference: 90% A-vs-B pairwise win-rate with Opus vs 54% with Sonnet, rubric mean 4.11 vs 3.40, slop 1.2 vs 1.8. The canonical baseline going forward is the Opus run. Raw data: `docs/design/fixtures/runs/full-abc-opus/`. Use Opus for both arm A and arm B unless explicitly testing a different model.
|
|
38
|
+
|
|
39
|
+
---
|
|
40
|
+
|
|
41
|
+
## Phase 0: Resolve inputs
|
|
42
|
+
|
|
43
|
+
1. Parse `$ARGUMENTS`: the brief name(s) and `--n` (default 3; **reject n<3** — n=1 is a lie). `all` = every
|
|
44
|
+
brief under `docs/design/fixtures/briefs/` (excluding `README.md`).
|
|
45
|
+
2. For each brief, load `docs/design/fixtures/briefs/<name>.md`. This is the design-blind input handed to arms
|
|
46
|
+
A and B.
|
|
47
|
+
3. Load the brief→reference(C) mapping from `docs/design/fixtures/briefs/README.md`. **The mapping is for the
|
|
48
|
+
judge/calibration record only — it is NEVER shown to the generating arms.** A real brand name leaking into a
|
|
49
|
+
brief measures recall, not taste, and inflates arm B.
|
|
50
|
+
|
|
51
|
+
---
|
|
52
|
+
|
|
53
|
+
## Phase 1: Brief (reference URL → design-blind brief)
|
|
54
|
+
|
|
55
|
+
The seed briefs are pre-stripped fixtures, so for the seed set this phase is a load, not a derivation.
|
|
56
|
+
|
|
57
|
+
**To add a new reference URL** (outside the seed set): fetch the page, strip every visual cue (color, type,
|
|
58
|
+
layout, "make it like X"), keep only what the product *does* / who it's for / what the page must communicate,
|
|
59
|
+
and assign an **anonymized** product name. Save as a new fixture and add its mapping row. Follow
|
|
60
|
+
`docs/design/fixtures/briefs/README.md`.
|
|
61
|
+
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
## Phase 2: Render arms (kill confounds)
|
|
65
|
+
|
|
66
|
+
Render **all three arms through the identical pipeline** so the judge compares design, not rendering artifacts.
|
|
67
|
+
|
|
68
|
+
For each of the `--n` runs per brief:
|
|
69
|
+
|
|
70
|
+
1. **Arm A** — run the brief through `/design` (#888) → self-contained HTML.
|
|
71
|
+
2. **Arm B** — one-shot the SAME model with the brief → self-contained HTML. No tools, no iteration.
|
|
72
|
+
3. **Arm C** — the real reference URL.
|
|
73
|
+
|
|
74
|
+
Render every arm with **Playwright** (milestone render dependency) at:
|
|
75
|
+
|
|
76
|
+
- **1440px** desktop and **390px** mobile,
|
|
77
|
+
- `networkidle` + fonts loaded before capture,
|
|
78
|
+
- **full-page** screenshot.
|
|
79
|
+
|
|
80
|
+
For arm C, trim cookie banners / hero video and **record any trim applied** in the run record (reproducibility).
|
|
81
|
+
|
|
82
|
+
> **Arm A is pluggable.** Until `/design` (#888) lands, arm A is produced by a clearly-labeled manual hook:
|
|
83
|
+
> generate arm A by hand using the harness method-of-the-day and drop its HTML into the run record marked
|
|
84
|
+
> `arm_a_source: manual`. The rig — brief loading, render protocol, judging, scorecard — is fully operational
|
|
85
|
+
> without #888. When #888 lands, replace the manual hook with the `/design` invocation; nothing else changes.
|
|
86
|
+
> **Do NOT block the benchmark on #888.**
|
|
87
|
+
|
|
88
|
+
---
|
|
89
|
+
|
|
90
|
+
## Phase 3: Judge (three layers)
|
|
91
|
+
|
|
92
|
+
The judge is a **named, independent stage**. See [Anti-Goodhart](#anti-goodhart) — it MUST NOT be the harness's
|
|
93
|
+
own critique loop (#882).
|
|
94
|
+
|
|
95
|
+
For each run, produce judge output in the [runs-file schema](../docs/design/abc-benchmark.md#scorecard-schema):
|
|
96
|
+
|
|
97
|
+
### Layer 1 — Blind randomized pairwise (primary)
|
|
98
|
+
Show the judge two **anonymized** screenshots in **shuffled** order and ask: *"Which reads more like a
|
|
99
|
+
professionally designed, well-funded SaaS landing page?"* Run **A-vs-C, B-vs-C, A-vs-B**. Record the winner
|
|
100
|
+
(`"A" | "B" | "C" | "tie"`) per comparison. Screenshots carry no arm labels.
|
|
101
|
+
|
|
102
|
+
### Layer 2 — Rubric (1–5 per arm)
|
|
103
|
+
Score each arm on: hierarchy, typography, color discipline, whitespace rhythm, originality-vs-slop, mobile.
|
|
104
|
+
|
|
105
|
+
### Layer 3 — Slop detector (binary)
|
|
106
|
+
Run the anti-slop negatives checklist (corpus #880) over each arm; record the count of negatives present
|
|
107
|
+
(lower is better).
|
|
108
|
+
|
|
109
|
+
Append each run to the brief's `runs[]` array in the runs file.
|
|
110
|
+
|
|
111
|
+
---
|
|
112
|
+
|
|
113
|
+
## Phase 4: Scorecard
|
|
114
|
+
|
|
115
|
+
Aggregate deterministically:
|
|
116
|
+
|
|
117
|
+
```bash
|
|
118
|
+
node scripts/bench-scorecard.mjs runs.json > scorecard.json
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
The aggregator (see `scripts/bench-scorecard.mjs`):
|
|
122
|
+
- **rejects any product with n<3 runs** (exit 1) — enforces the no-point-estimate rule,
|
|
123
|
+
- emits **win-rate vs C** per arm (overall + per-product), the **A-vs-B harness delta**, **rubric mean+stdev**
|
|
124
|
+
per arm/dimension, **mean slop** per arm,
|
|
125
|
+
- runs the **judge-calibration check**: C must beat A and B in the pairwise; any run where C loses is listed in
|
|
126
|
+
`judge_calibration.miscalibrated_runs` and the command exits 2 with a warning. A real reference losing to a
|
|
127
|
+
generated arm is a judge defect, not a harness win — treat those runs as suspect.
|
|
128
|
+
|
|
129
|
+
Post the result as a `FORGE:BENCH_SCORECARD` annotation (registered in
|
|
130
|
+
[`docs/FORGE-PROTOCOL.md`](../docs/FORGE-PROTOCOL.md)) on the tracking issue:
|
|
131
|
+
|
|
132
|
+
```bash
|
|
133
|
+
gh issue comment <ISSUE> --body "<!-- FORGE:BENCH_SCORECARD -->
|
|
134
|
+
## ABC Benchmark Scorecard
|
|
135
|
+
|
|
136
|
+
**Corpus version**: <ver> · **Generation model (A & B)**: <model> (canonical: claude-opus-4-6) · **Judge model**: <independent judge>
|
|
137
|
+
**Products**: <n_products> · **Runs/product**: <n>
|
|
138
|
+
|
|
139
|
+
| Arm | Win-rate vs C | A-vs-B | Mean slop |
|
|
140
|
+
|-----|---------------|--------|-----------|
|
|
141
|
+
| A | <wr> | <a_vs_b> | <slop_A> |
|
|
142
|
+
| B | <wr> | — | <slop_B> |
|
|
143
|
+
|
|
144
|
+
**Judge calibration**: <ok | MISCALIBRATED — N runs where C lost (suspect)>
|
|
145
|
+
|
|
146
|
+
\`\`\`json
|
|
147
|
+
<scorecard.json>
|
|
148
|
+
\`\`\`
|
|
149
|
+
|
|
150
|
+
<!-- FORGE:BENCH_SCORECARD:COMPLETE -->"
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
---
|
|
154
|
+
|
|
155
|
+
## Two non-negotiable rules
|
|
156
|
+
|
|
157
|
+
### n=1 is a lie
|
|
158
|
+
Taste output is high-variance. Run **3–5 generations per arm across 3–5 products** and report **distributions**.
|
|
159
|
+
The aggregator hard-fails on n<3.
|
|
160
|
+
|
|
161
|
+
### Anti-Goodhart
|
|
162
|
+
The benchmark judge **MUST be independent from the harness's own critique loop (#882)**. Optimizing against the
|
|
163
|
+
same judge that scores the benchmark is training to the test — the number rises while real quality does not. The
|
|
164
|
+
benchmark judge uses its own prompt and is a distinct stage; it does not reuse the #882 critique judge.
|
|
165
|
+
|
|
166
|
+
---
|
|
167
|
+
|
|
168
|
+
## Acceptance (issue #878)
|
|
169
|
+
|
|
170
|
+
- [x] Reproducible command/workflow: reference URL → brief → render arms → judge → scorecard.
|
|
171
|
+
- [x] Arms A/B/C defined; same generation model for A and B; C = gold standard + judge calibration.
|
|
172
|
+
- [x] Three judging layers: blind randomized pairwise, 1–5 rubric, binary slop detector.
|
|
173
|
+
- [x] n=3–5 across 3–5 products; distributions reported; n=1 rejected by the aggregator.
|
|
174
|
+
- [x] Benchmark judge independent from the #882 critique loop (anti-Goodhart).
|
|
175
|
+
- [x] Five seed briefs as fixtures (Cadence/Tender/Slipstream/Voltage/Plume), anonymized + mapped to C.
|
|
176
|
+
- [x] Arm A pluggable — rig runs now, plugs into `/design` (#888) when it lands.
|
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: Render a design from its FORGE:DESIGN_SPEC, critique the rendered pixels against the perceptual negatives, and iterate until it passes or the budget is spent — the missing reward signal.
|
|
3
|
+
argument-hint: [<issue-number>|<spec-path>] [--max-iters <n>] [--viewports desktop,mobile]
|
|
4
|
+
---
|
|
5
|
+
<!-- SPDX-FileCopyrightText: Copyright (c) RapierCraft Studios -->
|
|
6
|
+
<!-- SPDX-License-Identifier: AGPL-3.0-or-later -->
|
|
7
|
+
|
|
8
|
+
# /design-render-critique-loop — render → vision-critique → iterate
|
|
9
|
+
|
|
10
|
+
**Input**: $ARGUMENTS — the design issue number (reads its `FORGE:DESIGN_SPEC`) or a path to a spec, with optional
|
|
11
|
+
`--max-iters <n>` (default 3) and `--viewports` (default `desktop,mobile`).
|
|
12
|
+
|
|
13
|
+
This is the **missing reward signal** for the UI Taste Harness (milestone #13, issue #882). Every other lever in the
|
|
14
|
+
harness can still raw-dog the output — write a page and defend it blind. This loop is the only stage that *looks at the
|
|
15
|
+
rendered pixels* and feeds a correction back in. Without it the harness has no feedback loop; with it, a page measurably
|
|
16
|
+
improves across iterations.
|
|
17
|
+
|
|
18
|
+
Read [`docs/design/render-critique-loop.md`](../docs/design/render-critique-loop.md) (methodology) before running —
|
|
19
|
+
the thesis, the iteration protocol, the deterministic-floor-first ordering, the anti-Goodhart independence rule, and the
|
|
20
|
+
pass/budget criteria all live there. This command is the executable procedure over that methodology.
|
|
21
|
+
|
|
22
|
+
**No application runtime.** ForgeDock is a set of command specs. The only executable helper is the deterministic
|
|
23
|
+
linter `scripts/design-system-lint.mjs` (#884); the render + critique steps are agent-driven via the Playwright MCP.
|
|
24
|
+
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
## Inputs (read before iterating)
|
|
28
|
+
|
|
29
|
+
| Input | Source | Role |
|
|
30
|
+
|-------|--------|------|
|
|
31
|
+
| `FORGE:DESIGN_SPEC` | the design issue (#881) | the committed intent the render must satisfy |
|
|
32
|
+
| `scripts/design-system-lint.mjs` | #884 | deterministic floor — cheap, runs first |
|
|
33
|
+
| Perceptual negatives N3, N8–N12 | [`reference-corpus.md`](../docs/design/reference-corpus.md) (#880) | what the vision critic judges (the linter declares these out of scope) |
|
|
34
|
+
| Render protocol | the #878 / #875 Playwright protocol | desktop + mobile capture, same viewports as the benchmark |
|
|
35
|
+
|
|
36
|
+
## The loop
|
|
37
|
+
|
|
38
|
+
```
|
|
39
|
+
iteration = 0
|
|
40
|
+
while iteration < max_iters:
|
|
41
|
+
iteration += 1
|
|
42
|
+
|
|
43
|
+
# 1. Deterministic floor FIRST (cheap — never spend a render on a page that fails the linter)
|
|
44
|
+
Run: node scripts/design-system-lint.mjs --spec <spec> --html <generated> [--strict]
|
|
45
|
+
if HARD findings: fix against the spec, continue # do not render a lint-failing page
|
|
46
|
+
|
|
47
|
+
# 2. Render (Playwright MCP) — desktop + mobile, same protocol as #878
|
|
48
|
+
Capture the page at each viewport.
|
|
49
|
+
|
|
50
|
+
# 3. Vision critique — judge the RENDERED pixels against the perceptual negatives ONLY
|
|
51
|
+
For N3 (centered/no asymmetry), N8 (stock/generic 3D), N9 (glassmorphism overuse),
|
|
52
|
+
N10 (no visual hierarchy), N11 (abstract copy / no product shown), N12 (decorative effects
|
|
53
|
+
that serve nothing): is the negative present in the render? Each hit is a finding.
|
|
54
|
+
Also check the render against the spec's own acceptance + effects_plan justification (#885).
|
|
55
|
+
|
|
56
|
+
# 4. Emit the per-iteration trail
|
|
57
|
+
Post a FORGE:CRITIQUE annotation (schema below) with findings + verdict.
|
|
58
|
+
|
|
59
|
+
if no perceptual findings AND lint clean: PASS — break
|
|
60
|
+
else: translate each finding into a concrete spec/markup correction and iterate.
|
|
61
|
+
|
|
62
|
+
if iteration == max_iters and not PASS:
|
|
63
|
+
Post the final FORGE:CRITIQUE with verdict BUDGET-EXHAUSTED and the residual findings.
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
## Ordering rule (cheap gate first)
|
|
67
|
+
|
|
68
|
+
The deterministic linter (#884) runs **before** every render. A page that fails a hard lint rule is fixed before a
|
|
69
|
+
render is ever captured — rendering and vision-critiquing a page the linter already rejects wastes the expensive step.
|
|
70
|
+
The critic owns **only** the perceptual negatives the linter declares out of scope (N3, N8–N12); it never re-checks a
|
|
71
|
+
deterministic rule. No duplication.
|
|
72
|
+
|
|
73
|
+
## Anti-Goodhart independence (non-negotiable)
|
|
74
|
+
|
|
75
|
+
The critic in this loop is **strictly independent** from the ABC benchmark judge (#878). The benchmark judge is the
|
|
76
|
+
*evaluator* — it scores finished arms blind against arm C to tell us whether the harness improved. If the loop
|
|
77
|
+
optimized against that same judge, the harness would be training on its own test set (Goodhart). The loop's critic
|
|
78
|
+
judges *one page against its own committed spec + the perceptual negatives*; the benchmark judge judges *arms against a
|
|
79
|
+
real reference*. They must never be the same judge. This is also distinct from the divergent-generation taste-judge
|
|
80
|
+
(#883), which scores pre-render candidate directions.
|
|
81
|
+
|
|
82
|
+
## `FORGE:CRITIQUE` annotation
|
|
83
|
+
|
|
84
|
+
Registered in [FORGE-PROTOCOL](../docs/FORGE-PROTOCOL.md). One per iteration, so the improvement trajectory is an
|
|
85
|
+
auditable artifact:
|
|
86
|
+
|
|
87
|
+
```markdown
|
|
88
|
+
<!-- FORGE:CRITIQUE -->
|
|
89
|
+
## Critique — {product} · iteration {i}/{max}
|
|
90
|
+
|
|
91
|
+
**Lint floor:** {PASS | fixed N hard findings}
|
|
92
|
+
**Render:** desktop + mobile captured
|
|
93
|
+
**Perceptual findings:**
|
|
94
|
+
- N{n}: {what was observed in the render} → {correction}
|
|
95
|
+
**Verdict:** {PASS | ITERATE | BUDGET-EXHAUSTED}
|
|
96
|
+
<!-- FORGE:CRITIQUE:COMPLETE -->
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
## Success measure
|
|
100
|
+
|
|
101
|
+
The loop works when a page **improves across iterations** and when, on the seed set, *A-with-loop* scores higher than
|
|
102
|
+
*A-without-loop* and higher than *raw B*, measured against the #879 baseline via the ABC benchmark (#878). The loop is
|
|
103
|
+
the reward signal; the benchmark is the independent scoreboard that confirms it moved.
|
|
@@ -0,0 +1,259 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: Route a design GitHub issue through the UI Taste Harness — investigate → architect → implement → critique loop → user-feedback loop → close — accumulating FORGE:DESIGN_* annotations and driving the design:* label state machine to design:shipped.
|
|
3
|
+
argument-hint: <issue-number> | <brief-name>
|
|
4
|
+
---
|
|
5
|
+
<!-- SPDX-FileCopyrightText: Copyright (c) RapierCraft Studios -->
|
|
6
|
+
<!-- SPDX-License-Identifier: AGPL-3.0-or-later -->
|
|
7
|
+
|
|
8
|
+
# /design — the design pipeline (the `/work-on` analog for design)
|
|
9
|
+
|
|
10
|
+
**Input**: $ARGUMENTS — a design issue number, or a seed brief name (`cadence` | `tender` | `slipstream` | `voltage` | `plume`) which is opened as a design issue first.
|
|
11
|
+
|
|
12
|
+
This is the **spine of the UI Taste Harness** (milestone #13, issue #888). Just as a code task is a GitHub issue
|
|
13
|
+
routed `investigate → build → review → close` by [`/work-on`](work-on.md) — accumulating `FORGE:*` annotations, closed
|
|
14
|
+
at the end — a **design task is a GitHub issue routed through this design pipeline**, accumulating `FORGE:DESIGN_*`
|
|
15
|
+
annotations and closed when the design ships and passes its gates. It is a parallel track on the same "GitHub as a
|
|
16
|
+
knowledge graph" model, not a new paradigm.
|
|
17
|
+
|
|
18
|
+
**It IS arm A of the ABC benchmark (#878).** Running a brief through `/design` produces the arm-A output; the benchmark
|
|
19
|
+
rig (`/design-bench`) feeds briefs in and screenshots the result, so this command is what makes the benchmark runnable
|
|
20
|
+
end to end.
|
|
21
|
+
|
|
22
|
+
**Agent model policy**: `model: "claude-opus-4-6"` — Opus is the validated generation model. Benchmark #878 (90% A-vs-B pairwise win-rate with Opus vs 54% with Sonnet) confirms a tier-level quality difference. Both arm A and arm B MUST use the same model to keep the benchmark valid (see [design-bench](design-bench.md)).
|
|
23
|
+
|
|
24
|
+
**No application runtime.** ForgeDock is a set of command specs. `/design` *sequences* the already-built stage specs —
|
|
25
|
+
it does not reimplement them. The only executable helper in the chain is the deterministic linter
|
|
26
|
+
`scripts/design-system-lint.mjs` (#884).
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## Universal Stage Dispatcher
|
|
31
|
+
|
|
32
|
+
<!-- This is the single source of truth for design:* transitions, mirroring /work-on's Universal Phase Dispatcher. -->
|
|
33
|
+
|
|
34
|
+
| Stage | design:* label | Terminal? | Reads | Produces | Spec it sequences |
|
|
35
|
+
|-------|----------------|-----------|-------|----------|-------------------|
|
|
36
|
+
| 1. design-investigate | `design:investigating` | No | the brief | `FORGE:DESIGN_CONTEXT` | corpus #880 (grammar) + memory #887 (what to diverge from) |
|
|
37
|
+
| 2. design-architect | `design:architecting` | No | `FORGE:DESIGN_CONTEXT` | `FORGE:DESIGN_RATIONALE` → `FORGE:DESIGN_CANDIDATES` → `FORGE:DESIGN_SPEC` | rationale #886, divergent generation #883, effects #885, schema #881 |
|
|
38
|
+
| 3. design-implement | `design:generating` | No | `FORGE:DESIGN_SPEC` | the generated page (artifact) | — (generate from the committed spec) |
|
|
39
|
+
| 4. design-critique | `design:critiquing` | No | the render + `FORGE:DESIGN_SPEC` | `FORGE:CRITIQUE` (per pass) | render→critique loop #882 (lint #884 floor + perf #875 + a11y) |
|
|
40
|
+
| 4.5. user-feedback | `design:awaiting-feedback` | No | `FORGE:CRITIQUE` + user input | `FORGE:USER_FEEDBACK` (per round) | this spec (#1044); bypassed in automated/benchmark runs |
|
|
41
|
+
| 5. design-close | `design:shipped` / `design:rejected` | **Yes** | `FORGE:USER_FEEDBACK` (latest) or `FORGE:CRITIQUE` | `FORGE:DESIGN_SHIPPED` | scorecard + write outcome to memory #887 |
|
|
42
|
+
|
|
43
|
+
**Continuation rule** (mirrors `/work-on`): after any stage completes, if the issue is not at a terminal label
|
|
44
|
+
(`design:shipped` or `design:rejected`), proceed to the next stage immediately. No intermediate stage is terminal.
|
|
45
|
+
`design:awaiting-feedback` is **non-terminal** — it resolves to Stage 5 when `FORGE:USER_FEEDBACK.satisfied == "yes"`
|
|
46
|
+
or when no user is present (automated runs bypass Stage 4.5 entirely).
|
|
47
|
+
|
|
48
|
+
---
|
|
49
|
+
|
|
50
|
+
## Stage 1 — design-investigate (`design:investigating`)
|
|
51
|
+
|
|
52
|
+
Parse the brief into message / audience / single objection. Pull the relevant grammar from the
|
|
53
|
+
[reference corpus](../docs/design/reference-corpus.md) (#880) and query [design-memory](../docs/design/design-memory.md)
|
|
54
|
+
(#887) for the recent signature moves / archetypes / palettes to **diverge** from. Post `FORGE:DESIGN_CONTEXT`.
|
|
55
|
+
|
|
56
|
+
## Stage 2 — design-architect (`design:architecting`)
|
|
57
|
+
|
|
58
|
+
Run the [design-architect rationale](../docs/design/design-architect-rationale.md) (#886): the 7-element diary →
|
|
59
|
+
`FORGE:DESIGN_RATIONALE`. Then [divergent generation](../docs/design/divergent-generation.md) (#883): commit one
|
|
60
|
+
archetype, diverge into N directions, independent taste-judge selects → `FORGE:DESIGN_CANDIDATES`. The winning
|
|
61
|
+
direction + the [effects-appropriateness](../docs/design/effects-appropriateness.md) (#885) `effects_plan` are
|
|
62
|
+
committed into `FORGE:DESIGN_SPEC` ([schema](../docs/design/design-spec-schema.md), #881).
|
|
63
|
+
|
|
64
|
+
## Stage 3 — design-implement (`design:generating`)
|
|
65
|
+
|
|
66
|
+
Generate the page from the committed `FORGE:DESIGN_SPEC`. The spec is the contract — no taste decisions are re-rolled
|
|
67
|
+
here; generation realizes the committed intent.
|
|
68
|
+
|
|
69
|
+
### Foundation CSS injection <!-- Added: forge#1048 -->
|
|
70
|
+
|
|
71
|
+
Before generating the page, inject the archetype's pre-built CSS foundation as the starting stylesheet. Foundation files live in `docs/design/foundations/` and are keyed by `meta.archetype` from `FORGE:DESIGN_SPEC`:
|
|
72
|
+
|
|
73
|
+
| `meta.archetype` | Foundation file |
|
|
74
|
+
|---|---|
|
|
75
|
+
| `editorial-typographic` | `docs/design/foundations/editorial.css` |
|
|
76
|
+
| `technical-dense` | `docs/design/foundations/technical.css` |
|
|
77
|
+
| `minimal-luxury` | `docs/design/foundations/minimal-luxury.css` |
|
|
78
|
+
| `bold-brutalist` | `docs/design/foundations/brutalist.css` |
|
|
79
|
+
| `warm-photographic` | `docs/design/foundations/warm-photo.css` |
|
|
80
|
+
|
|
81
|
+
**Injection pattern**: read the foundation file content and include it in the generation prompt as the starting `<style>` block. Instruct the generator to:
|
|
82
|
+
1. Use the foundation's component classes (`.btn`, `.card`, `.badge`, `.input`, `.reveal`) where applicable — do NOT replace them with generic Tailwind equivalents.
|
|
83
|
+
2. Override `--color-accent` and `--color-bg` custom properties with the spec's `color.accent` and `color.background` values to apply the committed palette to the foundation.
|
|
84
|
+
3. Extend, don't replace — additional styles may be added on top of the foundation, but the foundation's radius system, component state management (hover/active/focus), and micro-detail rules (::selection, scrollbar, focus rings) must be preserved.
|
|
85
|
+
|
|
86
|
+
The foundation is the professional baseline. The generator adds layout and content on top.
|
|
87
|
+
|
|
88
|
+
## Stage 4 — design-critique (`design:critiquing`)
|
|
89
|
+
|
|
90
|
+
Run [`/design-render-critique-loop`](design-render-critique-loop.md) (#882): deterministic lint floor (#884) → Playwright
|
|
91
|
+
render (desktop + mobile, #875/#878) → vision critique of the perceptual negatives → iterate. Each pass posts a
|
|
92
|
+
`FORGE:CRITIQUE`. The loop continues until PASS or budget.
|
|
93
|
+
|
|
94
|
+
## Stage 4.5 — user-feedback loop (`design:awaiting-feedback`) <!-- Added: forge#1044 -->
|
|
95
|
+
|
|
96
|
+
**Entry condition**: Stage 4 (design-critique) has completed — `FORGE:CRITIQUE` posted with verdict `PASS` or
|
|
97
|
+
`BUDGET-EXHAUSTED`. **Automated/benchmark bypass**: if no user is present (e.g., running as part of the ABC benchmark
|
|
98
|
+
arm A), skip this stage entirely and proceed to Stage 5.
|
|
99
|
+
|
|
100
|
+
**Purpose**: The automated critique loop (#882) closes the perceptual-quality loop. This stage closes the **brand
|
|
101
|
+
fit** loop — the part only the user can judge. Brand assets are user-supplied, emotional register is subjective, and
|
|
102
|
+
the best heroes are refined, not generated once. Stage 4.5 collects that input in a structured way and routes it to
|
|
103
|
+
surgical section-level re-generation rather than full-page regeneration.
|
|
104
|
+
|
|
105
|
+
### Step 1 — Set label
|
|
106
|
+
|
|
107
|
+
```bash
|
|
108
|
+
gh issue edit {NUMBER} {GH_FLAG} --add-label "design:awaiting-feedback" \
|
|
109
|
+
--remove-label "design:critiquing" 2>/dev/null || true
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
### Step 2 — Prompt the user with structured hero-specific questions
|
|
113
|
+
|
|
114
|
+
After Stage 4 completes, post a comment on the design issue asking the user the following questions. Present them
|
|
115
|
+
clearly — the user's answer drives the `FORGE:USER_FEEDBACK` structured fields.
|
|
116
|
+
|
|
117
|
+
```
|
|
118
|
+
## Hero feedback — your turn
|
|
119
|
+
|
|
120
|
+
The automated critique is complete. Now it's yours to shape.
|
|
121
|
+
|
|
122
|
+
Please answer any or all of the following:
|
|
123
|
+
|
|
124
|
+
1. **Demo video or animation** — Do you have a product demo video or animation to feature in the hero?
|
|
125
|
+
(paste a URL, or say "none")
|
|
126
|
+
|
|
127
|
+
2. **Primary emotion** — What's the primary feeling you want the hero to evoke?
|
|
128
|
+
Options: trust / speed / power / craft / play — or describe in your own words.
|
|
129
|
+
|
|
130
|
+
3. **Brand assets** — Any assets to integrate?
|
|
131
|
+
- Logo SVG (paste URL or say "none")
|
|
132
|
+
- Brand hex colors (e.g. "#1a1a2e, #e94560" or "none")
|
|
133
|
+
- Product screenshots (paste URLs or say "none")
|
|
134
|
+
|
|
135
|
+
4. **Anything that feels off?** — Any section, element, or overall impression that feels wrong or
|
|
136
|
+
off-brand? Describe it.
|
|
137
|
+
|
|
138
|
+
If you're happy with the design as-is, reply: **"looks good, ship it"** (or any affirmative) — that
|
|
139
|
+
sets `satisfied: yes` and moves to Stage 5 without re-generation.
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
### Step 3 — Parse the user's response into `FORGE:USER_FEEDBACK`
|
|
143
|
+
|
|
144
|
+
Read the user's reply and produce a structured `FORGE:USER_FEEDBACK` annotation. Map natural-language input to
|
|
145
|
+
the structured fields:
|
|
146
|
+
|
|
147
|
+
| User says | Maps to |
|
|
148
|
+
|---|---|
|
|
149
|
+
| Pastes a video URL | `feedback_type: asset`, `asset_url: <url>`, `section_target: "hero"` |
|
|
150
|
+
| "make it feel faster / more urgent" | `feedback_type: emotion`, `emotion_target: speed`, `modification: "increase motion intensity; tighten headline tracking"` |
|
|
151
|
+
| "add our logo" + SVG URL | `feedback_type: asset`, `asset_url: <url>`, `section_target: "nav"`, `modification: "replace text logo with SVG; adjust sizing"` |
|
|
152
|
+
| Provides brand hex colors (e.g. "#1a1a2e") | `feedback_type: asset`, `asset_url: none`, `section_target: "all"`, `modification: "update CSS custom properties: --color-accent: #… --color-bg: #…"` |
|
|
153
|
+
| Describes a layout change | `feedback_type: direction`, `modification: <parsed description>` |
|
|
154
|
+
| Freeform complaint | `feedback_type: freeform`, `freeform_notes: <verbatim>`, `modification: <best-effort parse>` |
|
|
155
|
+
| "looks good" / "ship it" / affirmative | `satisfied: yes` → proceed to Stage 5 without re-generation |
|
|
156
|
+
|
|
157
|
+
```bash
|
|
158
|
+
gh issue comment {NUMBER} {GH_FLAG} --body "<!-- FORGE:USER_FEEDBACK -->
|
|
159
|
+
## User Feedback — {product} · round {n}
|
|
160
|
+
|
|
161
|
+
**Section target:** {section_target}
|
|
162
|
+
**Feedback type:** {asset | emotion | direction | freeform}
|
|
163
|
+
**Asset URL:** {URL or "none"}
|
|
164
|
+
**Modification:** {structured description of what to change}
|
|
165
|
+
**Emotion target:** {trust | speed | power | craft | play | unchanged}
|
|
166
|
+
**Satisfied:** {yes | no}
|
|
167
|
+
**Freeform notes:** {verbatim user input not captured above}
|
|
168
|
+
<!-- FORGE:USER_FEEDBACK:COMPLETE -->"
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
### Step 4 — Route on `satisfied`
|
|
172
|
+
|
|
173
|
+
**If `satisfied: yes`** (user approved, or no feedback given): proceed immediately to Stage 5 (design-close).
|
|
174
|
+
No re-generation needed.
|
|
175
|
+
|
|
176
|
+
**If `satisfied: no`** (feedback given): proceed to Step 5 (surgical re-generation), then loop back to Step 2
|
|
177
|
+
for another feedback round.
|
|
178
|
+
|
|
179
|
+
### Step 5 — Surgical re-generation (only when `satisfied: no`)
|
|
180
|
+
|
|
181
|
+
Use the `section_target` from `FORGE:USER_FEEDBACK` and the section IDs in `FORGE:DESIGN_SPEC →
|
|
182
|
+
layout_grammar.sections` to re-generate only the targeted section. See
|
|
183
|
+
[`docs/design/design-spec-schema.md`](../docs/design/design-spec-schema.md) for the full surgical re-generation
|
|
184
|
+
contract.
|
|
185
|
+
|
|
186
|
+
**Inputs for the re-generation agent**:
|
|
187
|
+
1. The full current HTML page
|
|
188
|
+
2. `FORGE:DESIGN_SPEC` (the committed spec — defines what all non-targeted sections must look like)
|
|
189
|
+
3. `FORGE:USER_FEEDBACK` (the modification + asset URL for the targeted section)
|
|
190
|
+
|
|
191
|
+
**Re-generation rules**:
|
|
192
|
+
- Modify **only** the targeted section's markup and styles
|
|
193
|
+
- Do NOT change any other section's content, layout, or CSS
|
|
194
|
+
- Do NOT re-roll the committed `meta.archetype` or the signature move from `FORGE:DESIGN_RATIONALE`
|
|
195
|
+
- If the feedback requires an archetype change, note the constraint and propose the closest variant within the committed direction
|
|
196
|
+
- After re-generation, run the deterministic linter on the modified section to catch any regressions (#884)
|
|
197
|
+
|
|
198
|
+
**Asset integration**:
|
|
199
|
+
|
|
200
|
+
| Asset | Integration procedure |
|
|
201
|
+
|---|---|
|
|
202
|
+
| Video URL (mp4/webm) | Add `<video autoplay loop muted playsinline>` scaffold in the hero visual area; adjust hero layout CSS to accommodate; keep LCP budget ≤ 2000ms |
|
|
203
|
+
| Video embed URL (YouTube/Vimeo) | `<iframe>` with `loading="lazy"` and aspect-ratio container (`padding-top: 56.25%`); no autoplay |
|
|
204
|
+
| Logo SVG | Fetch the SVG; inline or `<img>` it at the nav/hero logo node; remove the text logo; set `width`/`height` to maintain proportional sizing; verify contrast against background (WCAG AA ≥ 4.5:1) |
|
|
205
|
+
| Brand hex colors | Update the relevant CSS custom properties (`--color-accent`, `--color-bg`, etc.); run contrast check on all text/background pairs; record the update in `effects_plan.per_section` if it affects a hero effect |
|
|
206
|
+
| Product screenshot/image | Replace the placeholder with `<img loading="lazy" decoding="async">` or a `background-image`; maintain existing aspect-ratio and sizing tokens from the spec |
|
|
207
|
+
|
|
208
|
+
**After re-generation**: loop back to Step 2 — prompt the user again for the next round.
|
|
209
|
+
|
|
210
|
+
### Budget
|
|
211
|
+
|
|
212
|
+
Maximum **3 user-feedback rounds** per design issue (configurable). If the budget is exhausted with `satisfied: no`,
|
|
213
|
+
post a summary of the rounds and proceed to Stage 5 with the best version. The design was not fully approved by the
|
|
214
|
+
user — note this in the `FORGE:DESIGN_SHIPPED` annotation.
|
|
215
|
+
|
|
216
|
+
---
|
|
217
|
+
|
|
218
|
+
## Stage 5 — design-close (`design:shipped` | `design:rejected`)
|
|
219
|
+
|
|
220
|
+
Apply the **definition of done**: critique-rubric threshold met **and** perf budget (#875) **and** a11y check **and**
|
|
221
|
+
the divergence check vs memory (#887) — not a reskin of a prior design. If all pass → write the realized outcome
|
|
222
|
+
(archetype, signature move, palette/type/effects, learnings) to [design-memory](../docs/design/design-memory.md) (#887),
|
|
223
|
+
post `FORGE:DESIGN_SHIPPED`, set `design:shipped`, close the issue. Otherwise loop (back to Stage 4) or, if the budget is
|
|
224
|
+
exhausted with residual findings, set `design:rejected` with the reasons on the issue.
|
|
225
|
+
|
|
226
|
+
---
|
|
227
|
+
|
|
228
|
+
## Label state machine
|
|
229
|
+
|
|
230
|
+
```
|
|
231
|
+
design:investigating → design:architecting → design:generating → design:critiquing → design:awaiting-feedback → design:shipped
|
|
232
|
+
│ ↑
|
|
233
|
+
└──── (loop ≤ 3) ───────┘
|
|
234
|
+
(automated bypass: skip awaiting-feedback)
|
|
235
|
+
└──────────────────────────────────────────→ design:rejected
|
|
236
|
+
```
|
|
237
|
+
|
|
238
|
+
Terminal labels: `design:shipped`, `design:rejected`. These mirror `workflow:merged` / `workflow:invalid` on the code
|
|
239
|
+
track. `design:awaiting-feedback` is **non-terminal** — it resolves to `design:shipped` (via Stage 5) when the user
|
|
240
|
+
approves or the feedback budget is exhausted.
|
|
241
|
+
|
|
242
|
+
## Definition of done
|
|
243
|
+
|
|
244
|
+
A design ships only when it passes **all** of: the critique rubric threshold, the perf budget (#875), the a11y check,
|
|
245
|
+
and the divergence check (#887). Anything short loops or is rejected with reasons recorded on the issue — so every
|
|
246
|
+
outcome, pass or fail, is auditable on the GitHub issue.
|
|
247
|
+
|
|
248
|
+
## Why it matters
|
|
249
|
+
|
|
250
|
+
1. **Traceability** — every design's thinking (`FORGE:DESIGN_RATIONALE`), the candidate set and the choice
|
|
251
|
+
(`FORGE:DESIGN_CANDIDATES`), each critique iteration (`FORGE:CRITIQUE`), each user-feedback round
|
|
252
|
+
(`FORGE:USER_FEEDBACK`), and the final outcome (`FORGE:DESIGN_SHIPPED`) live on the issue. You can read *why* a
|
|
253
|
+
page looks the way it does — the designer's diary, made auditable, reviewable by a non-designer.
|
|
254
|
+
2. **It is arm A.** Driving the seed briefs (Cadence / Tender / Slipstream / Voltage / Plume) through `/design`
|
|
255
|
+
produces the arm-A outputs the benchmark (#878) scores against the real references — so building this makes the ABC
|
|
256
|
+
benchmark runnable end to end.
|
|
257
|
+
3. **Brand fit.** The automated critique loop (#882) closes the perceptual-quality loop. Stage 4.5 closes the brand-fit
|
|
258
|
+
loop — the part only the user can judge. Brand assets are user-supplied, emotional register is subjective, and the
|
|
259
|
+
best heroes are refined through iteration. The feedback annotations make that iteration traceable and repeatable.
|