forgedock 1.0.17 → 1.0.18
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +34 -1
- package/bin/cross-publish.mjs +344 -0
- package/bin/forgedock.mjs +229 -17
- package/commands/autopilot.md +77 -0
- package/commands/design-bench.md +176 -0
- package/commands/design-render-critique-loop.md +103 -0
- package/commands/design.md +259 -0
- package/commands/issue.md +7 -3
- package/commands/orchestrate.md +121 -0
- package/commands/qa-sweep.md +37 -2
- package/commands/quality-gate.md +30 -0
- package/commands/review-pr-staging.md +158 -2
- package/commands/test-gate.md +929 -0
- package/commands/work-on/build/implement.md +5 -3
- package/commands/work-on.md +53 -3
- package/docs/CI.md +177 -0
- package/docs/FORGE-PROTOCOL.md +324 -0
- package/docs/articles/b-vs-c-baseline.md +210 -0
- package/docs/articles/full-abc-benchmark.md +205 -0
- package/docs/articles/selective-spec-loading-token-savings.md +140 -0
- package/docs/design/abc-benchmark.md +195 -0
- package/docs/design/design-architect-rationale.md +123 -0
- package/docs/design/design-memory.md +61 -0
- package/docs/design/design-spec-schema.md +248 -0
- package/docs/design/design-system-lint.md +150 -0
- package/docs/design/divergent-generation.md +106 -0
- package/docs/design/effects-appropriateness.md +133 -0
- package/docs/design/fixtures/briefs/README.md +47 -0
- package/docs/design/fixtures/briefs/cadence.md +39 -0
- package/docs/design/fixtures/briefs/plume.md +39 -0
- package/docs/design/fixtures/briefs/slipstream.md +39 -0
- package/docs/design/fixtures/briefs/tender.md +39 -0
- package/docs/design/fixtures/briefs/voltage.md +39 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/cadence/arm-b/run-1.html +364 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/cadence/arm-b/run-2.html +420 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/cadence/arm-b/run-3.html +506 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/plume/arm-b/run-1.html +611 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/plume/arm-b/run-2.html +594 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/plume/arm-b/run-3.html +817 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/runs.json +175 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/scorecard.json +138 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/slipstream/arm-b/run-1.html +373 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/slipstream/arm-b/run-2.html +398 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/slipstream/arm-b/run-3.html +595 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/tender/arm-b/run-1.html +531 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/tender/arm-b/run-2.html +565 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/tender/arm-b/run-3.html +674 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/voltage/arm-b/run-1.html +388 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/voltage/arm-b/run-2.html +463 -0
- package/docs/design/fixtures/runs/b-vs-c-baseline/voltage/arm-b/run-3.html +568 -0
- package/docs/design/fixtures/runs/full-abc/cadence/arm-a/run-1.html +562 -0
- package/docs/design/fixtures/runs/full-abc/cadence/arm-a/run-2.html +618 -0
- package/docs/design/fixtures/runs/full-abc/cadence/arm-a/run-3.html +618 -0
- package/docs/design/fixtures/runs/full-abc/cadence/arm-b/run-1.html +364 -0
- package/docs/design/fixtures/runs/full-abc/cadence/arm-b/run-2.html +420 -0
- package/docs/design/fixtures/runs/full-abc/cadence/arm-b/run-3.html +506 -0
- package/docs/design/fixtures/runs/full-abc/plume/arm-a/run-1.html +527 -0
- package/docs/design/fixtures/runs/full-abc/plume/arm-a/run-2.html +445 -0
- package/docs/design/fixtures/runs/full-abc/plume/arm-a/run-3.html +483 -0
- package/docs/design/fixtures/runs/full-abc/plume/arm-b/run-1.html +611 -0
- package/docs/design/fixtures/runs/full-abc/plume/arm-b/run-2.html +594 -0
- package/docs/design/fixtures/runs/full-abc/plume/arm-b/run-3.html +817 -0
- package/docs/design/fixtures/runs/full-abc/runs.json +190 -0
- package/docs/design/fixtures/runs/full-abc/scorecard.json +144 -0
- package/docs/design/fixtures/runs/full-abc/slipstream/arm-a/run-1.html +496 -0
- package/docs/design/fixtures/runs/full-abc/slipstream/arm-a/run-2.html +447 -0
- package/docs/design/fixtures/runs/full-abc/slipstream/arm-a/run-3.html +564 -0
- package/docs/design/fixtures/runs/full-abc/slipstream/arm-b/run-1.html +373 -0
- package/docs/design/fixtures/runs/full-abc/slipstream/arm-b/run-2.html +398 -0
- package/docs/design/fixtures/runs/full-abc/slipstream/arm-b/run-3.html +595 -0
- package/docs/design/fixtures/runs/full-abc/tender/arm-a/run-1.html +622 -0
- package/docs/design/fixtures/runs/full-abc/tender/arm-a/run-2.html +475 -0
- package/docs/design/fixtures/runs/full-abc/tender/arm-a/run-3.html +660 -0
- package/docs/design/fixtures/runs/full-abc/tender/arm-b/run-1.html +531 -0
- package/docs/design/fixtures/runs/full-abc/tender/arm-b/run-2.html +565 -0
- package/docs/design/fixtures/runs/full-abc/tender/arm-b/run-3.html +674 -0
- package/docs/design/fixtures/runs/full-abc/voltage/arm-a/run-1.html +525 -0
- package/docs/design/fixtures/runs/full-abc/voltage/arm-a/run-2.html +446 -0
- package/docs/design/fixtures/runs/full-abc/voltage/arm-a/run-3.html +463 -0
- package/docs/design/fixtures/runs/full-abc/voltage/arm-b/run-1.html +388 -0
- package/docs/design/fixtures/runs/full-abc/voltage/arm-b/run-2.html +463 -0
- package/docs/design/fixtures/runs/full-abc/voltage/arm-b/run-3.html +568 -0
- package/docs/design/fixtures/runs/full-abc-opus/cadence/arm-a/run-1.html +1068 -0
- package/docs/design/fixtures/runs/full-abc-opus/cadence/arm-a/run-2.html +1423 -0
- package/docs/design/fixtures/runs/full-abc-opus/cadence/arm-a/run-3.html +1245 -0
- package/docs/design/fixtures/runs/full-abc-opus/cadence/arm-b/run-1.html +305 -0
- package/docs/design/fixtures/runs/full-abc-opus/cadence/arm-b/run-2.html +327 -0
- package/docs/design/fixtures/runs/full-abc-opus/cadence/arm-b/run-3.html +428 -0
- package/docs/design/fixtures/runs/full-abc-opus/plume/arm-a/run-1.html +1270 -0
- package/docs/design/fixtures/runs/full-abc-opus/plume/arm-a/run-2.html +1172 -0
- package/docs/design/fixtures/runs/full-abc-opus/plume/arm-a/run-3.html +1532 -0
- package/docs/design/fixtures/runs/full-abc-opus/plume/arm-b/run-1.html +315 -0
- package/docs/design/fixtures/runs/full-abc-opus/plume/arm-b/run-2.html +342 -0
- package/docs/design/fixtures/runs/full-abc-opus/plume/arm-b/run-3.html +353 -0
- package/docs/design/fixtures/runs/full-abc-opus/runs.json +190 -0
- package/docs/design/fixtures/runs/full-abc-opus/scorecard.json +144 -0
- package/docs/design/fixtures/runs/full-abc-opus/slipstream/arm-a/run-1.html +345 -0
- package/docs/design/fixtures/runs/full-abc-opus/slipstream/arm-a/run-2.html +396 -0
- package/docs/design/fixtures/runs/full-abc-opus/slipstream/arm-a/run-3.html +502 -0
- package/docs/design/fixtures/runs/full-abc-opus/slipstream/arm-b/run-1.html +408 -0
- package/docs/design/fixtures/runs/full-abc-opus/slipstream/arm-b/run-2.html +446 -0
- package/docs/design/fixtures/runs/full-abc-opus/slipstream/arm-b/run-3.html +496 -0
- package/docs/design/fixtures/runs/full-abc-opus/tender/arm-a/run-1.html +813 -0
- package/docs/design/fixtures/runs/full-abc-opus/tender/arm-a/run-2.html +711 -0
- package/docs/design/fixtures/runs/full-abc-opus/tender/arm-a/run-3.html +721 -0
- package/docs/design/fixtures/runs/full-abc-opus/tender/arm-b/run-1.html +280 -0
- package/docs/design/fixtures/runs/full-abc-opus/tender/arm-b/run-2.html +291 -0
- package/docs/design/fixtures/runs/full-abc-opus/tender/arm-b/run-3.html +366 -0
- package/docs/design/fixtures/runs/full-abc-opus/voltage/arm-a/run-1.html +1010 -0
- package/docs/design/fixtures/runs/full-abc-opus/voltage/arm-a/run-2.html +836 -0
- package/docs/design/fixtures/runs/full-abc-opus/voltage/arm-a/run-3.html +930 -0
- package/docs/design/fixtures/runs/full-abc-opus/voltage/arm-b/run-1.html +407 -0
- package/docs/design/fixtures/runs/full-abc-opus/voltage/arm-b/run-2.html +396 -0
- package/docs/design/fixtures/runs/full-abc-opus/voltage/arm-b/run-3.html +529 -0
- package/docs/design/foundations/brutalist.css +294 -0
- package/docs/design/foundations/editorial.css +300 -0
- package/docs/design/foundations/minimal-luxury.css +295 -0
- package/docs/design/foundations/technical.css +293 -0
- package/docs/design/foundations/warm-photo.css +299 -0
- package/docs/design/reference-corpus.md +1726 -0
- package/docs/design/render-critique-loop.md +57 -0
- package/docs/spec-graph-schema.md +397 -0
- package/package.json +1 -1
- package/scripts/bench-scorecard.mjs +239 -0
- package/scripts/build-spec-graph.mjs +509 -0
- package/scripts/classify-lane.sh +0 -7
- package/scripts/design-system-lint.mjs +993 -0
- package/scripts/graph-query.sh +486 -0
- package/scripts/transition-label.sh +10 -2
- package/scripts/validate-spec-graph.sh +423 -0
- package/templates/workflows/forgedock-review.yml +96 -0
|
@@ -243,9 +243,11 @@ gh issue comment {NUMBER} {GH_FLAG} --body "<!-- FORGE:BUILDER -->
|
|
|
243
243
|
{Checklist of each criterion from the contract, marked ✅ or ❌}
|
|
244
244
|
|
|
245
245
|
### Testing Checklist
|
|
246
|
-
- [ ] {Test scenario 1}
|
|
247
|
-
- [ ] {Test scenario 2}
|
|
248
|
-
- [ ] {Test scenario 3}
|
|
246
|
+
- [ ] {Test scenario 1} [type:api]
|
|
247
|
+
- [ ] {Test scenario 2} [type:unit]
|
|
248
|
+
- [ ] {Test scenario 3} [type:e2e]
|
|
249
|
+
|
|
250
|
+
> **Test-type annotation** (optional): Append `[type:api]`, `[type:unit]`, `[type:e2e]`, or `[type:manual]` to each checklist item. The test gate reads this annotation directly and skips regex inference. Omit it to rely on regex classification fallback.
|
|
249
251
|
|
|
250
252
|
<!-- FORGE:BUILDER:COMPLETE -->"
|
|
251
253
|
```
|
package/commands/work-on.md
CHANGED
|
@@ -162,8 +162,17 @@ fi
|
|
|
162
162
|
**Script resolution** — Use the following `resolve_script()` function whenever calling a pipeline script. It enforces the 4-level precedence hierarchy (see `devdocs/project/architecture.md → Script Precedence`):
|
|
163
163
|
|
|
164
164
|
```bash
|
|
165
|
-
|
|
165
|
+
ADAPTIVE_DIR_RAW="${REPO_PATH}/$(yq '.adaptive_scripts.directory // ".forgedock/scripts"' forge.yaml 2>/dev/null || echo '.forgedock/scripts')"
|
|
166
|
+
ADAPTIVE_DIR=$(realpath -m "$ADAPTIVE_DIR_RAW" 2>/dev/null || echo "$ADAPTIVE_DIR_RAW")
|
|
166
167
|
ADAPTIVE_ENABLED=$(yq '.adaptive_scripts.enabled // "true"' forge.yaml 2>/dev/null || echo 'true')
|
|
168
|
+
# Bounds check: reject adaptive_scripts.directory values that escape the repo root.
|
|
169
|
+
# Normalize REPO_PATH the same way ADAPTIVE_DIR is normalized (realpath -m) so a trailing
|
|
170
|
+
# slash in paths.root does not inject a '//' into the glob and trigger a false positive.
|
|
171
|
+
REPO_PATH_NORM=$(realpath -m "$REPO_PATH" 2>/dev/null || echo "$REPO_PATH")
|
|
172
|
+
if [[ "$ADAPTIVE_DIR" != "${REPO_PATH_NORM}/"* ]]; then
|
|
173
|
+
echo "WARNING: adaptive_scripts.directory resolves outside repo root ('$ADAPTIVE_DIR') — adaptive tier disabled" >&2
|
|
174
|
+
ADAPTIVE_ENABLED=false
|
|
175
|
+
fi
|
|
167
176
|
UNIVERSAL_DIR="${FORGEDOCK_HOME:-$(dirname "$(which classify-lane.sh 2>/dev/null || echo 'scripts')")}/scripts"
|
|
168
177
|
|
|
169
178
|
resolve_script() {
|
|
@@ -254,6 +263,45 @@ LEARNED_COMMIT_STYLE=$(yq '.learned.commit_style // ""' forge.yaml 2>/dev/null |
|
|
|
254
263
|
### 0C: Sync to Project board
|
|
255
264
|
Add issue to project, set Status=In Progress, Lane, Component, Priority, Workflow=Investigating.
|
|
256
265
|
|
|
266
|
+
### 0C.5: Resolve minimal spec set (selective spec loading)
|
|
267
|
+
|
|
268
|
+
Rather than loading the full ~27-command corpus (~1.1 MB / ~276K tokens) into
|
|
269
|
+
context, resolve the **minimal spec set** this run actually needs from the spec
|
|
270
|
+
knowledge graph. Use the universal-tier `graph-query` script via the canonical
|
|
271
|
+
`resolve_script` tier-dispatch pattern:
|
|
272
|
+
|
|
273
|
+
```bash
|
|
274
|
+
# Forward-transitive reachability: work-on + its reachable sub-phases (CONTAINS)
|
|
275
|
+
# + required devdocs (REQUIRES), as repo-relative file paths.
|
|
276
|
+
RESOLUTION=$(resolve_script 'graph-query')
|
|
277
|
+
TIER="${RESOLUTION%%:*}"
|
|
278
|
+
SCRIPT_PATH="${RESOLUTION#*:}"
|
|
279
|
+
case "$TIER" in
|
|
280
|
+
adaptive|universal)
|
|
281
|
+
SPEC_SET=$(bash "$SCRIPT_PATH" load-set work-on 2>/dev/null || echo '[]')
|
|
282
|
+
echo "$SPEC_SET" | jq -r '.[]'
|
|
283
|
+
;;
|
|
284
|
+
prose)
|
|
285
|
+
# Prose fallback: graph-query.sh unavailable — read specs on demand as each
|
|
286
|
+
# Skill(...) is invoked. Selective loading is an optimization, not required.
|
|
287
|
+
: ;;
|
|
288
|
+
esac
|
|
289
|
+
```
|
|
290
|
+
|
|
291
|
+
**Read ONLY the files returned by `load-set work-on`** when you need full spec
|
|
292
|
+
text during this run — these are the work-on orchestrator plus the sub-phases
|
|
293
|
+
and devdocs reachable from it (e.g. `commands/work-on/build/*`, `commands/work-on/review.md`,
|
|
294
|
+
`commands/review-pr.md`). Do **not** broadly read unrelated command specs
|
|
295
|
+
(`pipeline-health.md`, `audit.md`, `geo-audit.md`, …) that are not in the set.
|
|
296
|
+
Sub-phases are still invoked normally via their existing `Skill(...)` calls; this
|
|
297
|
+
step only narrows what is *pre-read* into context, it does not remove any phase.
|
|
298
|
+
|
|
299
|
+
This is the inverse of `graph-query.sh impact` (forward instead of reverse
|
|
300
|
+
reachability). It is read-only and auto-builds the graph if the gitignored JSON
|
|
301
|
+
is absent — no committed graph is required. The prose tier above handles older
|
|
302
|
+
installs without the scripts layer: selective loading is an optimization, never
|
|
303
|
+
a hard dependency.
|
|
304
|
+
|
|
257
305
|
---
|
|
258
306
|
|
|
259
307
|
## Phase 1: Investigation
|
|
@@ -1325,8 +1373,10 @@ gh issue comment {NUMBER} {GH_FLAG} --body "<!-- FORGE:BUILDER -->
|
|
|
1325
1373
|
{checklist from contract, marked pass/fail}
|
|
1326
1374
|
|
|
1327
1375
|
### Testing Checklist
|
|
1328
|
-
- [ ] {scenario 1}
|
|
1329
|
-
- [ ] {scenario 2}
|
|
1376
|
+
- [ ] {scenario 1} [type:api]
|
|
1377
|
+
- [ ] {scenario 2} [type:unit]
|
|
1378
|
+
|
|
1379
|
+
> **Test-type annotation** (optional): Append `[type:api]`, `[type:unit]`, `[type:e2e]`, or `[type:manual]` to each checklist item. The test gate reads this annotation directly and skips regex inference. Omit it to rely on regex classification fallback.
|
|
1330
1380
|
|
|
1331
1381
|
<!-- FORGE:BUILDER:COMPLETE -->"
|
|
1332
1382
|
```
|
package/docs/CI.md
ADDED
|
@@ -0,0 +1,177 @@
|
|
|
1
|
+
# CI Integration — ForgeDock PR Review in GitHub Actions
|
|
2
|
+
|
|
3
|
+
This guide explains how to integrate ForgeDock's `/review-pr` pipeline into GitHub Actions so every PR gets automatic multi-domain AI review without requiring a manual Claude Code session.
|
|
4
|
+
|
|
5
|
+
<!-- publish trigger: bot-token merges do not fire path-filtered workflows -->
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## What This Does
|
|
10
|
+
|
|
11
|
+
The `forgedock-review.yml` workflow template triggers on every pull request (opened, synchronized, or reopened) and runs ForgeDock's full review pipeline:
|
|
12
|
+
|
|
13
|
+
- Security, auth, billing, database, concurrency, frontend, API, performance, and infrastructure agents all review the PR diff
|
|
14
|
+
- Review findings are posted as comments on the PR
|
|
15
|
+
- Each finding becomes a tracked GitHub issue in your repository
|
|
16
|
+
- No human needs to remember to run `/review-pr` — it happens automatically
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## Prerequisites
|
|
21
|
+
|
|
22
|
+
1. **Anthropic API key** — Claude Code runs against the Anthropic API. Get one at [console.anthropic.com](https://console.anthropic.com).
|
|
23
|
+
2. **ForgeDock installed** — The workflow installs ForgeDock automatically via `npx forgedock install`.
|
|
24
|
+
3. **GitHub Actions enabled** — Standard for all public repos; enabled by default for private repos.
|
|
25
|
+
|
|
26
|
+
---
|
|
27
|
+
|
|
28
|
+
## Setup
|
|
29
|
+
|
|
30
|
+
### 1. Add the API key as a repository secret
|
|
31
|
+
|
|
32
|
+
In your repository: **Settings → Secrets and variables → Actions → New repository secret**
|
|
33
|
+
|
|
34
|
+
| Name | Value |
|
|
35
|
+
|------|-------|
|
|
36
|
+
| `ANTHROPIC_API_KEY` | Your Anthropic API key (`sk-ant-...`) |
|
|
37
|
+
|
|
38
|
+
Never commit the API key to your repository. GitHub Actions masks secrets in logs, but only secrets stored via Settings are protected.
|
|
39
|
+
|
|
40
|
+
### 2. Copy the workflow template
|
|
41
|
+
|
|
42
|
+
Copy `templates/workflows/forgedock-review.yml` from your ForgeDock installation to your repository:
|
|
43
|
+
|
|
44
|
+
```bash
|
|
45
|
+
# From your repository root
|
|
46
|
+
mkdir -p .github/workflows
|
|
47
|
+
cp "$(npx forgedock which-dir)/templates/workflows/forgedock-review.yml" .github/workflows/forgedock-review.yml
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
Or copy it manually — the file is in `~/.claude/commands/../templates/workflows/forgedock-review.yml` after running `npx forgedock install`.
|
|
51
|
+
|
|
52
|
+
### 3. Commit and push
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
git add .github/workflows/forgedock-review.yml
|
|
56
|
+
git commit -m "feat(ci): add ForgeDock automated PR review workflow"
|
|
57
|
+
git push
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
The workflow activates immediately. Open or synchronize any PR to trigger the first review.
|
|
61
|
+
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
## Security Model
|
|
65
|
+
|
|
66
|
+
### API key handling
|
|
67
|
+
|
|
68
|
+
The `ANTHROPIC_API_KEY` is passed exclusively via the `env:` block in the workflow YAML. It is never:
|
|
69
|
+
- Echoed to logs with `echo` or `printf`
|
|
70
|
+
- Passed as a CLI argument (visible in process lists)
|
|
71
|
+
- Written to disk during the workflow run
|
|
72
|
+
|
|
73
|
+
GitHub Actions masks the secret value in all log output automatically.
|
|
74
|
+
|
|
75
|
+
### GITHUB_TOKEN scope
|
|
76
|
+
|
|
77
|
+
The workflow uses the built-in `GITHUB_TOKEN`, which is:
|
|
78
|
+
- Automatically provided by GitHub Actions for every workflow run
|
|
79
|
+
- Scoped to the repository
|
|
80
|
+
- Revoked after the workflow run completes
|
|
81
|
+
- Constrained to `contents: read` and `pull-requests: write` by the `permissions:` block
|
|
82
|
+
|
|
83
|
+
No additional tokens or personal access tokens are required.
|
|
84
|
+
|
|
85
|
+
### Runner isolation
|
|
86
|
+
|
|
87
|
+
Each workflow run uses a fresh ephemeral GitHub-hosted runner (`ubuntu-latest`). The `--dangerously-skip-permissions` flag passed to Claude Code is safe in this context because:
|
|
88
|
+
- The runner is discarded after the job completes
|
|
89
|
+
- No persistent state is left behind
|
|
90
|
+
- The workflow's `permissions:` block constrains what the `GITHUB_TOKEN` can do
|
|
91
|
+
|
|
92
|
+
---
|
|
93
|
+
|
|
94
|
+
## Fork PR Limitation
|
|
95
|
+
|
|
96
|
+
**Pull requests from forks do not have access to repository secrets.**
|
|
97
|
+
|
|
98
|
+
This is a GitHub security restriction: a forked repository's PR cannot read secrets from the upstream repository's Settings. The `pull_request` event that triggers this workflow will have `ANTHROPIC_API_KEY` as an empty string for fork PRs, and the Claude Code step will fail silently or error.
|
|
99
|
+
|
|
100
|
+
### Option 1: Accept the limitation (recommended for open-source repos)
|
|
101
|
+
|
|
102
|
+
Leave the workflow as-is. Team members' PRs get reviewed automatically. External contributor PRs are reviewed manually when a maintainer runs `/review-pr` in Claude Code.
|
|
103
|
+
|
|
104
|
+
### Option 2: Use `pull_request_target` (advanced — security implications)
|
|
105
|
+
|
|
106
|
+
`pull_request_target` runs in the context of the base repository (not the fork), so secrets are available. However, it also runs untrusted code from the fork in a privileged context, which is a security risk if the workflow checks out and runs code from the fork.
|
|
107
|
+
|
|
108
|
+
If you use `pull_request_target`, you MUST:
|
|
109
|
+
1. Never check out the fork's code and run it
|
|
110
|
+
2. Scope what Claude Code can access carefully
|
|
111
|
+
3. Review the [GitHub security hardening guide for `pull_request_target`](https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)
|
|
112
|
+
|
|
113
|
+
This is not recommended without understanding the security implications.
|
|
114
|
+
|
|
115
|
+
---
|
|
116
|
+
|
|
117
|
+
## Customization
|
|
118
|
+
|
|
119
|
+
### Skip review for certain PRs
|
|
120
|
+
|
|
121
|
+
Add conditions to the `if:` field on the job:
|
|
122
|
+
|
|
123
|
+
```yaml
|
|
124
|
+
# Skip bot PRs and draft PRs
|
|
125
|
+
if: |
|
|
126
|
+
github.actor != 'dependabot[bot]' &&
|
|
127
|
+
github.actor != 'github-actions[bot]' &&
|
|
128
|
+
github.event.pull_request.draft == false
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
### Target only certain branches
|
|
132
|
+
|
|
133
|
+
```yaml
|
|
134
|
+
on:
|
|
135
|
+
pull_request:
|
|
136
|
+
types: [opened, synchronize, reopened]
|
|
137
|
+
branches:
|
|
138
|
+
- main
|
|
139
|
+
- staging
|
|
140
|
+
- 'milestone/**'
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
### Pass additional flags to /review-pr
|
|
144
|
+
|
|
145
|
+
The `claude --print` invocation accepts `/review-pr` arguments. For example, to skip auto-merge (default) or pass a custom flag:
|
|
146
|
+
|
|
147
|
+
```yaml
|
|
148
|
+
run: |
|
|
149
|
+
claude --print "/review-pr $PR_NUMBER --model opus" \
|
|
150
|
+
--dangerously-skip-permissions
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
### Enable commit status badge
|
|
154
|
+
|
|
155
|
+
Uncomment the `Set Commit Status` step in the workflow template to add a pass/fail badge to PRs. This uses the GitHub API via `actions/github-script` to post a commit status after the review completes.
|
|
156
|
+
|
|
157
|
+
---
|
|
158
|
+
|
|
159
|
+
## Troubleshooting
|
|
160
|
+
|
|
161
|
+
**Review job fails immediately with `ANTHROPIC_API_KEY not set`**
|
|
162
|
+
→ The secret is not configured. Go to Settings → Secrets and variables → Actions and add `ANTHROPIC_API_KEY`.
|
|
163
|
+
|
|
164
|
+
**Review passes but no comments appear on the PR**
|
|
165
|
+
→ Check that `permissions: pull-requests: write` is present in the workflow. Without it, the `gh` CLI calls from `/review-pr` will receive 403 errors.
|
|
166
|
+
|
|
167
|
+
**Fork PRs show empty `ANTHROPIC_API_KEY`**
|
|
168
|
+
→ Expected behavior. See the Fork PR Limitation section above.
|
|
169
|
+
|
|
170
|
+
**`claude: command not found`**
|
|
171
|
+
→ The `npm install -g @anthropic-ai/claude-code` step failed. Check the job logs. The `ubuntu-latest` runner includes Node.js but the global npm install may fail if the runner's Node version is too old. Ensure the `Set up Node.js` step runs first and targets Node.js 18+.
|
|
172
|
+
|
|
173
|
+
**`npx forgedock install` reports `command not found: forgedock`**
|
|
174
|
+
→ `npx forgedock install` runs the installer from npm. If it fails, verify the ForgeDock package name is correct: `npm show forgedock version` should return a version number.
|
|
175
|
+
|
|
176
|
+
**`claude: unknown flag --output-format` or similar unrecognized flag error**
|
|
177
|
+
→ Your local Claude Code installation is outdated. The `--output-format` flag is available in all recent versions of Claude Code. Run `npm update -g @anthropic-ai/claude-code` to update to the latest version. Note: the workflow template itself no longer passes `--output-format` (the flag was redundant — `--print` defaults to text output). If you copied the template before this fix, remove the `--output-format text` line from your workflow file.
|
package/docs/FORGE-PROTOCOL.md
CHANGED
|
@@ -398,6 +398,330 @@ gh api repos/{OWNER}/{REPO}/issues/{NUMBER}/comments \
|
|
|
398
398
|
|
|
399
399
|
---
|
|
400
400
|
|
|
401
|
+
### Design Pipeline Annotations
|
|
402
|
+
|
|
403
|
+
These annotations drive the UI Taste Harness — the design-generation pipeline that produces a landing page from a design-blind product brief. They carry design intent across the architect → generate → critique → close stages, exactly as the issue-pipeline annotations carry code-context across investigate → build → review.
|
|
404
|
+
|
|
405
|
+
---
|
|
406
|
+
|
|
407
|
+
#### `FORGE:DESIGN_CONTEXT`
|
|
408
|
+
|
|
409
|
+
**Phase**: Design — Investigate (`/design` stage 1, #888) — the first `FORGE:DESIGN_*` annotation in the chain
|
|
410
|
+
**Written by**: `/design` design-investigate stage
|
|
411
|
+
**Read by**: Design-architect stage (grounds the rationale), Close phase (audit trail)
|
|
412
|
+
**Location**: Issue comment
|
|
413
|
+
|
|
414
|
+
The design analog of `FORGE:CONTEXT`: the parsed brief (message / audience / single objection), the grammar pulled
|
|
415
|
+
from the [reference corpus](design/reference-corpus.md) (#880), and the recent signature moves / archetypes / palettes
|
|
416
|
+
pulled from [design-memory](design/design-memory.md) (#887) that this design must **diverge** from. Opens the
|
|
417
|
+
`FORGE:DESIGN_*` chain that `FORGE:DESIGN_RATIONALE` → `FORGE:DESIGN_CANDIDATES` → `FORGE:DESIGN_SPEC` → `FORGE:CRITIQUE`
|
|
418
|
+
→ `FORGE:DESIGN_SHIPPED` continues. See [`commands/design.md`](../commands/design.md).
|
|
419
|
+
|
|
420
|
+
**Schema**:
|
|
421
|
+
|
|
422
|
+
```markdown
|
|
423
|
+
<!-- FORGE:DESIGN_CONTEXT -->
|
|
424
|
+
## Design Context — {product}
|
|
425
|
+
|
|
426
|
+
**Message:** {one thing the page must say}
|
|
427
|
+
**Audience / objection:** {who} — must overcome: {objection}
|
|
428
|
+
**Corpus grammar:** {relevant traits / archetype priors from #880}
|
|
429
|
+
**Diverge from (memory):** {recent signature moves / archetypes / palettes to avoid — #887}
|
|
430
|
+
<!-- FORGE:DESIGN_CONTEXT:COMPLETE -->
|
|
431
|
+
```
|
|
432
|
+
|
|
433
|
+
**Detection query**:
|
|
434
|
+
```bash
|
|
435
|
+
gh api repos/{OWNER}/{REPO}/issues/{NUMBER}/comments \
|
|
436
|
+
--jq '.[] | select(.body | contains("FORGE:DESIGN_CONTEXT")) | .body'
|
|
437
|
+
```
|
|
438
|
+
|
|
439
|
+
---
|
|
440
|
+
|
|
441
|
+
#### `FORGE:DESIGN_RATIONALE`
|
|
442
|
+
|
|
443
|
+
**Phase**: Design — Architecture (design-architect, #886) — emitted *before* `FORGE:DESIGN_SPEC`
|
|
444
|
+
**Written by**: Design-architect agent
|
|
445
|
+
**Read by**: Generate agent (the spec it produces), render → vision-critique loop (#882), and a non-designer reviewer (the reasoning is reviewable without judging pixels)
|
|
446
|
+
**Location**: Issue comment
|
|
447
|
+
|
|
448
|
+
The designer's diary — the reasoning-before-generation step. Captures the seven-element chain of thought (intent/feeling, audience/objection, communication hierarchy, direction + rejected alternatives, signature move, what's being tried this time, non-goals) and **produces** the `FORGE:DESIGN_SPEC` rather than letting the spec appear from nowhere. The format is the structured contract: it MUST carry at least one explicitly **rejected alternative**, a named **signature move**, and a **produces-DESIGN_SPEC link**. See [`docs/design/design-architect-rationale.md`](design/design-architect-rationale.md) for the full doctrine.
|
|
449
|
+
|
|
450
|
+
**Schema**:
|
|
451
|
+
|
|
452
|
+
```markdown
|
|
453
|
+
<!-- FORGE:DESIGN_RATIONALE -->
|
|
454
|
+
## Design Rationale — {product}
|
|
455
|
+
|
|
456
|
+
**Intent / feeling:** {one message} · {one feeling}
|
|
457
|
+
**Audience / objection:** {who} — must overcome: {objection}
|
|
458
|
+
**Communication hierarchy:** 1) {…} 2) {…} 3) {…}
|
|
459
|
+
**Direction:** {archetype} — because {reasoning}
|
|
460
|
+
- Considered & rejected: {alt A} (because {…}); {alt B} (because {…})
|
|
461
|
+
**Signature move:** {the one non-obvious idea}
|
|
462
|
+
**Trying this time:** {technique/learning} (from memory: avoiding {prior move})
|
|
463
|
+
**Non-goals:** {what this won't do}
|
|
464
|
+
|
|
465
|
+
→ Produces DESIGN_SPEC: {link}
|
|
466
|
+
```
|
|
467
|
+
|
|
468
|
+
**Detection query**:
|
|
469
|
+
```bash
|
|
470
|
+
gh api repos/{OWNER}/{REPO}/issues/{NUMBER}/comments \
|
|
471
|
+
--jq '.[] | select(.body | contains("FORGE:DESIGN_RATIONALE")) | .body'
|
|
472
|
+
```
|
|
473
|
+
|
|
474
|
+
---
|
|
475
|
+
|
|
476
|
+
#### `FORGE:DESIGN_CANDIDATES`
|
|
477
|
+
|
|
478
|
+
**Phase**: Design — Divergent generation (#883) — emitted *after* `FORGE:DESIGN_RATIONALE`, *before* `FORGE:DESIGN_SPEC`
|
|
479
|
+
**Written by**: Generate agent (divergent-generation step)
|
|
480
|
+
**Read by**: Taste-judge (selects the winner), Close phase (audit trail), design-memory (#887, to diverge from past winners)
|
|
481
|
+
**Location**: Issue comment
|
|
482
|
+
|
|
483
|
+
The variance lever's audit record: the one **committed archetype** (one of the five corpus ids — #880, never blended),
|
|
484
|
+
the **N distinct directions** generated within it, and the **independent taste-judge's** scores + selection. The
|
|
485
|
+
winner hands off into `FORGE:DESIGN_SPEC`. The selection judge is deliberately distinct from the critique loop (#882)
|
|
486
|
+
and the benchmark judge (#878) — anti-Goodhart. See [`docs/design/divergent-generation.md`](design/divergent-generation.md).
|
|
487
|
+
|
|
488
|
+
**Schema**:
|
|
489
|
+
|
|
490
|
+
```markdown
|
|
491
|
+
<!-- FORGE:DESIGN_CANDIDATES -->
|
|
492
|
+
## Design Candidates — {product}
|
|
493
|
+
|
|
494
|
+
**Archetype (committed):** {one of the 5 corpus ids}
|
|
495
|
+
**Directions:**
|
|
496
|
+
1. {concept} — signature: {move} — {distinguishing grammar/tokens}
|
|
497
|
+
2. {concept} — signature: {move} — {…}
|
|
498
|
+
**Judge scores:** 1) {score} 2) {score} …
|
|
499
|
+
**Selected:** #{n} — because {reason}
|
|
500
|
+
|
|
501
|
+
→ Winner produces DESIGN_SPEC: {link}
|
|
502
|
+
```
|
|
503
|
+
|
|
504
|
+
**Detection query**:
|
|
505
|
+
```bash
|
|
506
|
+
gh api repos/{OWNER}/{REPO}/issues/{NUMBER}/comments \
|
|
507
|
+
--jq '.[] | select(.body | contains("FORGE:DESIGN_CANDIDATES")) | .body'
|
|
508
|
+
```
|
|
509
|
+
|
|
510
|
+
---
|
|
511
|
+
|
|
512
|
+
#### `FORGE:DESIGN_SPEC`
|
|
513
|
+
|
|
514
|
+
**Phase**: Design — Architecture (design-architect, #886)
|
|
515
|
+
**Written by**: Design-architect agent
|
|
516
|
+
**Read by**: Generate agent (constrains output), deterministic anti-slop linter (#884), render → vision-critique loop (#882), Close phase agent (persists realized spec to design-memory, #887)
|
|
517
|
+
|
|
518
|
+
A structured, **machine-checkable** representation of one page's design language. Carried across design-pipeline stages as a `FORGE:DESIGN_SPEC` annotation so taste decisions **persist** instead of being re-rolled per generation, and so a deterministic linter and a vision critic both check the rendered output against the *same* committed intent.
|
|
519
|
+
|
|
520
|
+
Critically, the spec is **produced by** a `FORGE:DESIGN_RATIONALE` (the reasoning) — not authored from nowhere. The flow is rationale → spec → page.
|
|
521
|
+
|
|
522
|
+
The full field-by-field schema (JSON shape, the per-field slop-tell defense table, and the lifecycle) lives in [`design/design-spec-schema.md`](design/design-spec-schema.md). The annotation body embeds that schema as a fenced `jsonc` block.
|
|
523
|
+
|
|
524
|
+
**Schema** (annotation envelope):
|
|
525
|
+
|
|
526
|
+
````
|
|
527
|
+
<!-- FORGE:DESIGN_SPEC -->
|
|
528
|
+
## Design Spec — {product}
|
|
529
|
+
|
|
530
|
+
```jsonc
|
|
531
|
+
{
|
|
532
|
+
"meta": { "product": "…", "archetype": "…", "corpus_version": "…", "rationale_ref": "…" },
|
|
533
|
+
"typography": { "display_family": "…", "body_family": "…", "scale_ratio": …, "weights": […] },
|
|
534
|
+
"color": { "mode": "…", "background": "#…", "foreground": "#…", "accent": "#…",
|
|
535
|
+
"rules": ["no-default-tailwind-palette", "contrast>=4.5"] },
|
|
536
|
+
"spacing": { "base_unit_px": …, "scale": […] },
|
|
537
|
+
"radius": { "scale": […] },
|
|
538
|
+
"shadow": { "tokens": […] },
|
|
539
|
+
"motion": { "vocabulary": […], "reduced_motion": "required" },
|
|
540
|
+
"layout_grammar":{ "sections": [ { "id": "…", "purpose": "…", "density": "…" } ], "rhythm": "…" },
|
|
541
|
+
"effects_plan": { "per_section": [ … ], "budget": { … }, "never": [ … ] }, // doctrine: docs/design/effects-appropriateness.md (#885)
|
|
542
|
+
"negatives": [ … ],
|
|
543
|
+
"acceptance": { "perf_budget": { … }, "a11y": { … }, "divergence_ref": "…" }
|
|
544
|
+
}
|
|
545
|
+
```
|
|
546
|
+
|
|
547
|
+
→ Produced by FORGE:DESIGN_RATIONALE: {link}
|
|
548
|
+
````
|
|
549
|
+
|
|
550
|
+
**Field reference**: see [`design/design-spec-schema.md`](design/design-spec-schema.md) for the complete schema, the "How each field defends against slop" table, and the lifecycle.
|
|
551
|
+
|
|
552
|
+
**Detection query**:
|
|
553
|
+
```bash
|
|
554
|
+
gh api repos/{OWNER}/{REPO}/issues/{NUMBER}/comments \
|
|
555
|
+
--jq '.[] | select(.body | contains("FORGE:DESIGN_SPEC")) | .body'
|
|
556
|
+
```
|
|
557
|
+
|
|
558
|
+
---
|
|
559
|
+
|
|
560
|
+
#### `FORGE:CRITIQUE`
|
|
561
|
+
|
|
562
|
+
**Phase**: Design — Render → vision-critique loop (`/design-render-critique-loop`, #882) — after `FORGE:DESIGN_SPEC`
|
|
563
|
+
**Written by**: Vision-critique loop (one per iteration)
|
|
564
|
+
**Read by**: The iterate step (consumes its own findings), Close phase (improvement trajectory audit), `/design` (#888)
|
|
565
|
+
**Location**: Issue comment
|
|
566
|
+
|
|
567
|
+
One annotation per render→critique iteration — the page's improvement trajectory as an auditable trail. Records the
|
|
568
|
+
deterministic lint floor result (#884), the desktop+mobile render, the perceptual findings (N3, N8–N12 — the negatives
|
|
569
|
+
the linter declares out of scope), and the verdict (`PASS` | `ITERATE` | `BUDGET-EXHAUSTED`). The critic is strictly
|
|
570
|
+
independent from the ABC benchmark judge (#878) — anti-Goodhart. See
|
|
571
|
+
[`docs/design/render-critique-loop.md`](design/render-critique-loop.md).
|
|
572
|
+
|
|
573
|
+
**Schema**:
|
|
574
|
+
|
|
575
|
+
```markdown
|
|
576
|
+
<!-- FORGE:CRITIQUE -->
|
|
577
|
+
## Critique — {product} · iteration {i}/{max}
|
|
578
|
+
|
|
579
|
+
**Lint floor:** {PASS | fixed N hard findings}
|
|
580
|
+
**Render:** desktop + mobile captured
|
|
581
|
+
**Perceptual findings:**
|
|
582
|
+
- N{n}: {what was observed in the render} → {correction}
|
|
583
|
+
**Verdict:** {PASS | ITERATE | BUDGET-EXHAUSTED}
|
|
584
|
+
<!-- FORGE:CRITIQUE:COMPLETE -->
|
|
585
|
+
```
|
|
586
|
+
|
|
587
|
+
**Detection query**:
|
|
588
|
+
```bash
|
|
589
|
+
gh api repos/{OWNER}/{REPO}/issues/{NUMBER}/comments \
|
|
590
|
+
--jq '.[] | select(.body | contains("FORGE:CRITIQUE")) | .body'
|
|
591
|
+
```
|
|
592
|
+
|
|
593
|
+
---
|
|
594
|
+
|
|
595
|
+
#### `FORGE:USER_FEEDBACK`
|
|
596
|
+
|
|
597
|
+
<!-- Added: forge#1044 — user-feedback loop -->
|
|
598
|
+
|
|
599
|
+
**Phase**: Design — User-feedback loop (`/design` stage 4.5, #1044) — after `FORGE:CRITIQUE`, before `FORGE:DESIGN_SHIPPED`
|
|
600
|
+
**Written by**: `/design` user-feedback stage (Stage 4.5) — records structured user input after the automated critique loop
|
|
601
|
+
**Read by**: Surgical re-generation step (same stage 4.5, consumes `section_target` + `modification`), Close phase (audit trail of user-driven changes)
|
|
602
|
+
**Location**: Issue comment
|
|
603
|
+
|
|
604
|
+
The structured record of one user-feedback round in Stage 4.5 of the `/design` pipeline. Posted once per feedback
|
|
605
|
+
exchange — there may be multiple `FORGE:USER_FEEDBACK` annotations on a single issue if the user iterates several
|
|
606
|
+
times. Each annotation carries a `section_target` (the section ID from `layout_grammar.sections` to re-generate), the
|
|
607
|
+
`feedback_type` (asset / emotion / direction / freeform), any asset URL, the structured modification derived from the
|
|
608
|
+
feedback, and a `satisfied` flag indicating whether the user is done iterating.
|
|
609
|
+
|
|
610
|
+
The annotation is **not produced** in automated (benchmark) runs — Stage 4.5 is bypassed when no user is present.
|
|
611
|
+
See [`docs/design/design-spec-schema.md`](design/design-spec-schema.md) for the section modularity contract that
|
|
612
|
+
governs surgical re-generation. See [`commands/design.md`](../commands/design.md) for the full Stage 4.5 procedure.
|
|
613
|
+
|
|
614
|
+
**Constraint**: `FORGE:USER_FEEDBACK` may modify a section's visual execution but MUST NOT change the committed
|
|
615
|
+
`meta.archetype` or the signature move from `FORGE:DESIGN_RATIONALE` — those choices are locked once the spec is
|
|
616
|
+
committed in Stage 2.
|
|
617
|
+
|
|
618
|
+
**Schema**:
|
|
619
|
+
|
|
620
|
+
````markdown
|
|
621
|
+
<!-- FORGE:USER_FEEDBACK -->
|
|
622
|
+
## User Feedback — {product} · round {n}
|
|
623
|
+
|
|
624
|
+
**Section target:** {section ID from layout_grammar.sections, e.g. "hero" | "all" for page-wide feedback}
|
|
625
|
+
**Feedback type:** {asset | emotion | direction | freeform}
|
|
626
|
+
**Asset URL:** {URL or "none"}
|
|
627
|
+
**Modification:** {structured description of what to change — derived from user's natural-language input}
|
|
628
|
+
**Emotion target:** {trust | speed | power | craft | play | unchanged} — maps to motion vocabulary
|
|
629
|
+
**Satisfied:** {yes | no — "no" loops back to 4.5 after re-generation}
|
|
630
|
+
**Freeform notes:** {verbatim user input not captured in the structured fields above}
|
|
631
|
+
<!-- FORGE:USER_FEEDBACK:COMPLETE -->
|
|
632
|
+
````
|
|
633
|
+
|
|
634
|
+
**Field notes**:
|
|
635
|
+
- `section_target`: must match an `id` in the committed `FORGE:DESIGN_SPEC → layout_grammar.sections`. Use `"all"` only for page-wide color/typography changes that cannot be scoped to one section.
|
|
636
|
+
- `feedback_type`: `asset` = user provided a URL (video/logo/image/colors); `emotion` = user described a feeling to change; `direction` = user described a layout or composition change; `freeform` = catch-all for input that doesn't fit the above.
|
|
637
|
+
- `modification`: the parsed, actionable version of the user's words — e.g., user says "make the hero feel more urgent" → modification: `"increase motion intensity in hero scroll-reveal; tighten headline tracking"`.
|
|
638
|
+
- `satisfied: yes` means no further feedback iteration — proceed to Stage 5 (design-close).
|
|
639
|
+
|
|
640
|
+
**Detection query**:
|
|
641
|
+
```bash
|
|
642
|
+
gh api repos/{OWNER}/{REPO}/issues/{NUMBER}/comments \
|
|
643
|
+
--jq '[.[] | select(.body | contains("FORGE:USER_FEEDBACK"))] | last | .body'
|
|
644
|
+
```
|
|
645
|
+
|
|
646
|
+
---
|
|
647
|
+
|
|
648
|
+
#### `FORGE:BENCH_SCORECARD`
|
|
649
|
+
|
|
650
|
+
**Phase**: Design — Benchmark (`/design-bench`, #878)
|
|
651
|
+
**Written by**: Benchmark rig (`/design-bench` command)
|
|
652
|
+
**Read by**: Harness developers (the fitness signal — "did arm A's win-rate vs C go up?"), milestone tracker, pipeline-health
|
|
653
|
+
|
|
654
|
+
The result of one ABC benchmark run. Arm A is the ForgeDock harness output, arm B is a raw one-shot model, and arm C is the real reference page (gold standard). The annotation carries the **win-rate of each arm against C**, the **A-vs-B harness delta**, rubric **distributions** (mean + stdev) across n runs, mean slop counts, and the **judge-calibration check** (C must beat A and B; otherwise the judge is miscalibrated and the result is suspect).
|
|
655
|
+
|
|
656
|
+
The scorecard is the fitness function for the UI Taste Harness milestone — it is built before the harness so each lever the harness adds is a measurable hypothesis. The methodology (three arms, same-model rule, three-layer judging, n>=3) lives in [`design/abc-benchmark.md`](design/abc-benchmark.md); the deterministic aggregator is [`../scripts/bench-scorecard.mjs`](../scripts/bench-scorecard.mjs).
|
|
657
|
+
|
|
658
|
+
Two invariants this annotation encodes: **n>=3** (taste output is high-variance — distributions, never a single number) and **judge independence** from the harness critique loop (#882) (anti-Goodhart).
|
|
659
|
+
|
|
660
|
+
**Schema** (annotation envelope):
|
|
661
|
+
|
|
662
|
+
````
|
|
663
|
+
<!-- FORGE:BENCH_SCORECARD -->
|
|
664
|
+
## ABC Benchmark Scorecard
|
|
665
|
+
|
|
666
|
+
**Corpus version**: {ver} · **Generation model (A & B)**: {model} · **Judge model**: {independent judge}
|
|
667
|
+
**Products**: {n_products} · **Runs/product**: {n}
|
|
668
|
+
|
|
669
|
+
| Arm | Win-rate vs C | A-vs-B | Mean slop |
|
|
670
|
+
|-----|---------------|--------|-----------|
|
|
671
|
+
| A | {wr} | {a_vs_b} | {slop_A} |
|
|
672
|
+
| B | {wr} | — | {slop_B} |
|
|
673
|
+
|
|
674
|
+
**Judge calibration**: {ok | MISCALIBRATED — N runs where C lost (suspect)}
|
|
675
|
+
|
|
676
|
+
```json
|
|
677
|
+
{scorecard.json from bench-scorecard.mjs}
|
|
678
|
+
```
|
|
679
|
+
|
|
680
|
+
<!-- FORGE:BENCH_SCORECARD:COMPLETE -->
|
|
681
|
+
````
|
|
682
|
+
|
|
683
|
+
**Detection query**:
|
|
684
|
+
```bash
|
|
685
|
+
gh api repos/{OWNER}/{REPO}/issues/{NUMBER}/comments \
|
|
686
|
+
--jq '.[] | select(.body | contains("FORGE:BENCH_SCORECARD")) | .body'
|
|
687
|
+
```
|
|
688
|
+
|
|
689
|
+
---
|
|
690
|
+
|
|
691
|
+
#### `FORGE:DESIGN_SHIPPED`
|
|
692
|
+
|
|
693
|
+
**Phase**: Design — Close (`/design` stage 5, #888) — the terminal `FORGE:DESIGN_*` annotation
|
|
694
|
+
**Written by**: `/design` design-close stage
|
|
695
|
+
**Read by**: [design-memory](design/design-memory.md) (#887, persists the realized outcome), milestone tracker, pipeline-health
|
|
696
|
+
**Location**: Issue comment
|
|
697
|
+
|
|
698
|
+
Closes the design pipeline. Posted only when the design passes the full definition of done — critique-rubric threshold,
|
|
699
|
+
perf budget (#875), a11y check, and the divergence check (#887). Records the final scorecard and the realized outcome
|
|
700
|
+
(archetype, signature move, palette/type/effects, learnings) that is written to design-memory so the *next* design can
|
|
701
|
+
diverge from it. The design analog of closing a code issue at `workflow:merged`; its label is `design:shipped` (the
|
|
702
|
+
rejected counterpart is `design:rejected`, set without this annotation). See [`commands/design.md`](../commands/design.md).
|
|
703
|
+
|
|
704
|
+
**Schema**:
|
|
705
|
+
|
|
706
|
+
```markdown
|
|
707
|
+
<!-- FORGE:DESIGN_SHIPPED -->
|
|
708
|
+
## Design Shipped — {product}
|
|
709
|
+
|
|
710
|
+
**Archetype:** {committed id} · **Signature move:** {the hook}
|
|
711
|
+
**Gates:** rubric {pass} · perf {pass} · a11y {pass} · divergence {distinct from prior — #887}
|
|
712
|
+
**Critique iterations:** {n}
|
|
713
|
+
**Written to memory:** {palette/type/effects/learnings summary}
|
|
714
|
+
<!-- FORGE:DESIGN_SHIPPED:COMPLETE -->
|
|
715
|
+
```
|
|
716
|
+
|
|
717
|
+
**Detection query**:
|
|
718
|
+
```bash
|
|
719
|
+
gh api repos/{OWNER}/{REPO}/issues/{NUMBER}/comments \
|
|
720
|
+
--jq '.[] | select(.body | contains("FORGE:DESIGN_SHIPPED")) | .body'
|
|
721
|
+
```
|
|
722
|
+
|
|
723
|
+
---
|
|
724
|
+
|
|
401
725
|
### Orchestration / Milestone Annotations
|
|
402
726
|
|
|
403
727
|
These annotations transfer investigation context across issue boundaries in decomposed or multi-issue milestone workflows.
|