devrites 2.2.0 → 2.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +18 -0
- package/README.md +30 -17
- package/docs/architecture.md +30 -24
- package/docs/command-map.md +15 -6
- package/docs/flow.md +19 -5
- package/docs/orchestration.md +17 -7
- package/docs/skills.md +32 -8
- package/pack/.claude/agents/devrites-code-reviewer.md +6 -1
- package/pack/.claude/agents/devrites-devex-reviewer.md +94 -0
- package/pack/.claude/agents/devrites-forge-judge.md +92 -0
- package/pack/.claude/agents/devrites-frontend-reviewer.md +8 -0
- package/pack/.claude/agents/devrites-retrospector.md +91 -0
- package/pack/.claude/agents/devrites-slice-wright.md +18 -5
- package/pack/.claude/rules/README.md +5 -2
- package/pack/.claude/rules/agents.md +30 -2
- package/pack/.claude/rules/anti-patterns.md +11 -0
- package/pack/.claude/rules/code-review.md +16 -6
- package/pack/.claude/rules/core.md +13 -4
- package/pack/.claude/rules/developer-experience.md +119 -0
- package/pack/.claude/rules/principles.md +158 -0
- package/pack/.claude/rules/spec-grammar.md +106 -0
- package/pack/.claude/skills/devrites-browser-proof/SKILL.md +42 -3
- package/pack/.claude/skills/devrites-lib/SKILL.md +9 -2
- package/pack/.claude/skills/devrites-lib/scripts/devrites.sh +8 -0
- package/pack/.claude/skills/devrites-lib/scripts/spec-validate.sh +130 -0
- package/pack/.claude/skills/rite-adopt/SKILL.md +13 -1
- package/pack/.claude/skills/rite-autocomplete/SKILL.md +3 -1
- package/pack/.claude/skills/rite-build/SKILL.md +36 -1
- package/pack/.claude/skills/rite-build/reference/anti-patterns.md +12 -0
- package/pack/.claude/skills/rite-build/reference/forge.md +171 -0
- package/pack/.claude/skills/rite-build/reference/one-slice-cycle.md +2 -0
- package/pack/.claude/skills/rite-build/reference/wright-dispatch.md +22 -2
- package/pack/.claude/skills/rite-define/SKILL.md +14 -1
- package/pack/.claude/skills/rite-frame/SKILL.md +7 -2
- package/pack/.claude/skills/rite-learn/SKILL.md +17 -4
- package/pack/.claude/skills/rite-polish/SKILL.md +3 -1
- package/pack/.claude/skills/rite-prove/SKILL.md +30 -1
- package/pack/.claude/skills/rite-quick/SKILL.md +9 -4
- package/pack/.claude/skills/rite-review/SKILL.md +5 -2
- package/pack/.claude/skills/rite-seal/SKILL.md +22 -4
- package/pack/.claude/skills/rite-ship/SKILL.md +21 -0
- package/pack/.claude/skills/rite-spec/SKILL.md +29 -5
- package/pack/.claude/skills/rite-spec/reference/spec-template.md +11 -1
- package/pack/.claude/skills/rite-status/SKILL.md +3 -1
- package/pack/.claude/skills/rite-vet/SKILL.md +46 -14
- package/package.json +1 -1
- package/scripts/check-cross-refs.py +4 -0
- package/scripts/run-behavioral-evals.sh +170 -0
- package/scripts/run-evals.sh +3 -0
- package/scripts/validate.sh +1 -1
|
@@ -21,6 +21,7 @@ if none, tell the user to run `/rite-spec <feature>`.
|
|
|
21
21
|
**Step 0:** Read `.claude/rules/core.md` first. The other rule files load on demand;
|
|
22
22
|
pull these via `Read` when the diff demands them:
|
|
23
23
|
- `code-review.md` — small PRs, severity labels, tests-first review focus.
|
|
24
|
+
- `principles.md` — declared project invariants (`.devrites/principles.md`); a diff that violates one with no recorded exception is a Critical, blocking finding.
|
|
24
25
|
- `testing.md` — confirm the tests prove the spec, not just pass.
|
|
25
26
|
- `agents.md` — when to fan out to which review subagent.
|
|
26
27
|
- `security.md` — when input / auth / data / integrations / secrets are in scope.
|
|
@@ -48,7 +49,8 @@ pull these via `Read` when the diff demands them:
|
|
|
48
49
|
[ -f "$P" ] && bash "$P" || echo "(orientation preamble unavailable on this install — read state.md directly to orient)"
|
|
49
50
|
```
|
|
50
51
|
1. Read `spec.md`, `tasks.md`, `state.md`, `decisions.md`, `evidence.md`,
|
|
51
|
-
`touched-files.md`,
|
|
52
|
+
`touched-files.md`, `.devrites/principles.md` (if present — the binding invariants to score
|
|
53
|
+
the diff against), and the `git diff`. For "what would this change break"
|
|
52
54
|
questions, prefer a code-intelligence index if available — codebase-memory-mcp first,
|
|
53
55
|
cross-checked with codegraph + graphify, else standard methods (LSP / Read/Grep/Glob); see
|
|
54
56
|
`.claude/rules/tooling.md` — over file reads;
|
|
@@ -77,7 +79,8 @@ pull these via `Read` when the diff demands them:
|
|
|
77
79
|
where 20 would do) and the silent-failure bugs (a missing value coerced to 0/''/[],
|
|
78
80
|
a dropped Result/err return, off-by-one / boundary, logic that contradicts the
|
|
79
81
|
comment/docstring/name). Per hunk, check whether working code was deleted that the
|
|
80
|
-
task did not ask to remove.
|
|
82
|
+
task did not ask to remove. Score the diff against `.devrites/principles.md` — a change
|
|
83
|
+
that breaks a declared invariant with no recorded, human-approved exception is a Critical."
|
|
81
84
|
- **Do NOT merge or re-rank** their findings. Present them under separate
|
|
82
85
|
`## Spec` and `## Code review` sub-sections in `review.md`. Surface contradictions
|
|
83
86
|
between the axes explicitly (e.g. "Spec axis says complete, Code-review axis says
|
|
@@ -17,6 +17,7 @@ live in `/rite-ship`, which refuses to run without a GO recorded here.
|
|
|
17
17
|
pull these via `Read` before sealing:
|
|
18
18
|
- `agents.md` — review-subagent fan-out at seal.
|
|
19
19
|
- `code-review.md` — severity labels (Critical / Important / Suggestion / Nit / FYI).
|
|
20
|
+
- `principles.md` — declared project invariants (`.devrites/principles.md`) are a pass/fail gate; a diff that violates one with no recorded, human-approved exception is a NO-GO.
|
|
20
21
|
- `documentation.md` — record decisions in `decisions.md` before sealing.
|
|
21
22
|
- `observability.md` — a runtime surface that ships blind is an Important finding.
|
|
22
23
|
- `deprecation.md` — when the diff removes / migrates code, API, or data (read with the
|
|
@@ -37,6 +38,8 @@ Read `review.md` and the latest reviewer outputs.
|
|
|
37
38
|
| `Critical == 0` and `Important > 0` and acceptance proven and drift resolved | Render interactive prompt: *"`Important > 0` open. Proceed to seal? [y/N]"*. Default **N**. If the user types `y`, GO; otherwise NO-GO with the open Important findings listed as blockers-by-policy. |
|
|
38
39
|
| `Critical > 0` | **NO-GO**, no exceptions. List every Critical with `file:line` and fix direction. |
|
|
39
40
|
| Any acceptance criterion unproven | **NO-GO**, list the unproven criteria. |
|
|
41
|
+
| Visual Verdict `FAIL` on an acceptance-mapped UI criterion (`browser-evidence.md`) | **NO-GO** — an unmet acceptance criterion. A declared-state `FAIL` is Important (the `Important > 0` row). UI build with a `design-brief.md` but no Visual Verdict → Important evidence gap. No brief → not applicable. |
|
|
42
|
+
| Diff violates a declared project principle (`.devrites/principles.md`) with no recorded, human-approved exception | **NO-GO**, list each violated principle with `file:line`. Same standing as an unproven criterion (absent / empty file → none declared → not a blocker). |
|
|
40
43
|
| Unresolved drift in `drift.md` | **NO-GO**, route through `/rite-plan` first. |
|
|
41
44
|
| Any `questions.md` entry with `gate: validating` and `status: open` | **NO-GO** regardless of behavior impact — an open validating gate is merge-blocking by definition. A slice marked `built (pending review)` is not done. |
|
|
42
45
|
|
|
@@ -53,7 +56,7 @@ Read `review.md` and the latest reviewer outputs.
|
|
|
53
56
|
Then read all artifacts: `brief.md`, `spec.md`, `plan.md`, `tasks.md`, `state.md`,
|
|
54
57
|
`decisions.md`, `assumptions.md`, `questions.md`, `drift.md`, `evidence.md`,
|
|
55
58
|
`browser-evidence.md`, `polish-report.md`, `review.md`, `design-brief.md` (if UI),
|
|
56
|
-
`strategy.md` (if present), and the **final diff**. If a code-intelligence index is available
|
|
59
|
+
`devex.md` (if a developer-facing surface), `strategy.md` (if present), and the **final diff**. If a code-intelligence index is available
|
|
57
60
|
(codebase-memory-mcp first, cross-checked with codegraph + graphify, else standard methods LSP / Read/Grep/Glob — see `.claude/rules/tooling.md`), use it for
|
|
58
61
|
blast-radius checks on the final diff in step 5; context7 if available can confirm a current
|
|
59
62
|
external-API signature a reviewer flags.
|
|
@@ -63,7 +66,11 @@ Read `review.md` and the latest reviewer outputs.
|
|
|
63
66
|
proof; the `devrites-spec-reviewer` + `devrites-test-analyst` fan-out in step 7 is the
|
|
64
67
|
independent cross-check (a verifier that never saw the optimistic narrative).
|
|
65
68
|
3. Verify tests, build/typecheck/lint, and browser proof are present and green for the
|
|
66
|
-
scope. Re-run if cheap and in doubt.
|
|
69
|
+
scope. Re-run if cheap and in doubt. **For a UI feature with a `design-brief.md`**, read the
|
|
70
|
+
`## Visual Verdict` table in `browser-evidence.md`: a `FAIL` on an **acceptance-mapped**
|
|
71
|
+
criterion is an unmet acceptance criterion (NO-GO, per the severity gate), a declared-state
|
|
72
|
+
`FAIL` is **Important**, and a UI build whose brief exists but whose verdict is **absent** is an
|
|
73
|
+
Important evidence gap (the scorecard should have been emitted at browser-proof).
|
|
67
74
|
4. Check unresolved **questions** and **drift** — any open item that changes product
|
|
68
75
|
behavior blocks. **Any `questions.md` entry with `gate: validating` and `status: open`
|
|
69
76
|
is a NO-GO regardless of behavior impact** (an open validating gate is merge-blocking by
|
|
@@ -73,10 +80,19 @@ Read `review.md` and the latest reviewer outputs.
|
|
|
73
80
|
`/rite-temper`), confirm its **top pre-mortem risks are mitigated** in the diff/evidence and
|
|
74
81
|
that no **Non-goal / deferred item crept into the diff** (scope creep) — either is a finding
|
|
75
82
|
(an unmitigated top risk or smuggled-in out-of-scope work).
|
|
83
|
+
- **Principles** (`principles.md`): score the final diff against each declared invariant in
|
|
84
|
+
`.devrites/principles.md`. A violation with no matching, human-approved exception in the
|
|
85
|
+
register is a **Critical** finding and a NO-GO; an exception that is stale (past its review
|
|
86
|
+
trigger) or wider than its stated scope is itself a finding. No file / no principles → skip.
|
|
76
87
|
- **Observability** (`observability.md`): if the diff added a runtime surface (endpoint,
|
|
77
88
|
job, integration, user flow, error path), a feature shipping with no way to debug it in
|
|
78
89
|
prod is an **Important** finding, not a pass — `evidence.md` should show telemetry observed
|
|
79
90
|
to emit (`/rite-prove` step 5b).
|
|
91
|
+
- **Developer experience** (`developer-experience.md`): if the diff ships a developer-facing
|
|
92
|
+
surface, reconcile `devex.md` (the `/rite-vet` predicted scorecard vs the `/rite-prove`
|
|
93
|
+
measured one — the boomerang). A broken public dev contract (a documented command that errors,
|
|
94
|
+
a getting-started flow that can't complete) or an unexplained measured DX regression is
|
|
95
|
+
**Important** — **Critical** on a frozen public surface (`principles.md`). No surface → skip.
|
|
80
96
|
- **Removal / migration** (`deprecation.md`): if the diff deletes or migrates code, an API,
|
|
81
97
|
or data, confirm it followed expand→contract, proved the old path unused before removing it,
|
|
82
98
|
and carries a rollback for every destructive step. A surprise deletion or a one-shot
|
|
@@ -98,8 +114,10 @@ Read `review.md` and the latest reviewer outputs.
|
|
|
98
114
|
`devrites-spec-reviewer` (does the diff implement
|
|
99
115
|
the spec?), `devrites-test-analyst` (do the tests prove acceptance?),
|
|
100
116
|
`devrites-code-reviewer`, `devrites-frontend-reviewer` (UI features),
|
|
101
|
-
`devrites-security-auditor` (input/auth/data/integrations),
|
|
102
|
-
`devrites-performance-reviewer` (perf-relevant)
|
|
117
|
+
`devrites-security-auditor` (input/auth/data/integrations),
|
|
118
|
+
`devrites-performance-reviewer` (perf-relevant), and `devrites-devex-reviewer`
|
|
119
|
+
(developer-facing surface — measure mode: grade the measured DX scorecard and reconcile
|
|
120
|
+
the boomerang against the `/rite-vet` prediction). Give each the workspace
|
|
103
121
|
path + diff *without the author's reasoning*. If subagents are unavailable,
|
|
104
122
|
run the equivalent reviews sequentially yourself.
|
|
105
123
|
The reviewer **AGENTS** here (fresh context, no author reasoning) are the seal
|
|
@@ -97,6 +97,26 @@ Refuses to ship unless `seal.md` records a **GO** verdict.
|
|
|
97
97
|
`bash .claude/skills/devrites-lib/scripts/close-out.sh <slug>` to archive
|
|
98
98
|
`.devrites/work/<slug>/` → `.devrites/archive/<slug>/` and clear `.devrites/ACTIVE`.
|
|
99
99
|
Every `.md` is preserved in the archive.
|
|
100
|
+
6a. **Cross-feature retro (automatic, cadence-gated, advisory).** The just-shipped feature is now in
|
|
101
|
+
the archive, so this is where the **cross-feature** learning loop closes on its own — the synthesis
|
|
102
|
+
that otherwise waits for a human to run `/rite-learn`. Run the cheap cadence gate first; it stays
|
|
103
|
+
silent unless a finding/drift class recurs across **>=2 shipped features** with new signal since the
|
|
104
|
+
last review (so it never fires on an early or one-off ship):
|
|
105
|
+
```bash
|
|
106
|
+
L=.claude/skills/devrites-lib/scripts/learnings.sh
|
|
107
|
+
[ -f "$L" ] || L="${CLAUDE_SKILL_DIR:-}/../devrites-lib/scripts/learnings.sh"
|
|
108
|
+
[ -f "$L" ] || L=pack/.claude/skills/devrites-lib/scripts/learnings.sh
|
|
109
|
+
[ -f "$L" ] && bash "$L" nudge || true
|
|
110
|
+
```
|
|
111
|
+
**If the nudge emits** (a recurring pattern crossed the threshold), dispatch the read-only
|
|
112
|
+
`devrites-retrospector` (`.claude/agents/`) over `.devrites/archive/` for the cross-feature
|
|
113
|
+
synthesis. Persist its digest to `.devrites/retro.md` (append a dated entry — the project-level
|
|
114
|
+
retro ledger, never rewritten) and surface the **graduation candidates** with a one-line pointer
|
|
115
|
+
to `/rite-learn`, which is where the human confirms a promotion to a rule / principle / convention.
|
|
116
|
+
**Propose, never impose:** retro **drafts**; it never auto-writes a rule or principle (a principle
|
|
117
|
+
is a gate, amended deliberately and dated — `principles.md` governance), and it never blocks the
|
|
118
|
+
ship, which has already happened. If the nudge is silent, skip — no retro this close. Then `touch
|
|
119
|
+
.devrites/.learnings-reviewed` only when the human acts on it via `/rite-learn`, not here.
|
|
100
120
|
|
|
101
121
|
> **Mid-flight discipline.** When tempted to ship without a GO seal, skip the type-GO,
|
|
102
122
|
> stage files outside `touched-files.md`, or delete the workspace instead of archiving
|
|
@@ -112,6 +132,7 @@ Commit: <sha> on <branch> Tag/PR: <ref | none>
|
|
|
112
132
|
Acceptance: <n/total> proven
|
|
113
133
|
Archived: .devrites/archive/<slug>/ · ACTIVE cleared
|
|
114
134
|
ship.md: .devrites/archive/<slug>/ship.md
|
|
135
|
+
Retro: <n graduation candidates drafted → .devrites/retro.md · promote with /rite-learn | quiet (no cross-feature signal yet)>
|
|
115
136
|
```
|
|
116
137
|
If the user declined type-GO: state that nothing shipped, the seal still reads GO, and
|
|
117
138
|
the resume command (`/rite-ship`).
|
|
@@ -21,7 +21,13 @@ are `/rite-define` and `/rite-build`.
|
|
|
21
21
|
## Rules consulted (read on demand from `.claude/rules/`)
|
|
22
22
|
**Step 0:** Read `.claude/rules/core.md` first. DevRites skills Read `.claude/rules/core.md`
|
|
23
23
|
as their first step; the other rule files load on demand. Pull `documentation.md` via `Read`
|
|
24
|
-
when capturing significant spec decisions (why-not-what, ADR-style notes in `decisions.md`)
|
|
24
|
+
when capturing significant spec decisions (why-not-what, ADR-style notes in `decisions.md`);
|
|
25
|
+
pull `principles.md` when the project has declared invariants (`.devrites/principles.md`) — a
|
|
26
|
+
new spec must respect them, and a requirement that can only be met by breaking one is a blocking gap.
|
|
27
|
+
Pull `spec-grammar.md` when writing acceptance for a behavioral / high-risk requirement (auth,
|
|
28
|
+
data model, state machine, public API, money, migration) — the structured `### Requirement:` /
|
|
29
|
+
`#### Scenario:` (SHALL · WHEN/THEN) form, lint-checked by `spec-validate.sh`. Simple criteria
|
|
30
|
+
stay flat `[ACn]` bullets; the grammar is opt-in by rigor, never forced.
|
|
25
31
|
|
|
26
32
|
## Operating rules (DevRites core)
|
|
27
33
|
- No silent assumptions · no guessing through confusion · prefer existing conventions ·
|
|
@@ -55,7 +61,8 @@ when capturing significant spec decisions (why-not-what, ADR-style notes in `dec
|
|
|
55
61
|
Also discover the project's **test / build/typecheck/lint** commands and the
|
|
56
62
|
frontend/backend systems; read `PRODUCT.md` / `DESIGN.md` / `CLAUDE.md` / `AGENTS.md` if
|
|
57
63
|
present (`AGENTS.md` is the cross-tool agent-conventions standard — treat it as project
|
|
58
|
-
conventions the build must follow, same standing as `CLAUDE.md`)
|
|
64
|
+
conventions the build must follow, same standing as `CLAUDE.md`), and read
|
|
65
|
+
`.devrites/principles.md` if present — the declared invariants the feature must respect.
|
|
59
66
|
3. **Gather design references (optional)** — [references-intake](reference/references-intake.md).
|
|
60
67
|
The human **may** attach screenshots, mockups, a Figma link, a video, or links — or
|
|
61
68
|
**none at all** (perfectly normal; skip this step then). If any are given: **view/fetch**
|
|
@@ -79,12 +86,19 @@ when capturing significant spec decisions (why-not-what, ADR-style notes in `dec
|
|
|
79
86
|
can predict them (don't interrogate). Aim for **zero blocking gaps**. *If a gap is genuinely undecidable on paper (state
|
|
80
87
|
machine that may deadlock, data shape ambiguity, "which UX wins") → suggest a
|
|
81
88
|
scoped detour to `/rite-prototype` to answer that ONE question before
|
|
82
|
-
continuing.*
|
|
89
|
+
continuing.* **Invariant conflict is a blocking gap:** if a requirement or acceptance
|
|
90
|
+
criterion can only be satisfied by breaking a declared principle (`.devrites/principles.md`),
|
|
91
|
+
surface it — the principle wins by default; breaking it needs a recorded, scoped exception a
|
|
92
|
+
human approves, never a spec that silently contradicts an invariant.
|
|
83
93
|
5. **Create the workspace** + set `.devrites/ACTIVE`
|
|
84
94
|
([state-workspace](reference/state-workspace.md)). Write `spec.md`
|
|
85
95
|
([spec-template](reference/spec-template.md)) — WHAT/WHY, technology-agnostic, with
|
|
86
96
|
**Placement & integration**, **Design references**, **Gaps/issues/decisions**, and
|
|
87
|
-
measurable acceptance ([acceptance-criteria](reference/acceptance-criteria.md)).
|
|
97
|
+
measurable acceptance ([acceptance-criteria](reference/acceptance-criteria.md)). For a
|
|
98
|
+
behavioral / high-risk requirement, write the acceptance as a structured
|
|
99
|
+
`### Requirement:` (SHALL) + `#### Scenario:` (WHEN/THEN) block per
|
|
100
|
+
[`spec-grammar.md`](../../rules/spec-grammar.md), nesting the `[ACn]` id inside each scenario
|
|
101
|
+
so `/rite-seal` still grades it; routine criteria stay flat `[ACn]` bullets. Also
|
|
88
102
|
write `brief.md`, `references.md`, `questions.md`, `decisions.md`, `assumptions.md`,
|
|
89
103
|
and an initial `state.md` (phase: spec). When the feature touches UI, `design-brief.md`
|
|
90
104
|
is written here too (by `devrites-ux-shape`, step 3a).
|
|
@@ -98,7 +112,16 @@ when capturing significant spec decisions (why-not-what, ADR-style notes in `dec
|
|
|
98
112
|
`[NEEDS CLARIFICATION]`, placement decided, all material gaps resolved, any design
|
|
99
113
|
references provided are saved, **UX/UI shaped into `design-brief.md` if the feature is
|
|
100
114
|
UI**, requirements testable, success criteria measurable, **every `checklists/<domain>.md` at
|
|
101
|
-
`Verdict: pass
|
|
115
|
+
`Verdict: pass`**, and **any structured requirement blocks are grammar-valid** — run
|
|
116
|
+
`spec-validate.sh` (resolved like the step-0 preamble), a non-zero exit is a blocking
|
|
117
|
+
failure to fix, not soften:
|
|
118
|
+
```bash
|
|
119
|
+
SV=.claude/skills/devrites-lib/scripts/spec-validate.sh
|
|
120
|
+
[ -f "$SV" ] || SV="${CLAUDE_SKILL_DIR:-}/../devrites-lib/scripts/spec-validate.sh"
|
|
121
|
+
[ -f "$SV" ] || SV=pack/.claude/skills/devrites-lib/scripts/spec-validate.sh
|
|
122
|
+
[ -f "$SV" ] && bash "$SV" ".devrites/work/<slug>" || echo "(spec-grammar validator unavailable — eyeball the Requirement/Scenario blocks)"
|
|
123
|
+
```
|
|
124
|
+
When it passes, write `Spec gate: passed <iso>` to `state.md`. **Stop** when
|
|
102
125
|
it passes.
|
|
103
126
|
|
|
104
127
|
> **Mid-flight discipline.** When tempted to skip investigation depth, gap-closing, or placement decisions — see [`anti-patterns`](reference/anti-patterns.md) (Common Rationalizations + Red Flags). Load it the moment you reach for the excuse.
|
|
@@ -114,6 +137,7 @@ References: <n saved | none provided>
|
|
|
114
137
|
Design brief: <design-brief.md shaped (compact|full) | n/a — not UI>
|
|
115
138
|
Gaps closed: <n> Open (non-blocking): <n>
|
|
116
139
|
Checklists: <n domains scored — all pass | BLOCKED: n CRITICAL open>
|
|
140
|
+
Grammar: <n requirements / m scenarios — valid | n/a (flat acceptance)>
|
|
117
141
|
Next: big / risky feature (auth · data model · public API · migration · multi-slice · ambiguous scope)?
|
|
118
142
|
→ /rite-temper (strategic review: scope mode + pre-mortem, hardens the spec) — then /rite-define.
|
|
119
143
|
Small / reversible / unambiguous? → /rite-define directly.
|
|
@@ -65,7 +65,16 @@ Observable outcomes that mean "this worked". Numbers where possible. No tech nam
|
|
|
65
65
|
|
|
66
66
|
## Acceptance criteria
|
|
67
67
|
Binary, evidence-backed checklist; each maps to a scenario/FR. See acceptance-criteria.md.
|
|
68
|
-
- [ ] <criterion> (FR-00x)
|
|
68
|
+
- [ ] [AC1] <criterion> (FR-00x)
|
|
69
|
+
|
|
70
|
+
For a behavioral / high-risk requirement (auth, data model, state machine, public API, money,
|
|
71
|
+
migration), write it as a structured Requirement/Scenario block instead — testable by
|
|
72
|
+
construction and lint-checked by spec-validate.sh (see spec-grammar.md). Keep the [ACn] id
|
|
73
|
+
inside the scenario so the seal still grades it; routine criteria stay flat bullets above.
|
|
74
|
+
### Requirement: <name>
|
|
75
|
+
The system SHALL <core observable behavior>.
|
|
76
|
+
#### Scenario: <name>
|
|
77
|
+
- [ ] [AC2] **WHEN** <trigger> **THEN** <observable outcome> (**AND** <extra condition>)
|
|
69
78
|
|
|
70
79
|
## Non-goals
|
|
71
80
|
Explicitly out of scope for this version.
|
|
@@ -119,6 +128,7 @@ List every open marker; blocking ones must be zero at the gate.
|
|
|
119
128
|
- [ ] Design references gathered + saved (if the human supplied any)
|
|
120
129
|
- [ ] **UX/UI shaped into `design-brief.md`** (if the feature touches UI) — direction, key states, interaction model decided
|
|
121
130
|
- [ ] Requirements are testable and unambiguous
|
|
131
|
+
- [ ] **Structured requirements are grammar-valid** (if any) — `### Requirement:` / `#### Scenario:` blocks pass `spec-validate.sh` (SHALL + WHEN/THEN, unique headers); flat `[ACn]` bullets need no check (see spec-grammar.md)
|
|
122
132
|
- [ ] Success criteria are measurable and technology-agnostic
|
|
123
133
|
- [ ] **Spec-quality checklists pass** — every `checklists/<domain>.md` reaches `Verdict: pass` (zero CRITICAL "unit-tests-for-English" fails; see [spec-checklists.md](spec-checklists.md))
|
|
124
134
|
```
|
|
@@ -49,7 +49,9 @@ Otherwise summarize from the loaded state, concisely:
|
|
|
49
49
|
6. **Open questions** — count by gate (from `questions.md`:
|
|
50
50
|
`n open: x blocking · y validating · z advisory`) + the blocking qids by id and one-line question.
|
|
51
51
|
7. **Unresolved drift** (`drift.md`).
|
|
52
|
-
8. **Risks** (from `state.md` / `spec.md`).
|
|
52
|
+
8. **Risks** (from `state.md` / `spec.md`); if `.devrites/principles.md` exists, note the count
|
|
53
|
+
of declared invariants and flag any exception in its register that is past its review/removal
|
|
54
|
+
trigger (a stale exception is a standing risk). Omit if there's no principles file.
|
|
53
55
|
9. **Ready for handoff?** — see section below.
|
|
54
56
|
|
|
55
57
|
Flag anything blocking: unresolved drift, failing evidence, `Status: awaiting_human`,
|
|
@@ -22,11 +22,16 @@ This is the engineering counterpart to `/rite-temper` (which is strategic, on th
|
|
|
22
22
|
Temper decides *the right thing*; vet decides *the right way to build it*.
|
|
23
23
|
|
|
24
24
|
## Rules consulted (read on demand from `.claude/rules/`)
|
|
25
|
-
**Step 0:** Read `.claude/rules/core.md` first. Pull on demand: `
|
|
25
|
+
**Step 0:** Read `.claude/rules/core.md` first. Pull on demand: `principles.md` (the project
|
|
26
|
+
invariants gate — how `.devrites/principles.md` is scored pass/fail), `patterns.md` +
|
|
26
27
|
`coding-style.md` (the over-engineering / reuse-first / YAGNI rubric — reuse the pack's
|
|
27
|
-
standard), `testing.md` (the test-coverage axis)
|
|
28
|
+
standard), `testing.md` (the test-coverage axis) + `spec-grammar.md` (when the spec uses
|
|
29
|
+
structured Requirement/Scenario blocks, each scenario is a coverage unit `test-plan.md` must
|
|
30
|
+
map), `performance.md` (the perf axis),
|
|
28
31
|
`error-handling.md` (failure-mode coverage), `development-workflow.md` (parallel lanes,
|
|
29
|
-
definition of done), `afk-hitl.md` (irreversible-risk list + gate ceiling)
|
|
32
|
+
definition of done), `afk-hitl.md` (irreversible-risk list + gate ceiling),
|
|
33
|
+
`developer-experience.md` (when the plan ships a developer-facing surface — API / CLI / SDK /
|
|
34
|
+
webhook / config / error messages / getting-started — predict the DX scorecard here).
|
|
30
35
|
|
|
31
36
|
## Operating rules
|
|
32
37
|
- **Review the plan, not the spec's ambition.** The spec's scope/ambition is `/rite-temper`'s
|
|
@@ -78,15 +83,22 @@ definition of done), `afk-hitl.md` (irreversible-risk list + gate ceiling).
|
|
|
78
83
|
against a built-in (dispatch `devrites-source-driven`); completeness check (with AI, full
|
|
79
84
|
coverage is ~100× cheaper than the human-hours saved by a shortcut — prefer complete); and a
|
|
80
85
|
distribution check for any new artifact.
|
|
81
|
-
2a. **Cross-artifact analyze gate + charter/conventions gate.** Before the axes, run
|
|
82
|
-
consistency+coverage pass over `spec.md` + `plan.md` + `tasks.md` (+ `coverage.md`
|
|
83
|
-
any **CRITICAL** — an acceptance criterion with no slice, a slice satisfying no
|
|
84
|
-
contradiction across artifacts — **blocks `/rite-build`** until resolved. Then score
|
|
85
|
-
|
|
86
|
-
(`.devrites/
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
86
|
+
2a. **Cross-artifact analyze gate + principles / charter / conventions gate.** Before the axes, run
|
|
87
|
+
one read-only consistency+coverage pass over `spec.md` + `plan.md` + `tasks.md` (+ `coverage.md`
|
|
88
|
+
if present); any **CRITICAL** — an acceptance criterion with no slice, a slice satisfying no
|
|
89
|
+
criterion, a contradiction across artifacts — **blocks `/rite-build`** until resolved. Then score
|
|
90
|
+
the three project gates as explicit **pass/fail** on the planned approach:
|
|
91
|
+
- **Principles** (`.devrites/principles.md`, rubric in [`principles.md`](../../rules/principles.md))
|
|
92
|
+
— the authored invariants the project will not break. A plan that bakes in a violation of a
|
|
93
|
+
declared principle with **no recorded, human-approved exception** is a **top-severity** finding,
|
|
94
|
+
walked **first**, and **blocks `/rite-build`**. Absent or empty file → none declared → passes;
|
|
95
|
+
**never block for the absence of principles**. A genuine need to break one routes to a scoped,
|
|
96
|
+
dated exception in the principles register — never a silent work-around (adding the exception is
|
|
97
|
+
an irreversible-risk decision: it always pauses for a human, even in AFK).
|
|
98
|
+
- **The anti-slop charter** (`coding-style.md` + `prose-style.md`) and **the conventions ledger**
|
|
99
|
+
(`.devrites/conventions.md`) — a plan that bakes in a god-module, a speculative abstraction with
|
|
100
|
+
no second caller, or a dependency where an in-repo option exists is a **top-severity** violation.
|
|
101
|
+
**Re-check all three after the axes harden the plan** (post-design). Write the result to `analysis.md`.
|
|
90
102
|
```bash
|
|
91
103
|
A=.claude/skills/devrites-lib/scripts/analyze.sh
|
|
92
104
|
[ -f "$A" ] || A="${CLAUDE_SKILL_DIR:-}/../devrites-lib/scripts/analyze.sh"
|
|
@@ -103,12 +115,31 @@ definition of done), `afk-hitl.md` (irreversible-risk list + gate ceiling).
|
|
|
103
115
|
coverage-increasing findings auto-apply; **anything that grows scope or changes acceptance is a
|
|
104
116
|
blocking pause**; irreversible-risk always pauses.)
|
|
105
117
|
4. **Required outputs** — the test-coverage diagram + per-gap test requirements (the **regression
|
|
106
|
-
rule** is mandatory, no question)
|
|
118
|
+
rule** is mandatory, no question). When the spec uses the structured grammar
|
|
119
|
+
(`spec-grammar.md`), the diagram maps **each `#### Scenario:` (WHEN/THEN) to ≥1 planned
|
|
120
|
+
test** — an unmapped scenario is a coverage gap the build must close. Then the failure-mode
|
|
121
|
+
table, "NOT in scope", "What already exists",
|
|
107
122
|
and the worktree parallelization strategy. Shapes in [`reference/review-axes.md`](reference/review-axes.md).
|
|
123
|
+
**Developer-facing surface?** If the plan ships one (`developer-experience.md` — API / CLI / SDK /
|
|
124
|
+
webhook / config / error messages / getting-started), predict the DX scorecard: the time-to-hello-world
|
|
125
|
+
estimate plus the getting-started, error-message, and ergonomics friction the plan bakes in. Write it
|
|
126
|
+
to `devex.md` as the **prediction the boomerang measures against** at `/rite-prove` / `/rite-seal`.
|
|
127
|
+
Absent surface → skip, no `devex.md` (greenfield no-op, like the principles gate).
|
|
108
128
|
Also a **PRP one-pass-implementable check** per slice brief (the build's pre-flight): confirm each
|
|
109
129
|
slice's Consumes/Produces, Known-Gotchas, validation commands, and reuse targets are present and
|
|
110
130
|
concrete — a brief that can't be built in one pass is a finding; harden the slice until it clears,
|
|
111
131
|
before `/rite-build`.
|
|
132
|
+
4a. **Forge gate (rare — confirm or clear).** For each slice carrying `Forge: yes` (proposed by
|
|
133
|
+
`/rite-define`), and any slice the architecture axis showed has **≥2 genuinely-viable approaches
|
|
134
|
+
with no clear winner at Complexity ≥4**, confirm the flag: name the 2–3 candidate strategies that
|
|
135
|
+
actually differ (different data shape, different seam, reuse-vs-build — not variations of one), and
|
|
136
|
+
confirm the slice's acceptance + `test-plan.md` give the judge an objective scorecard. **Clear**
|
|
137
|
+
`Forge: yes` back to `no` when the review settled on one approach, the slice is below the complexity
|
|
138
|
+
bar, or you can't name two real strategies — competing a decided or trivial slice burns K× the build
|
|
139
|
+
for nothing. Forge is a **build-cost** decision, not an irreversible one: it never bypasses a gate,
|
|
140
|
+
and under AFK its K candidates count against the slice budget. Record the confirmed strategies in
|
|
141
|
+
the slice brief so `/rite-build` competes them
|
|
142
|
+
([`rite-build/reference/forge.md`](../rite-build/reference/forge.md)). No flagged slice → nothing to do.
|
|
112
143
|
5. **Write `eng-review.md` + `test-plan.md`, fold back** — [`reference/artifacts.md`](reference/artifacts.md).
|
|
113
144
|
`eng-review.md` is the durable record; `test-plan.md` is the build-readable coverage target
|
|
114
145
|
(`/rite-build` and `/rite-prove` read it). Harden `plan.md` / `tasks.md` directly for
|
|
@@ -144,9 +175,10 @@ Depth: light | full (<trigger that escalated it>)
|
|
|
144
175
|
Scope: reuse <n found> / minimum-diff <ok|trimmed N> / complexity <ok|smell: N files, M new services → asked>
|
|
145
176
|
Axes (floor → verdict): Architecture <band> · Code-quality <band> · Tests <band> · Performance <band>
|
|
146
177
|
Findings: <Critical n / Important n / Suggestion n> (suppressed low-confidence: n)
|
|
147
|
-
Coverage: <x/y paths> planned · GAPS closed <n> · regressions flagged <n> → test-plan.md
|
|
178
|
+
Coverage: <x/y paths> planned (<s/t scenarios mapped | n/a>) · GAPS closed <n> · regressions flagged <n> → test-plan.md
|
|
148
179
|
Failure modes: <n> mapped (<n critical: no test + no handling + silent>)
|
|
149
180
|
Parallelization: <n lanes — n parallel / n sequential> | sequential (no opportunity)
|
|
181
|
+
Forge: <n slices flagged — strategies named | none> (competed candidate builds at /rite-build)
|
|
150
182
|
Reviewer loop: <n> iter · cross-model: ran (codex) | off
|
|
151
183
|
Plan: hardened in place | <n> deltas routed via Spec Drift Guard → /rite-plan repair
|
|
152
184
|
Next: /rite-build (builds the vetted plan)
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "devrites",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.3.0",
|
|
4
4
|
"description": "DevRites — disciplined senior-engineer workflow skills pack for Claude Code",
|
|
5
5
|
"license": "SEE LICENSE IN LICENSE",
|
|
6
6
|
"homepage": "https://github.com/ViktorsBaikers/DevRites#readme",
|
|
@@ -29,7 +29,11 @@ WORKSPACE_ARTIFACTS = {
|
|
|
29
29
|
"questions.md", "drift.md", "evidence.md", "browser-evidence.md", "touched-files.md",
|
|
30
30
|
"review.md", "seal.md", "ship.md", "brief.md", "design-brief.md", "strategy.md",
|
|
31
31
|
"polish-report.md", "handoff.md", "eng-review.md", "test-plan.md", "references.md",
|
|
32
|
+
# forge competition record (work/<slug>/forge-report.md) — written when a Forge: yes slice competes candidates
|
|
33
|
+
"forge-report.md",
|
|
32
34
|
"conventions.md", "coverage.md", "analysis.md", "learnings.md",
|
|
35
|
+
# developer-experience scorecard (work/<slug>/devex.md) + project-level retro ledger (.devrites/retro.md)
|
|
36
|
+
"devex.md", "retro.md",
|
|
33
37
|
# spec-quality checklists (.devrites/work/<slug>/checklists/<domain>.md)
|
|
34
38
|
"functional.md", "data-model.md", "interaction.md", "non-functional.md", "edge-cases.md",
|
|
35
39
|
}
|
|
@@ -0,0 +1,170 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
# scripts/run-behavioral-evals.sh — validate the SHAPE of DevRites behavioral evals.
|
|
3
|
+
#
|
|
4
|
+
# Trigger evals (run-evals.sh) test whether the right skill *fires*. Behavioral
|
|
5
|
+
# evals test whether a gating skill's discipline *holds under pressure* — does it
|
|
6
|
+
# resist the rationalizations it documents in `rules/anti-patterns.md` (and its own
|
|
7
|
+
# `reference/anti-patterns.md`), or does the agent talk itself past the gate. Each
|
|
8
|
+
# scenario pairs a pressure prompt with the resistance a holding response shows and
|
|
9
|
+
# the capitulation markers a failed one shows.
|
|
10
|
+
#
|
|
11
|
+
# This script is the DETERMINISTIC, zero-token CI gate — the analog of run-evals.sh's
|
|
12
|
+
# schema path and spec-validate.sh: it checks that every behavioral eval is well-formed
|
|
13
|
+
# so a malformed one can't reach the live grader. It does NOT invoke a model. Executing
|
|
14
|
+
# the scenarios against a live Claude (does the skill actually resist?) is the labeled /
|
|
15
|
+
# nightly rung documented in evals/behavioral/README.md.
|
|
16
|
+
#
|
|
17
|
+
# Usage:
|
|
18
|
+
# scripts/run-behavioral-evals.sh # validate every evals/behavioral/*.json
|
|
19
|
+
# scripts/run-behavioral-evals.sh evals/behavioral/rite-prove.json # validate one file
|
|
20
|
+
#
|
|
21
|
+
# Exit: 0 all valid (or no behavioral evals present — opt-in, never a failure) ·
|
|
22
|
+
# 1 shape violation(s) · 2 missing parser
|
|
23
|
+
|
|
24
|
+
set -euo pipefail
|
|
25
|
+
|
|
26
|
+
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
|
27
|
+
# Overridable so the test harness can exercise the opt-in no-op path against a
|
|
28
|
+
# scratch directory; defaults to the real behavioral-eval set.
|
|
29
|
+
BEHAVIORAL_DIR="${DEVRITES_BEHAVIORAL_DIR:-$ROOT/evals/behavioral}"
|
|
30
|
+
|
|
31
|
+
if [[ $# -gt 0 ]]; then
|
|
32
|
+
FILES=("$@")
|
|
33
|
+
else
|
|
34
|
+
if [[ ! -d "$BEHAVIORAL_DIR" ]]; then
|
|
35
|
+
echo "No evals/behavioral/ directory — behavioral evals are opt-in; nothing to validate."
|
|
36
|
+
exit 0
|
|
37
|
+
fi
|
|
38
|
+
FILES=()
|
|
39
|
+
while IFS= read -r f; do
|
|
40
|
+
[[ "$f" == */README.md ]] && continue
|
|
41
|
+
FILES+=("$f")
|
|
42
|
+
done < <(find "$BEHAVIORAL_DIR" -type f -name '*.json' | sort)
|
|
43
|
+
fi
|
|
44
|
+
|
|
45
|
+
if [[ ${#FILES[@]} -eq 0 ]]; then
|
|
46
|
+
echo "No behavioral eval files — behavioral evals are opt-in; nothing to validate."
|
|
47
|
+
exit 0
|
|
48
|
+
fi
|
|
49
|
+
|
|
50
|
+
if command -v python3 >/dev/null 2>&1; then
|
|
51
|
+
PARSER="python3"
|
|
52
|
+
elif command -v jq >/dev/null 2>&1; then
|
|
53
|
+
PARSER="jq"
|
|
54
|
+
else
|
|
55
|
+
echo "Need python3 or jq to validate JSON." >&2
|
|
56
|
+
exit 2
|
|
57
|
+
fi
|
|
58
|
+
|
|
59
|
+
FAILED=0
|
|
60
|
+
TOTAL=0
|
|
61
|
+
SCENARIOS=0
|
|
62
|
+
|
|
63
|
+
for file in "${FILES[@]}"; do
|
|
64
|
+
TOTAL=$((TOTAL + 1))
|
|
65
|
+
printf '== %s ==\n' "$file"
|
|
66
|
+
|
|
67
|
+
if [[ "$PARSER" == "python3" ]]; then
|
|
68
|
+
if OUT=$(python3 - "$file" <<'PY'
|
|
69
|
+
import json, sys, pathlib
|
|
70
|
+
path = pathlib.Path(sys.argv[1])
|
|
71
|
+
try:
|
|
72
|
+
data = json.loads(path.read_text())
|
|
73
|
+
except Exception as e:
|
|
74
|
+
print(f" FAIL: INVALID JSON: {e}")
|
|
75
|
+
sys.exit(1)
|
|
76
|
+
|
|
77
|
+
errors = []
|
|
78
|
+
for key in ("skill", "description", "scenarios"):
|
|
79
|
+
if key not in data:
|
|
80
|
+
errors.append(f"missing top-level key: {key}")
|
|
81
|
+
|
|
82
|
+
scenarios = data.get("scenarios", [])
|
|
83
|
+
if not isinstance(scenarios, list):
|
|
84
|
+
errors.append("scenarios is not a list")
|
|
85
|
+
scenarios = []
|
|
86
|
+
elif len(scenarios) == 0:
|
|
87
|
+
errors.append("scenarios is empty — a behavioral eval needs at least one")
|
|
88
|
+
|
|
89
|
+
seen_ids = {}
|
|
90
|
+
for i, s in enumerate(scenarios):
|
|
91
|
+
if not isinstance(s, dict):
|
|
92
|
+
errors.append(f"scenario[{i}] not an object")
|
|
93
|
+
continue
|
|
94
|
+
sid = s.get("id")
|
|
95
|
+
if not sid or not isinstance(sid, str):
|
|
96
|
+
errors.append(f"scenario[{i}] missing non-empty string id")
|
|
97
|
+
elif sid in seen_ids:
|
|
98
|
+
errors.append(f"scenario[{i}] duplicate id {sid!r} (also scenario[{seen_ids[sid]}])")
|
|
99
|
+
else:
|
|
100
|
+
seen_ids[sid] = i
|
|
101
|
+
where = sid or f"[{i}]"
|
|
102
|
+
# Non-empty string fields.
|
|
103
|
+
for k in ("rationalization", "pressure", "source"):
|
|
104
|
+
v = s.get(k)
|
|
105
|
+
if not isinstance(v, str) or not v.strip():
|
|
106
|
+
errors.append(f"scenario {where} missing non-empty {k}")
|
|
107
|
+
# Non-empty list-of-string fields.
|
|
108
|
+
for k in ("expected_resistance", "capitulation_markers"):
|
|
109
|
+
v = s.get(k)
|
|
110
|
+
if not isinstance(v, list) or len(v) == 0:
|
|
111
|
+
errors.append(f"scenario {where} missing non-empty {k} list")
|
|
112
|
+
elif not all(isinstance(x, str) and x.strip() for x in v):
|
|
113
|
+
errors.append(f"scenario {where} {k} has an empty / non-string entry")
|
|
114
|
+
|
|
115
|
+
if errors:
|
|
116
|
+
for e in errors:
|
|
117
|
+
print(f" FAIL: {e}")
|
|
118
|
+
sys.exit(1)
|
|
119
|
+
|
|
120
|
+
print(f" skill: {data['skill']}")
|
|
121
|
+
print(f" scenarios: {len(scenarios)}")
|
|
122
|
+
print(f"__COUNT__ {len(scenarios)}")
|
|
123
|
+
PY
|
|
124
|
+
); then rc=0; else rc=$?; fi
|
|
125
|
+
else
|
|
126
|
+
if OUT=$(jq -r '
|
|
127
|
+
if (.skill and .description and (.scenarios|type=="array") and (.scenarios|length > 0)) then
|
|
128
|
+
( [ .scenarios[]
|
|
129
|
+
| select(
|
|
130
|
+
((.id|type=="string") and (.id|length>0))
|
|
131
|
+
and ((.rationalization|type=="string") and (.rationalization|length>0))
|
|
132
|
+
and ((.pressure|type=="string") and (.pressure|length>0))
|
|
133
|
+
and ((.source|type=="string") and (.source|length>0))
|
|
134
|
+
and ((.expected_resistance|type=="array") and (.expected_resistance|length>0))
|
|
135
|
+
and ((.capitulation_markers|type=="array") and (.capitulation_markers|length>0))
|
|
136
|
+
) ] | length
|
|
137
|
+
) as $ok
|
|
138
|
+
| if $ok == (.scenarios|length)
|
|
139
|
+
then " skill: \(.skill)\n scenarios: \(.scenarios|length)\n__COUNT__ \(.scenarios|length)"
|
|
140
|
+
else " FAIL: \((.scenarios|length) - $ok) scenario(s) missing required fields"
|
|
141
|
+
end
|
|
142
|
+
else
|
|
143
|
+
" FAIL: missing required keys or empty scenarios"
|
|
144
|
+
end
|
|
145
|
+
' "$file"); then rc=0; else rc=$?; fi
|
|
146
|
+
fi
|
|
147
|
+
|
|
148
|
+
COUNT=$(printf '%s\n' "$OUT" | sed -n 's/^__COUNT__ //p')
|
|
149
|
+
OUT=$(printf '%s\n' "$OUT" | grep -v '^__COUNT__' || true)
|
|
150
|
+
printf '%s\n' "$OUT"
|
|
151
|
+
|
|
152
|
+
if [[ ${rc:-0} -ne 0 ]] || [[ "$OUT" == *"FAIL"* ]]; then
|
|
153
|
+
FAILED=$((FAILED + 1))
|
|
154
|
+
else
|
|
155
|
+
SCENARIOS=$((SCENARIOS + ${COUNT:-0}))
|
|
156
|
+
fi
|
|
157
|
+
done
|
|
158
|
+
|
|
159
|
+
echo
|
|
160
|
+
printf 'Validated %d behavioral eval file(s); %d scenario(s); %d failed.\n' "$TOTAL" "$SCENARIOS" "$FAILED"
|
|
161
|
+
|
|
162
|
+
if [[ $FAILED -gt 0 ]]; then
|
|
163
|
+
exit 1
|
|
164
|
+
fi
|
|
165
|
+
|
|
166
|
+
echo
|
|
167
|
+
echo "Note: this is shape validation only — it does not execute the scenarios."
|
|
168
|
+
echo "To grade whether the skills actually resist under pressure (live model):"
|
|
169
|
+
echo " see evals/behavioral/README.md — the labeled / nightly rung."
|
|
170
|
+
exit 0
|
package/scripts/run-evals.sh
CHANGED
|
@@ -26,6 +26,9 @@ else
|
|
|
26
26
|
FILES=()
|
|
27
27
|
while IFS= read -r f; do
|
|
28
28
|
[[ "$f" == */README.md ]] && continue
|
|
29
|
+
# Behavioral evals have their own schema + validator (run-behavioral-evals.sh);
|
|
30
|
+
# they are not 20-query trigger evals, so keep them out of this scan.
|
|
31
|
+
[[ "$f" == */behavioral/* ]] && continue
|
|
29
32
|
FILES+=("$f")
|
|
30
33
|
done < <(find "$EVALS_DIR" -type f -name '*.json' | sort)
|
|
31
34
|
fi
|
package/scripts/validate.sh
CHANGED
|
@@ -14,7 +14,7 @@ good() { printf 'ok: %s\n' "$*"; }
|
|
|
14
14
|
|
|
15
15
|
PUBLIC="rite rite-spec rite-adopt rite-temper rite-define rite-vet rite-plan rite-build rite-prove rite-polish rite-review rite-seal rite-ship rite-status rite-doctor rite-resolve rite-pressure-test rite-zoom-out rite-prototype rite-handoff rite-autocomplete"
|
|
16
16
|
INTERNAL="devrites-interview devrites-source-driven devrites-doubt devrites-ux-shape devrites-frontend-craft devrites-browser-proof devrites-debug-recovery devrites-api-interface devrites-audit"
|
|
17
|
-
AGENT_FILES="devrites-spec-reviewer devrites-code-reviewer devrites-test-analyst devrites-frontend-reviewer devrites-security-auditor devrites-performance-reviewer devrites-doubt-reviewer devrites-simplifier-reviewer devrites-strategy-reviewer devrites-plan-reviewer devrites-slice-wright"
|
|
17
|
+
AGENT_FILES="devrites-spec-reviewer devrites-code-reviewer devrites-test-analyst devrites-frontend-reviewer devrites-security-auditor devrites-performance-reviewer devrites-devex-reviewer devrites-doubt-reviewer devrites-simplifier-reviewer devrites-strategy-reviewer devrites-plan-reviewer devrites-retrospector devrites-slice-wright"
|
|
18
18
|
|
|
19
19
|
# ---- 1. bash -n on every shell script ------------------------------------
|
|
20
20
|
section "bash syntax (bash -n)"
|