tiny-spec 0.2.1__tar.gz → 0.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (27) hide show
  1. {tiny_spec-0.2.1 → tiny_spec-0.3.0}/PKG-INFO +75 -19
  2. {tiny_spec-0.2.1 → tiny_spec-0.3.0}/README.md +74 -18
  3. {tiny_spec-0.2.1 → tiny_spec-0.3.0}/agents/tiny-spec-build-executor.md +22 -2
  4. tiny_spec-0.3.0/agents/tiny-spec-build-reviewer.md +125 -0
  5. tiny_spec-0.3.0/docs/eval/README.md +119 -0
  6. {tiny_spec-0.2.1 → tiny_spec-0.3.0}/pyproject.toml +5 -1
  7. {tiny_spec-0.2.1 → tiny_spec-0.3.0}/tiny-spec-breakdown/SKILL.md +70 -6
  8. {tiny_spec-0.2.1 → tiny_spec-0.3.0}/tiny-spec-build/SKILL.md +47 -7
  9. tiny_spec-0.3.0/tiny-spec-create/SKILL.md +428 -0
  10. tiny_spec-0.3.0/tiny-spec-plan/SKILL.md +148 -0
  11. tiny_spec-0.3.0/tiny-spec-prd/SKILL.md +160 -0
  12. tiny_spec-0.3.0/tiny-spec-run/SKILL.md +228 -0
  13. {tiny_spec-0.2.1 → tiny_spec-0.3.0}/tiny-spec-tasks/SKILL.md +64 -6
  14. {tiny_spec-0.2.1 → tiny_spec-0.3.0}/tiny_spec/__init__.py +1 -1
  15. {tiny_spec-0.2.1 → tiny_spec-0.3.0}/tiny_spec/manifest.json +2 -0
  16. tiny_spec-0.2.1/agents/tiny-spec-build-reviewer.md +0 -69
  17. tiny_spec-0.2.1/tiny-spec-breakdown/templates/BREAKDOWN.template.md +0 -37
  18. tiny_spec-0.2.1/tiny-spec-build/templates/memory.template.md +0 -17
  19. tiny_spec-0.2.1/tiny-spec-create/SKILL.md +0 -133
  20. tiny_spec-0.2.1/tiny-spec-create/templates/SPEC.template.md +0 -51
  21. tiny_spec-0.2.1/tiny-spec-create/templates/constitution.template.md +0 -34
  22. tiny_spec-0.2.1/tiny-spec-plan/SKILL.md +0 -70
  23. tiny_spec-0.2.1/tiny-spec-plan/templates/PLAN.template.md +0 -45
  24. tiny_spec-0.2.1/tiny-spec-tasks/templates/tasks.template.md +0 -28
  25. {tiny_spec-0.2.1 → tiny_spec-0.3.0}/.gitignore +0 -0
  26. {tiny_spec-0.2.1 → tiny_spec-0.3.0}/LICENSE +0 -0
  27. {tiny_spec-0.2.1 → tiny_spec-0.3.0}/tiny_spec/cli.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: tiny-spec
3
- Version: 0.2.1
3
+ Version: 0.3.0
4
4
  Summary: A tiny, opinionated take on spec-driven development.
5
5
  Project-URL: Homepage, https://github.com/GrayMa77er/tiny-spec
6
6
  Project-URL: Source, https://github.com/GrayMa77er/tiny-spec
@@ -54,22 +54,30 @@ builds the work one task at a time. Every task is implemented by one agent and
54
54
  graded by an independent reviewer that runs the real tests before anything is
55
55
  committed.
56
56
 
57
- It is four skills and two agents plus an optional Phase-0 on-ramp,
58
- `tiny-spec-breakdown`, for carving a PRD into stories. No orchestrator, no config
59
- file, no build step.
57
+ That core is **four skills and two agents**. In front of it sit **two optional
58
+ planning on-ramps** — `tiny-spec-prd` (idea PRD) and `tiny-spec-breakdown`
59
+ (PRD stories) — for when you're starting from an idea rather than a ready ticket.
60
+ `tiny-spec-run` walks the three planning steps in one command. No config file, no
61
+ build step.
60
62
 
61
63
  ```
62
- tiny-spec-breakdown ⇢ tiny-spec-create → tiny-spec-plan → tiny-spec-tasks → tiny-spec-build
63
- (optional) intent design tasks per-task loop
64
- PRD → stories SPEC.md PLAN.md + tasks.md plan → implement → review → commit
65
- BREAKDOWN.md constitution
64
+ ┌───────────────── tiny-spec-run ─────────────────┐
65
+ optional: one command, stops before build │
66
+ ▼ ▼
67
+ PLANNING (optional on-ramps) EXECUTION (the core loop, one story at a time)
68
+ tiny-spec-prd ⇢ tiny-spec-breakdown ⇢ tiny-spec-create → tiny-spec-plan → tiny-spec-tasks → tiny-spec-build
69
+ idea → PRD PRD → stories intent design tasks per-task loop
70
+ PRD.md BREAKDOWN.md SPEC.md PLAN.md + tasks.md plan → implement → review → commit
71
+ constitution
66
72
  ```
67
73
 
68
- `tiny-spec-breakdown` is optional: skip it and start at `tiny-spec-create` for a
69
- single known piece of work; reach for it when a PRD needs carving into many stories.
70
- It reads a PRD plus wireframes/notes and writes a `BREAKDOWN.md` at your project root
71
- — a flat list of Features → Stories with draft acceptance criteria which
72
- `tiny-spec-create` then reads one story at a time.
74
+ The two on-ramps are **optional** and stack. Have nothing written down? Run
75
+ `tiny-spec-prd` to interview your idea into a `PRD.md`. Have a PRD already? Run
76
+ `tiny-spec-breakdown` to carve it into a `BREAKDOWN.md` a flat list of
77
+ Features → Stories with draft acceptance criteria. Have a single known piece of
78
+ work? Skip both and start at `tiny-spec-create`. Both on-ramps write a regenerable
79
+ file at your project root (not under `.spec/`); `tiny-spec-create` then reads the
80
+ breakdown one story at a time.
73
81
 
74
82
  ## New to spec-driven development?
75
83
 
@@ -93,6 +101,7 @@ uvx tiny-spec install
93
101
  Restart Claude Code so it picks up the new skills, then run the flow in your project:
94
102
 
95
103
  ```
104
+ /tiny-spec-prd # optional: interview a rough idea into a PRD (PRD.md)
96
105
  /tiny-spec-breakdown # optional: carve a PRD + wireframes into stories (BREAKDOWN.md)
97
106
  /tiny-spec-create # capture intent and requirements (binds a ticket, optional)
98
107
  /tiny-spec-plan # turn the spec into a design and harden the constitution
@@ -100,6 +109,18 @@ Restart Claude Code so it picks up the new skills, then run the flow in your pro
100
109
  /tiny-spec-build # build each task: implement, review, commit
101
110
  ```
102
111
 
112
+ Or collapse the three planning steps into one and go straight to building:
113
+
114
+ ```
115
+ /tiny-spec-run # create → plan → tasks in one pass; stops before build
116
+ /tiny-spec-build # build each task: implement, review, commit
117
+ ```
118
+
119
+ `tiny-spec-run` resolves where your ticket stands and invokes whichever of
120
+ `create` / `plan` / `tasks` moves it forward, reconciling anything stale first. It
121
+ **stops before `tiny-spec-build`** — that's where you actually review the work — and
122
+ it writes nothing itself, it only delegates.
123
+
103
124
  Re-run `install` any time to update; `tiny-spec uninstall` removes only what it
104
125
  installed. Each skill is copied (not symlinked) so every install is
105
126
  self-contained.
@@ -115,7 +136,7 @@ git clone https://github.com/GrayMa77er/tiny-spec.git
115
136
  cd tiny-spec
116
137
 
117
138
  mkdir -p "$HOME/.claude/skills" "$HOME/.claude/agents"
118
- for s in tiny-spec-breakdown tiny-spec-create tiny-spec-plan tiny-spec-tasks tiny-spec-build; do
139
+ for s in tiny-spec-prd tiny-spec-breakdown tiny-spec-run tiny-spec-create tiny-spec-plan tiny-spec-tasks tiny-spec-build; do
119
140
  cp -R "$s" "$HOME/.claude/skills/$s"
120
141
  done
121
142
  cp agents/*.md "$HOME/.claude/agents/"
@@ -133,6 +154,33 @@ short interview, `tiny-spec-plan` hardens it with concrete engineering rules, an
133
154
  `tiny-spec-build` injects it whole into every task. It holds your style, standards,
134
155
  invariants, definition of done, and verification commands.
135
156
 
157
+ Because it is project-wide it can also go missing — deleted, or never committed —
158
+ while your specs survive. Re-running `tiny-spec-create` then repairs it: it rebuilds
159
+ the constitution from whatever is already written down and marks completed tasks
160
+ stale, since they were reviewed against a document that wasn't there.
161
+
162
+ ### Designs, if you have them
163
+
164
+ Drop your exported wireframes in a `design/` directory and `tiny-spec-create` will
165
+ look at them — actually look, they are read as images. From all of them together it
166
+ proposes one coherent **design system** (spacing scale, type scale, color roles) for
167
+ your approval and writes it into the constitution, then describes each screen as a
168
+ `D<n>` entry in `SPEC.md`: its layout, its elements with a selector each, and the
169
+ states it must render — all in those token names.
170
+
171
+ The point is what happens at build time. Tag a task with `design: D1` and the
172
+ reviewer renders that surface, measures the selectors the entry names, and **fails the
173
+ task** on a value that isn't on your scale, an element that never got built, or a
174
+ state the design calls for and the code doesn't render. It measures numbers rather
175
+ than diffing screenshots, because pixel diffs go flaky and get muted. Tasks without a
176
+ `design:` tag are graded exactly as before.
177
+
178
+ No Figma token, no plugin, no design SaaS — a view-only Figma works fine, since the
179
+ committed export is what the agents read and the `source:` link keeps the trail back.
180
+ Change an export and its recorded hash stops matching, which marks the spec stale the
181
+ same way editing a requirement does. Skip the whole thing for a CLI or a library; the
182
+ constitution simply has no design section.
183
+
136
184
  `tiny-spec-build` walks the task list top to bottom. Each task runs through one loop:
137
185
 
138
186
  1. Plan the task against the constitution (inline, brief).
@@ -156,11 +204,14 @@ flowchart TB
156
204
 
157
205
  CON([constitution.md]) -.-> P & I & R
158
206
  MEM([memory.md]) -.-> I & R
207
+ DES([SPEC.md D-n + design/ export]) -.->|only on a design: task| I & R
159
208
  ```
160
209
 
161
210
  Solid arrows are the flow. Dotted arrows show the persistent context injected into
162
211
  a step: the `constitution.md` goes into planning, implementation, and review, while
163
- `memory.md` is handed to the executor and reviewer.
212
+ `memory.md` is handed to the executor and reviewer. A task tagged `design:` also
213
+ carries its screen's `D<n>` entry into both agents — and the review step then runs
214
+ the `visual:` gate on top of the usual one.
164
215
 
165
216
  A small `memory.md` carries operational lessons between runs, so the executor and
166
217
  reviewer (which start fresh each time) don't relearn the same pitfalls.
@@ -199,7 +250,7 @@ The case for staying small:
199
250
  optional by design — add shape where it pays, skip it where it doesn't.
200
251
  - **More moving parts is more to maintain.** Orchestrators, ownership contracts,
201
252
  checkpoint matrices, and config files are themselves a system you have to learn
202
- and keep in sync. Four skills and two agents are not.
253
+ and keep in sync. A few small skills and two agents are not.
203
254
  - **Generated docs can fake rigor.** A folder of polished planning artifacts looks
204
255
  like progress, but it isn't proof. The proof is the reviewer running your real
205
256
  tests before each commit.
@@ -209,9 +260,10 @@ independent reviewer and stops.
209
260
 
210
261
  ## Project layout
211
262
 
212
- Each skill is self-contained. It carries its own templates and refers to them by
213
- relative path, with no absolute paths and no shared parent required at runtime, so
214
- a skill folder works wherever you drop it.
263
+ Each skill is one self-contained `SKILL.md`, with every document skeleton inline in
264
+ it — no companion template files, no absolute paths, and no shared parent required
265
+ at runtime, so a skill works wherever you drop it. (It also means a run never stops
266
+ to ask permission to read a template out of your Claude config directory.)
215
267
 
216
268
  tiny-spec creates a `.spec/` directory in your project root, never inside a skill.
217
269
  It is namespaced per ticket, with a shared spine at the root:
@@ -224,6 +276,10 @@ It is namespaced per ticket, with a shared spine at the root:
224
276
  SPEC.md PLAN.md tasks.md decisions.md
225
277
  ```
226
278
 
279
+ `PRD.md`, `BREAKDOWN.md`, and `design/` sit at your project root rather than inside
280
+ `.spec/`, because they are yours: the first two are regenerable pre-spec planning
281
+ files, and the design exports are project files no skill ever writes.
282
+
227
283
  `CONTRACTS.md` documents the formats for maintainers. The skills do not read it at
228
284
  runtime; each is self-sufficient.
229
285
 
@@ -17,22 +17,30 @@ builds the work one task at a time. Every task is implemented by one agent and
17
17
  graded by an independent reviewer that runs the real tests before anything is
18
18
  committed.
19
19
 
20
- It is four skills and two agents plus an optional Phase-0 on-ramp,
21
- `tiny-spec-breakdown`, for carving a PRD into stories. No orchestrator, no config
22
- file, no build step.
20
+ That core is **four skills and two agents**. In front of it sit **two optional
21
+ planning on-ramps** — `tiny-spec-prd` (idea PRD) and `tiny-spec-breakdown`
22
+ (PRD stories) — for when you're starting from an idea rather than a ready ticket.
23
+ `tiny-spec-run` walks the three planning steps in one command. No config file, no
24
+ build step.
23
25
 
24
26
  ```
25
- tiny-spec-breakdown ⇢ tiny-spec-create → tiny-spec-plan → tiny-spec-tasks → tiny-spec-build
26
- (optional) intent design tasks per-task loop
27
- PRD → stories SPEC.md PLAN.md + tasks.md plan → implement → review → commit
28
- BREAKDOWN.md constitution
27
+ ┌───────────────── tiny-spec-run ─────────────────┐
28
+ optional: one command, stops before build │
29
+ ▼ ▼
30
+ PLANNING (optional on-ramps) EXECUTION (the core loop, one story at a time)
31
+ tiny-spec-prd ⇢ tiny-spec-breakdown ⇢ tiny-spec-create → tiny-spec-plan → tiny-spec-tasks → tiny-spec-build
32
+ idea → PRD PRD → stories intent design tasks per-task loop
33
+ PRD.md BREAKDOWN.md SPEC.md PLAN.md + tasks.md plan → implement → review → commit
34
+ constitution
29
35
  ```
30
36
 
31
- `tiny-spec-breakdown` is optional: skip it and start at `tiny-spec-create` for a
32
- single known piece of work; reach for it when a PRD needs carving into many stories.
33
- It reads a PRD plus wireframes/notes and writes a `BREAKDOWN.md` at your project root
34
- — a flat list of Features → Stories with draft acceptance criteria which
35
- `tiny-spec-create` then reads one story at a time.
37
+ The two on-ramps are **optional** and stack. Have nothing written down? Run
38
+ `tiny-spec-prd` to interview your idea into a `PRD.md`. Have a PRD already? Run
39
+ `tiny-spec-breakdown` to carve it into a `BREAKDOWN.md` a flat list of
40
+ Features → Stories with draft acceptance criteria. Have a single known piece of
41
+ work? Skip both and start at `tiny-spec-create`. Both on-ramps write a regenerable
42
+ file at your project root (not under `.spec/`); `tiny-spec-create` then reads the
43
+ breakdown one story at a time.
36
44
 
37
45
  ## New to spec-driven development?
38
46
 
@@ -56,6 +64,7 @@ uvx tiny-spec install
56
64
  Restart Claude Code so it picks up the new skills, then run the flow in your project:
57
65
 
58
66
  ```
67
+ /tiny-spec-prd # optional: interview a rough idea into a PRD (PRD.md)
59
68
  /tiny-spec-breakdown # optional: carve a PRD + wireframes into stories (BREAKDOWN.md)
60
69
  /tiny-spec-create # capture intent and requirements (binds a ticket, optional)
61
70
  /tiny-spec-plan # turn the spec into a design and harden the constitution
@@ -63,6 +72,18 @@ Restart Claude Code so it picks up the new skills, then run the flow in your pro
63
72
  /tiny-spec-build # build each task: implement, review, commit
64
73
  ```
65
74
 
75
+ Or collapse the three planning steps into one and go straight to building:
76
+
77
+ ```
78
+ /tiny-spec-run # create → plan → tasks in one pass; stops before build
79
+ /tiny-spec-build # build each task: implement, review, commit
80
+ ```
81
+
82
+ `tiny-spec-run` resolves where your ticket stands and invokes whichever of
83
+ `create` / `plan` / `tasks` moves it forward, reconciling anything stale first. It
84
+ **stops before `tiny-spec-build`** — that's where you actually review the work — and
85
+ it writes nothing itself, it only delegates.
86
+
66
87
  Re-run `install` any time to update; `tiny-spec uninstall` removes only what it
67
88
  installed. Each skill is copied (not symlinked) so every install is
68
89
  self-contained.
@@ -78,7 +99,7 @@ git clone https://github.com/GrayMa77er/tiny-spec.git
78
99
  cd tiny-spec
79
100
 
80
101
  mkdir -p "$HOME/.claude/skills" "$HOME/.claude/agents"
81
- for s in tiny-spec-breakdown tiny-spec-create tiny-spec-plan tiny-spec-tasks tiny-spec-build; do
102
+ for s in tiny-spec-prd tiny-spec-breakdown tiny-spec-run tiny-spec-create tiny-spec-plan tiny-spec-tasks tiny-spec-build; do
82
103
  cp -R "$s" "$HOME/.claude/skills/$s"
83
104
  done
84
105
  cp agents/*.md "$HOME/.claude/agents/"
@@ -96,6 +117,33 @@ short interview, `tiny-spec-plan` hardens it with concrete engineering rules, an
96
117
  `tiny-spec-build` injects it whole into every task. It holds your style, standards,
97
118
  invariants, definition of done, and verification commands.
98
119
 
120
+ Because it is project-wide it can also go missing — deleted, or never committed —
121
+ while your specs survive. Re-running `tiny-spec-create` then repairs it: it rebuilds
122
+ the constitution from whatever is already written down and marks completed tasks
123
+ stale, since they were reviewed against a document that wasn't there.
124
+
125
+ ### Designs, if you have them
126
+
127
+ Drop your exported wireframes in a `design/` directory and `tiny-spec-create` will
128
+ look at them — actually look, they are read as images. From all of them together it
129
+ proposes one coherent **design system** (spacing scale, type scale, color roles) for
130
+ your approval and writes it into the constitution, then describes each screen as a
131
+ `D<n>` entry in `SPEC.md`: its layout, its elements with a selector each, and the
132
+ states it must render — all in those token names.
133
+
134
+ The point is what happens at build time. Tag a task with `design: D1` and the
135
+ reviewer renders that surface, measures the selectors the entry names, and **fails the
136
+ task** on a value that isn't on your scale, an element that never got built, or a
137
+ state the design calls for and the code doesn't render. It measures numbers rather
138
+ than diffing screenshots, because pixel diffs go flaky and get muted. Tasks without a
139
+ `design:` tag are graded exactly as before.
140
+
141
+ No Figma token, no plugin, no design SaaS — a view-only Figma works fine, since the
142
+ committed export is what the agents read and the `source:` link keeps the trail back.
143
+ Change an export and its recorded hash stops matching, which marks the spec stale the
144
+ same way editing a requirement does. Skip the whole thing for a CLI or a library; the
145
+ constitution simply has no design section.
146
+
99
147
  `tiny-spec-build` walks the task list top to bottom. Each task runs through one loop:
100
148
 
101
149
  1. Plan the task against the constitution (inline, brief).
@@ -119,11 +167,14 @@ flowchart TB
119
167
 
120
168
  CON([constitution.md]) -.-> P & I & R
121
169
  MEM([memory.md]) -.-> I & R
170
+ DES([SPEC.md D-n + design/ export]) -.->|only on a design: task| I & R
122
171
  ```
123
172
 
124
173
  Solid arrows are the flow. Dotted arrows show the persistent context injected into
125
174
  a step: the `constitution.md` goes into planning, implementation, and review, while
126
- `memory.md` is handed to the executor and reviewer.
175
+ `memory.md` is handed to the executor and reviewer. A task tagged `design:` also
176
+ carries its screen's `D<n>` entry into both agents — and the review step then runs
177
+ the `visual:` gate on top of the usual one.
127
178
 
128
179
  A small `memory.md` carries operational lessons between runs, so the executor and
129
180
  reviewer (which start fresh each time) don't relearn the same pitfalls.
@@ -162,7 +213,7 @@ The case for staying small:
162
213
  optional by design — add shape where it pays, skip it where it doesn't.
163
214
  - **More moving parts is more to maintain.** Orchestrators, ownership contracts,
164
215
  checkpoint matrices, and config files are themselves a system you have to learn
165
- and keep in sync. Four skills and two agents are not.
216
+ and keep in sync. A few small skills and two agents are not.
166
217
  - **Generated docs can fake rigor.** A folder of polished planning artifacts looks
167
218
  like progress, but it isn't proof. The proof is the reviewer running your real
168
219
  tests before each commit.
@@ -172,9 +223,10 @@ independent reviewer and stops.
172
223
 
173
224
  ## Project layout
174
225
 
175
- Each skill is self-contained. It carries its own templates and refers to them by
176
- relative path, with no absolute paths and no shared parent required at runtime, so
177
- a skill folder works wherever you drop it.
226
+ Each skill is one self-contained `SKILL.md`, with every document skeleton inline in
227
+ it — no companion template files, no absolute paths, and no shared parent required
228
+ at runtime, so a skill works wherever you drop it. (It also means a run never stops
229
+ to ask permission to read a template out of your Claude config directory.)
178
230
 
179
231
  tiny-spec creates a `.spec/` directory in your project root, never inside a skill.
180
232
  It is namespaced per ticket, with a shared spine at the root:
@@ -187,6 +239,10 @@ It is namespaced per ticket, with a shared spine at the root:
187
239
  SPEC.md PLAN.md tasks.md decisions.md
188
240
  ```
189
241
 
242
+ `PRD.md`, `BREAKDOWN.md`, and `design/` sit at your project root rather than inside
243
+ `.spec/`, because they are yours: the first two are regenerable pre-spec planning
244
+ files, and the design exports are project files no skill ever writes.
245
+
190
246
  `CONTRACTS.md` documents the formats for maintainers. The skills do not read it at
191
247
  runtime; each is self-sufficient.
192
248
 
@@ -18,9 +18,12 @@ Everything you need and nothing you don't:
18
18
  - the **task id**, **description**, and **acceptance** (the outcome that proves it done);
19
19
  - a **`files:` hint** — likely paths to touch (guidance, not a hard boundary);
20
20
  - the full **constitution** (`constitution.md`): Style, Engineering standards,
21
- Guiding invariants, Glossary, Layout, Definition of Done, Verification commands;
21
+ Guiding invariants, Glossary, Layout, Definition of Done, Verification commands,
22
+ and — for projects with a visual surface — the **Design system** token table;
22
23
  - the project's **memory** if any (`memory.md`) — operational lessons; honor them
23
24
  so you don't re-learn a pitfall a past run already paid for;
25
+ - **if the task carries `design:`** — that screen's `D<n>` entry from `SPEC.md` and
26
+ the path to its committed export;
24
27
  - the specific existing files that are your starting point, named explicitly.
25
28
 
26
29
  You are **blind to the workflow, not to the codebase.** You don't get the plan,
@@ -41,7 +44,24 @@ violated.
41
44
  task genuinely needs a nearby file the hint missed, that's fine (you're
42
45
  sequential, no one else is writing). But do **not** refactor unrelated code or
43
46
  implement adjacent tasks — that's scope creep, not thoroughness.
44
- 5. You MAY run a **narrow self-check** of your own work (the one test file you
47
+ 5. **If the task carries a `design:` reference**, build that surface against the
48
+ `D<n>` entry: **look at the export image** (`Read` renders it), then implement its
49
+ `layout:`, every row of its `elements:`, and **every state it names** — empty,
50
+ loading, error, success. Missing states are the most common way generated UI
51
+ passes a functional check and is still wrong.
52
+ - **The `elements:` selectors are a contract, not a hint** — unlike `files:`. The
53
+ reviewer measures exactly those selectors, so a `[data-testid="signup-email"]`
54
+ row means your markup carries that attribute verbatim. Rename one and the check
55
+ silently measures nothing. If a selector is genuinely wrong for this codebase
56
+ (a component library owns the markup, the id collides), that is a **blocker** —
57
+ report it so the spec gets fixed. Never quietly substitute your own.
58
+ - **Reference tokens, never raw values.** `space.4`, `color.text.muted` — not
59
+ `16px`, not `#6B7280`. A value that isn't on the constitution's scale will fail
60
+ review, and inventing a token the Design system doesn't define is a blocker,
61
+ not a judgement call: report it and let the constitution be fixed.
62
+ - The image is the *intent*; the `D<n>` entry and the token table are the
63
+ *contract*. Where they disagree, the contract wins and you say so in `DECISIONS`.
64
+ 6. You MAY run a **narrow self-check** of your own work (the one test file you
45
65
  wrote, a syntax/import check). You do **not** need to run the full gate — the
46
66
  independent **reviewer** runs the authoritative Verification commands next.
47
67
  Leave the tree in a clean, buildable state for it.
@@ -0,0 +1,125 @@
1
+ ---
2
+ name: tiny-spec-build-reviewer
3
+ description: Independently reviews a single finished task — runs the project's real gate end-to-end and checks the code against the constitution and the task's acceptance. Blind to how the code was written. Returns PASS/FAIL plus findings. Spawned (one per task) by tiny-spec-build. Does not fix code, plan, spawn agents, or invoke skills.
4
+ tools: Read, Write, Edit, Bash, Grep, Glob
5
+ ---
6
+
7
+ # tiny-spec-build-reviewer
8
+
9
+ You independently review **one finished task** from a build. You did
10
+ **not** write this code and you have no memory of how it was written — that
11
+ independence is the whole point. Your final message **is** the structured verdict
12
+ back to `tiny-spec-build`; return data, not pleasantries.
13
+
14
+ ## What you receive (the context contract)
15
+
16
+ - the **task id**, **description**, and **acceptance** (the outcome that must hold);
17
+ - the full **constitution** (`constitution.md`) — especially **Guiding invariants**,
18
+ **Definition of Done**, and **Verification commands**, plus the **Design system**
19
+ token table if the project has one;
20
+ - the project's **memory** if any (`memory.md`) — operational lessons (e.g. the
21
+ gate needs the package installed first); honor them so you don't false-fail on a
22
+ known precondition;
23
+ - **if the task carries `design:`** — that screen's `D<n>` entry from `SPEC.md` and
24
+ the path to its committed export;
25
+ - the list of **changed files** to review.
26
+
27
+ ## How to review
28
+
29
+ Your job is to answer one question honestly: **does this task actually satisfy its
30
+ acceptance and the constitution — verified, not inferred?**
31
+
32
+ 1. **Read the changed code.** Check it against the constitution: does it honor the
33
+ **Guiding invariants**, match the **Style** and **Layout**, meet the
34
+ **Definition of Done**? Note any violation as a finding.
35
+ 2. **Run the real gate.** Execute the constitution's **Verification commands**
36
+ end-to-end (install → lint → test → build → run, as applicable) from a clean
37
+ state, after the documented setup — not a test-runner shortcut. Capture the
38
+ real output.
39
+ 3. **Exercise the acceptance.** Trigger the task's stated outcome the most
40
+ black-box way available (CLI > HTTP > public API) with realistic input,
41
+ including a negative case if the acceptance implies a boundary or rejection.
42
+ The acceptance is met only if the **observed** effect is the one it names — not
43
+ an adjacent or merely-plausible behavior. "A unit test exists" or "the code
44
+ looks right" is **not** evidence.
45
+ 4. **Measure the design — only if the task carries `design:`.** Skip this step
46
+ entirely otherwise; a task without the field is graded exactly as steps 1–3.
47
+
48
+ First check the constitution has a `visual:` verification command. If it does
49
+ **not**, stop and report a **blocker** (`VERDICT: FAIL`, findings naming the
50
+ missing command). Do not fall back to eyeballing and do not pass the task — a
51
+ design gate that silently doesn't run is worse than no gate, because the checked
52
+ box then claims a verification that never happened.
53
+
54
+ Otherwise:
55
+ - Run the `visual:` command against **every selector in the entry's `elements:`
56
+ list** and read back the **real numbers** — `getComputedStyle()` and
57
+ `getBoundingClientRect()`. Measure the selectors the entry names; do not
58
+ substitute ones you think are equivalent, and do not measure a sample.
59
+ - **A selector that matches nothing is a `FAIL`, never a skip.** Either the code
60
+ didn't build the element or it named it something else — both are real, and both
61
+ are invisible if you quietly move on. Report the selector and that it was absent.
62
+ - Compare each element's numbers to the tokens its row names. Report concrete
63
+ deltas ("heading is 28px, `type.heading.lg` is 24px"; "padding 19px is not on
64
+ the `space.*` scale"). **Do not pixel-diff the screenshot** — font antialiasing
65
+ makes image comparison flaky enough that the check gets ignored, which is
66
+ exactly how visual gates die.
67
+ - **Check `layout:`** — the arrangement, max width, and the **order** it names.
68
+ Use the bounding rectangles: elements listed in order should appear in that
69
+ order down the page (or across it, for a row). Every token can be correct on an
70
+ element that is in the wrong place.
71
+ - **Exercise every state the entry names** — empty, loading, error, success.
72
+ Drive the UI into each one and observe what changes; **finding the word in the
73
+ source is not evidence**, and it false-passes routinely (a comment saying the
74
+ loading state is missing contains "loading"). A surface that renders its happy
75
+ path and nothing else is a fail, not a nit.
76
+ - Finally, `Read` the export image as a cross-check for what numbers can't catch
77
+ (a missing element, wrong order, wrong hierarchy).
78
+
79
+ ## Verdict rules
80
+
81
+ - **`PASS`** — the gate is green AND you exercised the acceptance end-to-end with
82
+ real input AND the observed effect matches AND no invariant/DoD violation — AND,
83
+ on a task carrying `design:`, step 4 ran and found nothing measurable wrong. Only
84
+ this is a pass.
85
+ - **`FAIL`** — anything short of the above: a red gate, an invariant violated, the
86
+ acceptance not observably met, or you couldn't exercise it end-to-end. When torn,
87
+ **fail** — never round up. List concrete, actionable findings so the executor
88
+ can fix them.
89
+
90
+ **The design step splits the verdict differently — this is deliberate.** On a task
91
+ carrying `design:`:
92
+
93
+ - **`FAIL` on what you measured**: a selector from `elements:` that matches nothing,
94
+ a value off the token scale, a token the Design system doesn't define, a hardcoded
95
+ color/spacing where a token exists, an order or arrangement that contradicts
96
+ `layout:`, or a state the `D<n>` entry names that doesn't render. These are
97
+ objective and an executor can fix them from your numbers.
98
+ - **Flag, don't fail, on taste.** "The hierarchy feels off", "spacing looks cramped
99
+ but is on-scale" — put it in `FINDINGS` prefixed `flag:` and pass if everything
100
+ measurable is green. The fix loop is bounded at two attempts; burning it on a
101
+ subjective disagreement means the task blocks on something no executor can resolve.
102
+ - The "when torn, fail" rule still governs steps 1–3 unchanged. It does **not** apply
103
+ to a subjective visual impression.
104
+
105
+ You are **read-only on the source** — you run commands and read files, but you do
106
+ **not** edit code, fix the task, or rewrite docs. If it's wrong, you report it; the
107
+ executor fixes it on the next attempt. (You have edit tools only so you can run
108
+ gates that scratch-write build output — never use them on source.)
109
+
110
+ Never spawn subagents or invoke skills.
111
+
112
+ ## Report back (your final message)
113
+
114
+ ```
115
+ TASK: <task id>
116
+ VERDICT: PASS | FAIL
117
+ GATE: <the Verification commands you ran + the real result (pass/fail + key output)>
118
+ ACCEPTANCE: <how you exercised it + the observed effect, or why you couldn't>
119
+ DESIGN: <omit unless the task carried `design:`. The D<n> checked, the measurements
120
+ you read back vs the tokens they should match, and which states you exercised —
121
+ or "blocker: no `visual:` command in the constitution">
122
+ FINDINGS:
123
+ - <each invariant/DoD/acceptance problem, concrete and actionable> (omit if PASS)
124
+ - flag: <subjective visual note — does not fail the task> (only with a DESIGN section)
125
+ ```
@@ -0,0 +1,119 @@
1
+ # tiny-spec evaluation
2
+
3
+ Three complementary evaluations of the tiny-spec suite, plus a shared score history.
4
+ All are **maintainer-only dev tools** — none of this is in `tiny_spec/manifest.json`,
5
+ so nothing here ships to users.
6
+
7
+ | Eval | Question it answers | How | Drives |
8
+ |------|---------------------|-----|--------|
9
+ | **Static review** — `/eval-suite` | Does the *design* have the right mechanisms? | Reads `CONTRACTS.md`, agents, SKILL.md; scores 0–3 against the rubric. | Dimensions **B, C, D** |
10
+ | **Empirical harness** — `harness/run.sh` | Does the *execution* stage produce working code, and does its own gate tell the truth? | Runs `create → plan → tasks → build` on benchmark tasks and grades the output with held-out tests. | Dimension **A** (auto-derived) |
11
+ | **Planning harness** — `harness/run-planning.sh` | Does the *planning* stage produce a sound, faithful hand-off? | Runs `tiny-spec-prd → tiny-spec-breakdown` on loose ideas; grades the PRD/BREAKDOWN with structural checks + an LLM judge. | `planning-results.jsonl` (standalone) |
12
+
13
+ The rubric, criteria, weights, and worked rationale live in
14
+ [`../sdd-evaluation-rubric.md`](../sdd-evaluation-rubric.md).
15
+
16
+ ## Files
17
+
18
+ ```
19
+ docs/eval/
20
+ scores.jsonl append-only scorecard history (the A–D score over time)
21
+ results.jsonl append-only per-run empirical detail (created on first run)
22
+ planning-results.jsonl append-only planning-eval detail (created on first run)
23
+ benchmark/<task>/ execution-stage tasks
24
+ TICKET.md the prompt handed to tiny-spec (NO hidden tests)
25
+ grade/test.py held-out grader — the suite never sees this
26
+ planning/<case>/ planning-stage cases
27
+ IDEA.md a loose product idea handed to the planning stage
28
+ harness/
29
+ run.sh execution: sandbox -> headless build -> held-out grade -> score
30
+ score.py metrics + Dimension A thresholds + merge into scores.jsonl
31
+ run-planning.sh planning: sandbox -> headless prd+breakdown -> grade -> score
32
+ grade_planning.py structural checks + LLM judge -> one result object per case
33
+ score_planning.py aggregate -> planning-results.jsonl (does NOT touch scores.jsonl)
34
+ ```
35
+
36
+ ## Running the empirical eval
37
+
38
+ Prereqs: `claude` CLI, `python3`, `git`. The harness vendors the repo's skills+agents
39
+ into each sandbox's local `.claude/`, so you do **not** need tiny-spec globally installed.
40
+
41
+ ```sh
42
+ docs/eval/harness/run.sh # all tasks
43
+ docs/eval/harness/run.sh roman duration # a subset
44
+ CLAUDE_MODEL=claude-sonnet-4-6 docs/eval/harness/run.sh roman # pick a model
45
+ KEEP_SANDBOX=1 docs/eval/harness/run.sh roman # keep sandbox to inspect
46
+ ```
47
+
48
+ Each task runs in a throwaway sandbox: a fresh git repo seeded only with `TICKET.md`
49
+ and the vendored skills. `claude -p` drives `create → plan → tasks → build`
50
+ autonomously; then the held-out grader runs against the produced `solution.py`.
51
+
52
+ ### What gets measured
53
+
54
+ - **held-out pass rate** — the real correctness signal (truth = the grader).
55
+ - **suite/truth agreement** & **FALSE-PASS rate** — does tiny-spec's *own* verdict (all
56
+ tasks ticked, no open blocker) match reality? A FALSE-PASS (suite says done, grader
57
+ fails) is the worst outcome; it directly tests the suite's core claim that "an
58
+ independent reviewer running the real gate is the safeguard."
59
+ - completion rate, blocker rate.
60
+
61
+ ### How Dimension A is derived (thresholds in `score.py`)
62
+
63
+ - **A1 measurability** → 3 once a pass-rate is produced.
64
+ - **A3 reproducible harness** → 3 once the harness has run.
65
+ - **A2 gate effectiveness** → 3 if 0 false-PASS & agreement ≥ 90%; 2 if ≤ 20% false-PASS
66
+ & agreement ≥ 70%; 1 otherwise; 0 if nothing completed.
67
+ - **A4 regression visibility** → 3 once ≥ 2 empirical runs are on record, else 2.
68
+
69
+ B/C/D are carried over from the latest `/eval-suite` static scorecard; the empirical run
70
+ swaps in the A scores and recomputes the weighted total, appending a new line tagged
71
+ `"scored_by": "eval-output"`.
72
+
73
+ ## Running the planning eval
74
+
75
+ ```sh
76
+ docs/eval/harness/run-planning.sh # all planning cases
77
+ docs/eval/harness/run-planning.sh snip # a subset
78
+ JUDGE_MODEL=claude-sonnet-4-6 docs/eval/harness/run-planning.sh # pick the judge model
79
+ KEEP_SANDBOX=1 docs/eval/harness/run-planning.sh snip # keep sandbox to inspect
80
+ ```
81
+
82
+ Each case runs in a throwaway sandbox seeded with `IDEA.md` and the vendored skills.
83
+ `claude -p` drives `tiny-spec-prd → tiny-spec-breakdown` (it must **not** scaffold
84
+ `.spec/` or run any later skill), then `grade_planning.py` scores the produced
85
+ `PRD.md` + `BREAKDOWN.md`.
86
+
87
+ ### What gets measured
88
+
89
+ - **structural conformance** (deterministic) — PRD has its required sections filled;
90
+ BREAKDOWN has a Decisions block, ≥1 Feature, Stories with a slug and ≥1 AC; the
91
+ planning skills left no `.spec/` behind.
92
+ - **hand-off integrity** (LLM judge) — **coverage** (every PRD capability lands in ≥1
93
+ story, nothing dropped) and **no fabrication** (every story traces to a capability,
94
+ nothing invented). These are the high-value signals; a case PASSes only if structural
95
+ conformance holds *and* the judge confirms both.
96
+ - **quality** (LLM judge, reported not gated) — atomicity / user-observable phrasing,
97
+ cross-cutting concerns placed in Decisions rather than as their own Feature, and
98
+ whether the PRD faithfully reflects the idea.
99
+
100
+ Results append to `planning-results.jsonl`. This scorecard is **standalone** — it does
101
+ not feed the A–D `scores.jsonl`, because planning quality and execution correctness are
102
+ separate claims.
103
+
104
+ ## Caveats
105
+
106
+ - **Cost/time:** each task runs the full multi-agent flow headlessly — minutes and real
107
+ tokens per task. Start with a subset.
108
+ - **Autonomy:** `--dangerously-skip-permissions` is used so the sandbox run is
109
+ non-interactive. It runs in a throwaway dir, never your repo.
110
+ - **Small benchmark:** 5 self-contained Python tasks. It measures the suite's loop on
111
+ small, well-specified work — not large-codebase performance. To approximate the
112
+ literature (Spec Kit Agents on SWE-bench Lite), add harder tasks under `benchmark/`;
113
+ the grader contract (`grade/test.py` reading `$SOLUTION_PATH`) is all a new task needs.
114
+ - **Judgment vs measurement:** Dimension A here is *measured*; B/C/D remain design
115
+ judgment from `/eval-suite`.
116
+ - **Planning eval uses an LLM judge:** coverage/fabrication/atomicity are judged by a
117
+ model, so verdicts are not bit-reproducible the way held-out tests are. The
118
+ deterministic structural layer *is* reproducible; the judge adds the semantic signal
119
+ the structure can't see. Add planning cases by dropping a new `planning/<case>/IDEA.md`.
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
4
4
 
5
5
  [project]
6
6
  name = "tiny-spec"
7
- version = "0.2.1"
7
+ version = "0.3.0"
8
8
  description = "A tiny, opinionated take on spec-driven development."
9
9
  readme = "README.md"
10
10
  requires-python = ">=3.9"
@@ -34,7 +34,9 @@ packages = ["tiny_spec"]
34
34
  # expects them). Bundle them into the wheel under tiny_spec/_bundle/ so the
35
35
  # installer can copy them at runtime, without restructuring the repo.
36
36
  [tool.hatch.build.targets.wheel.force-include]
37
+ "tiny-spec-prd" = "tiny_spec/_bundle/tiny-spec-prd"
37
38
  "tiny-spec-breakdown" = "tiny_spec/_bundle/tiny-spec-breakdown"
39
+ "tiny-spec-run" = "tiny_spec/_bundle/tiny-spec-run"
38
40
  "tiny-spec-create" = "tiny_spec/_bundle/tiny-spec-create"
39
41
  "tiny-spec-plan" = "tiny_spec/_bundle/tiny-spec-plan"
40
42
  "tiny-spec-tasks" = "tiny_spec/_bundle/tiny-spec-tasks"
@@ -44,6 +46,8 @@ packages = ["tiny_spec"]
44
46
  [tool.hatch.build.targets.sdist]
45
47
  include = [
46
48
  "tiny_spec",
49
+ "tiny-spec-prd",
50
+ "tiny-spec-run",
47
51
  "tiny-spec-breakdown",
48
52
  "tiny-spec-create",
49
53
  "tiny-spec-plan",