spec-eval 0.2.0__tar.gz → 0.2.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (22) hide show
  1. {spec_eval-0.2.0/spec_eval.egg-info → spec_eval-0.2.1}/PKG-INFO +38 -20
  2. {spec_eval-0.2.0 → spec_eval-0.2.1}/README.md +37 -19
  3. {spec_eval-0.2.0 → spec_eval-0.2.1}/spec_eval/__init__.py +1 -1
  4. {spec_eval-0.2.0 → spec_eval-0.2.1/spec_eval.egg-info}/PKG-INFO +38 -20
  5. {spec_eval-0.2.0 → spec_eval-0.2.1}/LICENSE +0 -0
  6. {spec_eval-0.2.0 → spec_eval-0.2.1}/pyproject.toml +0 -0
  7. {spec_eval-0.2.0 → spec_eval-0.2.1}/setup.cfg +0 -0
  8. {spec_eval-0.2.0 → spec_eval-0.2.1}/spec_eval/__main__.py +0 -0
  9. {spec_eval-0.2.0 → spec_eval-0.2.1}/spec_eval/audit.py +0 -0
  10. {spec_eval-0.2.0 → spec_eval-0.2.1}/spec_eval/authoring.py +0 -0
  11. {spec_eval-0.2.0 → spec_eval-0.2.1}/spec_eval/cli.py +0 -0
  12. {spec_eval-0.2.0 → spec_eval-0.2.1}/spec_eval/coverage.py +0 -0
  13. {spec_eval-0.2.0 → spec_eval-0.2.1}/spec_eval/providers.py +0 -0
  14. {spec_eval-0.2.0 → spec_eval-0.2.1}/spec_eval/report.py +0 -0
  15. {spec_eval-0.2.0 → spec_eval-0.2.1}/spec_eval/rubric.py +0 -0
  16. {spec_eval-0.2.0 → spec_eval-0.2.1}/spec_eval/runlog.py +0 -0
  17. {spec_eval-0.2.0 → spec_eval-0.2.1}/spec_eval/sufficiency.py +0 -0
  18. {spec_eval-0.2.0 → spec_eval-0.2.1}/spec_eval.egg-info/SOURCES.txt +0 -0
  19. {spec_eval-0.2.0 → spec_eval-0.2.1}/spec_eval.egg-info/dependency_links.txt +0 -0
  20. {spec_eval-0.2.0 → spec_eval-0.2.1}/spec_eval.egg-info/entry_points.txt +0 -0
  21. {spec_eval-0.2.0 → spec_eval-0.2.1}/spec_eval.egg-info/requires.txt +0 -0
  22. {spec_eval-0.2.0 → spec_eval-0.2.1}/spec_eval.egg-info/top_level.txt +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: spec-eval
3
- Version: 0.2.0
3
+ Version: 0.2.1
4
4
  Summary: A trust-eval for your specs: coverage, drift, and sufficiency of markdown specs against code.
5
5
  Author: Ben Jones
6
6
  License: MIT
@@ -35,7 +35,7 @@ Dynamic: license-file
35
35
 
36
36
  *A trust-eval for your specs — coverage, drift, and sufficiency, measured against the code.*
37
37
 
38
- [Get started](#try-it) · [Tutorial](GETTING-STARTED.md) · [No API key?](#run-via-prompt-chat-no-setup) · [Second opinion](#higher-stakes-get-a-second-opinion) · [FAQ](FAQ.md)
38
+ [Get started](#try-it) · [Tutorial](https://github.com/benjaminjones/spec-eval/blob/main/GETTING-STARTED.md) · [No API key?](#run-via-prompt-chat-no-setup) · [Second opinion](#higher-stakes-get-a-second-opinion) · [FAQ](https://github.com/benjaminjones/spec-eval/blob/main/FAQ.md)
39
39
 
40
40
  A **spec** is short for **specification** — a plain-English description of what your code should do. **spec-eval**
41
41
  keeps one beside your code — per file, or per folder — and checks the two still agree, so the spec stays worth
@@ -98,7 +98,7 @@ Everything it writes is plain markdown, right next to your code:
98
98
 
99
99
  ## Try it
100
100
 
101
- Prefer a guided walkthrough? **[Getting-started tutorial →](GETTING-STARTED.md)**
101
+ Prefer a guided walkthrough? **[Getting-started tutorial →](https://github.com/benjaminjones/spec-eval/blob/main/GETTING-STARTED.md)**
102
102
 
103
103
  **The commands** — in the order you'll typically use them:
104
104
 
@@ -116,12 +116,30 @@ Three ways to run them, least setup first:
116
116
  ### Run via prompt chat (no setup)
117
117
 
118
118
  Ask the coding agent you already have (Claude Code, Copilot, Cursor, …) — nothing to install, no key, answers
119
- land in the chat:
119
+ land in the chat. Two skills, one per job — point your agent at the right one:
120
+
121
+ **Write specs** — the [authoring skill](https://github.com/benjaminjones/spec-eval/blob/main/skills/spec-authoring/SKILL.md) carries the spec structure
122
+ (the `INV-*` invariant and `AC-*` acceptance-criteria tables come from its rubric and templates):
123
+ ```text
124
+ Read path/to/spec-eval/skills/spec-authoring/SKILL.md — and the templates
125
+ in its templates/ folder — and follow it: write specs for my code.
126
+ ```
127
+
128
+ **Check specs** — the [checking skill](https://github.com/benjaminjones/spec-eval/blob/main/skills/spec-check/SKILL.md) grades what exists (drift + sufficiency):
120
129
  ```text
121
130
  Read path/to/spec-eval/skills/spec-check/SKILL.md and follow it —
122
131
  first check coverage, then check my specs against my code.
123
132
  ```
124
- The path is wherever you cloned this repo — no clone? Give your agent [the file's link](skills/spec-check/SKILL.md).
133
+
134
+ > [!NOTE]
135
+ > `path/to/spec-eval` is wherever you cloned this repo. No clone? Give your agent the skill's link by name:
136
+ > [`spec-authoring/SKILL.md`](https://github.com/benjaminjones/spec-eval/blob/main/skills/spec-authoring/SKILL.md) to write specs,
137
+ > [`spec-check/SKILL.md`](https://github.com/benjaminjones/spec-eval/blob/main/skills/spec-check/SKILL.md) to check them.
138
+
139
+ > [!TIP]
140
+ > **Author and check in separate sessions.** A checker that still remembers its authoring decisions tends to
141
+ > echo them instead of re-reading the code — the same reason a PR isn't reviewed by its author. Write specs in
142
+ > one session; open a fresh one to check them.
125
143
 
126
144
  Optionally save reports — add:
127
145
  ```text
@@ -129,7 +147,7 @@ and save the results to spec-reports/
129
147
  ```
130
148
  The agent writes them itself (its own scores; the terminal adds the exact coverage % and a run history).
131
149
 
132
- **Make it a standing command** *(optional, one-time)* — copy the [`skills/`](skills/) into your agent's skills
150
+ **Make it a standing command** *(optional, one-time)* — copy the [`skills/`](https://github.com/benjaminjones/spec-eval/tree/main/skills/) into your agent's skills
133
151
  folder (Claude Code: `.claude/skills/`). Then the prompt is the command:
134
152
  ```bash
135
153
  mkdir -p .claude/skills && cp -r path/to/spec-eval/skills/* .claude/skills/
@@ -164,7 +182,7 @@ export ANTHROPIC_API_KEY=sk-ant-... # or OPENAI_API_KEY / GOOGLE_API_KEY
164
182
 
165
183
  > [!NOTE]
166
184
  > A typical first run is `coverage → generate → sufficiency`, then at every milestone after: `audit` — plus
167
- > `sufficiency` when you've added behavior. The [tutorial](GETTING-STARTED.md) has step-by-step guides for
185
+ > `sufficiency` when you've added behavior. The [tutorial](https://github.com/benjaminjones/spec-eval/blob/main/GETTING-STARTED.md) has step-by-step guides for
168
186
  > starting with no specs and for existing docs.
169
187
 
170
188
  > [!TIP]
@@ -175,7 +193,7 @@ export ANTHROPIC_API_KEY=sk-ant-... # or OPENAI_API_KEY / GOOGLE_API_KEY
175
193
  A few things worth knowing:
176
194
 
177
195
  - **No config needed** — by default each spec pairs with the code file of the same name beside it (`parser.md` ↔ `parser.py`).
178
- Prefer one spec per folder? `--layout per-dir` ([FAQ](FAQ.md#can-i-use-one-spec-per-folder-instead-of-one-per-file)).
196
+ Prefer one spec per folder? `--layout per-dir` ([FAQ](https://github.com/benjaminjones/spec-eval/blob/main/FAQ.md#can-i-use-one-spec-per-folder-instead-of-one-per-file)).
179
197
  - **Any language, three providers** — swap the model with `--model openai:…`, `google:…`, or `claude-code`
180
198
  (Anthropic is the default); code is read as plain text. Want another provider? [Open an issue](#feedback).
181
199
  - **Your other docs are safe** — by default only a `.md` next to code with the same name counts as a spec, so your
@@ -183,7 +201,7 @@ A few things worth knowing:
183
201
  - **What's shared** — to grade a spec, its code and text go to your AI provider. That's it. (`coverage` sends nothing — it runs fully on your machine.)
184
202
 
185
203
  **Keep it honest on every commit** — add one line to a git hook (a script git runs before each commit) so new
186
- code always needs a spec; copy-paste setup in [the tutorial](GETTING-STARTED.md#make-it-routine):
204
+ code always needs a spec; copy-paste setup in [the tutorial](https://github.com/benjaminjones/spec-eval/blob/main/GETTING-STARTED.md#make-it-routine):
187
205
  ```bash
188
206
  spec-eval coverage . --min 90 # fails the commit if coverage drops below 90% (free — no AI)
189
207
  ```
@@ -214,20 +232,20 @@ the pair against a fixed rubric — one model call per pair. Two things follow f
214
232
  > your coding agent's own usage if you asked in chat. Every CLI run prints its exact call and token counts.
215
233
 
216
234
  **Want the exact rules behind each score?** spec-eval specs its own scoring — read
217
- [`spec_eval/coverage.md`](spec_eval/coverage.md) (how coverage counts),
218
- [`spec_eval/rubric.md`](spec_eval/rubric.md) (the drift rules), and
219
- [`spec_eval/sufficiency.md`](spec_eval/sufficiency.md) (how the 0–1 score is decided).
235
+ [`spec_eval/coverage.md`](https://github.com/benjaminjones/spec-eval/blob/main/spec_eval/coverage.md) (how coverage counts),
236
+ [`spec_eval/rubric.md`](https://github.com/benjaminjones/spec-eval/blob/main/spec_eval/rubric.md) (the drift rules), and
237
+ [`spec_eval/sufficiency.md`](https://github.com/benjaminjones/spec-eval/blob/main/spec_eval/sufficiency.md) (how the 0–1 score is decided).
220
238
 
221
239
  ## Reading the scores
222
240
 
223
241
  Each run writes a short report to `spec-reports/` (a `.md` you read, a `.json` for tools):
224
242
 
225
- - **coverage** — the % of your code that has a spec, and which files don't. *(live example: [coverage.md](spec-reports/coverage.md))*
226
- - **drift** — each place a spec and the code disagree, with a suggested fix. `0` is clean. *(live example: [report.md](spec-reports/report.md))*
243
+ - **coverage** — the % of your code that has a spec, and which files don't. *(live example: [coverage.md](https://github.com/benjaminjones/spec-eval/blob/main/spec-reports/coverage.md))*
244
+ - **drift** — each place a spec and the code disagree, with a suggested fix. `0` is clean. *(live example: [report.md](https://github.com/benjaminjones/spec-eval/blob/main/spec-reports/report.md))*
227
245
  - **sufficiency** — a `0`–`1` score per spec (worst first), listing what's missing with a searchable code pointer
228
- (`file.py (function)`). `1.0` = the grader found nothing missing — guidance, not a guarantee. *(live example: [sufficiency.md](spec-reports/sufficiency.md))*
246
+ (`file.py (function)`). `1.0` = the grader found nothing missing — guidance, not a guarantee. *(live example: [sufficiency.md](https://github.com/benjaminjones/spec-eval/blob/main/spec-reports/sufficiency.md))*
229
247
 
230
- The three examples are **spec-eval grading itself**, rolled up in [SPEC-HEALTH.md](SPEC-HEALTH.md).
248
+ The three examples are **spec-eval grading itself**, rolled up in [SPEC-HEALTH.md](https://github.com/benjaminjones/spec-eval/blob/main/SPEC-HEALTH.md).
231
249
 
232
250
  Every command also appends a line to `runs.jsonl` — timestamp, git commit, scores — so you can track change over
233
251
  time.
@@ -258,8 +276,8 @@ so put both keys in one `.env` and each run picks up the one it needs. (Want one
258
276
 
259
277
  ## Proof it works
260
278
 
261
- - **spec-eval specs itself** — every module in [`spec_eval/`](spec_eval/) has its own spec:
262
- **coverage 100% · drift 0 · sufficiency ≈0.86** ([receipt](SPEC-HEALTH.md) — the exact, dated score).
279
+ - **spec-eval specs itself** — every module in [`spec_eval/`](https://github.com/benjaminjones/spec-eval/tree/main/spec_eval/) has its own spec:
280
+ **coverage 100% · drift 0 · sufficiency ≈0.86** ([receipt](https://github.com/benjaminjones/spec-eval/blob/main/SPEC-HEALTH.md) — the exact, dated score).
263
281
 
264
282
  ## More
265
283
 
@@ -271,13 +289,13 @@ so put both keys in one `.env` and each run picks up the one it needs. (Want one
271
289
  agent writes the code; the check is where they meet), so the spec is *born accurate* instead of written up
272
290
  afterward.
273
291
  - **Just want to vibe code?** Fine — build first, check later, automatically: a pre-commit gate and a GitHub
274
- Action can run the checks for you — see [Make it routine](GETTING-STARTED.md#make-it-routine).
292
+ Action can run the checks for you — see [Make it routine](https://github.com/benjaminjones/spec-eval/blob/main/GETTING-STARTED.md#make-it-routine).
275
293
  - **Higher-stakes spec?** Run it past two vendors and compare — see [Higher stakes? Get a second opinion](#higher-stakes-get-a-second-opinion) above.
276
294
 
277
295
  ## FAQ
278
296
 
279
297
  Common questions — from *"which command do I run first?"* through costs, layouts, and score accuracy — live in
280
- **[FAQ.md](FAQ.md)**.
298
+ **[FAQ.md](https://github.com/benjaminjones/spec-eval/blob/main/FAQ.md)**.
281
299
 
282
300
  ## Feedback
283
301
 
@@ -6,7 +6,7 @@
6
6
 
7
7
  *A trust-eval for your specs — coverage, drift, and sufficiency, measured against the code.*
8
8
 
9
- [Get started](#try-it) · [Tutorial](GETTING-STARTED.md) · [No API key?](#run-via-prompt-chat-no-setup) · [Second opinion](#higher-stakes-get-a-second-opinion) · [FAQ](FAQ.md)
9
+ [Get started](#try-it) · [Tutorial](https://github.com/benjaminjones/spec-eval/blob/main/GETTING-STARTED.md) · [No API key?](#run-via-prompt-chat-no-setup) · [Second opinion](#higher-stakes-get-a-second-opinion) · [FAQ](https://github.com/benjaminjones/spec-eval/blob/main/FAQ.md)
10
10
 
11
11
  A **spec** is short for **specification** — a plain-English description of what your code should do. **spec-eval**
12
12
  keeps one beside your code — per file, or per folder — and checks the two still agree, so the spec stays worth
@@ -69,7 +69,7 @@ Everything it writes is plain markdown, right next to your code:
69
69
 
70
70
  ## Try it
71
71
 
72
- Prefer a guided walkthrough? **[Getting-started tutorial →](GETTING-STARTED.md)**
72
+ Prefer a guided walkthrough? **[Getting-started tutorial →](https://github.com/benjaminjones/spec-eval/blob/main/GETTING-STARTED.md)**
73
73
 
74
74
  **The commands** — in the order you'll typically use them:
75
75
 
@@ -87,12 +87,30 @@ Three ways to run them, least setup first:
87
87
  ### Run via prompt chat (no setup)
88
88
 
89
89
  Ask the coding agent you already have (Claude Code, Copilot, Cursor, …) — nothing to install, no key, answers
90
- land in the chat:
90
+ land in the chat. Two skills, one per job — point your agent at the right one:
91
+
92
+ **Write specs** — the [authoring skill](https://github.com/benjaminjones/spec-eval/blob/main/skills/spec-authoring/SKILL.md) carries the spec structure
93
+ (the `INV-*` invariant and `AC-*` acceptance-criteria tables come from its rubric and templates):
94
+ ```text
95
+ Read path/to/spec-eval/skills/spec-authoring/SKILL.md — and the templates
96
+ in its templates/ folder — and follow it: write specs for my code.
97
+ ```
98
+
99
+ **Check specs** — the [checking skill](https://github.com/benjaminjones/spec-eval/blob/main/skills/spec-check/SKILL.md) grades what exists (drift + sufficiency):
91
100
  ```text
92
101
  Read path/to/spec-eval/skills/spec-check/SKILL.md and follow it —
93
102
  first check coverage, then check my specs against my code.
94
103
  ```
95
- The path is wherever you cloned this repo — no clone? Give your agent [the file's link](skills/spec-check/SKILL.md).
104
+
105
+ > [!NOTE]
106
+ > `path/to/spec-eval` is wherever you cloned this repo. No clone? Give your agent the skill's link by name:
107
+ > [`spec-authoring/SKILL.md`](https://github.com/benjaminjones/spec-eval/blob/main/skills/spec-authoring/SKILL.md) to write specs,
108
+ > [`spec-check/SKILL.md`](https://github.com/benjaminjones/spec-eval/blob/main/skills/spec-check/SKILL.md) to check them.
109
+
110
+ > [!TIP]
111
+ > **Author and check in separate sessions.** A checker that still remembers its authoring decisions tends to
112
+ > echo them instead of re-reading the code — the same reason a PR isn't reviewed by its author. Write specs in
113
+ > one session; open a fresh one to check them.
96
114
 
97
115
  Optionally save reports — add:
98
116
  ```text
@@ -100,7 +118,7 @@ and save the results to spec-reports/
100
118
  ```
101
119
  The agent writes them itself (its own scores; the terminal adds the exact coverage % and a run history).
102
120
 
103
- **Make it a standing command** *(optional, one-time)* — copy the [`skills/`](skills/) into your agent's skills
121
+ **Make it a standing command** *(optional, one-time)* — copy the [`skills/`](https://github.com/benjaminjones/spec-eval/tree/main/skills/) into your agent's skills
104
122
  folder (Claude Code: `.claude/skills/`). Then the prompt is the command:
105
123
  ```bash
106
124
  mkdir -p .claude/skills && cp -r path/to/spec-eval/skills/* .claude/skills/
@@ -135,7 +153,7 @@ export ANTHROPIC_API_KEY=sk-ant-... # or OPENAI_API_KEY / GOOGLE_API_KEY
135
153
 
136
154
  > [!NOTE]
137
155
  > A typical first run is `coverage → generate → sufficiency`, then at every milestone after: `audit` — plus
138
- > `sufficiency` when you've added behavior. The [tutorial](GETTING-STARTED.md) has step-by-step guides for
156
+ > `sufficiency` when you've added behavior. The [tutorial](https://github.com/benjaminjones/spec-eval/blob/main/GETTING-STARTED.md) has step-by-step guides for
139
157
  > starting with no specs and for existing docs.
140
158
 
141
159
  > [!TIP]
@@ -146,7 +164,7 @@ export ANTHROPIC_API_KEY=sk-ant-... # or OPENAI_API_KEY / GOOGLE_API_KEY
146
164
  A few things worth knowing:
147
165
 
148
166
  - **No config needed** — by default each spec pairs with the code file of the same name beside it (`parser.md` ↔ `parser.py`).
149
- Prefer one spec per folder? `--layout per-dir` ([FAQ](FAQ.md#can-i-use-one-spec-per-folder-instead-of-one-per-file)).
167
+ Prefer one spec per folder? `--layout per-dir` ([FAQ](https://github.com/benjaminjones/spec-eval/blob/main/FAQ.md#can-i-use-one-spec-per-folder-instead-of-one-per-file)).
150
168
  - **Any language, three providers** — swap the model with `--model openai:…`, `google:…`, or `claude-code`
151
169
  (Anthropic is the default); code is read as plain text. Want another provider? [Open an issue](#feedback).
152
170
  - **Your other docs are safe** — by default only a `.md` next to code with the same name counts as a spec, so your
@@ -154,7 +172,7 @@ A few things worth knowing:
154
172
  - **What's shared** — to grade a spec, its code and text go to your AI provider. That's it. (`coverage` sends nothing — it runs fully on your machine.)
155
173
 
156
174
  **Keep it honest on every commit** — add one line to a git hook (a script git runs before each commit) so new
157
- code always needs a spec; copy-paste setup in [the tutorial](GETTING-STARTED.md#make-it-routine):
175
+ code always needs a spec; copy-paste setup in [the tutorial](https://github.com/benjaminjones/spec-eval/blob/main/GETTING-STARTED.md#make-it-routine):
158
176
  ```bash
159
177
  spec-eval coverage . --min 90 # fails the commit if coverage drops below 90% (free — no AI)
160
178
  ```
@@ -185,20 +203,20 @@ the pair against a fixed rubric — one model call per pair. Two things follow f
185
203
  > your coding agent's own usage if you asked in chat. Every CLI run prints its exact call and token counts.
186
204
 
187
205
  **Want the exact rules behind each score?** spec-eval specs its own scoring — read
188
- [`spec_eval/coverage.md`](spec_eval/coverage.md) (how coverage counts),
189
- [`spec_eval/rubric.md`](spec_eval/rubric.md) (the drift rules), and
190
- [`spec_eval/sufficiency.md`](spec_eval/sufficiency.md) (how the 0–1 score is decided).
206
+ [`spec_eval/coverage.md`](https://github.com/benjaminjones/spec-eval/blob/main/spec_eval/coverage.md) (how coverage counts),
207
+ [`spec_eval/rubric.md`](https://github.com/benjaminjones/spec-eval/blob/main/spec_eval/rubric.md) (the drift rules), and
208
+ [`spec_eval/sufficiency.md`](https://github.com/benjaminjones/spec-eval/blob/main/spec_eval/sufficiency.md) (how the 0–1 score is decided).
191
209
 
192
210
  ## Reading the scores
193
211
 
194
212
  Each run writes a short report to `spec-reports/` (a `.md` you read, a `.json` for tools):
195
213
 
196
- - **coverage** — the % of your code that has a spec, and which files don't. *(live example: [coverage.md](spec-reports/coverage.md))*
197
- - **drift** — each place a spec and the code disagree, with a suggested fix. `0` is clean. *(live example: [report.md](spec-reports/report.md))*
214
+ - **coverage** — the % of your code that has a spec, and which files don't. *(live example: [coverage.md](https://github.com/benjaminjones/spec-eval/blob/main/spec-reports/coverage.md))*
215
+ - **drift** — each place a spec and the code disagree, with a suggested fix. `0` is clean. *(live example: [report.md](https://github.com/benjaminjones/spec-eval/blob/main/spec-reports/report.md))*
198
216
  - **sufficiency** — a `0`–`1` score per spec (worst first), listing what's missing with a searchable code pointer
199
- (`file.py (function)`). `1.0` = the grader found nothing missing — guidance, not a guarantee. *(live example: [sufficiency.md](spec-reports/sufficiency.md))*
217
+ (`file.py (function)`). `1.0` = the grader found nothing missing — guidance, not a guarantee. *(live example: [sufficiency.md](https://github.com/benjaminjones/spec-eval/blob/main/spec-reports/sufficiency.md))*
200
218
 
201
- The three examples are **spec-eval grading itself**, rolled up in [SPEC-HEALTH.md](SPEC-HEALTH.md).
219
+ The three examples are **spec-eval grading itself**, rolled up in [SPEC-HEALTH.md](https://github.com/benjaminjones/spec-eval/blob/main/SPEC-HEALTH.md).
202
220
 
203
221
  Every command also appends a line to `runs.jsonl` — timestamp, git commit, scores — so you can track change over
204
222
  time.
@@ -229,8 +247,8 @@ so put both keys in one `.env` and each run picks up the one it needs. (Want one
229
247
 
230
248
  ## Proof it works
231
249
 
232
- - **spec-eval specs itself** — every module in [`spec_eval/`](spec_eval/) has its own spec:
233
- **coverage 100% · drift 0 · sufficiency ≈0.86** ([receipt](SPEC-HEALTH.md) — the exact, dated score).
250
+ - **spec-eval specs itself** — every module in [`spec_eval/`](https://github.com/benjaminjones/spec-eval/tree/main/spec_eval/) has its own spec:
251
+ **coverage 100% · drift 0 · sufficiency ≈0.86** ([receipt](https://github.com/benjaminjones/spec-eval/blob/main/SPEC-HEALTH.md) — the exact, dated score).
234
252
 
235
253
  ## More
236
254
 
@@ -242,13 +260,13 @@ so put both keys in one `.env` and each run picks up the one it needs. (Want one
242
260
  agent writes the code; the check is where they meet), so the spec is *born accurate* instead of written up
243
261
  afterward.
244
262
  - **Just want to vibe code?** Fine — build first, check later, automatically: a pre-commit gate and a GitHub
245
- Action can run the checks for you — see [Make it routine](GETTING-STARTED.md#make-it-routine).
263
+ Action can run the checks for you — see [Make it routine](https://github.com/benjaminjones/spec-eval/blob/main/GETTING-STARTED.md#make-it-routine).
246
264
  - **Higher-stakes spec?** Run it past two vendors and compare — see [Higher stakes? Get a second opinion](#higher-stakes-get-a-second-opinion) above.
247
265
 
248
266
  ## FAQ
249
267
 
250
268
  Common questions — from *"which command do I run first?"* through costs, layouts, and score accuracy — live in
251
- **[FAQ.md](FAQ.md)**.
269
+ **[FAQ.md](https://github.com/benjaminjones/spec-eval/blob/main/FAQ.md)**.
252
270
 
253
271
  ## Feedback
254
272
 
@@ -1,2 +1,2 @@
1
1
  """spec-eval — portable code↔doc drift audit + fingerprint."""
2
- __version__ = "0.2.0" # THE single source of version truth; pyproject.toml reads this via dynamic version
2
+ __version__ = "0.2.1" # THE single source of version truth; pyproject.toml reads this via dynamic version
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: spec-eval
3
- Version: 0.2.0
3
+ Version: 0.2.1
4
4
  Summary: A trust-eval for your specs: coverage, drift, and sufficiency of markdown specs against code.
5
5
  Author: Ben Jones
6
6
  License: MIT
@@ -35,7 +35,7 @@ Dynamic: license-file
35
35
 
36
36
  *A trust-eval for your specs — coverage, drift, and sufficiency, measured against the code.*
37
37
 
38
- [Get started](#try-it) · [Tutorial](GETTING-STARTED.md) · [No API key?](#run-via-prompt-chat-no-setup) · [Second opinion](#higher-stakes-get-a-second-opinion) · [FAQ](FAQ.md)
38
+ [Get started](#try-it) · [Tutorial](https://github.com/benjaminjones/spec-eval/blob/main/GETTING-STARTED.md) · [No API key?](#run-via-prompt-chat-no-setup) · [Second opinion](#higher-stakes-get-a-second-opinion) · [FAQ](https://github.com/benjaminjones/spec-eval/blob/main/FAQ.md)
39
39
 
40
40
  A **spec** is short for **specification** — a plain-English description of what your code should do. **spec-eval**
41
41
  keeps one beside your code — per file, or per folder — and checks the two still agree, so the spec stays worth
@@ -98,7 +98,7 @@ Everything it writes is plain markdown, right next to your code:
98
98
 
99
99
  ## Try it
100
100
 
101
- Prefer a guided walkthrough? **[Getting-started tutorial →](GETTING-STARTED.md)**
101
+ Prefer a guided walkthrough? **[Getting-started tutorial →](https://github.com/benjaminjones/spec-eval/blob/main/GETTING-STARTED.md)**
102
102
 
103
103
  **The commands** — in the order you'll typically use them:
104
104
 
@@ -116,12 +116,30 @@ Three ways to run them, least setup first:
116
116
  ### Run via prompt chat (no setup)
117
117
 
118
118
  Ask the coding agent you already have (Claude Code, Copilot, Cursor, …) — nothing to install, no key, answers
119
- land in the chat:
119
+ land in the chat. Two skills, one per job — point your agent at the right one:
120
+
121
+ **Write specs** — the [authoring skill](https://github.com/benjaminjones/spec-eval/blob/main/skills/spec-authoring/SKILL.md) carries the spec structure
122
+ (the `INV-*` invariant and `AC-*` acceptance-criteria tables come from its rubric and templates):
123
+ ```text
124
+ Read path/to/spec-eval/skills/spec-authoring/SKILL.md — and the templates
125
+ in its templates/ folder — and follow it: write specs for my code.
126
+ ```
127
+
128
+ **Check specs** — the [checking skill](https://github.com/benjaminjones/spec-eval/blob/main/skills/spec-check/SKILL.md) grades what exists (drift + sufficiency):
120
129
  ```text
121
130
  Read path/to/spec-eval/skills/spec-check/SKILL.md and follow it —
122
131
  first check coverage, then check my specs against my code.
123
132
  ```
124
- The path is wherever you cloned this repo — no clone? Give your agent [the file's link](skills/spec-check/SKILL.md).
133
+
134
+ > [!NOTE]
135
+ > `path/to/spec-eval` is wherever you cloned this repo. No clone? Give your agent the skill's link by name:
136
+ > [`spec-authoring/SKILL.md`](https://github.com/benjaminjones/spec-eval/blob/main/skills/spec-authoring/SKILL.md) to write specs,
137
+ > [`spec-check/SKILL.md`](https://github.com/benjaminjones/spec-eval/blob/main/skills/spec-check/SKILL.md) to check them.
138
+
139
+ > [!TIP]
140
+ > **Author and check in separate sessions.** A checker that still remembers its authoring decisions tends to
141
+ > echo them instead of re-reading the code — the same reason a PR isn't reviewed by its author. Write specs in
142
+ > one session; open a fresh one to check them.
125
143
 
126
144
  Optionally save reports — add:
127
145
  ```text
@@ -129,7 +147,7 @@ and save the results to spec-reports/
129
147
  ```
130
148
  The agent writes them itself (its own scores; the terminal adds the exact coverage % and a run history).
131
149
 
132
- **Make it a standing command** *(optional, one-time)* — copy the [`skills/`](skills/) into your agent's skills
150
+ **Make it a standing command** *(optional, one-time)* — copy the [`skills/`](https://github.com/benjaminjones/spec-eval/tree/main/skills/) into your agent's skills
133
151
  folder (Claude Code: `.claude/skills/`). Then the prompt is the command:
134
152
  ```bash
135
153
  mkdir -p .claude/skills && cp -r path/to/spec-eval/skills/* .claude/skills/
@@ -164,7 +182,7 @@ export ANTHROPIC_API_KEY=sk-ant-... # or OPENAI_API_KEY / GOOGLE_API_KEY
164
182
 
165
183
  > [!NOTE]
166
184
  > A typical first run is `coverage → generate → sufficiency`, then at every milestone after: `audit` — plus
167
- > `sufficiency` when you've added behavior. The [tutorial](GETTING-STARTED.md) has step-by-step guides for
185
+ > `sufficiency` when you've added behavior. The [tutorial](https://github.com/benjaminjones/spec-eval/blob/main/GETTING-STARTED.md) has step-by-step guides for
168
186
  > starting with no specs and for existing docs.
169
187
 
170
188
  > [!TIP]
@@ -175,7 +193,7 @@ export ANTHROPIC_API_KEY=sk-ant-... # or OPENAI_API_KEY / GOOGLE_API_KEY
175
193
  A few things worth knowing:
176
194
 
177
195
  - **No config needed** — by default each spec pairs with the code file of the same name beside it (`parser.md` ↔ `parser.py`).
178
- Prefer one spec per folder? `--layout per-dir` ([FAQ](FAQ.md#can-i-use-one-spec-per-folder-instead-of-one-per-file)).
196
+ Prefer one spec per folder? `--layout per-dir` ([FAQ](https://github.com/benjaminjones/spec-eval/blob/main/FAQ.md#can-i-use-one-spec-per-folder-instead-of-one-per-file)).
179
197
  - **Any language, three providers** — swap the model with `--model openai:…`, `google:…`, or `claude-code`
180
198
  (Anthropic is the default); code is read as plain text. Want another provider? [Open an issue](#feedback).
181
199
  - **Your other docs are safe** — by default only a `.md` next to code with the same name counts as a spec, so your
@@ -183,7 +201,7 @@ A few things worth knowing:
183
201
  - **What's shared** — to grade a spec, its code and text go to your AI provider. That's it. (`coverage` sends nothing — it runs fully on your machine.)
184
202
 
185
203
  **Keep it honest on every commit** — add one line to a git hook (a script git runs before each commit) so new
186
- code always needs a spec; copy-paste setup in [the tutorial](GETTING-STARTED.md#make-it-routine):
204
+ code always needs a spec; copy-paste setup in [the tutorial](https://github.com/benjaminjones/spec-eval/blob/main/GETTING-STARTED.md#make-it-routine):
187
205
  ```bash
188
206
  spec-eval coverage . --min 90 # fails the commit if coverage drops below 90% (free — no AI)
189
207
  ```
@@ -214,20 +232,20 @@ the pair against a fixed rubric — one model call per pair. Two things follow f
214
232
  > your coding agent's own usage if you asked in chat. Every CLI run prints its exact call and token counts.
215
233
 
216
234
  **Want the exact rules behind each score?** spec-eval specs its own scoring — read
217
- [`spec_eval/coverage.md`](spec_eval/coverage.md) (how coverage counts),
218
- [`spec_eval/rubric.md`](spec_eval/rubric.md) (the drift rules), and
219
- [`spec_eval/sufficiency.md`](spec_eval/sufficiency.md) (how the 0–1 score is decided).
235
+ [`spec_eval/coverage.md`](https://github.com/benjaminjones/spec-eval/blob/main/spec_eval/coverage.md) (how coverage counts),
236
+ [`spec_eval/rubric.md`](https://github.com/benjaminjones/spec-eval/blob/main/spec_eval/rubric.md) (the drift rules), and
237
+ [`spec_eval/sufficiency.md`](https://github.com/benjaminjones/spec-eval/blob/main/spec_eval/sufficiency.md) (how the 0–1 score is decided).
220
238
 
221
239
  ## Reading the scores
222
240
 
223
241
  Each run writes a short report to `spec-reports/` (a `.md` you read, a `.json` for tools):
224
242
 
225
- - **coverage** — the % of your code that has a spec, and which files don't. *(live example: [coverage.md](spec-reports/coverage.md))*
226
- - **drift** — each place a spec and the code disagree, with a suggested fix. `0` is clean. *(live example: [report.md](spec-reports/report.md))*
243
+ - **coverage** — the % of your code that has a spec, and which files don't. *(live example: [coverage.md](https://github.com/benjaminjones/spec-eval/blob/main/spec-reports/coverage.md))*
244
+ - **drift** — each place a spec and the code disagree, with a suggested fix. `0` is clean. *(live example: [report.md](https://github.com/benjaminjones/spec-eval/blob/main/spec-reports/report.md))*
227
245
  - **sufficiency** — a `0`–`1` score per spec (worst first), listing what's missing with a searchable code pointer
228
- (`file.py (function)`). `1.0` = the grader found nothing missing — guidance, not a guarantee. *(live example: [sufficiency.md](spec-reports/sufficiency.md))*
246
+ (`file.py (function)`). `1.0` = the grader found nothing missing — guidance, not a guarantee. *(live example: [sufficiency.md](https://github.com/benjaminjones/spec-eval/blob/main/spec-reports/sufficiency.md))*
229
247
 
230
- The three examples are **spec-eval grading itself**, rolled up in [SPEC-HEALTH.md](SPEC-HEALTH.md).
248
+ The three examples are **spec-eval grading itself**, rolled up in [SPEC-HEALTH.md](https://github.com/benjaminjones/spec-eval/blob/main/SPEC-HEALTH.md).
231
249
 
232
250
  Every command also appends a line to `runs.jsonl` — timestamp, git commit, scores — so you can track change over
233
251
  time.
@@ -258,8 +276,8 @@ so put both keys in one `.env` and each run picks up the one it needs. (Want one
258
276
 
259
277
  ## Proof it works
260
278
 
261
- - **spec-eval specs itself** — every module in [`spec_eval/`](spec_eval/) has its own spec:
262
- **coverage 100% · drift 0 · sufficiency ≈0.86** ([receipt](SPEC-HEALTH.md) — the exact, dated score).
279
+ - **spec-eval specs itself** — every module in [`spec_eval/`](https://github.com/benjaminjones/spec-eval/tree/main/spec_eval/) has its own spec:
280
+ **coverage 100% · drift 0 · sufficiency ≈0.86** ([receipt](https://github.com/benjaminjones/spec-eval/blob/main/SPEC-HEALTH.md) — the exact, dated score).
263
281
 
264
282
  ## More
265
283
 
@@ -271,13 +289,13 @@ so put both keys in one `.env` and each run picks up the one it needs. (Want one
271
289
  agent writes the code; the check is where they meet), so the spec is *born accurate* instead of written up
272
290
  afterward.
273
291
  - **Just want to vibe code?** Fine — build first, check later, automatically: a pre-commit gate and a GitHub
274
- Action can run the checks for you — see [Make it routine](GETTING-STARTED.md#make-it-routine).
292
+ Action can run the checks for you — see [Make it routine](https://github.com/benjaminjones/spec-eval/blob/main/GETTING-STARTED.md#make-it-routine).
275
293
  - **Higher-stakes spec?** Run it past two vendors and compare — see [Higher stakes? Get a second opinion](#higher-stakes-get-a-second-opinion) above.
276
294
 
277
295
  ## FAQ
278
296
 
279
297
  Common questions — from *"which command do I run first?"* through costs, layouts, and score accuracy — live in
280
- **[FAQ.md](FAQ.md)**.
298
+ **[FAQ.md](https://github.com/benjaminjones/spec-eval/blob/main/FAQ.md)**.
281
299
 
282
300
  ## Feedback
283
301
 
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes