spec-receipt 1.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- spec_receipt-1.0.0/LICENSE +21 -0
- spec_receipt-1.0.0/PKG-INFO +598 -0
- spec_receipt-1.0.0/README.md +573 -0
- spec_receipt-1.0.0/pyproject.toml +52 -0
- spec_receipt-1.0.0/setup.cfg +4 -0
- spec_receipt-1.0.0/src/spec_receipt.egg-info/PKG-INFO +598 -0
- spec_receipt-1.0.0/src/spec_receipt.egg-info/SOURCES.txt +10 -0
- spec_receipt-1.0.0/src/spec_receipt.egg-info/dependency_links.txt +1 -0
- spec_receipt-1.0.0/src/spec_receipt.egg-info/entry_points.txt +2 -0
- spec_receipt-1.0.0/src/spec_receipt.egg-info/top_level.txt +1 -0
- spec_receipt-1.0.0/src/spec_receipt.py +2646 -0
- spec_receipt-1.0.0/tests/test_spec_receipt.py +1279 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Jasveer Singh
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,598 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: spec-receipt
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: Receipt a repo's design spec against its code — every spec item IMPLEMENTED, MISSING, CONTRADICTED or UNTRACED, with file-and-line evidence.
|
|
5
|
+
Author: Jasveer Singh
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/jasveerai88/spec-receipt
|
|
8
|
+
Project-URL: Repository, https://github.com/jasveerai88/spec-receipt
|
|
9
|
+
Project-URL: Issues, https://github.com/jasveerai88/spec-receipt/issues
|
|
10
|
+
Project-URL: Changelog, https://github.com/jasveerai88/spec-receipt/blob/main/CHANGELOG.md
|
|
11
|
+
Keywords: spec,drift,audit,static-analysis,cli,documentation,ai-agents,developer-tools
|
|
12
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
13
|
+
Classifier: Environment :: Console
|
|
14
|
+
Classifier: Intended Audience :: Developers
|
|
15
|
+
Classifier: Operating System :: OS Independent
|
|
16
|
+
Classifier: Programming Language :: Python :: 3
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
20
|
+
Classifier: Topic :: Software Development :: Quality Assurance
|
|
21
|
+
Requires-Python: >=3.10
|
|
22
|
+
Description-Content-Type: text/markdown
|
|
23
|
+
License-File: LICENSE
|
|
24
|
+
Dynamic: license-file
|
|
25
|
+
|
|
26
|
+
<h1 align="center">spec-receipt</h1>
|
|
27
|
+
|
|
28
|
+
<p align="center"><strong>Your spec says <code>POST /items</code>. Receipt it — every spec item IMPLEMENTED, MISSING, CONTRADICTED or UNTRACED, every code surface DOCUMENTED or UNDOCUMENTED, with file-and-line evidence.</strong></p>
|
|
29
|
+
|
|
30
|
+
<p align="center">
|
|
31
|
+
<img alt="tests: 87 passing" src="https://img.shields.io/badge/tests-87%20passing-brightgreen">
|
|
32
|
+
<img alt="python 3.10 | 3.11 | 3.12" src="https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12-blue">
|
|
33
|
+
<img alt="version 1.0.0" src="https://img.shields.io/badge/version-1.0.0-orange">
|
|
34
|
+
<img alt="dependencies: none" src="https://img.shields.io/badge/dependencies-0-brightgreen">
|
|
35
|
+
<img alt="license: MIT" src="https://img.shields.io/badge/license-MIT-blue">
|
|
36
|
+
</p>
|
|
37
|
+
|
|
38
|
+
<p align="center"><img alt="spec-receipt demo: the drifted fixture's receipt — a changed default, a deleted flag, a forbidden flag and a POST endpoint the code only registers as GET" src="demo.gif" width="880"></p>
|
|
39
|
+
|
|
40
|
+
A spec is a promise made at design time. Then the code
|
|
41
|
+
moves: a flag is deleted, a default quietly changes, an
|
|
42
|
+
endpoint's method flips, a forbidden flag sneaks back in,
|
|
43
|
+
a feature ships that no document ever mentioned. Review
|
|
44
|
+
catches some of it, memory catches less, and the spec
|
|
45
|
+
slowly becomes fiction — while still being the document
|
|
46
|
+
new contributors, agents, and reviewers trust first.
|
|
47
|
+
|
|
48
|
+
spec-receipt prints the receipt instead. It extracts what
|
|
49
|
+
the spec **declares** (flags, commands, endpoints, config
|
|
50
|
+
keys, environment variables, requirement IDs — by
|
|
51
|
+
deterministic patterns, from any Markdown spec), extracts
|
|
52
|
+
what the code **declares** (the same five surface classes,
|
|
53
|
+
by Python AST), and prints a per-item verdict with the
|
|
54
|
+
evidence cited on both sides. No LLM. No score. No
|
|
55
|
+
network. Nothing uploaded.
|
|
56
|
+
|
|
57
|
+
**Static and read-only — it executes nothing.** The
|
|
58
|
+
target repo is read as Markdown text and Python AST only:
|
|
59
|
+
no file is run, imported, compiled, or evaluated. Its own
|
|
60
|
+
source imports no network or subprocess module, and the
|
|
61
|
+
test suite proves the read-only claim: a fixture module
|
|
62
|
+
that would write a marker file if it ran never runs, and
|
|
63
|
+
every fixture tree comes out byte-identical after a
|
|
64
|
+
check.
|
|
65
|
+
|
|
66
|
+
Verdicts are stingy by design. **MISSING** is issued
|
|
67
|
+
only when a surface class was *completely enumerated*
|
|
68
|
+
and the item is absent from it. **CONTRADICTED** needs
|
|
69
|
+
hard evidence on both sides — a stated default against a
|
|
70
|
+
different code default, a "there is no `--legacy`" against
|
|
71
|
+
the flag's own AST node. A prose requirement is never
|
|
72
|
+
judged implemented or absent: it is **TRACED** to an
|
|
73
|
+
anchor in code or tests, or honestly **UNTRACED**.
|
|
74
|
+
Natural language cannot prove a behaviour absent, and
|
|
75
|
+
the receipt never pretends it can.
|
|
76
|
+
|
|
77
|
+
## The receipt family
|
|
78
|
+
|
|
79
|
+
Three tools, one grammar — *document = claims, code =
|
|
80
|
+
evidence, every verdict cited*:
|
|
81
|
+
|
|
82
|
+
- **[skill-claims](https://github.com/jasveerai88/skill-claims)
|
|
83
|
+
— is the package honest?** Audits an agent skill's
|
|
84
|
+
prose promises against its static behaviour.
|
|
85
|
+
- **spec-receipt — does the code still match the
|
|
86
|
+
spec?** Audits a repo's engineering spec against its
|
|
87
|
+
declared implementation surfaces.
|
|
88
|
+
- **[done-claim-auditor](https://github.com/jasveerai88/done-claim-auditor)
|
|
89
|
+
— did the agent do what it claimed?** The same
|
|
90
|
+
claim-vs-evidence logic pointed at session
|
|
91
|
+
transcripts.
|
|
92
|
+
|
|
93
|
+
And before any of them:
|
|
94
|
+
**[skill-vet](https://github.com/jasveerai88/skill-vet) —
|
|
95
|
+
is it dangerous?** Stealing, spying, hidden sending,
|
|
96
|
+
each proven as a combination of evidence.
|
|
97
|
+
|
|
98
|
+
## Quick start
|
|
99
|
+
|
|
100
|
+
```bash
|
|
101
|
+
pipx install spec-receipt # or: pip install spec-receipt
|
|
102
|
+
spec-receipt check path/to/repo
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
*(The PyPI name goes live when v1.0.0 is published;
|
|
106
|
+
until then use a source checkout:
|
|
107
|
+
`python3 src/spec_receipt.py check <repo>`, or
|
|
108
|
+
`pipx install git+https://github.com/jasveerai88/spec-receipt`.)*
|
|
109
|
+
|
|
110
|
+
One command, one receipt, an exit code you can gate
|
|
111
|
+
on — **1** = drift found (at least one item MISSING or
|
|
112
|
+
CONTRADICTED), **0** = no drift, **2** = input error.
|
|
113
|
+
Add `--json` for a machine-readable receipt. Python
|
|
114
|
+
3.10+, standard library only, no dependencies.
|
|
115
|
+
|
|
116
|
+
Which file is the spec, in priority order:
|
|
117
|
+
|
|
118
|
+
1. Explicit `--spec` arguments (repeatable).
|
|
119
|
+
2. A `.spec-receipt.toml` manifest in the repo root
|
|
120
|
+
listing `specs = [...]`.
|
|
121
|
+
3. Auto-discovery: `SPEC.md` / `spec.md` at root,
|
|
122
|
+
`docs/spec*.md`, `docs/SPEC.md`,
|
|
123
|
+
`specs/**/spec.md`, `DESIGN.md`, `API.md`.
|
|
124
|
+
Nothing found → exit 2.
|
|
125
|
+
|
|
126
|
+
## What it checks
|
|
127
|
+
|
|
128
|
+
Five surface classes, extracted from the spec and
|
|
129
|
+
indexed from the code, judged in both directions:
|
|
130
|
+
|
|
131
|
+
| Class | Declared in the spec as | Indexed in the code from |
|
|
132
|
+
|---|---|---|
|
|
133
|
+
| **CLI flags** | `--name` in backticks, with a stated default (`(default: fast)`) or choice set (``one of `low\|high` ``) | `argparse` `add_argument` and `click` `@click.option`, with defaults and choices, via AST |
|
|
134
|
+
| **CLI commands** | A `## Command: name` heading, a backticked invocation line, an invocation line in a fenced block | `add_parser` subcommands, `click` commands and groups, via AST |
|
|
135
|
+
| **HTTP routes** | A backticked `` `GET /path` `` span, a row in an Endpoints/API table, an exclusivity statement ("only `GET` on `/items`") | Flask `@*.route`, FastAPI method decorators and `api_route`, Django `path()` / `re_path()` in `urls.py` |
|
|
136
|
+
| **Config keys** | A backticked dotted key (`server.port`) under a Configuration heading, or a key in a fenced TOML/INI example | Keys of example/sample/config-named `.toml` / `.ini` / `.json` files, and `config[...]` / `config.get(...)` reads with constant keys |
|
|
137
|
+
| **Environment variables** | A backticked `SCREAMING_CASE` name; negations ("never reads `AWS_*`") | `os.environ[...]`, `os.getenv(...)`, `os.environ.get(...)` with constant names, incl. the from-import spellings |
|
|
138
|
+
|
|
139
|
+
Plus **requirement items**: a line carrying an ID
|
|
140
|
+
(`FR-###`, `REQ-<word>`, `R-##`) is traced to an anchor
|
|
141
|
+
— the ID verbatim in code or tests (scoped to the ID's
|
|
142
|
+
own spec), or two or more of its distinctive statement
|
|
143
|
+
terms co-occurring in one in-scope file.
|
|
144
|
+
|
|
145
|
+
The verdicts:
|
|
146
|
+
|
|
147
|
+
| Verdict | Applies to | Meaning |
|
|
148
|
+
|---|---|---|
|
|
149
|
+
| `IMPLEMENTED` | Spec item | The named surface exists in code as specified — AST node cited; any stated default/choices also match. For a negation item: the surface is absent from a completely enumerated class, so the negation holds |
|
|
150
|
+
| `MISSING` | Spec item | A structured surface item has no counterpart in code — issued **only** on complete enumeration of its class. Never for requirements, never on partial enumeration |
|
|
151
|
+
| `CONTRADICTED` | Spec item | Code proves the opposite of the spec statement, hard evidence on both sides: default vs different default, choices vs different choices, "there is no `--x`" vs the flag's AST node, a spec'd path registered with other methods only, an env negation vs a matching read |
|
|
152
|
+
| `TRACED` | Requirement item | The ID has an anchor in code or tests, scoped to its own spec. TRACED never means "correctly implemented" — the receipt says so every time |
|
|
153
|
+
| `UNTRACED` | Requirement item (or partial class) | No anchor, or the class could only be partially enumerated. The honest residue — **not** MISSING |
|
|
154
|
+
| `DOCUMENTED` | Code surface | A spec item names this surface (a surface named by a negation is DOCUMENTED — named, as forbidden; the item carries the CONTRADICTED) |
|
|
155
|
+
| `UNDOCUMENTED` | Code surface | The surface exists in code; no spec item names it |
|
|
156
|
+
|
|
157
|
+
[Quick start](#quick-start) · [What it checks](#what-it-checks) ·
|
|
158
|
+
[How it compares](#how-it-compares) · [Reference](#reference)
|
|
159
|
+
|
|
160
|
+
## How it compares
|
|
161
|
+
|
|
162
|
+
| | **spec-receipt** | chenhz01/spec-drift | lilabrooks/spec-drift | spec-kit `/speckit.analyze` |
|
|
163
|
+
|---|---|---|---|---|
|
|
164
|
+
| The question | Does the **code** still match the **spec**? | Do spec-kit specs and their test anchors agree? | What changed in this diff, per the governing docs? | Do the spec, plan and tasks documents agree? |
|
|
165
|
+
| Scope | Whole-repo snapshot, post-hoc | Spec-kit layout only (`specs/NNN/spec.md`, `FR-###`) | A diff range (`base..HEAD`) | In-flow, during authoring |
|
|
166
|
+
| What is checked | Five surface classes (flags, commands, routes, config, env) + requirement tracing, **both directions** | FR↔test anchors and spec-vs-spec supersession | The changed files, against their governing documents | Spec ↔ plan ↔ tasks — documents only, never the implementation |
|
|
167
|
+
| Method | Deterministic patterns + AST — same repo, same receipt, forever | Lexical, zero-dependency, no LLM | An LLM judges (Anthropic/OpenAI keys, per-file model calls, API cost) | The authoring agent, in the loop |
|
|
168
|
+
| Output | Per-item verdicts — MISSING / CONTRADICTED only on complete enumeration or direct conflict — with file + line on both sides | "Candidates, never verdicts" (its own README's words) | LLM drift judgements on the diff | Consistency findings inside the authoring flow |
|
|
169
|
+
| Licence / state | MIT, maintained | Apache-2.0 | Proprietary; its README marks the experiment as concluding / being retired | Part of github/spec-kit |
|
|
170
|
+
|
|
171
|
+
*(Comparable descriptions from their own public pages
|
|
172
|
+
and READMEs, checked 2026-11-06. chenhz01/spec-drift is
|
|
173
|
+
the closest neighbour and honest prior art — the
|
|
174
|
+
stopword + document-frequency filtering its README
|
|
175
|
+
found mandatory for term tracing is used here too. The
|
|
176
|
+
gap spec-receipt fills is the one `/speckit.analyze`
|
|
177
|
+
leaves by design: post-hoc, against the code, in any
|
|
178
|
+
Markdown spec format.)*
|
|
179
|
+
|
|
180
|
+
## Honest limits (the short version)
|
|
181
|
+
|
|
182
|
+
Static patterns are evidence, not proof. Surfaces built
|
|
183
|
+
purely at runtime may not appear in the index; a class
|
|
184
|
+
that cannot be fully enumerated is marked partial and
|
|
185
|
+
its absence verdicts degrade to UNTRACED rather than
|
|
186
|
+
being upgraded to MISSING; prose requirements are
|
|
187
|
+
traced, never semantically judged. Extraction is
|
|
188
|
+
deliberately scoped — a spec's *examples of its own
|
|
189
|
+
notation* are not declarations, and some declaration
|
|
190
|
+
shapes are missed as the price of that (each one is
|
|
191
|
+
named in the full limits). Every receipt prints its
|
|
192
|
+
limits in full; the complete list is
|
|
193
|
+
[below](#honest-limits).
|
|
194
|
+
|
|
195
|
+
---
|
|
196
|
+
|
|
197
|
+
## Reference
|
|
198
|
+
|
|
199
|
+
**Contents:** [The drifted fixture](#the-drifted-fixture--the-demo) ·
|
|
200
|
+
[The clean fixture](#the-clean-fixture--the-other-direction) ·
|
|
201
|
+
[The dogfood gate](#the-dogfood-gate--this-tool-was-caught-lying-first) ·
|
|
202
|
+
[For agents](#for-agents) · [Tests](#tests) ·
|
|
203
|
+
[Honest limits](#honest-limits)
|
|
204
|
+
|
|
205
|
+
### The drifted fixture — the demo
|
|
206
|
+
|
|
207
|
+
`tests/fixtures/drifted/` is a constructed repo whose
|
|
208
|
+
code evolved against a frozen `SPEC.md`: one spec'd
|
|
209
|
+
flag deleted, one default changed, a forbidden flag
|
|
210
|
+
added, one endpoint's method changed from POST to GET,
|
|
211
|
+
one undocumented flag grown, one requirement's tests
|
|
212
|
+
renamed away. Verbatim output:
|
|
213
|
+
|
|
214
|
+
```
|
|
215
|
+
$ spec-receipt check tests/fixtures/drifted
|
|
216
|
+
spec-receipt 1.0.0 — spec drift receipt
|
|
217
|
+
Repo: drifted
|
|
218
|
+
Spec: SPEC.md
|
|
219
|
+
Classes: flags complete (4 surfaces), commands complete (1 surface), routes complete (1 surface), config complete (5 surfaces), env complete (2 surfaces)
|
|
220
|
+
|
|
221
|
+
SPEC ITEMS
|
|
222
|
+
CONTRADICTED flag --mode spec SPEC.md:5 · code app.py:19
|
|
223
|
+
spec states default `fast`; code default is `slow`
|
|
224
|
+
MISSING flag --level spec SPEC.md:6 · code —
|
|
225
|
+
the flags class was completely enumerated and --level is absent from it
|
|
226
|
+
IMPLEMENTED flag --verbose spec SPEC.md:7 · code app.py:20
|
|
227
|
+
surface declared in code
|
|
228
|
+
CONTRADICTED negation --legacy spec SPEC.md:8 · code app.py:21
|
|
229
|
+
spec states there is no --legacy; the flag is declared in code
|
|
230
|
+
IMPLEMENTED command export spec SPEC.md:10 · code app.py:24
|
|
231
|
+
command declared in code
|
|
232
|
+
TRACED requirement FR-001 spec SPEC.md:16 · code tests/test_app.py:1
|
|
233
|
+
ID FR-001 appears verbatim in code/tests
|
|
234
|
+
UNTRACED requirement FR-002 spec SPEC.md:17 · code —
|
|
235
|
+
no verbatim ID and no file carries two distinctive statement terms
|
|
236
|
+
IMPLEMENTED endpoint GET /items spec SPEC.md:21 · code app.py:35
|
|
237
|
+
route declared in code with this method
|
|
238
|
+
CONTRADICTED endpoint POST /items spec SPEC.md:22 · code app.py:35
|
|
239
|
+
spec states POST /items; the path is registered in code with GET only
|
|
240
|
+
IMPLEMENTED config server spec SPEC.md:28 · code config.example.toml:2
|
|
241
|
+
declared in the config index (example config or code read)
|
|
242
|
+
IMPLEMENTED config server.host spec SPEC.md:29 · code config.example.toml:3
|
|
243
|
+
declared in the config index (example config or code read)
|
|
244
|
+
IMPLEMENTED config server.port spec SPEC.md:30 · code config.example.toml:4
|
|
245
|
+
declared in the config index (example config or code read)
|
|
246
|
+
IMPLEMENTED config database spec SPEC.md:31 · code config.example.toml:6
|
|
247
|
+
declared in the config index (example config or code read)
|
|
248
|
+
IMPLEMENTED config database.url spec SPEC.md:32 · code config.example.toml:7
|
|
249
|
+
declared in the config index (example config or code read)
|
|
250
|
+
IMPLEMENTED env ITEMS_API_KEY spec SPEC.md:48 · code app.py:48
|
|
251
|
+
read in code
|
|
252
|
+
IMPLEMENTED env ITEMS_REGION spec SPEC.md:49 · code app.py:49
|
|
253
|
+
read in code
|
|
254
|
+
IMPLEMENTED env-negation AWS_* spec SPEC.md:50 · code —
|
|
255
|
+
negation holds: no matching variable is read in the completely enumerated env class
|
|
256
|
+
|
|
257
|
+
CODE SURFACES
|
|
258
|
+
DOCUMENTED flag --mode code app.py:19 · spec SPEC.md:5
|
|
259
|
+
DOCUMENTED flag --verbose code app.py:20 · spec SPEC.md:7
|
|
260
|
+
DOCUMENTED flag --legacy code app.py:21 · spec SPEC.md:8
|
|
261
|
+
UNDOCUMENTED flag --extra code app.py:22 · spec —
|
|
262
|
+
DOCUMENTED command export code app.py:24 · spec SPEC.md:10
|
|
263
|
+
DOCUMENTED endpoint GET /items code app.py:35 · spec SPEC.md:21
|
|
264
|
+
DOCUMENTED config server code config.example.toml:2 · spec SPEC.md:28
|
|
265
|
+
DOCUMENTED config server.host code config.example.toml:3 · spec SPEC.md:29
|
|
266
|
+
DOCUMENTED config server.port code config.example.toml:4 · spec SPEC.md:30
|
|
267
|
+
DOCUMENTED config database code config.example.toml:6 · spec SPEC.md:31
|
|
268
|
+
DOCUMENTED config database.url code config.example.toml:7 · spec SPEC.md:32
|
|
269
|
+
DOCUMENTED env ITEMS_API_KEY code app.py:48 · spec SPEC.md:48
|
|
270
|
+
DOCUMENTED env ITEMS_REGION code app.py:49 · spec SPEC.md:49
|
|
271
|
+
|
|
272
|
+
Summary: 17 items (IMPLEMENTED 11, MISSING 1, CONTRADICTED 3, TRACED 1, UNTRACED 1); 13 surfaces (DOCUMENTED 12, UNDOCUMENTED 1)
|
|
273
|
+
Verdict: DRIFT
|
|
274
|
+
|
|
275
|
+
Limits:
|
|
276
|
+
- Stage 3 indexes five surface classes: CLI flags and commands (argparse + click, via AST), HTTP routes (Flask, FastAPI and Django URL declarations, via AST), config keys, and environment variables. Declaration forms outside those patterns are not indexed and never judged.
|
|
277
|
+
- Routes: only declared methods are indexed - a Flask route declared without methods counts as GET (its framework default); the automatic HEAD/OPTIONS a framework adds at runtime are not declarations and are not indexed. Django URL routes declare no method, so they are indexed as ANY-method surfaces: they can satisfy, but never contradict, a method-specific spec item. Blueprint/APIRouter constructor prefixes, register_blueprint prefixes and include_router prefixes are applied only when constant and declared in the same module; add_url_rule registrations are not indexed, and their presence marks the routes class partial.
|
|
278
|
+
- Config keys: presence only - values in example configs are never compared with anything. Indexed: keys of example/sample/config-named .toml, .ini and .json files (pyproject.toml and the manifest excluded) and config[...] / config.get(...) reads with constant-string keys, on a receiver named config. YAML configs cannot be parsed with the standard library and are not indexed; their presence marks the config class partial.
|
|
279
|
+
- Environment variables: only os.environ[...], os.getenv(...) and os.environ.get(...) with constant-string names (including the from-import spellings) are indexed. Names built at runtime mark the env class partial.
|
|
280
|
+
- Extraction scope: sections whose headings announce examples or definitions of the notation itself (Examples, Definitions, Glossary, Notation, 'spec item' forms) and lines that present an example (e.g. / for example) or describe the notation (a backticked token, of the form, written as, a placeholder) are illustrative, not declarative: no items are extracted from them. Text inside straight double quotes is quotation — the document talking about wording, not wording that declares — and is likewise never extracted. Template placeholders are not declarations either: an endpoint path whose every segment is a placeholder word (path, x, foo, ...) and a token naming a capitalisation style (SCREAMING_CASE) name the notation, not a surface. A name the document itself defines as one of its terms — in a definition table with a Meaning column, or in an enumerated list of terms with glosses — is vocabulary, not an environment-variable declaration.
|
|
281
|
+
- Fenced blocks: an untagged block counts as config only when it contains at least one key = value assignment, and a [section] header declares a key only when it stands alone on its line — sample output quoted in a spec is not configuration. Invocation lines in fenced blocks do declare commands, but a line invoking a generic shell or package tool (pip, pipx, python, git, ...) names that tool's own operand, never a command of the tool being spec'd.
|
|
282
|
+
- TRACED means a requirement ID has an anchor in code or tests; it never means the requirement is correctly implemented. Anchors are scoped to the requirement's own spec: an anchor must carry the spec's feature qualifier, sit in a file of the feature, or — when the ID is declared by only one spec in its tree — sit in that tree. A comment block that lists an ID as unanchored, or enumerates several IDs, is a denial or a listing, never an anchor — and its words are not term evidence either. An ID found only outside its scope is UNTRACED, not traced to the wrong place. UNTRACED is not MISSING: natural language cannot prove a behaviour absent.
|
|
283
|
+
- MISSING is issued only when the item's surface class was completely enumerated and the item is absent from it; on partial enumeration an absent item is UNTRACED.
|
|
284
|
+
- Static evidence only: nothing in the target repo is executed, imported or run. Surfaces built purely at runtime may not appear in the index.
|
|
285
|
+
- Python code only. A repo with no Python surfaces reports its classes enumerated to empty.
|
|
286
|
+
(exit code 1)
|
|
287
|
+
```
|
|
288
|
+
|
|
289
|
+
Note the verdict that gives this tool its name in the
|
|
290
|
+
author's own launch post: the spec says `POST /items`;
|
|
291
|
+
the path is registered in code with GET only —
|
|
292
|
+
CONTRADICTED, cited on both sides (SPEC.md:22 against
|
|
293
|
+
app.py:35). And note what is *not* overclaimed:
|
|
294
|
+
FR-002's tests were renamed away, so it is UNTRACED —
|
|
295
|
+
reported, counted, and not upgraded to MISSING,
|
|
296
|
+
because natural language cannot prove a behaviour
|
|
297
|
+
absent.
|
|
298
|
+
|
|
299
|
+
### The clean fixture — the other direction
|
|
300
|
+
|
|
301
|
+
`tests/fixtures/clean/` declares every surface exactly
|
|
302
|
+
— 3 flags (one with a stated default, one with
|
|
303
|
+
choices), a command, 2 requirements, a Flask route
|
|
304
|
+
pair, 5 config keys, 2 env vars and an env negation —
|
|
305
|
+
and the code matches, surface for surface. Verbatim
|
|
306
|
+
output (the Limits block, printed identically on every
|
|
307
|
+
run, is elided here only):
|
|
308
|
+
|
|
309
|
+
```
|
|
310
|
+
$ spec-receipt check tests/fixtures/clean
|
|
311
|
+
spec-receipt 1.0.0 — spec drift receipt
|
|
312
|
+
Repo: clean
|
|
313
|
+
Spec: SPEC.md
|
|
314
|
+
Classes: flags complete (3 surfaces), commands complete (1 surface), routes complete (2 surfaces), config complete (5 surfaces), env complete (2 surfaces)
|
|
315
|
+
|
|
316
|
+
SPEC ITEMS
|
|
317
|
+
IMPLEMENTED flag --mode spec SPEC.md:5 · code app.py:16
|
|
318
|
+
surface declared in code; stated default and choices match
|
|
319
|
+
IMPLEMENTED flag --level spec SPEC.md:6 · code app.py:17
|
|
320
|
+
surface declared in code; stated default and choices match
|
|
321
|
+
IMPLEMENTED flag --verbose spec SPEC.md:7 · code app.py:18
|
|
322
|
+
surface declared in code
|
|
323
|
+
IMPLEMENTED negation --legacy spec SPEC.md:8 · code —
|
|
324
|
+
negation holds: --legacy is absent from the completely enumerated flags class
|
|
325
|
+
IMPLEMENTED command export spec SPEC.md:10 · code app.py:20
|
|
326
|
+
command declared in code
|
|
327
|
+
TRACED requirement FR-001 spec SPEC.md:16 · code tests/test_app.py:1
|
|
328
|
+
ID FR-001 appears verbatim in code/tests
|
|
329
|
+
TRACED requirement FR-002 spec SPEC.md:17 · code tests/test_app.py:1
|
|
330
|
+
ID FR-002 appears verbatim in code/tests
|
|
331
|
+
IMPLEMENTED endpoint GET /items spec SPEC.md:21 · code app.py:31
|
|
332
|
+
route declared in code with this method
|
|
333
|
+
IMPLEMENTED endpoint POST /items spec SPEC.md:22 · code app.py:36
|
|
334
|
+
route declared in code with this method
|
|
335
|
+
IMPLEMENTED config server spec SPEC.md:28 · code config.example.toml:2
|
|
336
|
+
declared in the config index (example config or code read)
|
|
337
|
+
IMPLEMENTED config server.host spec SPEC.md:29 · code config.example.toml:3
|
|
338
|
+
declared in the config index (example config or code read)
|
|
339
|
+
IMPLEMENTED config server.port spec SPEC.md:30 · code config.example.toml:4
|
|
340
|
+
declared in the config index (example config or code read)
|
|
341
|
+
IMPLEMENTED config database spec SPEC.md:31 · code config.example.toml:6
|
|
342
|
+
declared in the config index (example config or code read)
|
|
343
|
+
IMPLEMENTED config database.url spec SPEC.md:32 · code config.example.toml:7
|
|
344
|
+
declared in the config index (example config or code read)
|
|
345
|
+
IMPLEMENTED env ITEMS_API_KEY spec SPEC.md:48 · code app.py:42
|
|
346
|
+
read in code
|
|
347
|
+
IMPLEMENTED env ITEMS_REGION spec SPEC.md:49 · code app.py:43
|
|
348
|
+
read in code
|
|
349
|
+
IMPLEMENTED env-negation AWS_* spec SPEC.md:50 · code —
|
|
350
|
+
negation holds: no matching variable is read in the completely enumerated env class
|
|
351
|
+
|
|
352
|
+
CODE SURFACES
|
|
353
|
+
DOCUMENTED flag --mode code app.py:16 · spec SPEC.md:5
|
|
354
|
+
DOCUMENTED flag --level code app.py:17 · spec SPEC.md:6
|
|
355
|
+
DOCUMENTED flag --verbose code app.py:18 · spec SPEC.md:7
|
|
356
|
+
DOCUMENTED command export code app.py:20 · spec SPEC.md:10
|
|
357
|
+
DOCUMENTED endpoint GET /items code app.py:31 · spec SPEC.md:21
|
|
358
|
+
DOCUMENTED endpoint POST /items code app.py:36 · spec SPEC.md:22
|
|
359
|
+
DOCUMENTED config server code config.example.toml:2 · spec SPEC.md:28
|
|
360
|
+
DOCUMENTED config server.host code config.example.toml:3 · spec SPEC.md:29
|
|
361
|
+
DOCUMENTED config server.port code config.example.toml:4 · spec SPEC.md:30
|
|
362
|
+
DOCUMENTED config database code config.example.toml:6 · spec SPEC.md:31
|
|
363
|
+
DOCUMENTED config database.url code config.example.toml:7 · spec SPEC.md:32
|
|
364
|
+
DOCUMENTED env ITEMS_API_KEY code app.py:42 · spec SPEC.md:48
|
|
365
|
+
DOCUMENTED env ITEMS_REGION code app.py:43 · spec SPEC.md:49
|
|
366
|
+
|
|
367
|
+
Summary: 17 items (IMPLEMENTED 15, MISSING 0, CONTRADICTED 0, TRACED 2, UNTRACED 0); 13 surfaces (DOCUMENTED 13, UNDOCUMENTED 0)
|
|
368
|
+
Verdict: NO_DRIFT
|
|
369
|
+
(exit code 0)
|
|
370
|
+
```
|
|
371
|
+
|
|
372
|
+
Any CONTRADICTED on this fixture is a release blocker;
|
|
373
|
+
the suite pins it.
|
|
374
|
+
|
|
375
|
+
### The dogfood gate — this tool was caught lying first
|
|
376
|
+
|
|
377
|
+
Before v1.0.0, spec-receipt was run against three real
|
|
378
|
+
spec'd repos — its own design document, skill-claims,
|
|
379
|
+
and the public spec-kit-layout project
|
|
380
|
+
chenhz01/spec-drift — with every CONTRADICTED and
|
|
381
|
+
every TRACED citation hand-checked against the actual
|
|
382
|
+
code. **Run 1 failed the precision bar.** Coverage
|
|
383
|
+
passed (67.5% of items received a deterministic
|
|
384
|
+
verdict, against a 50% kill line), but all 3
|
|
385
|
+
CONTRADICTED verdicts were false in intent: the
|
|
386
|
+
extractor had read the design document's *examples of
|
|
387
|
+
spec-item forms* — an illustrative `--mode` with a
|
|
388
|
+
default, a quoted "no `--legacy`" — as declarations,
|
|
389
|
+
and formed verdicts against pairs that should never
|
|
390
|
+
have existed. Requirement anchors were also matched
|
|
391
|
+
repo-wide, so one feature's `FR-001` was "traced" to
|
|
392
|
+
another feature's tests, and two IDs were traced to a
|
|
393
|
+
comment block that explicitly lists them as having
|
|
394
|
+
*no* anchor.
|
|
395
|
+
|
|
396
|
+
The fix was one class — extraction scoping, not the
|
|
397
|
+
verdict engine: illustrative sections and lines,
|
|
398
|
+
template placeholders, the document's own defined
|
|
399
|
+
terms, quoted mentions, and sample output no longer
|
|
400
|
+
form items; fenced invocation lines do form commands;
|
|
401
|
+
anchors are scoped to the requirement's own spec, and
|
|
402
|
+
a comment block that denies anchoring is never an
|
|
403
|
+
anchor. Every failure shape became a regression test.
|
|
404
|
+
**The rerun, unchanged targets and commands: zero
|
|
405
|
+
false CONTRADICTED on any target, every TRACED
|
|
406
|
+
citation correctly scoped or honestly UNTRACED, mean
|
|
407
|
+
coverage 92.6%.** The full reports are in the
|
|
408
|
+
project's working notes (`DOGFOOD-SPEC-RECEIPT.md`,
|
|
409
|
+
`DOGFOOD-RERUN.md`); the disclosed trade-offs of the
|
|
410
|
+
scoping rules are in [Honest limits](#honest-limits).
|
|
411
|
+
|
|
412
|
+
### For agents
|
|
413
|
+
|
|
414
|
+
`--json` emits one object: the tool and repo record,
|
|
415
|
+
the per-class enumeration state (`complete` /
|
|
416
|
+
`surfaces` count, with `index_notes` explaining every
|
|
417
|
+
partial marking), the items (each with `kind`, `name`,
|
|
418
|
+
spec file + line, code file + line or null, `verdict`,
|
|
419
|
+
and `basis`; flags may carry `stated_default` /
|
|
420
|
+
`stated_choices`; requirements carry `text`; endpoints
|
|
421
|
+
carry `method` and `path`), the surfaces (each with
|
|
422
|
+
its verdict and both citations), summary counts that
|
|
423
|
+
recount exactly against the items and surfaces, the
|
|
424
|
+
overall verdict, the verdict rule, and the limits
|
|
425
|
+
statement. Input errors — including argument errors —
|
|
426
|
+
are a single JSON object with an `error` key and exit
|
|
427
|
+
2. [SKILL.md](SKILL.md) states the full verdict rules
|
|
428
|
+
and the JSON contract — and makes spec-receipt itself
|
|
429
|
+
installable as a skill: point your agent at this
|
|
430
|
+
repo's SKILL.md and it will receipt spec'd repos
|
|
431
|
+
before trusting them.
|
|
432
|
+
|
|
433
|
+
### Tests
|
|
434
|
+
|
|
435
|
+
```
|
|
436
|
+
python3 -m unittest discover -s tests
|
|
437
|
+
```
|
|
438
|
+
|
|
439
|
+
87 tests: both fixtures pinned to the exact verdict
|
|
440
|
+
multisets above (clean: 17 items — 15 IMPLEMENTED, 2
|
|
441
|
+
TRACED — 13 surfaces all DOCUMENTED, exit 0; drifted:
|
|
442
|
+
17 items — 11 IMPLEMENTED, 1 MISSING, 3 CONTRADICTED,
|
|
443
|
+
1 TRACED, 1 UNTRACED — 13 surfaces, 1 UNDOCUMENTED,
|
|
444
|
+
exit 1); unit tests for every extractor and verdict
|
|
445
|
+
rule across all five classes (defaults, choices,
|
|
446
|
+
method sets, negations, exclusivity, prefix
|
|
447
|
+
resolution); partial-enumeration degradation (dynamic
|
|
448
|
+
surfaces degrade absence to UNTRACED, never MISSING);
|
|
449
|
+
the error contract (no spec, unreadable spec,
|
|
450
|
+
unparseable files — clean `{"error": ...}`, exit 2,
|
|
451
|
+
never a traceback, in text and `--json` modes); the
|
|
452
|
+
stage-3 regression tests, one per dogfood failure
|
|
453
|
+
shape; and the safety proofs — nothing in a target
|
|
454
|
+
repo is ever executed, fixture trees are byte-identical
|
|
455
|
+
after runs, output is byte-identical across runs, and
|
|
456
|
+
the source's imports are pinned to a stdlib set with
|
|
457
|
+
no network or subprocess module.
|
|
458
|
+
|
|
459
|
+
### Honest limits
|
|
460
|
+
|
|
461
|
+
- **Static evidence only — runtime behaviour is out
|
|
462
|
+
of scope.** Receipt reads the repo's text and AST;
|
|
463
|
+
it never runs anything. Surfaces built purely at
|
|
464
|
+
runtime (routes registered from data, flags built
|
|
465
|
+
in loops from non-constant names) may not appear in
|
|
466
|
+
the index. Where the tool can see a class cannot be
|
|
467
|
+
fully enumerated, it marks the class partial and
|
|
468
|
+
absence verdicts for it degrade to UNTRACED.
|
|
469
|
+
- **CONTRADICTED is reserved for hard evidence.** A
|
|
470
|
+
spec statement flips only on a direct conflict
|
|
471
|
+
cited on both sides. A stated parameter whose code
|
|
472
|
+
value is not statically resolvable does not
|
|
473
|
+
contradict — the surface is reported as matching,
|
|
474
|
+
with the parameter unverifiable by construction
|
|
475
|
+
(non-constant choices never contradict a stated
|
|
476
|
+
set; an absent restriction does, because the code
|
|
477
|
+
provably accepts more than the spec allows).
|
|
478
|
+
- **Requirements are traced, never judged.** There is
|
|
479
|
+
no verdict for "this prose requirement is correctly
|
|
480
|
+
implemented" and none for "it is absent". TRACED
|
|
481
|
+
means an anchor exists, scoped to the ID's own
|
|
482
|
+
spec — never that the requirement is met.
|
|
483
|
+
- **Routes: only declared methods are indexed.** A
|
|
484
|
+
Flask route declared without `methods` counts as
|
|
485
|
+
GET (its framework default); the automatic
|
|
486
|
+
HEAD/OPTIONS a framework adds at runtime are not
|
|
487
|
+
declarations. Django routes declare no method and
|
|
488
|
+
are indexed as ANY-method surfaces: they can
|
|
489
|
+
satisfy, but never contradict, a method-specific
|
|
490
|
+
item. Blueprint/APIRouter constructor prefixes,
|
|
491
|
+
`register_blueprint` and `include_router` prefixes
|
|
492
|
+
are applied only when constant and in the same
|
|
493
|
+
module; `add_url_rule` registrations are not
|
|
494
|
+
indexed, and mark the routes class partial.
|
|
495
|
+
- **Config keys are presence only.** Values are
|
|
496
|
+
never compared with anything. Only example/sample/
|
|
497
|
+
config-named `.toml`, `.ini` and `.json` files and
|
|
498
|
+
`config[...]` / `config.get(...)` reads with
|
|
499
|
+
constant-string keys on a receiver named `config`
|
|
500
|
+
are indexed; other receiver names are refused, and
|
|
501
|
+
YAML configs cannot be parsed with the standard
|
|
502
|
+
library — their presence marks the class partial.
|
|
503
|
+
- **Environment variables: constant names only.**
|
|
504
|
+
Only `os.environ[...]`, `os.getenv(...)` and
|
|
505
|
+
`os.environ.get(...)` with constant-string names
|
|
506
|
+
(including the from-import spellings) are indexed.
|
|
507
|
+
Names built at runtime mark the env class partial.
|
|
508
|
+
- **Extraction scope — the stage-3 trade-offs,
|
|
509
|
+
disclosed.** Sections whose headings announce
|
|
510
|
+
examples or definitions of the notation itself
|
|
511
|
+
(Examples, Definitions, Glossary, Notation,
|
|
512
|
+
"spec item" forms) and lines that present an
|
|
513
|
+
example ("e.g." / "for example") or describe the
|
|
514
|
+
notation are illustrative, not declarative — no
|
|
515
|
+
items are extracted from them; the price is that a
|
|
516
|
+
genuine declaration sharing a line with "e.g." is
|
|
517
|
+
missed too. Text inside straight double quotes is
|
|
518
|
+
quotation and is never extracted. Template
|
|
519
|
+
placeholders are not declarations: an endpoint path
|
|
520
|
+
whose every segment is a placeholder word (`/path`,
|
|
521
|
+
`/x`) names the notation, not a surface — so a
|
|
522
|
+
literal route made only of placeholder words is
|
|
523
|
+
template text to the tool. A name the document
|
|
524
|
+
itself defines as one of its terms (a definition
|
|
525
|
+
table with a Meaning column, an enumerated list of
|
|
526
|
+
terms with glosses) is vocabulary, not an env-var
|
|
527
|
+
declaration — and a term enumeration with
|
|
528
|
+
parenthetical glosses is read the same way, so env
|
|
529
|
+
vars declared in exactly that shape are missed.
|
|
530
|
+
Sample output quoted in a spec is not
|
|
531
|
+
configuration: an untagged fence counts as config
|
|
532
|
+
only with at least one `key = value` assignment,
|
|
533
|
+
and a `[section]` header declares a key only when
|
|
534
|
+
it stands alone on its line.
|
|
535
|
+
- **Requirement anchors are scoped.** An anchor must
|
|
536
|
+
carry the spec's feature qualifier (e.g.
|
|
537
|
+
`FR-001 (001)`), sit in a file of the feature, or —
|
|
538
|
+
when the ID is declared by only one spec in its
|
|
539
|
+
tree — sit in that tree. A comment block that lists
|
|
540
|
+
an ID as unanchored, or enumerates several IDs, is
|
|
541
|
+
a denial or a listing, never an anchor — and its
|
|
542
|
+
words are not term evidence either. An ID found
|
|
543
|
+
only outside its scope is UNTRACED, not traced to
|
|
544
|
+
the wrong place.
|
|
545
|
+
- **Python implementations only.** Other languages
|
|
546
|
+
are not half-scanned; a repo with no Python
|
|
547
|
+
surfaces reports its classes enumerated to empty,
|
|
548
|
+
plainly.
|
|
549
|
+
- **No git/diff mode, no test running, no auto-fix,
|
|
550
|
+
no spec rewriting, no PR comments, no score.** A
|
|
551
|
+
receipt, not a repair tool — and never a drift
|
|
552
|
+
percentage or a grade.
|
|
553
|
+
|
|
554
|
+
## Contributing
|
|
555
|
+
|
|
556
|
+
See [CONTRIBUTING.md](CONTRIBUTING.md). The evidence
|
|
557
|
+
discipline is the product: a verdict is issued only
|
|
558
|
+
on hard evidence, cited on both sides, and
|
|
559
|
+
contributions are held to that. Found a repo
|
|
560
|
+
spec-receipt misjudges — a declaration it misses, a
|
|
561
|
+
verdict the evidence doesn't support? Open an issue
|
|
562
|
+
with the repo (or a minimal fixture); that is exactly
|
|
563
|
+
the feedback that improves the extraction library.
|
|
564
|
+
|
|
565
|
+
## Security
|
|
566
|
+
|
|
567
|
+
spec-receipt is itself an audit tool, so its own
|
|
568
|
+
posture is simple: it has no network code, no
|
|
569
|
+
subprocess code, no dependencies, and it never
|
|
570
|
+
executes what it scans. See [SECURITY.md](SECURITY.md)
|
|
571
|
+
for how to report a problem.
|
|
572
|
+
|
|
573
|
+
## License
|
|
574
|
+
|
|
575
|
+
MIT — see [LICENSE](LICENSE).
|
|
576
|
+
|
|
577
|
+
## Related
|
|
578
|
+
|
|
579
|
+
- [skill-claims](https://github.com/jasveerai88/skill-claims)
|
|
580
|
+
— the honesty half of the receipt family: does a
|
|
581
|
+
skill package do what its description claims?
|
|
582
|
+
- [done-claim-auditor](https://github.com/jasveerai88/done-claim-auditor)
|
|
583
|
+
— the same claim-vs-evidence logic pointed at agent
|
|
584
|
+
sessions: audits a coding agent's completion claims
|
|
585
|
+
against its own transcript.
|
|
586
|
+
- [skill-vet](https://github.com/jasveerai88/skill-vet)
|
|
587
|
+
— the pre-install danger gate: stealing, spying,
|
|
588
|
+
hidden sending, proven with file-and-line evidence.
|
|
589
|
+
- [permission-audit](https://github.com/jasveerai88/permission-audit)
|
|
590
|
+
— audits a Claude Code permission allowlist for
|
|
591
|
+
wildcard grants and dangerous modes before they hurt
|
|
592
|
+
you.
|
|
593
|
+
|
|
594
|
+
## Support
|
|
595
|
+
|
|
596
|
+
If spec-receipt catches your code drifting away from
|
|
597
|
+
your spec, consider
|
|
598
|
+
[sponsoring](https://github.com/sponsors/jasveerai88).
|