@agent-ix/quoin 0.12.0 → 0.12.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/{base-DGA47VMM.js → base-DD0ONCEf.js} +1 -1
- package/dist/cli.d.ts +15 -0
- package/dist/cli.d.ts.map +1 -1
- package/dist/cli.js +11 -7
- package/dist/commands/catalog/list.js +2 -2
- package/dist/commands/catalog/show.js +2 -2
- package/dist/commands/catalog/validate.js +2 -2
- package/dist/commands/config/doctor.js +1 -1
- package/dist/commands/config/edit.js +1 -1
- package/dist/commands/config/get.js +1 -1
- package/dist/commands/config/set.js +1 -1
- package/dist/commands/matrix.js +1 -1
- package/dist/commands/module/ensure-defaults.js +3 -3
- package/dist/commands/module/install.js +2 -2
- package/dist/commands/module/list.js +2 -2
- package/dist/commands/module/remove.js +2 -2
- package/dist/commands/review.js +1 -1
- package/dist/commands/to-plan.js +1 -1
- package/dist/commands/update.js +2 -2
- package/dist/commands/write.js +3 -3
- package/dist/{flow-command-CPS23AjI.js → flow-command-DK5LDrtg.js} +1 -1
- package/dist/hooks/command-not-found.d.ts +23 -0
- package/dist/hooks/command-not-found.d.ts.map +1 -0
- package/dist/hooks/command-not-found.js +8 -0
- package/dist/index.js +6 -6
- package/dist/{modules-CCvL41O_.js → modules-_4_rnXpw.js} +2 -2
- package/dist/{plugins-ByvihMa0.js → plugins-BO1uLsfi.js} +1 -1
- package/dist/{version-Be_NtSa4.js → version-DIJF4enj.js} +1 -1
- package/dist/{write-xs9kemgB.js → write-D462q_kk.js} +1 -1
- package/package.json +5 -2
- package/skills/gap-analysis/SKILL.md +5 -3
- package/skills/gap-analysis/references/step-3-matrix-verification.md +94 -24
- package/skills/gap-analysis/references/step-6-specreview-artifact.md +6 -1
- package/skills/spec-correctness/SKILL.md +11 -13
- package/skills/spec-correctness/assets/strategy-table.md +3 -3
- package/skills/spec-correctness/assets/templates/fast-check.ts.md +41 -25
- package/skills/spec-correctness/assets/templates/hypothesis.py.md +16 -15
- package/skills/spec-correctness/assets/templates/proptest.rs.md +12 -9
- package/skills/spec-correctness/references/step-0-scope-and-harness.md +6 -6
- package/skills/spec-correctness/references/step-1-census.md +1 -1
- package/skills/spec-correctness/references/step-2-clause-grounding.md +2 -2
- package/skills/spec-correctness/references/step-3-strategy-selection.md +13 -11
- package/skills/spec-correctness/references/step-4-generate-tests.md +43 -25
- package/skills/spec-correctness/references/step-5-second-pass.md +11 -9
- package/skills/spec-correctness/references/step-6-review-artifact.md +121 -0
- package/skills/spec-correctness/references/step-7-report-and-handoff.md +38 -17
- package/skills/spec-correctness/assets/property-queue-template.md +0 -78
- package/skills/spec-correctness/references/step-6-review-queue.md +0 -64
|
@@ -5,21 +5,22 @@ reconcile every tag this run emitted.
|
|
|
5
5
|
|
|
6
6
|
## The run report
|
|
7
7
|
|
|
8
|
-
|
|
8
|
+
The `## Census` section of the review artifact (step 6), and the same counts to the user.
|
|
9
|
+
Counts only:
|
|
9
10
|
|
|
10
11
|
```
|
|
11
12
|
spec-correctness — <repo> — <YYYY-MM-DD>
|
|
12
13
|
quire-cli <version> · harness <name> · <N> criteria
|
|
13
14
|
|
|
14
|
-
emitted
|
|
15
|
-
|
|
16
|
-
|
|
15
|
+
emitted 37 extractable, grounded, no finding
|
|
16
|
+
emitted + finding 22 candidate 0 · second-pass 18 · downgraded 1 · dep-missing 3
|
|
17
|
+
not settled 14 symbol-not-found 6 · oracle-is-adjectival 5 · unimplemented 3
|
|
17
18
|
already covered 9 hand-written tests already carry the row_id
|
|
18
19
|
witnesses 5 Unit tests, not property coverage
|
|
19
20
|
```
|
|
20
21
|
|
|
21
|
-
`emitted
|
|
22
|
-
with a `row_id`. If it does not, a record was dropped — find it before reporting.
|
|
22
|
+
`emitted + emitted-with-finding + not-settled + already-covered` must equal the number of
|
|
23
|
+
records with a `row_id`. If it does not, a record was dropped — find it before reporting.
|
|
23
24
|
|
|
24
25
|
No thresholds, no grades, no rewording suggestions. Same rule as step 1.
|
|
25
26
|
|
|
@@ -36,28 +37,48 @@ already runs the matrix workflow; hand them over.
|
|
|
36
37
|
it disagrees with what you write, the manifest wins.
|
|
37
38
|
- `Traces To` — the `row_id`, exactly (`FR-027-AC-1`). Never a range you invented, never
|
|
38
39
|
another TC.
|
|
39
|
-
- `Status` — `✅` only for
|
|
40
|
-
|
|
41
|
-
matrix.
|
|
40
|
+
- `Status` — `✅` only for a test that actually passed in this run. A criterion with no
|
|
41
|
+
test gets **no row**; it is a finding in the review artifact, not a claim in the matrix.
|
|
42
42
|
|
|
43
|
-
##
|
|
43
|
+
## Binding check
|
|
44
44
|
|
|
45
|
-
Before finishing, prove
|
|
45
|
+
Before finishing, prove every emitted tag actually **binds** — not that it exists.
|
|
46
46
|
|
|
47
47
|
```
|
|
48
|
-
quire
|
|
48
|
+
quire coverage --scope <repo> --json
|
|
49
49
|
```
|
|
50
|
-
then run `gap-analysis` step 3's index over the test tree and confirm:
|
|
51
50
|
|
|
52
|
-
|
|
51
|
+
For each `row_id` this run emitted, confirm the id appears among the backed ids: its
|
|
52
|
+
minting document's group must count it, and it must not appear in `untracked_symbols`.
|
|
53
|
+
|
|
54
|
+
```
|
|
55
|
+
quire coverage --scope <repo> --json \
|
|
56
|
+
| jq -r '.untracked_symbols[] | "\(.trace_id)\t\(.path)\t\(.symbol)"'
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
An emitted `row_id` that is *not* backed means the tag is in the file but attached to no
|
|
60
|
+
test symbol — almost always TypeScript placement, a tag above `describe(` instead of above
|
|
61
|
+
`it(` (step 4). Fix the placement and re-run; do not report the criterion as covered.
|
|
62
|
+
|
|
63
|
+
**This check replaces a grep, deliberately.** The previous version of this step confirmed
|
|
64
|
+
the tags were greppable, and a grep matches a comment wherever it sits. Six generated files
|
|
65
|
+
passed that check with every tag bound to nothing (agent-ix/quoin#61). Only the engine that
|
|
66
|
+
consumes the tags can tell you a tag works.
|
|
67
|
+
|
|
68
|
+
Then confirm, as before:
|
|
69
|
+
|
|
53
70
|
- no matrix row this run added is `✅` while its test is skipped;
|
|
54
71
|
- no emitted tag names a `row_id` absent from the `quire properties` output.
|
|
55
72
|
|
|
73
|
+
If `quire coverage` reports `no rows matched`, the module in scope declares no traceability
|
|
74
|
+
model for this repo's layout — that is an environment gap, not a clean run. Say so rather
|
|
75
|
+
than reporting the run as reconciled.
|
|
76
|
+
|
|
56
77
|
A mismatch here is a bug in this run, not in `gap-analysis`.
|
|
57
78
|
|
|
58
79
|
## Finish by saying
|
|
59
80
|
|
|
60
81
|
- What now has tests, in concrete terms: which FRs, how many criteria.
|
|
61
|
-
-
|
|
62
|
-
- The one next action — usually "
|
|
63
|
-
dev-dependencies
|
|
82
|
+
- How many findings are waiting and roughly how long reading them would take.
|
|
83
|
+
- The one next action — usually "read `reviews/<date>-<slug>.md`" or "add `<lib>` to
|
|
84
|
+
dev-dependencies, then re-run to emit the tests it blocked".
|
|
@@ -1,78 +0,0 @@
|
|
|
1
|
-
# Property Test Queue
|
|
2
|
-
|
|
3
|
-
<!-- Written by the spec-correctness skill. Not a spec artifact; not validated by quire. -->
|
|
4
|
-
|
|
5
|
-
spec-correctness — `<repo>` — `<YYYY-MM-DD>`
|
|
6
|
-
quire-cli `<version>` · harness `<name>` · `<N>` criteria
|
|
7
|
-
|
|
8
|
-
## Run report
|
|
9
|
-
|
|
10
|
-
```
|
|
11
|
-
emitted unattended <n> (extractable, grounded)
|
|
12
|
-
queued <n> candidate <n> · second-pass <n> · downgraded <n> · dep-missing <n>
|
|
13
|
-
refused <n> symbol-not-found <n> · oracle-is-adjectival <n> · unimplemented <n>
|
|
14
|
-
already covered <n> hand-written tests already carry the row_id
|
|
15
|
-
witnesses <n> Unit tests, not property coverage
|
|
16
|
-
```
|
|
17
|
-
|
|
18
|
-
`emitted unattended + queued + refused + already covered` = records carrying a `row_id`.
|
|
19
|
-
|
|
20
|
-
## Queued — awaiting review
|
|
21
|
-
|
|
22
|
-
Each row is inert in the runner until accepted. Accepting is: delete the skip marker, move
|
|
23
|
-
the file out of `_review/`, change `review=required` to `review=accepted`, run the suite,
|
|
24
|
-
flip the matrix row from 🚧 to ✅. The tracking tag never changes.
|
|
25
|
-
|
|
26
|
-
| row_id | property | origin | confidence | file | why it is queued |
|
|
27
|
-
| --- | --- | --- | --- | --- | --- |
|
|
28
|
-
| FR-018-AC-3 | invariant | regex-candidate | — | `tests/props/_review/fr-018.prop.test.ts` | metamorphic label not corroborated by the structural pass |
|
|
29
|
-
| FR-005-AC-2 | example | llm-second-pass | medium | `tests/props/_review/fr-005.prop.test.ts` | reclassified as error-case over a negative domain |
|
|
30
|
-
| FR-050-AC-1 | concurrency | llm-second-pass | low | `tests/props/_review/test_fr_050.py` | hypothesis has no interleaving scheduler; interleavings not exhaustive |
|
|
31
|
-
|
|
32
|
-
## Witnesses — Unit tests, not property coverage
|
|
33
|
-
|
|
34
|
-
| row_id | file | the case it pins |
|
|
35
|
-
| --- | --- | --- |
|
|
36
|
-
| FR-002-AC-1 | `tests/props/_review/fr-002.prop.test.ts` | `version`, `--version`, `-v` each print the package version |
|
|
37
|
-
|
|
38
|
-
## Refused — no test written
|
|
39
|
-
|
|
40
|
-
Not findings. A refusal says this run could not ground the criterion; several of these are
|
|
41
|
-
criteria that are correctly verified by inspection rather than by test.
|
|
42
|
-
|
|
43
|
-
| row_id | reason | note |
|
|
44
|
-
| --- | --- | --- |
|
|
45
|
-
| NFR-003-AC-2 | oracle-is-adjectival | "actionable" has no observable in `## Outputs`; verified by inspection |
|
|
46
|
-
| FR-022-AC-4 | unimplemented | `selfUpdate` is not in the source yet |
|
|
47
|
-
| FR-011-AC-6 | ambiguous-symbol | `list` matches two exported functions; no `Verification` column to disambiguate |
|
|
48
|
-
|
|
49
|
-
## Rejected — do not re-propose
|
|
50
|
-
|
|
51
|
-
A re-run skips these `row_id`s unless the criterion's `statement` changes.
|
|
52
|
-
|
|
53
|
-
| row_id | rejected on | reason |
|
|
54
|
-
| --- | --- | --- |
|
|
55
|
-
|
|
56
|
-
## Dependency remedies
|
|
57
|
-
|
|
58
|
-
No test file is written when the generator library is absent — an import of a missing
|
|
59
|
-
library breaks test collection even for a skipped test. The proposals sit below as code
|
|
60
|
-
blocks until the dependency is added and the skill is re-run.
|
|
61
|
-
|
|
62
|
-
| harness | remedy |
|
|
63
|
-
| --- | --- |
|
|
64
|
-
| fast-check | `add fast-check to devDependencies, then re-run spec-correctness to emit these` |
|
|
65
|
-
|
|
66
|
-
### Proposals awaiting the dependency
|
|
67
|
-
|
|
68
|
-
<details><summary>FR-012-AC-1 · ordering · extractable</summary>
|
|
69
|
-
|
|
70
|
-
```ts
|
|
71
|
-
// tests/props/fr-012.prop.test.ts
|
|
72
|
-
/**
|
|
73
|
-
* Trace: FR-012-AC-1 — declaring modules are listed in sorted order.
|
|
74
|
-
* spec-correctness: row=FR-012-AC-1 property=ordering extraction=extractable origin=regex review=none
|
|
75
|
-
*/
|
|
76
|
-
```
|
|
77
|
-
|
|
78
|
-
</details>
|
|
@@ -1,64 +0,0 @@
|
|
|
1
|
-
# Step 6: The Review Queue
|
|
2
|
-
|
|
3
|
-
**Goal**: everything not settled unattended lands somewhere a person can accept or reject
|
|
4
|
-
it, and lands **inert** so it cannot be mistaken for coverage.
|
|
5
|
-
|
|
6
|
-
## Where it lives
|
|
7
|
-
|
|
8
|
-
`tests/props/QUEUE.md`, in the test tree. Template:
|
|
9
|
-
[`../assets/property-queue-template.md`](../assets/property-queue-template.md).
|
|
10
|
-
|
|
11
|
-
It is **not** a `SpecReview`. `SpecReview.analysis` is a closed enum in
|
|
12
|
-
`spec-artifacts-process` with no `spec-correctness` value, so a file under `reviews/` would
|
|
13
|
-
fail `quire validate`. Keeping the queue in the test tree also keeps it out of every
|
|
14
|
-
`spec/**` validation glob. Adding the enum value upstream is a reasonable follow-up; it is
|
|
15
|
-
not a prerequisite for this skill.
|
|
16
|
-
|
|
17
|
-
## What goes in it
|
|
18
|
-
|
|
19
|
-
| Lane | Reason it is queued |
|
|
20
|
-
| --- | --- |
|
|
21
|
-
| `extraction: candidate` | Metamorphic label the structural pass did not corroborate |
|
|
22
|
-
| Second-pass reclassify | An LLM read, not a deterministic one |
|
|
23
|
-
| Second-pass witness | An example test, not property coverage |
|
|
24
|
-
| `singleton-domain` witness | Classified `extractable`, but grounding found a one-element domain — a `Unit` test, not property coverage |
|
|
25
|
-
| Grounded but harness-downgraded | e.g. Python `concurrency` |
|
|
26
|
-
| Refused | No test written; the reason is the row |
|
|
27
|
-
| Generator library missing | **No file written** — the proposal is a code block in the queue. An import of a missing library breaks collection even for a skipped test |
|
|
28
|
-
|
|
29
|
-
## Inert by construction
|
|
30
|
-
|
|
31
|
-
Every queued test carries its harness's skip marker (see step 4). Two consequences worth
|
|
32
|
-
being explicit about:
|
|
33
|
-
|
|
34
|
-
- **Its matrix row is `🚧`, never `✅`.** `gap-analysis` treats a `✅` row backed by a
|
|
35
|
-
skipped test as a *status lie* and raises a `high` finding — correctly. Queued rows are
|
|
36
|
-
in-progress rows.
|
|
37
|
-
- **A skipped test still carries its tag**, so `gap-analysis` sees the trace and reports it
|
|
38
|
-
as present-but-not-passing rather than as unbacked.
|
|
39
|
-
|
|
40
|
-
## Acceptance procedure
|
|
41
|
-
|
|
42
|
-
For each accepted row:
|
|
43
|
-
|
|
44
|
-
1. Delete the skip marker line.
|
|
45
|
-
2. Move the file out of the review location: `tests/props/_review/` → `tests/props/`
|
|
46
|
-
for TypeScript and Python, `tests/props_review_fr_NNN.rs` → `tests/props_fr_NNN.rs`
|
|
47
|
-
for Rust, whose paths are flat (step 4).
|
|
48
|
-
3. Change the provenance line's `review=required` to `review=accepted`, leaving `row=` and
|
|
49
|
-
`origin=` untouched.
|
|
50
|
-
4. Run the suite.
|
|
51
|
-
5. Flip the matrix row from `🚧` to `✅`.
|
|
52
|
-
|
|
53
|
-
The tracking tag is **byte-identical** through all of this — that is the point. Nothing
|
|
54
|
-
about reconciliation changes when a test is accepted.
|
|
55
|
-
|
|
56
|
-
Rejection is simpler: delete the file, and leave the queue row with `status: rejected` and
|
|
57
|
-
a one-line reason so the next run does not re-propose it. A re-run reads existing
|
|
58
|
-
`rejected` rows and skips those `row_id`s unless the criterion's `statement` has changed.
|
|
59
|
-
|
|
60
|
-
## What the queue must not contain
|
|
61
|
-
|
|
62
|
-
- A verdict, a score, or a completion percentage framed as a target.
|
|
63
|
-
- A recommendation to edit any spec file.
|
|
64
|
-
- A `row_id` absent from the classification output.
|