@agent-ix/quoin 0.13.0 → 0.15.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/.codex-plugin/plugin.json +1 -1
- package/dist/advisor/advise.d.ts +57 -0
- package/dist/advisor/advise.d.ts.map +1 -0
- package/dist/advisor/index.d.ts +10 -0
- package/dist/advisor/index.d.ts.map +1 -0
- package/dist/advisor/methods.d.ts +50 -0
- package/dist/advisor/methods.d.ts.map +1 -0
- package/dist/advisor-D35K3_iI.js +50 -0
- package/dist/auditor/audit.d.ts +52 -0
- package/dist/auditor/audit.d.ts.map +1 -0
- package/dist/auditor/index.d.ts +8 -0
- package/dist/auditor/index.d.ts.map +1 -0
- package/dist/{base-D-_JMqkO.js → base-CtpwIg4M.js} +2 -2
- package/dist/catalog.d.ts +1 -0
- package/dist/catalog.d.ts.map +1 -1
- package/dist/cli.js +1 -1
- package/dist/commands/catalog/list.js +2 -2
- package/dist/commands/catalog/methods.d.ts +12 -0
- package/dist/commands/catalog/methods.d.ts.map +1 -0
- package/dist/commands/catalog/methods.js +43 -0
- package/dist/commands/catalog/show.js +2 -2
- package/dist/commands/catalog/validate.js +2 -2
- package/dist/commands/config/doctor.js +1 -1
- package/dist/commands/config/edit.js +1 -1
- package/dist/commands/config/get.js +1 -1
- package/dist/commands/config/set.js +1 -1
- package/dist/commands/evidence/affirm.d.ts +17 -0
- package/dist/commands/evidence/affirm.d.ts.map +1 -0
- package/dist/commands/evidence/affirm.js +81 -0
- package/dist/commands/evidence/audit.d.ts +15 -0
- package/dist/commands/evidence/audit.d.ts.map +1 -0
- package/dist/commands/evidence/audit.js +192 -0
- package/dist/commands/evidence/gc.d.ts +13 -0
- package/dist/commands/evidence/gc.d.ts.map +1 -0
- package/dist/commands/evidence/gc.js +34 -0
- package/dist/commands/evidence/index.d.ts +8 -0
- package/dist/commands/evidence/index.d.ts.map +1 -0
- package/dist/commands/evidence/index.js +16 -0
- package/dist/commands/evidence/record.d.ts +18 -0
- package/dist/commands/evidence/record.d.ts.map +1 -0
- package/dist/commands/evidence/record.js +85 -0
- package/dist/commands/matrix.js +1 -1
- package/dist/commands/module/ensure-defaults.js +3 -3
- package/dist/commands/module/install.js +2 -2
- package/dist/commands/module/list.js +2 -2
- package/dist/commands/module/remove.js +2 -2
- package/dist/commands/review.js +1 -1
- package/dist/commands/to-plan.js +1 -1
- package/dist/commands/update.js +2 -2
- package/dist/commands/write.js +3 -3
- package/dist/evidence/index.d.ts +13 -0
- package/dist/evidence/index.d.ts.map +1 -0
- package/dist/evidence/record.d.ts +52 -0
- package/dist/evidence/record.d.ts.map +1 -0
- package/dist/evidence/store.d.ts +77 -0
- package/dist/evidence/store.d.ts.map +1 -0
- package/dist/evidence/types.d.ts +122 -0
- package/dist/evidence/types.d.ts.map +1 -0
- package/dist/evidence-BvUyw-Yv.js +171 -0
- package/dist/{flow-command-BPwQwyEV.js → flow-command-BSMqxass.js} +1 -1
- package/dist/index.js +6 -6
- package/dist/{modules-Dn6ac8th.js → modules-Ckfaq5p4.js} +2 -2
- package/dist/{plugins-BCCcBgCW.js → plugins-CLuXx2Jr.js} +1 -1
- package/dist/quire/contract.d.ts +82 -0
- package/dist/quire/contract.d.ts.map +1 -0
- package/dist/quire/index.d.ts +11 -0
- package/dist/quire/index.d.ts.map +1 -0
- package/dist/quire/types.d.ts +155 -0
- package/dist/quire/types.d.ts.map +1 -0
- package/dist/quire/validate.d.ts +34 -0
- package/dist/quire/validate.d.ts.map +1 -0
- package/dist/quire-MbMdffPh.js +4872 -0
- package/dist/schemas/coverage-v1.schema.json +191 -0
- package/dist/schemas/properties-v1.schema.json +134 -0
- package/dist/{version-BGJINLrJ.js → version-By2YFRZQ.js} +1 -1
- package/dist/{write-FTRN1yh6.js → write-8JJ8Dumo.js} +1 -1
- package/package.json +6 -2
- package/skills/gap-analysis/SKILL.md +12 -0
- package/skills/gap-analysis/references/step-3-matrix-verification.md +24 -1
- package/skills/gap-analysis/references/step-6-specreview-artifact.md +7 -0
- package/skills/spec-correctness/references/step-6-review-artifact.md +13 -0
- package/skills/spec-correctness/references/step-7-report-and-handoff.md +21 -3
- package/skills/spec-ears-analysis/SKILL.md +13 -0
- package/skills/spec-evidence-analysis/SKILL.md +64 -52
- package/skills/spec-matrix/SKILL.md +13 -0
- package/skills/spec-review/SKILL.md +13 -0
- package/skills/spec-to-plan/SKILL.md +13 -0
|
@@ -1,78 +1,90 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: spec-evidence-analysis
|
|
3
|
-
description:
|
|
3
|
+
description: Recommend verification methods per obligation from the declared catalog, and flag mismatches with what was authored.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Verification & Evidence Strategy
|
|
7
7
|
|
|
8
|
-
|
|
8
|
+
Every requirement needs a defined verification method and a concrete artifact
|
|
9
|
+
that proves it was verified. This skill recommends the method **from the
|
|
10
|
+
declared catalog**, and flags where the authored choice disagrees.
|
|
11
|
+
|
|
12
|
+
> **This skill no longer carries its own method table.** It used to: a
|
|
13
|
+
> skill-local list of `test | analysis | inspection | demonstration` plus a few
|
|
14
|
+
> evidence kinds, declared in no manifest and read by no code. The result was
|
|
15
|
+
> `Verification` columns defaulting to `Test` by habit, with nothing ever
|
|
16
|
+
> advising DAST for an attack surface, monitors for a temporal property, or
|
|
17
|
+
> fault injection for a reliability NFR. The methods now live in module data
|
|
18
|
+
> (quire-rs FR-054, `spec-artifacts-process` FR-007) and are read from there.
|
|
9
19
|
|
|
10
20
|
## When to use
|
|
11
21
|
|
|
12
|
-
-
|
|
13
|
-
-
|
|
14
|
-
- Whenever a new NFR is added
|
|
22
|
+
- At **Spec Review**, to confirm each obligation's method before the matrix is built.
|
|
23
|
+
- At **Matrix** time, to plan which suites the chosen methods imply.
|
|
24
|
+
- Whenever a new NFR is added — NFRs are the ones most prone to a defaulted method.
|
|
15
25
|
|
|
16
|
-
##
|
|
26
|
+
## Read the catalog, do not recall it
|
|
17
27
|
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
| `demonstration` | The requirement is satisfied by observing operation in a realistic environment without instrumentation (e.g. UX flow walkthrough). |
|
|
28
|
+
```bash
|
|
29
|
+
quoin catalog methods # human-readable, grouped by class
|
|
30
|
+
quoin catalog methods --json # the merged catalog, for scripting
|
|
31
|
+
quoin catalog methods --class Analysis
|
|
32
|
+
```
|
|
24
33
|
|
|
25
|
-
|
|
34
|
+
The catalog is merged first-wins across active modules, exactly as quire-rs
|
|
35
|
+
merges it — so the method you are advised is the method the auditor will later
|
|
36
|
+
check conformance against.
|
|
26
37
|
|
|
27
|
-
|
|
28
|
-
|----------|------------|
|
|
29
|
-
| `test_case` | `test` — points to a TC-XXX file or test path |
|
|
30
|
-
| `analysis_report` | `analysis` — points to a doc/notebook |
|
|
31
|
-
| `inspection_checklist` | `inspection` — points to a checklist file |
|
|
32
|
-
| `demo_recording` | `demonstration` — points to a recording or scripted walkthrough |
|
|
33
|
-
| `metric` | any — observability metric that proves runtime behavior |
|
|
34
|
-
| `log` | any — log event proving the path executed |
|
|
38
|
+
## Process
|
|
35
39
|
|
|
36
|
-
**
|
|
40
|
+
1. **Derive the obligations.** `quire coverage --json` carries them
|
|
41
|
+
(quire-rs FR-053): id, statement, statement hash, authored method,
|
|
42
|
+
criticality. Do not restate them by hand.
|
|
37
43
|
|
|
38
|
-
|
|
44
|
+
2. **Take the deterministic recommendations first.** They come from the
|
|
45
|
+
catalog's `applicability` rules matched against facts about the obligation:
|
|
46
|
+
its statement's characteristics, its FR-052 property shape, the object types
|
|
47
|
+
near it, its archetype. Rules match or they do not.
|
|
39
48
|
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
49
|
+
3. **Judge only the residue, and label it.** Where the rules are inconclusive
|
|
50
|
+
the advisor says so and stops rather than guessing. That is where your
|
|
51
|
+
judgement belongs — and it must be recorded *as* judgement. Never present an
|
|
52
|
+
LLM conclusion as a verdict (the ADR-0010 discipline).
|
|
43
53
|
|
|
44
|
-
|
|
54
|
+
4. **Confirm or correct the authored method.** A flagged mismatch is advisory:
|
|
55
|
+
the human decides. The confirmed method lands in the spec, becomes the
|
|
56
|
+
obligation's method, and is what `quoin evidence` records discharge against.
|
|
45
57
|
|
|
46
|
-
|
|
58
|
+
5. **Plan the suite.** A method implies an evidence kind; a suite in
|
|
59
|
+
`spec/evidence/suites.md` produces that kind. A recommended method with no
|
|
60
|
+
suite that can produce its evidence is a gap in the plan, not a gap in the
|
|
61
|
+
spec.
|
|
47
62
|
|
|
48
|
-
|
|
49
|
-
verification_method: test # test | analysis | inspection | demonstration
|
|
50
|
-
evidence:
|
|
51
|
-
- kind: test_case
|
|
52
|
-
ref: tests/integration/test_login.py::test_invalid_credentials
|
|
53
|
-
- kind: metric
|
|
54
|
-
ref: auth_failure_total
|
|
55
|
-
```
|
|
63
|
+
## What "inconclusive" means, and why it is not a failure
|
|
56
64
|
|
|
57
|
-
|
|
65
|
+
An obligation the rules cannot place is a normal outcome — the catalog's
|
|
66
|
+
applicability rules are deliberately narrow, because a rule that matches
|
|
67
|
+
everything advises nothing. Report it as inconclusive and choose a method
|
|
68
|
+
yourself. **Do not default to `Test` to make the report look complete**: that is
|
|
69
|
+
precisely the habit this skill was rebuilt to end.
|
|
58
70
|
|
|
59
|
-
|
|
60
|
-
# Verification & Evidence Summary
|
|
71
|
+
## Deliverable
|
|
61
72
|
|
|
62
|
-
|
|
63
|
-
|-----|--------|----------|
|
|
64
|
-
| FR-003 | test | tests/integration/test_login.py::test_login_happy_path |
|
|
65
|
-
| FR-010 | analysis | docs/analysis/storage-bound.md |
|
|
66
|
-
| NFR-002 | demonstration | docs/demos/uptime-walkthrough.md |
|
|
67
|
-
| NFR-005 | test + metric | tests/perf/test_p99_latency.py, metric `http_request_duration_seconds` |
|
|
73
|
+
A `SpecReview` with `analysis: evidence`, under `reviews/YY-MM-DD-<slug>.md`.
|
|
68
74
|
|
|
69
|
-
|
|
75
|
+
The `Findings` table carries one row per obligation whose method needs
|
|
76
|
+
attention — a mismatch, or an inconclusive recommendation the author must
|
|
77
|
+
settle. An obligation whose authored method matches the recommendation needs no
|
|
78
|
+
row; a review listing every passing obligation is a review nobody reads to the
|
|
79
|
+
end.
|
|
70
80
|
|
|
71
|
-
|
|
81
|
+
```markdown
|
|
82
|
+
| ID | Severity | Summary | Refs |
|
|
83
|
+
|----|----------|---------|------|
|
|
84
|
+
| FND-001 | medium | NFR-009-AC-1 is authored `Inspection`; the statement's `reliability` characteristic recommends `fault-injection` (Test). | NFR-009-AC-1 |
|
|
85
|
+
| FND-002 | low | FR-001-AC-9 matched no applicability rule; method chosen by judgement, not by the catalog. | FR-001-AC-9 |
|
|
72
86
|
```
|
|
73
87
|
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
- No `evidence: TBD`, no missing refs.
|
|
78
|
-
- The summary table has zero open gaps before `spec-to-plan` is invoked.
|
|
88
|
+
Record the per-requirement outcome in the requirement's own `Verification`
|
|
89
|
+
cell — that cell **is** the obligation's method (quire-rs FR-053), so editing it
|
|
90
|
+
is what changes the plan. There is no separate evidence file to keep in sync.
|
|
@@ -27,6 +27,19 @@ matrix, run the EARS requirement-grammar check:
|
|
|
27
27
|
quire validate --scope . "spec/**/*.md" --summary
|
|
28
28
|
```
|
|
29
29
|
|
|
30
|
+
> **`--scope` is the repository root, and must be passed explicitly.** Since quire-cli
|
|
31
|
+
> v0.16.0 (quire-rs CR-045) the command derives **two roots** from it and never
|
|
32
|
+
> interchanges them: spec documents are read from `<repo>/spec` only, trace tags from
|
|
33
|
+
> the source tree at `<repo>` excluding `spec/`. A repo with no `spec/` exits with a
|
|
34
|
+
> diagnostic naming the missing document root rather than scanning the whole tree, and a
|
|
35
|
+
> matrix outside `spec/` (a fixture, a `plan/` copy) mints nothing. A relative glob
|
|
36
|
+
> resolves under `--scope` only in scoped mode (no `--module`); with `--module` it
|
|
37
|
+
> resolves against the process working directory, and an omitted `--scope` defaults to
|
|
38
|
+
> `.` — so a run launched from a parent directory validates the **wrong tree** and exits
|
|
39
|
+
> 0 for whatever it matched. Check `quire --version` ≥ 0.16.0 before relying on any of
|
|
40
|
+
> this; ≤ 0.15.0 has the pre-split traversal semantics.
|
|
41
|
+
|
|
42
|
+
|
|
30
43
|
Treat these `[ears:…]` findings as **un-mappable requirements to fix first**,
|
|
31
44
|
not coverage you can claim:
|
|
32
45
|
|
|
@@ -52,6 +52,19 @@ The seven analyses:
|
|
|
52
52
|
`low`/`medium`/`high`).
|
|
53
53
|
4. Validate + record:
|
|
54
54
|
- Run `quire validate --scope <repo> "spec/**/*.md"` and fix any errors.
|
|
55
|
+
|
|
56
|
+
> **`--scope` is the repository root, and must be passed explicitly.** Since quire-cli
|
|
57
|
+
> v0.16.0 (quire-rs CR-045) the command derives **two roots** from it and never
|
|
58
|
+
> interchanges them: spec documents are read from `<repo>/spec` only, trace tags from
|
|
59
|
+
> the source tree at `<repo>` excluding `spec/`. A repo with no `spec/` exits with a
|
|
60
|
+
> diagnostic naming the missing document root rather than scanning the whole tree, and a
|
|
61
|
+
> matrix outside `spec/` (a fixture, a `plan/` copy) mints nothing. A relative glob
|
|
62
|
+
> resolves under `--scope` only in scoped mode (no `--module`); with `--module` it
|
|
63
|
+
> resolves against the process working directory, and an omitted `--scope` defaults to
|
|
64
|
+
> `.` — so a run launched from a parent directory validates the **wrong tree** and exits
|
|
65
|
+
> 0 for whatever it matched. Check `quire --version` ≥ 0.16.0 before relying on any of
|
|
66
|
+
> this; ≤ 0.15.0 has the pre-split traversal semantics.
|
|
67
|
+
|
|
55
68
|
- The docs sync into filament-core as `SpecReview` artifacts; do not proceed to
|
|
56
69
|
implementation until reviewed.
|
|
57
70
|
|
|
@@ -67,6 +67,19 @@ This skill produces or updates a **plan bundle** at `<project_root>/plan/<Plan-i
|
|
|
67
67
|
Validate the bundle with:
|
|
68
68
|
`quire validate --scope <project_root> "plan/**/*.md"`
|
|
69
69
|
|
|
70
|
+
> **`--scope` is the repository root, and must be passed explicitly.** Since quire-cli
|
|
71
|
+
> v0.16.0 (quire-rs CR-045) the command derives **two roots** from it and never
|
|
72
|
+
> interchanges them: spec documents are read from `<repo>/spec` only, trace tags from
|
|
73
|
+
> the source tree at `<repo>` excluding `spec/`. A repo with no `spec/` exits with a
|
|
74
|
+
> diagnostic naming the missing document root rather than scanning the whole tree, and a
|
|
75
|
+
> matrix outside `spec/` (a fixture, a `plan/` copy) mints nothing. A relative glob
|
|
76
|
+
> resolves under `--scope` only in scoped mode (no `--module`); with `--module` it
|
|
77
|
+
> resolves against the process working directory, and an omitted `--scope` defaults to
|
|
78
|
+
> `.` — so a run launched from a parent directory validates the **wrong tree** and exits
|
|
79
|
+
> 0 for whatever it matched. Check `quire --version` ≥ 0.16.0 before relying on any of
|
|
80
|
+
> this; ≤ 0.15.0 has the pre-split traversal semantics.
|
|
81
|
+
|
|
82
|
+
|
|
70
83
|
## Open in the review cockpit
|
|
71
84
|
|
|
72
85
|
After the bundle validates, run `filament open <plan-bundle>/plan.md` to open the
|