@agent-ix/quoin 0.17.0 → 0.20.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/.codex-plugin/plugin.json +1 -1
- package/default-modules.yaml +15 -6
- package/dist/advisor/advise.d.ts +14 -1
- package/dist/advisor/advise.d.ts.map +1 -1
- package/dist/{advisor-Dhy9S7PS.js → advisor-B7X0YTVo.js} +1 -1
- package/dist/assurance/graph.d.ts +79 -0
- package/dist/assurance/graph.d.ts.map +1 -0
- package/dist/assurance/index.d.ts +9 -0
- package/dist/assurance/index.d.ts.map +1 -0
- package/dist/assurance/render.d.ts +3 -0
- package/dist/assurance/render.d.ts.map +1 -0
- package/dist/auditor/audit.d.ts +23 -2
- package/dist/auditor/audit.d.ts.map +1 -1
- package/dist/auditor/combinatorial.d.ts +53 -0
- package/dist/auditor/combinatorial.d.ts.map +1 -0
- package/dist/auditor-C9dLCmDh.js +188 -0
- package/dist/{base-D_-Jy4cE.js → base-q8brKRlC.js} +1 -1
- package/dist/cli.js +1 -1
- package/dist/combinatorial-Cq01XfhV.js +80 -0
- package/dist/commands/advise.js +35 -32
- package/dist/commands/assurance.d.ts +14 -0
- package/dist/commands/assurance.d.ts.map +1 -0
- package/dist/commands/assurance.js +179 -0
- package/dist/commands/catalog/list.js +2 -2
- package/dist/commands/catalog/methods.js +3 -3
- package/dist/commands/catalog/show.js +2 -2
- package/dist/commands/catalog/validate.js +2 -2
- package/dist/commands/completeness.d.ts +16 -0
- package/dist/commands/completeness.d.ts.map +1 -0
- package/dist/commands/completeness.js +49 -0
- package/dist/commands/config/doctor.js +1 -1
- package/dist/commands/config/edit.js +1 -1
- package/dist/commands/config/get.js +1 -1
- package/dist/commands/config/set.js +1 -1
- package/dist/commands/evidence/affirm.js +3 -3
- package/dist/commands/evidence/audit.d.ts +10 -0
- package/dist/commands/evidence/audit.d.ts.map +1 -1
- package/dist/commands/evidence/audit.js +54 -36
- package/dist/commands/evidence/baseline.js +12 -12
- package/dist/commands/evidence/gc.js +2 -2
- package/dist/commands/evidence/index.js +1 -1
- package/dist/commands/evidence/record.d.ts +2 -0
- package/dist/commands/evidence/record.d.ts.map +1 -1
- package/dist/commands/evidence/record.js +89 -32
- package/dist/commands/matrix.js +1 -1
- package/dist/commands/module/ensure-defaults.js +3 -3
- package/dist/commands/module/install.js +2 -2
- package/dist/commands/module/list.js +2 -2
- package/dist/commands/module/remove.js +2 -2
- package/dist/commands/review.js +1 -1
- package/dist/commands/to-plan.js +1 -1
- package/dist/commands/update.js +2 -2
- package/dist/commands/write.js +3 -3
- package/dist/completeness/assess.d.ts +82 -0
- package/dist/completeness/assess.d.ts.map +1 -0
- package/dist/completeness/bundle.d.ts +52 -0
- package/dist/completeness/bundle.d.ts.map +1 -0
- package/dist/completeness/declarations.d.ts +46 -0
- package/dist/completeness/declarations.d.ts.map +1 -0
- package/dist/completeness/index.d.ts +11 -0
- package/dist/completeness/index.d.ts.map +1 -0
- package/dist/completeness/run.d.ts +35 -0
- package/dist/completeness/run.d.ts.map +1 -0
- package/dist/completeness-Bic6298v.js +228 -0
- package/dist/evidence/adapters/agent-eval.d.ts +26 -0
- package/dist/evidence/adapters/agent-eval.d.ts.map +1 -0
- package/dist/evidence/adapters/audit-script.d.ts +25 -0
- package/dist/evidence/adapters/audit-script.d.ts.map +1 -0
- package/dist/evidence/adapters/cargo-mutants.d.ts +3 -0
- package/dist/evidence/adapters/cargo-mutants.d.ts.map +1 -0
- package/dist/evidence/adapters/junit.d.ts +17 -0
- package/dist/evidence/adapters/junit.d.ts.map +1 -0
- package/dist/evidence/adapters/registry.d.ts +63 -0
- package/dist/evidence/adapters/registry.d.ts.map +1 -0
- package/dist/evidence/adapters/sarif.d.ts +35 -0
- package/dist/evidence/adapters/sarif.d.ts.map +1 -0
- package/dist/evidence/adapters/sbom.d.ts +35 -0
- package/dist/evidence/adapters/sbom.d.ts.map +1 -0
- package/dist/evidence/adapters/types.d.ts +55 -0
- package/dist/evidence/adapters/types.d.ts.map +1 -0
- package/dist/evidence/index.d.ts +10 -0
- package/dist/evidence/index.d.ts.map +1 -1
- package/dist/evidence/record.d.ts.map +1 -1
- package/dist/evidence/store.d.ts +48 -2
- package/dist/evidence/store.d.ts.map +1 -1
- package/dist/evidence/types.d.ts +84 -0
- package/dist/evidence/types.d.ts.map +1 -1
- package/dist/evidence-OIkRqQrU.js +599 -0
- package/dist/{flow-command-B-ZBn9Sd.js → flow-command-k7eYqzBE.js} +1 -1
- package/dist/index.js +6 -6
- package/dist/{modules-DSkUXhHn.js → modules-E0AaYTTp.js} +2 -2
- package/dist/{plugins-DiS0OXoo.js → plugins-xe5e7dMl.js} +1 -1
- package/dist/quire/contract.d.ts +2 -2
- package/dist/quire/contract.d.ts.map +1 -1
- package/dist/quire/types.d.ts +11 -0
- package/dist/quire/types.d.ts.map +1 -1
- package/dist/{quire-D7ra5rUF.js → quire-BS-EW-0Y.js} +2 -2
- package/dist/schemas/coverage-v1.schema.json +6 -1
- package/dist/{version-DmZHxX4n.js → version-0hg-SgVf.js} +1 -1
- package/dist/{write-CPdSNIkF.js → write-Dig0zXLM.js} +1 -1
- package/package.json +2 -1
- package/skills/spec-fuzz/SKILL.md +94 -0
- package/skills/spec-fuzz/assets/templates/atheris.py.md +74 -0
- package/skills/spec-fuzz/assets/templates/cargo-fuzz.rs.md +83 -0
- package/skills/spec-fuzz/assets/templates/fast-check.ts.md +76 -0
- package/skills/spec-fuzz/references/step-0-scope-and-harness.md +51 -0
- package/skills/spec-fuzz/references/step-1-select-obligations.md +47 -0
- package/skills/spec-fuzz/references/step-2-ground-entry-point.md +55 -0
- package/skills/spec-fuzz/references/step-3-emit-targets.md +65 -0
- package/skills/spec-fuzz/references/step-4-report.md +56 -0
- package/dist/audit-CudnEJmh.js +0 -127
- package/dist/evidence-QUw0KyMH.js +0 -209
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"types.d.ts","sourceRoot":"","sources":["../../src/quire/types.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;GAeG;AAEH,8EAA8E;AAC9E,MAAM,WAAW,WAAW;IAC1B,SAAS,EAAE,MAAM,CAAC;IAClB,QAAQ,EAAE,MAAM,CAAC;IACjB,MAAM,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IACvB,UAAU,EAAE,MAAM,EAAE,CAAC;CACtB;AAED,qEAAqE;AACrE,MAAM,WAAW,SAAS;IACxB,SAAS,EAAE,MAAM,CAAC;IAClB,QAAQ,EAAE,MAAM,CAAC;IACjB,MAAM,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IACvB,MAAM,EAAE,MAAM,CAAC;IACf,UAAU,EAAE,MAAM,EAAE,CAAC;CACtB;AAED,iFAAiF;AACjF,MAAM,WAAW,WAAW;IAC1B,SAAS,EAAE,MAAM,CAAC;IAClB,QAAQ,EAAE,MAAM,CAAC;IACjB,MAAM,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IACvB,SAAS,EAAE,MAAM,CAAC;IAClB,UAAU,EAAE,MAAM,EAAE,CAAC;CACtB;AAED,oEAAoE;AACpE,MAAM,WAAW,eAAe;IAC9B,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE,MAAM,CAAC;IACf,QAAQ,EAAE,MAAM,CAAC;CAClB;AAED,oEAAoE;AACpE,MAAM,WAAW,WAAW;IAC1B,QAAQ,EAAE,MAAM,CAAC;IACjB,MAAM,EAAE,MAAM,CAAC;IACf,MAAM,EAAE,MAAM,CAAC;IACf,KAAK,EAAE,MAAM,CAAC;CACf;AAED,mDAAmD;AACnD,MAAM,WAAW,cAAc;IAC7B,QAAQ,EAAE,MAAM,CAAC;IACjB,SAAS,EAAE,MAAM,CAAC;IAClB,QAAQ,EAAE,MAAM,CAAC;IACjB,eAAe,EAAE,MAAM,CAAC;IACxB,WAAW,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;CACrC;AAED,6DAA6D;AAC7D,MAAM,WAAW,kBAAkB;IACjC,WAAW,EAAE,MAAM,CAAC;IACpB,mEAAmE;IACnE,MAAM,EAAE,MAAM,CAAC;IACf,OAAO,EAAE,MAAM,CAAC;IAChB,IAAI,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;CACtB;AAED;;;;;GAKG;AACH,MAAM,WAAW,UAAU;IACzB,MAAM,EAAE,MAAM,CAAC;IACf,EAAE,EAAE,MAAM,CAAC;IACX,QAAQ,EAAE,MAAM,CAAC;IACjB,SAAS,EAAE,MAAM,CAAC;IAClB,cAAc,EAAE,MAAM,CAAC;IACvB,MAAM,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IACvB,UAAU,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IACpC,WAAW,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;
|
|
1
|
+
{"version":3,"file":"types.d.ts","sourceRoot":"","sources":["../../src/quire/types.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;GAeG;AAEH,8EAA8E;AAC9E,MAAM,WAAW,WAAW;IAC1B,SAAS,EAAE,MAAM,CAAC;IAClB,QAAQ,EAAE,MAAM,CAAC;IACjB,MAAM,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IACvB,UAAU,EAAE,MAAM,EAAE,CAAC;CACtB;AAED,qEAAqE;AACrE,MAAM,WAAW,SAAS;IACxB,SAAS,EAAE,MAAM,CAAC;IAClB,QAAQ,EAAE,MAAM,CAAC;IACjB,MAAM,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IACvB,MAAM,EAAE,MAAM,CAAC;IACf,UAAU,EAAE,MAAM,EAAE,CAAC;CACtB;AAED,iFAAiF;AACjF,MAAM,WAAW,WAAW;IAC1B,SAAS,EAAE,MAAM,CAAC;IAClB,QAAQ,EAAE,MAAM,CAAC;IACjB,MAAM,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IACvB,SAAS,EAAE,MAAM,CAAC;IAClB,UAAU,EAAE,MAAM,EAAE,CAAC;CACtB;AAED,oEAAoE;AACpE,MAAM,WAAW,eAAe;IAC9B,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE,MAAM,CAAC;IACf,QAAQ,EAAE,MAAM,CAAC;CAClB;AAED,oEAAoE;AACpE,MAAM,WAAW,WAAW;IAC1B,QAAQ,EAAE,MAAM,CAAC;IACjB,MAAM,EAAE,MAAM,CAAC;IACf,MAAM,EAAE,MAAM,CAAC;IACf,KAAK,EAAE,MAAM,CAAC;CACf;AAED,mDAAmD;AACnD,MAAM,WAAW,cAAc;IAC7B,QAAQ,EAAE,MAAM,CAAC;IACjB,SAAS,EAAE,MAAM,CAAC;IAClB,QAAQ,EAAE,MAAM,CAAC;IACjB,eAAe,EAAE,MAAM,CAAC;IACxB,WAAW,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;CACrC;AAED,6DAA6D;AAC7D,MAAM,WAAW,kBAAkB;IACjC,WAAW,EAAE,MAAM,CAAC;IACpB,mEAAmE;IACnE,MAAM,EAAE,MAAM,CAAC;IACf,OAAO,EAAE,MAAM,CAAC;IAChB,IAAI,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;CACtB;AAED;;;;;GAKG;AACH,MAAM,WAAW,UAAU;IACzB,MAAM,EAAE,MAAM,CAAC;IACf,EAAE,EAAE,MAAM,CAAC;IACX,QAAQ,EAAE,MAAM,CAAC;IACjB,SAAS,EAAE,MAAM,CAAC;IAClB,cAAc,EAAE,MAAM,CAAC;IACvB,MAAM,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IACvB,UAAU,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IACpC,WAAW,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAC5B;;;;;;;;;OASG;IACH,UAAU,CAAC,EAAE,MAAM,EAAE,CAAC;CACvB;AAED,+DAA+D;AAC/D,MAAM,WAAW,cAAc;IAC7B,MAAM,EAAE,MAAM,CAAC;IACf,KAAK,EAAE,MAAM,CAAC;IACd,QAAQ,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IACzB,eAAe,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;CACjC;AAED,gDAAgD;AAChD,MAAM,WAAW,cAAc;IAC7B,aAAa,EAAE,WAAW,EAAE,CAAC;IAC7B,WAAW,EAAE,SAAS,EAAE,CAAC;IACzB,6EAA6E;IAC7E,cAAc,CAAC,EAAE,WAAW,EAAE,CAAC;IAC/B,iBAAiB,EAAE,eAAe,EAAE,CAAC;IACrC,MAAM,EAAE,WAAW,EAAE,CAAC;IACtB,QAAQ,CAAC,EAAE,cAAc,EAAE,CAAC;IAC5B,WAAW,CAAC,EAAE,kBAAkB,EAAE,CAAC;IACnC,WAAW,CAAC,EAAE,UAAU,EAAE,CAAC;IAC3B,MAAM,EAAE,cAAc,CAAC;CACxB;AAED,yDAAyD;AACzD,MAAM,WAAW,YAAY;IAC3B,KAAK,EAAE,MAAM,CAAC;IACd,GAAG,EAAE,MAAM,CAAC;IACZ,IAAI,EAAE,MAAM,CAAC;CACd;AAED,mEAAmE;AACnE,MAAM,MAAM,OAAO,GACjB,WAAW,GAAG,YAAY,GAAG,iBAAiB,GAAG,cAAc,CAAC;AAElE,8DAA8D;AAC9D,MAAM,MAAM,aAAa,GACrB,YAAY,GACZ,aAAa,GACb,UAAU,GACV,WAAW,GACX,YAAY,GACZ,WAAW,GACX,aAAa,GACb,WAAW,GACX,SAAS,GACT,cAAc,CAAC;AAEnB,yDAAyD;AACzD,MAAM,MAAM,UAAU,GAAG,aAAa,GAAG,WAAW,GAAG,iBAAiB,CAAC;AAEzE;;;;;GAKG;AACH,MAAM,WAAW,mBAAmB;IAClC,MAAM,EAAE,MAAM,CAAC;IACf,cAAc,EAAE,MAAM,CAAC;IACvB,MAAM,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IACvB,UAAU,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IACpC,WAAW,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;CAC7B;AAED,2CAA2C;AAC3C,MAAM,WAAW,SAAS;IACxB,MAAM,EAAE,MAAM,GAAG,IAAI,CAAC;IACtB,SAAS,EAAE,MAAM,CAAC;IAClB,IAAI,EAAE,MAAM,GAAG,IAAI,CAAC;IACpB,KAAK,EAAE,OAAO,CAAC;IACf,QAAQ,EAAE,aAAa,CAAC;IACxB,WAAW,EAAE,OAAO,CAAC;IACrB,UAAU,EAAE,UAAU,CAAC;IACvB,MAAM,EAAE,YAAY,GAAG,IAAI,CAAC;IAC5B,YAAY,EAAE,YAAY,GAAG,IAAI,CAAC;IAClC,MAAM,EAAE,YAAY,GAAG,IAAI,CAAC;IAC5B,OAAO,EAAE,MAAM,EAAE,CAAC;IAClB,UAAU,EAAE,mBAAmB,GAAG,IAAI,CAAC;CACxC;AAED,0CAA0C;AAC1C,MAAM,WAAW,kBAAkB;IACjC,QAAQ,EAAE,MAAM,CAAC;IACjB,SAAS,EAAE,MAAM,CAAC;IAClB,QAAQ,EAAE,SAAS,EAAE,CAAC;CACvB;AAED,kDAAkD;AAClD,MAAM,WAAW,gBAAgB;IAC/B,SAAS,EAAE,kBAAkB,EAAE,CAAC;CACjC"}
|
|
@@ -25,11 +25,11 @@ var a = Object.defineProperty, o = Object.getOwnPropertyDescriptor, s = Object.g
|
|
|
25
25
|
});
|
|
26
26
|
return e;
|
|
27
27
|
}, p = (e) => c.call(e, "module.exports") ? e["module.exports"] : f(a({}, "__esModule", { value: !0 }), e), m = {
|
|
28
|
-
sourceTag: "v0.
|
|
28
|
+
sourceTag: "v0.36.0",
|
|
29
29
|
contractVersion: "v1",
|
|
30
30
|
minimumCli: "0.21.0",
|
|
31
31
|
hashes: {
|
|
32
|
-
"coverage-v1.schema.json": "
|
|
32
|
+
"coverage-v1.schema.json": "5669991ed60f2dbf3db82f7b43e12893ecc328b63e3c290b84b41d08c92881e6",
|
|
33
33
|
"properties-v1.schema.json": "d81f1ec85abdecd1f1664ded15e081f9aecb6c0f054d6b332b71e904cf826292"
|
|
34
34
|
}
|
|
35
35
|
}, h = t(r(import.meta.url));
|
|
@@ -169,7 +169,12 @@
|
|
|
169
169
|
"additionalProperties": { "type": "string" },
|
|
170
170
|
"description": "Declared parameters. An absent key means the cell was empty — never defaulted."
|
|
171
171
|
},
|
|
172
|
-
"criticality": { "type": ["string", "null"] }
|
|
172
|
+
"criticality": { "type": ["string", "null"] },
|
|
173
|
+
"target_ids": {
|
|
174
|
+
"type": "array",
|
|
175
|
+
"items": { "type": "string", "minLength": 1 },
|
|
176
|
+
"description": "Test-case ids the method cell names, e.g. `Test (TC-707)`. Additive in v1: absent on a payload from an engine before quire-rs v0.35.0, and omitted where the cell names none. A consumer binding evidence keyed on a TEST CASE rather than on the criterion needs this join, which the engine parsed and discarded (agent-ix/quire-rs#180)."
|
|
177
|
+
}
|
|
173
178
|
}
|
|
174
179
|
},
|
|
175
180
|
"CoverageTotals": {
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { i as e } from "./base-
|
|
1
|
+
import { i as e } from "./base-q8brKRlC.js";
|
|
2
2
|
import { n as t, r as n, t as r } from "./config-schema-Dp-vlxuR.js";
|
|
3
3
|
import { ConfigService as i } from "@agent-ix/ix-cli-core";
|
|
4
4
|
import { existsSync as a, readFileSync as o, statSync as s } from "node:fs";
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@agent-ix/quoin",
|
|
3
3
|
"description": "Spec-driven development for Claude Code — author validated, ISO/IEC/IEEE 29148-aligned specs, then plan and build against them.",
|
|
4
|
-
"version": "0.
|
|
4
|
+
"version": "0.20.0",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
7
7
|
"files": [
|
|
@@ -97,6 +97,7 @@
|
|
|
97
97
|
"zod": ">=4.4.3"
|
|
98
98
|
},
|
|
99
99
|
"devDependencies": {
|
|
100
|
+
"@agent-ix/quire-cli": "^0.25.1",
|
|
100
101
|
"@types/node": "26.1.2",
|
|
101
102
|
"@typescript-eslint/eslint-plugin": "8.66.0",
|
|
102
103
|
"@typescript-eslint/parser": "8.66.0",
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: spec-fuzz
|
|
3
|
+
description: Turn Fuzz-kind obligations into runnable fuzz targets. Consumes `quoin advise
|
|
4
|
+
--json`, selects the obligations whose verification method carries `evidence_kind: Fuzz`,
|
|
5
|
+
grounds each one's entry-point symbol in the repository's source, and emits targets in the
|
|
6
|
+
repo's own harness (cargo-fuzz / atheris / fast-check) keyed on the obligation id. Writes
|
|
7
|
+
nothing where the tooling is absent or the entry point cannot be grounded — a finding
|
|
8
|
+
instead. Emits a validated SpecReview recording every obligation it could not serve.
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
# Spec Fuzz
|
|
12
|
+
|
|
13
|
+
Use this skill to **generate fuzz targets from robustness requirements you already wrote**.
|
|
14
|
+
|
|
15
|
+
*"Never panic on arbitrary input"* and *"reject malformed frontmatter"* are property-shaped
|
|
16
|
+
over an input surface. `spec-correctness` generates example and property tests, not fuzz
|
|
17
|
+
targets, so a consumer that parses anything gets nothing from its spec toward a harness —
|
|
18
|
+
the method can be advised from the catalog and materializing it is entirely manual.
|
|
19
|
+
|
|
20
|
+
Specified by [FR-038](../../spec/functional/FR-038-generate-fuzz-harnesses.md).
|
|
21
|
+
|
|
22
|
+
## The rule that is not negotiable
|
|
23
|
+
|
|
24
|
+
**Writing a target discharges nothing.**
|
|
25
|
+
|
|
26
|
+
This is where fuzz differs from every other generator in this family. A property test that
|
|
27
|
+
exists and passes is evidence. A fuzz target that exists has proved nothing — fuzzing is a
|
|
28
|
+
search, and an unrun target is a search never started.
|
|
29
|
+
|
|
30
|
+
The obligation is discharged by a **recorded run** (FR-030), in its own suite with
|
|
31
|
+
`Evidence Kind: Fuzz`. A generated target that has never run is precisely the
|
|
32
|
+
`vacuous-evidence` shape FR-034 exists to name. Report generated targets as **undischarged**,
|
|
33
|
+
always, and never imply otherwise in the handoff.
|
|
34
|
+
|
|
35
|
+
Three more:
|
|
36
|
+
|
|
37
|
+
- **Never install anything.** No `cargo install`, no dependency added, no fuzz workspace
|
|
38
|
+
scaffolded. Absent tooling is a finding. (CON-2.)
|
|
39
|
+
- **Never decide which methods are fuzz methods.** That is the catalog's, read at run time.
|
|
40
|
+
(CON-1.)
|
|
41
|
+
- **Never write a framework name into `spec/**`.** Inherited from FR-028-CON-2.
|
|
42
|
+
|
|
43
|
+
## When to use
|
|
44
|
+
|
|
45
|
+
- After `quoin advise` recommends `fuzzing` for an obligation and nothing implements it.
|
|
46
|
+
- On a repository that parses untrusted or structured input and has no fuzz suite.
|
|
47
|
+
- To extend an existing fuzz suite to surfaces the spec names and the suite misses.
|
|
48
|
+
|
|
49
|
+
Not for: authoring the requirement (`specify`), choosing the method (`quoin advise`), or
|
|
50
|
+
running the fuzzer. Running is the consumer's CI — quoin transcribes, it does not execute.
|
|
51
|
+
|
|
52
|
+
## Inputs
|
|
53
|
+
|
|
54
|
+
- The target repository.
|
|
55
|
+
- `quoin advise --repo <repo> --json` — obligations with their authored method and
|
|
56
|
+
recommendations.
|
|
57
|
+
- The active `verification_catalog` — which methods are Fuzz-kind, and the `tooling` each
|
|
58
|
+
names. **Read it; do not carry a list here.**
|
|
59
|
+
|
|
60
|
+
## Steps
|
|
61
|
+
|
|
62
|
+
0. **[Scope and harness](references/step-0-scope-and-harness.md)** — detect the repository's
|
|
63
|
+
language and fuzz tooling from its manifest. Absent tooling stops the run here, with
|
|
64
|
+
findings.
|
|
65
|
+
1. **[Select the obligations](references/step-1-select-obligations.md)** — those whose
|
|
66
|
+
method has `evidence_kind: Fuzz`, from the catalog rather than by name.
|
|
67
|
+
2. **[Ground the entry point](references/step-2-ground-entry-point.md)** — find the symbol
|
|
68
|
+
the target will call, in the source. Ungrounded means a finding, not a guess.
|
|
69
|
+
3. **[Emit the targets](references/step-3-emit-targets.md)** — harness-native, two trace
|
|
70
|
+
carriers, one provenance line.
|
|
71
|
+
4. **[Report](references/step-4-report.md)** — a validated `SpecReview` naming every
|
|
72
|
+
obligation not served and why, and the handoff that says the targets are undischarged.
|
|
73
|
+
|
|
74
|
+
## What "could not be served" looks like
|
|
75
|
+
|
|
76
|
+
Three distinct outcomes, and they are not interchangeable:
|
|
77
|
+
|
|
78
|
+
| Outcome | Meaning | What to write |
|
|
79
|
+
|---|---|---|
|
|
80
|
+
| **no tooling** | the repo has no fuzz harness | finding per obligation; **no files** |
|
|
81
|
+
| **ungrounded** | no entry-point symbol found in the source | finding naming the obligation; **no file for it** |
|
|
82
|
+
| **already covered** | an existing target already claims the id | nothing; record as covered |
|
|
83
|
+
|
|
84
|
+
Never collapse these into "skipped". The first is a decision for the repository owner, the
|
|
85
|
+
second is a gap in the spec or the code, and the third is success.
|
|
86
|
+
|
|
87
|
+
## Why there is no `quoin fuzz` command
|
|
88
|
+
|
|
89
|
+
The selection is mechanical and the emission is not. Choosing the entry point for
|
|
90
|
+
*"the parser SHALL NOT panic on arbitrary input"* means reading the parser, and a command
|
|
91
|
+
that guessed would produce targets that do not compile.
|
|
92
|
+
|
|
93
|
+
`quoin advise` supplies the mechanical half as JSON. This skill is the judgement half, and
|
|
94
|
+
it records which is which in the provenance line so a reviewer can tell them apart.
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# atheris target template
|
|
2
|
+
|
|
3
|
+
One file per obligation, at `fuzz/fuzz_<obligation_slug>_<surface>.py`, executable.
|
|
4
|
+
|
|
5
|
+
Atheris instruments at import time, so **`atheris.instrument_imports()` must wrap the import
|
|
6
|
+
of the module under test**. An import outside that block is uninstrumented: the fuzzer runs,
|
|
7
|
+
reports coverage, finds nothing, and looks like it worked.
|
|
8
|
+
|
|
9
|
+
## Bytes in, no interpretation
|
|
10
|
+
|
|
11
|
+
```python
|
|
12
|
+
#!/usr/bin/env python3
|
|
13
|
+
"""The config loader does not raise on arbitrary input.
|
|
14
|
+
|
|
15
|
+
Trace: NFR-003-M-1
|
|
16
|
+
|
|
17
|
+
spec-fuzz: obligation=NFR-003-M-1 harness=atheris entry=myapp.config.parse_config origin=advised
|
|
18
|
+
"""
|
|
19
|
+
|
|
20
|
+
import sys
|
|
21
|
+
|
|
22
|
+
import atheris
|
|
23
|
+
|
|
24
|
+
with atheris.instrument_imports():
|
|
25
|
+
from myapp.config import parse_config
|
|
26
|
+
|
|
27
|
+
|
|
28
|
+
def test_one_input(data: bytes) -> None:
|
|
29
|
+
fdp = atheris.FuzzedDataProvider(data)
|
|
30
|
+
text = fdp.ConsumeUnicodeNoSurrogates(fdp.remaining_bytes())
|
|
31
|
+
try:
|
|
32
|
+
parse_config(text)
|
|
33
|
+
except (ValueError, KeyError):
|
|
34
|
+
# The declared rejection path. Raising a *declared* error on garbage is
|
|
35
|
+
# the requirement being met, so it must not read as a crash. Catch the
|
|
36
|
+
# exception types the requirement names — never bare `except Exception`,
|
|
37
|
+
# which would swallow the AttributeError and IndexError this exists to find.
|
|
38
|
+
pass
|
|
39
|
+
|
|
40
|
+
|
|
41
|
+
atheris.Setup(sys.argv, test_one_input)
|
|
42
|
+
atheris.Fuzz()
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
The narrow `except` is the whole design of the file. `except Exception` turns a target that
|
|
46
|
+
finds real bugs into one that can never fail, and it looks more careful.
|
|
47
|
+
|
|
48
|
+
## Which exceptions to catch
|
|
49
|
+
|
|
50
|
+
From the requirement, not from the code. If the requirement says *"rejects malformed input"*
|
|
51
|
+
and the code raises `ConfigError`, catch `ConfigError`. If the code also raises `TypeError`
|
|
52
|
+
on some path, that is a **finding** — the requirement says reject, and `TypeError` is a
|
|
53
|
+
crash, not a rejection.
|
|
54
|
+
|
|
55
|
+
Do not widen the catch to make the target quiet. That is the same move as widening a rule to
|
|
56
|
+
lower a finding count.
|
|
57
|
+
|
|
58
|
+
## Structured input
|
|
59
|
+
|
|
60
|
+
`FuzzedDataProvider` builds typed values without a hand-rolled decoder:
|
|
61
|
+
|
|
62
|
+
```python
|
|
63
|
+
def test_one_input(data: bytes) -> None:
|
|
64
|
+
fdp = atheris.FuzzedDataProvider(data)
|
|
65
|
+
document = fdp.ConsumeUnicodeNoSurrogates(fdp.ConsumeIntInRange(0, 4096))
|
|
66
|
+
selector = fdp.ConsumeUnicodeNoSurrogates(fdp.remaining_bytes())
|
|
67
|
+
try:
|
|
68
|
+
run_query(selector, document)
|
|
69
|
+
except QueryError:
|
|
70
|
+
pass
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
Consume the bounded field first and the unbounded one last, or the second field is almost
|
|
74
|
+
always empty and half the input surface is never reached.
|
|
@@ -0,0 +1,83 @@
|
|
|
1
|
+
# cargo-fuzz target template
|
|
2
|
+
|
|
3
|
+
One file per obligation, at `fuzz/fuzz_targets/fuzz_<obligation-slug>_<surface>.rs`, plus a
|
|
4
|
+
`[[bin]]` block in `fuzz/Cargo.toml`.
|
|
5
|
+
|
|
6
|
+
**The `[[bin]]` block is not optional.** cargo-fuzz discovers targets from `Cargo.toml`, not
|
|
7
|
+
from the directory — a `.rs` file with no block is a file nobody runs, which is the same
|
|
8
|
+
"green over nothing" shape this whole program exists to catch.
|
|
9
|
+
|
|
10
|
+
## Bytes in, no interpretation
|
|
11
|
+
|
|
12
|
+
```rust
|
|
13
|
+
#![no_main]
|
|
14
|
+
|
|
15
|
+
use libfuzzer_sys::fuzz_target;
|
|
16
|
+
|
|
17
|
+
use quire_rs::loader::manifest::parse_manifest;
|
|
18
|
+
|
|
19
|
+
/// The manifest loader does not panic on arbitrary input.
|
|
20
|
+
///
|
|
21
|
+
/// Trace: NFR-003-M-1
|
|
22
|
+
///
|
|
23
|
+
/// spec-fuzz: obligation=NFR-003-M-1 harness=cargo-fuzz entry=loader::manifest::parse_manifest origin=advised
|
|
24
|
+
fuzz_target!(|data: &[u8]| {
|
|
25
|
+
// `from_utf8` rather than `from_utf8_lossy`: lossy conversion silently
|
|
26
|
+
// repairs the invalid sequences that are the most interesting inputs a
|
|
27
|
+
// byte fuzzer produces, so the parser never sees them.
|
|
28
|
+
if let Ok(text) = std::str::from_utf8(data) {
|
|
29
|
+
let _ = parse_manifest(text);
|
|
30
|
+
}
|
|
31
|
+
});
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
Note what is absent: no `unwrap()`, no assertion on the result, no `expect`. `parse_manifest`
|
|
35
|
+
returning `Err` on garbage **is the requirement being met**. The `let _ =` is deliberate.
|
|
36
|
+
|
|
37
|
+
## Structured input, via `arbitrary`
|
|
38
|
+
|
|
39
|
+
Where the surface takes a typed value rather than bytes, derive `Arbitrary` instead of
|
|
40
|
+
hand-rolling a decoder — a hand-rolled one spends most of the corpus failing its own parse
|
|
41
|
+
before reaching the code under test.
|
|
42
|
+
|
|
43
|
+
```rust
|
|
44
|
+
#![no_main]
|
|
45
|
+
|
|
46
|
+
use arbitrary::Arbitrary;
|
|
47
|
+
use libfuzzer_sys::fuzz_target;
|
|
48
|
+
|
|
49
|
+
use quire_rs::query::{run_query, Query};
|
|
50
|
+
|
|
51
|
+
#[derive(Arbitrary, Debug)]
|
|
52
|
+
struct Input {
|
|
53
|
+
document: String,
|
|
54
|
+
selector: String,
|
|
55
|
+
}
|
|
56
|
+
|
|
57
|
+
/// The query engine does not panic on any selector against any document.
|
|
58
|
+
///
|
|
59
|
+
/// Trace: FR-010-AC-3
|
|
60
|
+
///
|
|
61
|
+
/// spec-fuzz: obligation=FR-010-AC-3 harness=cargo-fuzz entry=query::run_query origin=authored
|
|
62
|
+
fuzz_target!(|input: Input| {
|
|
63
|
+
if let Ok(query) = Query::parse(&input.selector) {
|
|
64
|
+
let _ = run_query(&query, &input.document);
|
|
65
|
+
}
|
|
66
|
+
});
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
`arbitrary` must be a declared dependency of the **fuzz** crate. If it is not, that is the
|
|
70
|
+
absent-tooling case from step 0 — add nothing, report it.
|
|
71
|
+
|
|
72
|
+
## The `[[bin]]` block
|
|
73
|
+
|
|
74
|
+
```toml
|
|
75
|
+
[[bin]]
|
|
76
|
+
name = "fuzz_nfr_003_m_1_parse_manifest"
|
|
77
|
+
path = "fuzz_targets/fuzz_nfr_003_m_1_parse_manifest.rs"
|
|
78
|
+
test = false
|
|
79
|
+
doc = false
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
`test = false` and `doc = false` keep the target out of `cargo test` and `cargo doc`, which
|
|
83
|
+
is what stops a nightly-only target from breaking a stable build.
|
|
@@ -0,0 +1,76 @@
|
|
|
1
|
+
# fast-check target template
|
|
2
|
+
|
|
3
|
+
TypeScript has no libFuzzer equivalent in ordinary use, so the harness is `fast-check`
|
|
4
|
+
running in the repo's existing test runner. That has one consequence worth stating: **this is
|
|
5
|
+
a bounded search, not a fuzzing campaign.** It runs for N cases and stops.
|
|
6
|
+
|
|
7
|
+
Say so in the report rather than implying parity with cargo-fuzz.
|
|
8
|
+
|
|
9
|
+
## Placement is the thing that bites
|
|
10
|
+
|
|
11
|
+
The `Trace:` carrier goes **immediately above `it(`** — never above `describe(`. A `describe`
|
|
12
|
+
block groups tests and registers no symbol of its own, so a tag above it binds to nothing
|
|
13
|
+
while reading correctly and matching a grep (`agent-ix/quoin#61`).
|
|
14
|
+
|
|
15
|
+
```ts
|
|
16
|
+
import fc from "fast-check";
|
|
17
|
+
import { describe, expect, it } from "vitest";
|
|
18
|
+
|
|
19
|
+
import { parseManifest } from "../src/manifest.js";
|
|
20
|
+
|
|
21
|
+
describe("the manifest parser under arbitrary input", () => {
|
|
22
|
+
// The parser does not throw an unexpected error on any string.
|
|
23
|
+
//
|
|
24
|
+
// Trace: NFR-003-M-1
|
|
25
|
+
//
|
|
26
|
+
// spec-fuzz: obligation=NFR-003-M-1 harness=fast-check entry=parseManifest origin=advised
|
|
27
|
+
it("NFR-003-M-1 never throws outside its declared rejection", () => {
|
|
28
|
+
fc.assert(
|
|
29
|
+
fc.property(fc.string(), (text) => {
|
|
30
|
+
try {
|
|
31
|
+
parseManifest(text);
|
|
32
|
+
} catch (error) {
|
|
33
|
+
// The declared rejection path. Any OTHER error is the bug this
|
|
34
|
+
// target exists to find, so it is rethrown rather than swallowed —
|
|
35
|
+
// a bare catch would make this target incapable of failing.
|
|
36
|
+
if (!(error instanceof ManifestError)) throw error;
|
|
37
|
+
}
|
|
38
|
+
}),
|
|
39
|
+
{ numRuns: 10_000 },
|
|
40
|
+
);
|
|
41
|
+
});
|
|
42
|
+
});
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
## Generators that reach past the first parse
|
|
46
|
+
|
|
47
|
+
`fc.string()` finds shallow crashes and then plateaus — almost every case fails validation in
|
|
48
|
+
the first few bytes and the code behind it is never reached. Where the surface takes
|
|
49
|
+
structured text, build the structure and mutate it:
|
|
50
|
+
|
|
51
|
+
```ts
|
|
52
|
+
const manifestish = fc.record({
|
|
53
|
+
name: fc.string(),
|
|
54
|
+
version: fc.oneof(fc.string(), fc.constant("1.0.0")),
|
|
55
|
+
entries: fc.array(fc.record({ id: fc.string(), ref: fc.string() }), { maxLength: 8 }),
|
|
56
|
+
});
|
|
57
|
+
|
|
58
|
+
fc.assert(
|
|
59
|
+
fc.property(manifestish, (value) => {
|
|
60
|
+
try {
|
|
61
|
+
parseManifest(JSON.stringify(value));
|
|
62
|
+
} catch (error) {
|
|
63
|
+
if (!(error instanceof ManifestError)) throw error;
|
|
64
|
+
}
|
|
65
|
+
}),
|
|
66
|
+
);
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
Mixing `fc.constant("1.0.0")` into an otherwise arbitrary field is deliberate: it keeps a
|
|
70
|
+
fraction of cases valid enough to get past the field's own check, which is where the
|
|
71
|
+
interesting behaviour is.
|
|
72
|
+
|
|
73
|
+
## Seeds and reproduction
|
|
74
|
+
|
|
75
|
+
`fc.assert` prints the failing counterexample and its seed. Put the seed in the bug, not a
|
|
76
|
+
copy of the input — the seed reproduces the shrink as well as the failure.
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
# Step 0: Scope and harness
|
|
2
|
+
|
|
3
|
+
**Goal**: know what the repository can actually run before generating anything for it.
|
|
4
|
+
|
|
5
|
+
## Detect the harness from the manifest, never from the requirement
|
|
6
|
+
|
|
7
|
+
The requirement says *"the parser SHALL NOT panic on arbitrary input"*. It does not say Rust.
|
|
8
|
+
The **repository's manifest** says Rust, and that is the only thing that may choose a harness
|
|
9
|
+
(FR-038-AC-7).
|
|
10
|
+
|
|
11
|
+
| Manifest | Harness | Present when |
|
|
12
|
+
|---|---|---|
|
|
13
|
+
| `Cargo.toml` | cargo-fuzz / libFuzzer | a `fuzz/Cargo.toml` exists with `cargo-fuzz = true`, or `cargo fuzz --version` resolves |
|
|
14
|
+
| `pyproject.toml` / `setup.py` | atheris | `atheris` is a declared dev-dependency |
|
|
15
|
+
| `package.json` | fast-check | `fast-check` is a declared dependency |
|
|
16
|
+
|
|
17
|
+
A repository with several manifests has several harnesses; scope each obligation to the one
|
|
18
|
+
its entry point lives in.
|
|
19
|
+
|
|
20
|
+
## Absent tooling stops the run
|
|
21
|
+
|
|
22
|
+
If the manifest is present but the harness is not, **stop here**. Emit one finding per
|
|
23
|
+
selected obligation and write no files (FR-038-AC-4).
|
|
24
|
+
|
|
25
|
+
Do not:
|
|
26
|
+
|
|
27
|
+
- run `cargo install cargo-fuzz`
|
|
28
|
+
- add `atheris` or `fast-check` to a dependency list
|
|
29
|
+
- create `fuzz/Cargo.toml`, a `[[bin]]` block, or a nightly toolchain file
|
|
30
|
+
|
|
31
|
+
A fuzz workspace is not a generated artifact. It is a dev-dependency, a nightly toolchain
|
|
32
|
+
requirement and a CI job, and adding all three to somebody's repository because their spec
|
|
33
|
+
mentioned a parser is a decision that belongs to them (ADR-0011: generated artifacts are
|
|
34
|
+
consumer-owned).
|
|
35
|
+
|
|
36
|
+
The finding is the deliverable in this case, and it is a good one — it says *"your spec
|
|
37
|
+
declares N fuzzable surfaces and this repository cannot fuzz"*, which is a fact nobody had
|
|
38
|
+
written down.
|
|
39
|
+
|
|
40
|
+
## Record what you detected
|
|
41
|
+
|
|
42
|
+
The provenance line carries it, so a re-run and a reviewer can both see which harness was
|
|
43
|
+
chosen and why:
|
|
44
|
+
|
|
45
|
+
```
|
|
46
|
+
spec-fuzz: obligation=NFR-012-M-2 harness=cargo-fuzz entry=quire_rs::parse_document origin=advised
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
`origin ∈ {advised, authored}` — whether the method came from `quoin advise`'s
|
|
50
|
+
recommendation or from the requirement's own `Verification` cell. They are different claims
|
|
51
|
+
and the distinction survives into the report.
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
# Step 1: Select the obligations
|
|
2
|
+
|
|
3
|
+
**Goal**: the set of obligations that want a fuzz target, decided by module data.
|
|
4
|
+
|
|
5
|
+
## The selector
|
|
6
|
+
|
|
7
|
+
An obligation is in scope when its verification method carries **`evidence_kind: Fuzz`** in
|
|
8
|
+
the active `verification_catalog`.
|
|
9
|
+
|
|
10
|
+
```
|
|
11
|
+
quoin advise --repo <repo> --json
|
|
12
|
+
quoin catalog methods --json # the catalog, with each method's evidence_kind
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
Take the method from the obligation's authored `Verification` cell if it has one, and from
|
|
16
|
+
the top recommendation otherwise. Record which in `origin=` (step 0).
|
|
17
|
+
|
|
18
|
+
## Do not carry a list of fuzz methods here
|
|
19
|
+
|
|
20
|
+
Today the catalog declares two — `fuzzing` and `grammar-based-fuzzing`. **Do not hardcode
|
|
21
|
+
those two names.** A module adding a third, or renaming one, must change this skill's
|
|
22
|
+
behaviour without changing this skill (FR-038-CON-1).
|
|
23
|
+
|
|
24
|
+
The test for "is this a fuzz method" is `evidence_kind === "Fuzz"`, every time.
|
|
25
|
+
|
|
26
|
+
## Two facts about today's catalog worth knowing
|
|
27
|
+
|
|
28
|
+
Neither changes the selector; both change what you should expect to see.
|
|
29
|
+
|
|
30
|
+
**`grammar-based-fuzzing` is never advised.** It is keyed on `structured-input` and
|
|
31
|
+
`grammar-declared`, and `characteristicsOf` mints neither, so no statement can reach it
|
|
32
|
+
(`agent-ix/quoin#128`). It arrives only as an authored method. If you were expecting the
|
|
33
|
+
advisor to suggest it for a grammar, that is why it did not.
|
|
34
|
+
|
|
35
|
+
**`fuzzing` is advised from two of its three keys.** `untrusted-input` and `parser` are
|
|
36
|
+
mintable; `deserializer` is not. An obligation about deserialization that says neither
|
|
37
|
+
"untrusted" nor "pars…" will not be advised, and is a legitimate authored-method case.
|
|
38
|
+
|
|
39
|
+
## What a good selection looks like
|
|
40
|
+
|
|
41
|
+
Obligations that survive this step describe an **input surface**: something that consumes
|
|
42
|
+
bytes or text from outside and must not crash. If a selected obligation is about a return
|
|
43
|
+
value, a threshold or an ordering, the method is wrong — say so in the report rather than
|
|
44
|
+
generating a target that fuzzes the wrong thing.
|
|
45
|
+
|
|
46
|
+
That is a finding about the method, not about the requirement. Do not reword the
|
|
47
|
+
requirement, and do not silently drop it (FR-038-AC-9).
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
# Step 2: Ground the entry point
|
|
2
|
+
|
|
3
|
+
**Goal**: the symbol the target will call. It must exist.
|
|
4
|
+
|
|
5
|
+
## Why this step cannot be skipped
|
|
6
|
+
|
|
7
|
+
A property test that grounds nothing is a weak test. **A fuzz target that calls nothing is a
|
|
8
|
+
build error in somebody's CI**, and it arrives in a harness most reviewers do not run
|
|
9
|
+
locally, so it fails in the one place nobody is watching.
|
|
10
|
+
|
|
11
|
+
The entry point is therefore *found in the source*, never inferred from the requirement's
|
|
12
|
+
wording (FR-038-AC-2).
|
|
13
|
+
|
|
14
|
+
## How to find it
|
|
15
|
+
|
|
16
|
+
In order, stopping at the first that resolves:
|
|
17
|
+
|
|
18
|
+
1. **The requirement's `## Inputs` section**, if it names a symbol or a module path. This is
|
|
19
|
+
the author telling you directly.
|
|
20
|
+
2. **An existing target in the same suite** covering a neighbouring surface — its imports
|
|
21
|
+
show the shape of a callable entry point in this repository.
|
|
22
|
+
3. **The public surface**: a function taking `&[u8]`, `&str`, `bytes` or `string` and
|
|
23
|
+
returning a `Result`/`Option`, exported from the module the requirement names.
|
|
24
|
+
|
|
25
|
+
Read the function. Confirm it takes the input the requirement describes and that calling it
|
|
26
|
+
has no side effect outside the process — a fuzz target that writes files or opens sockets
|
|
27
|
+
will be killed by the fuzzer's own sanitizers and produce nothing but noise.
|
|
28
|
+
|
|
29
|
+
## Ungrounded is a finding, not a guess
|
|
30
|
+
|
|
31
|
+
If none of the three resolves, **write no file for that obligation** and record a finding
|
|
32
|
+
naming it and what you looked for (FR-038-AC-3).
|
|
33
|
+
|
|
34
|
+
This is the honest outcome and it is often the useful one. *"NFR-003 declares the config
|
|
35
|
+
loader must not panic on arbitrary input, and there is no function in `src/` that takes
|
|
36
|
+
arbitrary bytes"* means either the surface does not exist yet or the requirement is about
|
|
37
|
+
something else. Both are worth a reviewer's attention; a generated target guessing at
|
|
38
|
+
`load_config()` is not.
|
|
39
|
+
|
|
40
|
+
## Do not fuzz through a wrapper
|
|
41
|
+
|
|
42
|
+
Where the requirement names a surface and the obvious public function wraps it in file I/O,
|
|
43
|
+
ground on the **inner** function that takes the bytes. Fuzzing the wrapper spends the whole
|
|
44
|
+
run on the filesystem and reaches the parser a few thousand times instead of a few hundred
|
|
45
|
+
million.
|
|
46
|
+
|
|
47
|
+
Record the choice in the provenance line, because it is a judgement:
|
|
48
|
+
|
|
49
|
+
```
|
|
50
|
+
spec-fuzz: obligation=NFR-003-M-1 harness=cargo-fuzz entry=loader::manifest::parse_manifest origin=advised
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
If you grounded on an inner function while the requirement named the outer one, say so in
|
|
54
|
+
the report. It is the kind of thing that is obviously right when explained and looks like an
|
|
55
|
+
error when not.
|
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
# Step 3: Emit the targets
|
|
2
|
+
|
|
3
|
+
**Goal**: a target that compiles, runs, and reconciles.
|
|
4
|
+
|
|
5
|
+
Templates: [cargo-fuzz](../assets/templates/cargo-fuzz.rs.md) ·
|
|
6
|
+
[atheris](../assets/templates/atheris.py.md) ·
|
|
7
|
+
[fast-check](../assets/templates/fast-check.ts.md)
|
|
8
|
+
|
|
9
|
+
## The tag contract
|
|
10
|
+
|
|
11
|
+
Inherited from `spec-correctness` step 4, unchanged, because `gap-analysis` reconciles by the
|
|
12
|
+
same grep. **Two independent carriers**, so one formatting change cannot break reconciliation:
|
|
13
|
+
|
|
14
|
+
1. `Trace: NFR-003-M-1` on its own line in the doc comment or docstring.
|
|
15
|
+
2. The target's own name containing the id — `fuzz_nfr_003_m_1_parse_manifest`.
|
|
16
|
+
|
|
17
|
+
### Placement, which is where this bites
|
|
18
|
+
|
|
19
|
+
A carrier binds to the **symbol whose source span encloses it**. For a libfuzzer target that
|
|
20
|
+
span is the `fuzz_target!` block — so the doc comment goes immediately above `fuzz_target!`,
|
|
21
|
+
not at the top of the file.
|
|
22
|
+
|
|
23
|
+
A file-header comment reads correctly, passes review, matches a grep, and **binds to
|
|
24
|
+
nothing**. Six generated files shipped that way in `@agent-ix/quoin@0.12.x` and every
|
|
25
|
+
criterion in them scored zero (`agent-ix/quoin#61`). Do not verify placement with grep — grep
|
|
26
|
+
does not care where a comment sits, which is exactly why that shipped.
|
|
27
|
+
|
|
28
|
+
## The provenance line
|
|
29
|
+
|
|
30
|
+
One per target, ignored by the reconciliation grep, read by this skill on re-runs:
|
|
31
|
+
|
|
32
|
+
```
|
|
33
|
+
spec-fuzz: obligation=<id> harness=<harness> entry=<symbol> origin=<advised|authored>
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
Add `wrapped=<outer-symbol>` when step 2 grounded on an inner function.
|
|
37
|
+
|
|
38
|
+
## Idempotent re-runs
|
|
39
|
+
|
|
40
|
+
Same obligation, same harness, same entry → **leave the file alone** (FR-038-AC-6).
|
|
41
|
+
|
|
42
|
+
A changed entry point is a real change: rewrite the target and say so in the report. A
|
|
43
|
+
changed requirement *statement* is not — the target calls a symbol, not a sentence, so a
|
|
44
|
+
reworded requirement with the same surface needs no new file.
|
|
45
|
+
|
|
46
|
+
## What the target must not do
|
|
47
|
+
|
|
48
|
+
- **No assertions about correctness.** Fuzzing looks for crashes, hangs and assertion
|
|
49
|
+
failures inside the code under test — not for wrong answers. An `assert_eq!` on the result
|
|
50
|
+
turns every interesting input into a false positive.
|
|
51
|
+
- **No `unwrap()` on the result.** The function returning `Err` on garbage is the requirement
|
|
52
|
+
being *met*. Unwrapping turns success into a crash and the fuzzer will find it in seconds.
|
|
53
|
+
- **No file, network or process access.** Sanitizers will kill it and the run produces noise.
|
|
54
|
+
- **No `panic!` as a way of reporting.** The fuzzer's crash is the report.
|
|
55
|
+
|
|
56
|
+
The body is therefore usually three lines: take the bytes, make the input the function
|
|
57
|
+
expects, call it and drop the result.
|
|
58
|
+
|
|
59
|
+
## Corpus and seeds
|
|
60
|
+
|
|
61
|
+
Out of scope. A seed corpus makes a fuzzer dramatically more effective and building one from
|
|
62
|
+
the repository's own fixtures is real work with its own decisions.
|
|
63
|
+
|
|
64
|
+
Say so in the report rather than shipping an empty `corpus/` directory that looks like it was
|
|
65
|
+
considered.
|