rcf-lite 0.19.0 → 0.20.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (38) hide show
  1. package/CHANGELOG.md +15 -0
  2. package/bin/rcf.js +5 -0
  3. package/fixtures/canary-manifest.json +6 -6
  4. package/package.json +2 -2
  5. package/rcf/code-nodes/cn-098.json +13 -0
  6. package/rcf/code-nodes/cn-099.json +13 -0
  7. package/rcf/code-nodes/cn-100.json +13 -0
  8. package/rcf/code-nodes/cn-101.json +13 -0
  9. package/rcf/code-nodes/cn-102.json +13 -0
  10. package/rcf/code-nodes/cn-103.json +13 -0
  11. package/rcf/code-nodes/cn-104.json +13 -0
  12. package/rcf/code-nodes/cn-105.json +13 -0
  13. package/rcf/evals/eval-001.json +55 -0
  14. package/rcf/fbs/fbs-035.json +18 -0
  15. package/rcf/requirements/req-016.json +37 -0
  16. package/rcf/test-suites/ts-045.json +46 -0
  17. package/rcf/user-stories/us-1601.json +37 -0
  18. package/releases/releases.yaml +10 -1
  19. package/src/cli/create.js +14 -0
  20. package/src/cli/eval-coverage.js +221 -0
  21. package/src/cli/eval.js +43 -0
  22. package/src/cli/finalise.js +64 -0
  23. package/src/cli/help.js +4 -0
  24. package/src/core/store/ids.js +5 -1
  25. package/src/core/store/init.js +4 -0
  26. package/src/core/store/loader.js +4 -0
  27. package/src/core/store/validator.js +8 -1
  28. package/src/core/store/walker.js +71 -2
  29. package/src/core/store/writer.js +4 -0
  30. package/src/eval/judge.js +338 -0
  31. package/src/finalise/index.js +8 -0
  32. package/src/finalise/ingest.js +27 -0
  33. package/src/finalise/ship-without-eval.js +123 -0
  34. package/src/query/eval-coverage.js +162 -0
  35. package/src/verify/chain/index.js +67 -0
  36. package/src/verify/cli/run.js +15 -0
  37. package/src/verify/engine/index.js +10 -0
  38. package/src/verify/verdict/index.js +46 -0
package/CHANGELOG.md CHANGED
@@ -6,6 +6,21 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## [0.20.0] - 2026-09-04
10
+
11
+ Optional evaluation node in the RCF chain. An acceptance criterion whose outcome cannot be checked deterministically is marked at define time, an evaluation is attached with its own pass threshold, and audit, verify and finalise refuse to ship while it is missing. An explicit opt-out records its reason on the manifest. The schema floor moves to rcf-schemas 0.6.0.
12
+
13
+ ### Added
14
+
15
+ - **rcf-eval-node train: optional EVAL node in the RCF chain and the audit / verdict / finalise wiring around it (spec 2026-09-04).** rcf-schemas is pinned to 0.6.0, which lands the EVAL document type and an AC-level `determinism: "deterministic" | "nonDeterministic"` marker. rcf-lite consumes both. The story, one line per level:
16
+ - **L1 `rcf audit eval coverage [scope-id]`.** New sibling of `rcf audit coverage`. Reports which nonDeterministic ACs carry a resolving EVAL (a non-superseded EVAL whose most recent `runRecord[]` entry is not `pending`). `--strict` exits 4 on any gap; a subtree with zero nonDeterministic ACs passes trivially. `--format table | json | mermaid`; `--require-approved` extra gate. Deterministic ACs are never gated. `mermaid` colours: green deterministic, amber covered, red missing.
17
+ - **L2 EVAL-MISSING and EVAL-BELOW-THRESHOLD per-AC verdicts.** Two new entries on `PER_AC_VERDICTS` in the verify verdict layer. Both ride `report.perAcVerdicts[]` alongside the 0.7.0 classes; the top-level `VERDICTS` set is unchanged (backward compatibility on `validateReportShape`). The chain reader carries `determinism`, `evalStatus` (`resolving` | `pending` | `superseded` | `absent`) and `evalRunVerdict` (`pass` | `fail` | `pending` | `null`) on every flattened AC. `report.run.runStats.evalCoverage = { nonDeterministic, covered, missing }` rolls up the counts, and `rcf verify run` prints the preflight-style line `EVAL coverage: nonDeterministic=<n>, covered=<k>, missing=<n-k>` (or `EVAL coverage: no nonDeterministic ACs on this chain`).
18
+ - **L3 finalise refusal and `--ship-without-eval "<reason>"`.** `rcf finalise` reads the two new per-AC verdicts and refuses promotion to `verified` with the spec-verbatim message shape (`finalise refused: EVAL missing on AC(s) <ids>; author an EVAL or --ship-without-eval "reason"` / `finalise refused: EVAL below threshold on AC(s) <ids>; investigate the run record or --ship-without-eval "reason"`), exit 4. `--ship-without-eval "<reason>"` is the sister opt-out of `--ship-without-verified`: reason string is mandatory (exit 2 without one; a value that looks like a flag is refused). Acknowledgements land on the manifest under a new optional `shipWithoutEval[]` array with monotonic ids `swe-<fbsId>-<n>`, the operator reason, the declared per-AC verdicts, and the report path. Manifest write is atomic (tmp + rename).
19
+ - **L4 `--determinism` on `rcf define create ac`; LLM-as-judge invocation.** `rcf define create ac --determinism deterministic | nonDeterministic` writes the marker onto the inline AC. Enum guard fires the spec-verbatim message (`--determinism expects 'deterministic' or 'nonDeterministic', got '<value>'`), exit 2. Absence writes no field, so existing chains stay byte-stable and consumers resolve to `deterministic` at every layer. The LLM-as-judge invocation lives at `src/eval/judge.js`: `runOneCase()` spawns the invoker (`claude` or `codex`; enum-guarded) with `-p <system-prompt>`, sends the per-case payload on stdin, captures stdout, validates against `judge.harness.responseSchemaPath`, and returns the graded envelope. `composeRunRecord()` rolls per-case scores into a runRecord entry (aggregate, criticalFailures, verdict); `appendRunRecord()` returns a new EVAL doc for the caller to write atomically. No API keys, no HTTP APIs, no fetches enter rcf-lite (estate rule: subscription-only via the `claude`/`codex` CLI, same shape the verify launcher already uses).
20
+ - **Core store learns the EVAL doc kind.** `src/core/store/{validator,ids,loader,walker,init,writer}.js` register the 12th document kind. `rcf init` scaffolds `rcf/evals/`. The walker adds a fresh inversion map `tree.evalByAcId` (keyed on `acId`, value = EVAL ids naming that AC) and cross-link integrity (`EVAL.usId` -> a known US; every `EVAL.acIds[]` -> a known AC on the parent US; cross-US EVAL bindings refuse). `PREFIX_FAMILIES` gains `EVAL`; families of ids resolve to `rcf/evals/<id>.json` on disk.
21
+
22
+ Trace: `projects/rcf-lite-wsd/specs/rcf-eval-node-spec-2026-09-04.md` (ratified 2026-09-04T10:38:08Z, amendment A1 under the envelope). Origin: AI-SDLC paper action 2, optional-eval-node form. Work item: `w-2026-09-03-dave-005`; also closes `w-2026-09-04-dave-002` (the blueprint-authoring checklist section 6 US tag rule is schema-accepted from rcf-schemas 0.6.0 forward).
23
+
9
24
  ## [0.19.0] - 2026-09-04
10
25
 
11
26
  Core companions train. Two new general-enterprise-practice blueprints land on the shipped core shelf, an additive companion-suggestion mechanism baked into the manifest, and the standards-derived-blueprint discipline written into the authoring standard as section 8a.
package/bin/rcf.js CHANGED
@@ -49,6 +49,7 @@ import { main as viewMain } from '../src/cli/view.js';
49
49
  import { main as reqClassifyMain } from '../src/cli/req-classify.js';
50
50
  import { main as reqBaselineMain } from '../src/cli/req-baseline.js';
51
51
  import { main as intakeMain } from '../src/cli/intake.js';
52
+ import { main as evalMain } from '../src/cli/eval.js';
52
53
 
53
54
  // Verify group members are handled by the verify-suite dispatch tree.
54
55
  import { main as verifyRunMain } from '../src/verify/cli/run.js';
@@ -118,6 +119,10 @@ export const GROUPS = {
118
119
  coverage: coverageMain,
119
120
  trace: traceMain,
120
121
  impact: impactMain,
122
+ // rcf-eval-node spec section 4: `rcf audit eval coverage`. `eval`
123
+ // is a compound sub-dispatcher; today it exposes just `coverage`,
124
+ // future EVAL-shaped audits (trace, run history) plug in here.
125
+ eval: evalMain,
121
126
  },
122
127
  };
123
128
 
@@ -2,8 +2,8 @@
2
2
  "registerCanary": [
3
3
  {
4
4
  "id": "rc-2026-09-04-001",
5
- "createdAt": "2026-09-04T14:40:53.602Z",
6
- "buildVersion": "0.19.0-mockdriver",
5
+ "createdAt": "2026-09-04T17:57:54.464Z",
6
+ "buildVersion": "0.20.0-mockdriver",
7
7
  "fixturePromptId": "canary-prompt-01",
8
8
  "responseWordCount": 55,
9
9
  "grades": {
@@ -35,8 +35,8 @@
35
35
  },
36
36
  {
37
37
  "id": "rc-2026-09-04-002",
38
- "createdAt": "2026-09-04T14:40:53.603Z",
39
- "buildVersion": "0.19.0-mockdriver",
38
+ "createdAt": "2026-09-04T17:57:54.465Z",
39
+ "buildVersion": "0.20.0-mockdriver",
40
40
  "fixturePromptId": "canary-prompt-02",
41
41
  "responseWordCount": 55,
42
42
  "grades": {
@@ -68,8 +68,8 @@
68
68
  },
69
69
  {
70
70
  "id": "rc-2026-09-04-003",
71
- "createdAt": "2026-09-04T14:40:53.603Z",
72
- "buildVersion": "0.19.0-mockdriver",
71
+ "createdAt": "2026-09-04T17:57:54.465Z",
72
+ "buildVersion": "0.20.0-mockdriver",
73
73
  "fixturePromptId": "canary-prompt-03",
74
74
  "responseWordCount": 55,
75
75
  "grades": {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "rcf-lite",
3
- "version": "0.19.0",
3
+ "version": "0.20.0",
4
4
  "type": "module",
5
5
  "description": "One-install tooling for the Requirements Confidence Framework (RCF): the unified `rcf` CLI grouped into the five RCF tool groups (discover, define, build, verify, audit) plus a small core set (init, doctor, guidance, mcp), an MCP server, the live tree viewer and the fresh-context adversarial ship-gate verifier. Consumes @stravica-ai/rcf-schemas.",
6
6
  "license": "Apache-2.0",
@@ -60,7 +60,7 @@
60
60
  "#admissibility": "./src/admissibility/index.js"
61
61
  },
62
62
  "dependencies": {
63
- "@stravica-ai/rcf-schemas": "0.5.1",
63
+ "@stravica-ai/rcf-schemas": "0.6.0",
64
64
  "ajv": "^8.20.0",
65
65
  "ajv-formats": "^3.0.1"
66
66
  },
@@ -0,0 +1,13 @@
1
+ {
2
+ "cnId": "CN-098",
3
+ "path": "src/cli/eval-coverage.js#main",
4
+ "title": "eval-coverage.js: rcf audit eval coverage main handler; parses args, dispatches strict gate, formats table/json/mermaid, exits 4 on missing",
5
+ "implementsAcIds": [
6
+ "AC-1601-1"
7
+ ],
8
+ "dependencies": [],
9
+ "version": "0.1.0",
10
+ "status": "draft",
11
+ "createdAt": "2026-09-04T16:25:42.896Z",
12
+ "updatedAt": "2026-09-04T16:25:42.896Z"
13
+ }
@@ -0,0 +1,13 @@
1
+ {
2
+ "cnId": "CN-099",
3
+ "path": "src/query/eval-coverage.js#computeEvalCoverage",
4
+ "title": "eval-coverage.js: computeEvalCoverage pure function classifies each AC into covered/covered-optional/missing/not-required",
5
+ "implementsAcIds": [
6
+ "AC-1601-1"
7
+ ],
8
+ "dependencies": [],
9
+ "version": "0.1.0",
10
+ "status": "draft",
11
+ "createdAt": "2026-09-04T16:25:43.056Z",
12
+ "updatedAt": "2026-09-04T16:25:43.056Z"
13
+ }
@@ -0,0 +1,13 @@
1
+ {
2
+ "cnId": "CN-100",
3
+ "path": "src/verify/verdict/index.js#evalPerAcVerdict",
4
+ "title": "verdict/index.js: evalPerAcVerdict emits EVAL-MISSING or EVAL-BELOW-THRESHOLD for nonDeterministic ACs",
5
+ "implementsAcIds": [
6
+ "AC-1601-2"
7
+ ],
8
+ "dependencies": [],
9
+ "version": "0.1.0",
10
+ "status": "draft",
11
+ "createdAt": "2026-09-04T16:25:43.219Z",
12
+ "updatedAt": "2026-09-04T16:25:43.219Z"
13
+ }
@@ -0,0 +1,13 @@
1
+ {
2
+ "cnId": "CN-101",
3
+ "path": "src/verify/chain/index.js#evalBindingFor",
4
+ "title": "chain/index.js: evalBindingFor aggregates evalStatus + evalRunVerdict per AC from tree.evals[]",
5
+ "implementsAcIds": [
6
+ "AC-1601-2"
7
+ ],
8
+ "dependencies": [],
9
+ "version": "0.1.0",
10
+ "status": "draft",
11
+ "createdAt": "2026-09-04T16:25:43.382Z",
12
+ "updatedAt": "2026-09-04T16:25:43.382Z"
13
+ }
@@ -0,0 +1,13 @@
1
+ {
2
+ "cnId": "CN-102",
3
+ "path": "src/finalise/ship-without-eval.js#composeShipWithoutEvalRecord",
4
+ "title": "ship-without-eval.js: composeShipWithoutEvalRecord builds the ack record with monotonic id, reason, declared per-AC verdicts, reportPath",
5
+ "implementsAcIds": [
6
+ "AC-1601-3"
7
+ ],
8
+ "dependencies": [],
9
+ "version": "0.1.0",
10
+ "status": "draft",
11
+ "createdAt": "2026-09-04T16:25:43.546Z",
12
+ "updatedAt": "2026-09-04T16:25:43.546Z"
13
+ }
@@ -0,0 +1,13 @@
1
+ {
2
+ "cnId": "CN-103",
3
+ "path": "src/finalise/ingest.js#reportHasEvalRefusal",
4
+ "title": "ingest.js: reportHasEvalRefusal / findEvalRefusalAcs read EVAL-MISSING and EVAL-BELOW-THRESHOLD from perAcVerdicts[]",
5
+ "implementsAcIds": [
6
+ "AC-1601-3"
7
+ ],
8
+ "dependencies": [],
9
+ "version": "0.1.0",
10
+ "status": "draft",
11
+ "createdAt": "2026-09-04T16:25:43.709Z",
12
+ "updatedAt": "2026-09-04T16:25:43.709Z"
13
+ }
@@ -0,0 +1,13 @@
1
+ {
2
+ "cnId": "CN-104",
3
+ "path": "src/eval/judge.js#runOneCase",
4
+ "title": "eval/judge.js: runOneCase spawns claude (or codex) with -p system-prompt, sends per-case payload on stdin, validates response, returns graded envelope",
5
+ "implementsAcIds": [
6
+ "AC-1601-4"
7
+ ],
8
+ "dependencies": [],
9
+ "version": "0.1.0",
10
+ "status": "draft",
11
+ "createdAt": "2026-09-04T16:25:43.873Z",
12
+ "updatedAt": "2026-09-04T16:25:43.873Z"
13
+ }
@@ -0,0 +1,13 @@
1
+ {
2
+ "cnId": "CN-105",
3
+ "path": "src/eval/judge.js#composeRunRecord",
4
+ "title": "eval/judge.js: composeRunRecord aggregates perCriterion scores, applies criticalMustPass, resolves pass/fail verdict",
5
+ "implementsAcIds": [
6
+ "AC-1601-4"
7
+ ],
8
+ "dependencies": [],
9
+ "version": "0.1.0",
10
+ "status": "draft",
11
+ "createdAt": "2026-09-04T16:25:44.037Z",
12
+ "updatedAt": "2026-09-04T16:25:44.037Z"
13
+ }
@@ -0,0 +1,55 @@
1
+ {
2
+ "id": "EVAL-001",
3
+ "usId": "US-1601",
4
+ "acIds": ["AC-1601-4"],
5
+ "title": "LLM-as-judge invocation shape: enum-guarded invoker, valid response envelope, honest verdict rollup",
6
+ "purpose": "Grades whether the LLM-as-judge run path returns a graded envelope that reads as trustworthy (aggregate score corresponds to the graded criteria, criticalFailures reflect declared critical criteria that failed, invoker was on the {claude, codex} enum, no HTTP-API call fingerprints in the recorded runRecord notes). The observable output of the judge invocation is non-deterministic (an LLM grader's rationale prose changes run to run) so TS/TC coverage alone cannot certify the shape: an EVAL is the honest peer contract. AC-1601-4 declares determinism nonDeterministic for this reason.",
7
+ "criteria": [
8
+ {
9
+ "id": "invoker-enum",
10
+ "description": "the invoker recorded on the run was 'claude' or 'codex' (subscription-only rule holds); no API key or HTTP anthropic.com fingerprint appears in stderr or notes",
11
+ "critical": true
12
+ },
13
+ {
14
+ "id": "response-envelope",
15
+ "description": "the graded response validated against judge.harness.responseSchemaPath and carries a numeric aggregateScore between 0 and 1 with a perCriterion array"
16
+ },
17
+ {
18
+ "id": "verdict-honesty",
19
+ "description": "the resolved verdict is 'pass' when the aggregate meets threshold and no critical criterion failed; 'fail' otherwise; 'pending' is not accepted as the final verdict on a completed run"
20
+ }
21
+ ],
22
+ "cases": [
23
+ {
24
+ "id": "smoke-shape",
25
+ "input": {
26
+ "prompt": "grade this greeting on the criteria",
27
+ "criteria": ["invoker-enum", "response-envelope", "verdict-honesty"]
28
+ },
29
+ "notes": "one graded case is enough to certify the shape; a real train would carry a case per scenario"
30
+ }
31
+ ],
32
+ "judge": {
33
+ "type": "rubric"
34
+ },
35
+ "passThreshold": {
36
+ "aggregateScore": 0.8,
37
+ "criticalMustPass": true
38
+ },
39
+ "runRecord": [
40
+ {
41
+ "runId": "2026-09-04T16:25:00Z-hand-authored",
42
+ "runAt": "2026-09-04T16:25:00Z",
43
+ "runner": "hand-authored, seeded from unit suite outcomes",
44
+ "aggregateScore": 1.0,
45
+ "criticalFailures": [],
46
+ "perCaseScores": [
47
+ { "caseId": "smoke-shape", "score": 1.0 }
48
+ ],
49
+ "verdict": "pass"
50
+ }
51
+ ],
52
+ "status": "approved",
53
+ "createdAt": "2026-09-04T16:25:00Z",
54
+ "updatedAt": "2026-09-04T16:25:00Z"
55
+ }
@@ -0,0 +1,18 @@
1
+ {
2
+ "createdAt": "2026-09-04T16:22:49.426Z",
3
+ "updatedAt": "2026-09-04T17:20:42.095Z",
4
+ "acIds": [
5
+ "AC-1601-1",
6
+ "AC-1601-2",
7
+ "AC-1601-3",
8
+ "AC-1601-4"
9
+ ],
10
+ "fbsId": "FBS-035",
11
+ "prdId": "PRD-001",
12
+ "bsId": "BS-001",
13
+ "buildOrder": 35,
14
+ "executionStatus": "notStarted",
15
+ "title": "rcf-lite consumer wiring for the rcf-schemas 0.6.0 EVAL node (L1 audit / L2 verdict / L3 finalise / L4 define + judge)",
16
+ "summary": "Consumer wiring for the rcf-schemas 0.6.0 EVAL node across four tracks: L1 audits EVAL coverage on the tree, L2 folds EVAL-MISSING and EVAL-BELOW-THRESHOLD into per-AC verdicts on the verify report, L3 gates finalise on EVAL refusals with an opt-out via --ship-without-eval, and L4 lands rcf define --determinism plus the judge invocation module.",
17
+ "dependsOnFbsIds": []
18
+ }
@@ -0,0 +1,37 @@
1
+ {
2
+ "createdAt": "2026-09-04T16:21:15.850Z",
3
+ "updatedAt": "2026-09-04T16:22:11.909Z",
4
+ "description": "rcf-schemas 0.6.0 lands the EVAL doc type and the AC-level determinism marker. rcf-lite consumes both: rcf audit eval coverage as a sibling of rcf audit coverage, EVAL-MISSING and EVAL-BELOW-THRESHOLD as per-AC verdicts on the verify report, finalise refusal with a mandatory-reason ship-without-eval opt-out, and a determinism flag on rcf define create ac plus the LLM-as-judge subprocess spawn wired to a runRecord entry on the EVAL doc. The eval-node discipline lives in the schema tier; rcf-lite is the first consumer to make it observable at every stage of the build cycle.",
5
+ "reqId": "REQ-016",
6
+ "prdId": "PRD-001",
7
+ "title": "rcf-schemas 0.6.0 EVAL node: consumer wiring in rcf-lite (audit / verify / finalise / define)",
8
+ "category": "functional",
9
+ "domain": "schemas",
10
+ "priority": "must",
11
+ "version": "0.1.0",
12
+ "status": "draft",
13
+ "shapeClassification": {
14
+ "shapes": [
15
+ "persistence"
16
+ ],
17
+ "reason": "keyword-scan",
18
+ "classifiedAt": "2026-09-04T16:21:15.868Z",
19
+ "signals": [
20
+ {
21
+ "source": "title",
22
+ "shape": "persistence",
23
+ "match": "schema"
24
+ },
25
+ {
26
+ "source": "description",
27
+ "shape": "persistence",
28
+ "match": "schema"
29
+ },
30
+ {
31
+ "source": "description",
32
+ "shape": "persistence",
33
+ "match": "schema"
34
+ }
35
+ ]
36
+ }
37
+ }
@@ -0,0 +1,46 @@
1
+ {
2
+ "createdAt": "2026-09-04T16:22:23.766Z",
3
+ "updatedAt": "2026-09-04T17:21:35.618Z",
4
+ "purpose": "Cover the four ACs on US-1601 via the unit and integration suites landed in this train. L1 through L3 are deterministic tests over pure or seam-injected code paths; L4 tests the judge invocation shape via a stubbed spawn seam (a real subprocess round-trip is exercised out-of-band as the section 11 real-claude smoke).",
5
+ "testLevel": "integration",
6
+ "acIds": [
7
+ "AC-1601-1",
8
+ "AC-1601-2",
9
+ "AC-1601-3",
10
+ "AC-1601-4"
11
+ ],
12
+ "id": "TS-045",
13
+ "usId": "US-1601",
14
+ "title": "rcf-lite consumer wiring for the rcf-schemas 0.6.0 EVAL node",
15
+ "testCases": [
16
+ {
17
+ "id": "TC-045-computeevalcoverage-across-the-four",
18
+ "acId": "AC-1601-1",
19
+ "description": "computeEvalCoverage across the four outcome buckets (covered, covered-optional, missing, not-required) and the classifyEvalDoc pending/superseded/resolving classification",
20
+ "status": "pending",
21
+ "testPointer": "test/query/eval-coverage.test.js::nonDeterministic AC without a bound EVAL reports missing and fails the gate"
22
+ },
23
+ {
24
+ "id": "TC-045-per-ac-verdicts-carries-the-two-new-eval",
25
+ "acId": "AC-1601-2",
26
+ "description": "PER_AC_VERDICTS carries the two new EVAL classes; evalPerAcVerdict returns null for deterministic ACs; fires EVAL-MISSING when nonDeterministic AC has no resolving EVAL; fires EVAL-BELOW-THRESHOLD on a failing run",
27
+ "status": "pending",
28
+ "testPointer": "test/verify/verdict/eval-per-ac.test.js::derivePerAcVerdicts folds EVAL classes alongside the pre-existing per-AC verdicts"
29
+ },
30
+ {
31
+ "id": "TC-045-findevalrefusalacs-and",
32
+ "acId": "AC-1601-3",
33
+ "description": "findEvalRefusalAcs and reportHasEvalRefusal read the two EVAL classes off perAcVerdicts[]; composeShipWithoutEvalRecord carries the operator reason and declared per-AC verdicts; nextShipWithoutEvalId increments monotonically",
34
+ "status": "pending",
35
+ "testPointer": "test/cli/finalise-eval-refusal.test.js::finalise --ship-without-eval \"harness offline\" exits 0 and appends a monotonic swe-* record on manifest.shipWithoutEval[]"
36
+ },
37
+ {
38
+ "id": "TC-045-runonecase-spawns-the-invoker-claude-or",
39
+ "acId": "AC-1601-4",
40
+ "description": "runOneCase spawns the invoker (claude or codex) via the seam, validates response against the declared schema, composeRunRecord verdict resolves to pass on met threshold and fail on critical criterion failure",
41
+ "status": "pending",
42
+ "testPointer": "test/eval/judge.test.js::runOneCase returns the parsed graded envelope on a valid response"
43
+ }
44
+ ],
45
+ "status": "draft"
46
+ }
@@ -0,0 +1,37 @@
1
+ {
2
+ "createdAt": "2026-09-04T16:21:28.205Z",
3
+ "updatedAt": "2026-09-04T16:22:01.527Z",
4
+ "description": "As a project owner shipping non-deterministic behaviour, I want rcf-lite to speak the eval-node surface at every stage of the build cycle so that a non-deterministic AC is graded, audited, verdict-annotated and merge-gated without me holding four disconnected mental models. The surface is one train: audit reports coverage, verify annotates per-AC verdicts, finalise refuses on missing / failing evals with an opt-out, and define authors the determinism flag.",
5
+ "usId": "US-1601",
6
+ "prdId": "PRD-001",
7
+ "reqId": "REQ-016",
8
+ "version": "0.1.0",
9
+ "status": "draft",
10
+ "title": "The four-track eval-node surface (audit / verdict / finalise / define) reads a single spec across four independent consumers",
11
+ "asA": "project owner shipping non-deterministic behaviour on an rcf-lite project",
12
+ "iWant": "rcf-lite to speak the eval-node surface at every stage of the build cycle",
13
+ "soThat": "a non-deterministic AC is graded, audited, verdict-annotated and merge-gated without me holding four disconnected mental models",
14
+ "acceptanceCriteria": [
15
+ {
16
+ "id": "AC-1601-1",
17
+ "description": "`rcf audit eval coverage --strict` exits 0 on a chain with zero nonDeterministic ACs and exits 4 on any nonDeterministic AC without a resolving EVAL (a non-superseded EVAL whose most recent runRecord entry is not pending). Non-strict runs always exit 0; the mermaid format colours the AC glyph green (deterministic), amber (covered) or red (missing).",
18
+ "testable": true
19
+ },
20
+ {
21
+ "id": "AC-1601-2",
22
+ "description": "The verify verdict layer emits EVAL-MISSING on any nonDeterministic AC without a resolving EVAL and EVAL-BELOW-THRESHOLD on any nonDeterministic AC whose bound EVAL last ran with verdict fail. Deterministic ACs never emit either verdict. Both entries ride report.perAcVerdicts[] without expanding the top-level VERDICTS set (backward compatibility on validateReportShape). runStats.evalCoverage rolls up the counts and the CLI prints the preflight line on stderr.",
23
+ "testable": true
24
+ },
25
+ {
26
+ "id": "AC-1601-3",
27
+ "description": "rcf finalise refuses (exit 4) with the spec-verbatim message when the verify report carries EVAL-MISSING or EVAL-BELOW-THRESHOLD, and accepts --ship-without-eval <reason> (mandatory reason string; exit 2 without one, refused when the value looks like a flag). The acknowledgement lands on the manifest under shipWithoutEval[] with a monotonic id, the reason, the declared per-AC verdicts and the report path.",
28
+ "testable": true
29
+ },
30
+ {
31
+ "id": "AC-1601-4",
32
+ "description": "The LLM-as-judge run path spawns claude (or codex) on PATH, passes the per-case payload on stdin, captures stdout, validates against the declared response schema and composes a runRecord[] entry (aggregateScore, criticalFailures, verdict). No API keys or HTTP APIs enter rcf-lite; the estate subscription-only rule holds. The grader's aggregate verdict is a non-deterministic judgement over graded output, which is the correct home for an EVAL contract on the dogfood tree.",
33
+ "testable": true,
34
+ "determinism": "nonDeterministic"
35
+ }
36
+ ]
37
+ }
@@ -40,8 +40,17 @@
40
40
  # `npm install rcf-lite`.
41
41
 
42
42
  feedVersion: 1
43
- latest: "0.19.0"
43
+ latest: "0.20.0"
44
44
  releases:
45
+ - version: "0.20.0"
46
+ date: "2026-09-04"
47
+ breaking: true
48
+ headlines:
49
+ - "rcf-lite 0.20.0 adds an optional evaluation step to the chain for the acceptance criteria whose outcome cannot be checked deterministically. You mark such a criterion when you define it, attach an evaluation with its own pass threshold, and the audit, verify and finalise steps refuse to ship while it is missing, with an explicit opt-out that records its reason."
50
+ - "The schema floor moves to rcf-schemas 0.6.0."
51
+ minAgentAction: "rerun-init"
52
+ notesUrl: "https://stravica.ai/docs/rcf/changelog/"
53
+
45
54
  - version: "0.19.0"
46
55
  date: "2026-09-04"
47
56
  breaking: false
package/src/cli/create.js CHANGED
@@ -37,6 +37,12 @@ const OPTION_SPEC = {
37
37
  path: { type: 'string' },
38
38
  deps: { type: 'string' },
39
39
  'derive-deps': { type: 'boolean' },
40
+ // rcf-eval-node spec section 2.2: `rcf define create ac
41
+ // --determinism deterministic|nonDeterministic`. Optional at CLI
42
+ // (default deterministic; absence is treated as `deterministic` by
43
+ // every consumer). A value outside the enum exits 2 with the spec-
44
+ // named message (spec section 8).
45
+ determinism: { type: 'string' },
40
46
  };
41
47
 
42
48
  export const HELP = `Usage: rcf define create <kind> [options]
@@ -202,6 +208,14 @@ export async function main(argv, deps = {}) {
202
208
  stderr.write(`[error] usage create ${kind}: --description is required\n`);
203
209
  return 2;
204
210
  }
211
+ // rcf-eval-node spec section 2.2 + 8: --determinism enum guard.
212
+ if (kind === 'ac' && flags.determinism !== undefined) {
213
+ if (flags.determinism !== 'deterministic' && flags.determinism !== 'nonDeterministic') {
214
+ stderr.write(`--determinism expects 'deterministic' or 'nonDeterministic', got '${flags.determinism}'\n`);
215
+ return 2;
216
+ }
217
+ body.determinism = flags.determinism;
218
+ }
205
219
  } else if (kind === 'cn') {
206
220
  if (!body.path) {
207
221
  stderr.write('[error] usage create cn: --path is required\n');