triad-plus 1.7.0 → 1.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +10 -0
- package/README.md +9 -0
- package/adapters/antigravity/.agents/agents/triad-evaluator/agent.md +5 -0
- package/adapters/claude-code/.claude/agents/triad-evaluator.md +5 -0
- package/adapters/copilot/.github/agents/triad-evaluator.agent.md +5 -0
- package/adapters/hermes/skills/triad/SKILL.md +5 -0
- package/adapters/opencode/.opencode/agents/triad-evaluator.md +6 -0
- package/docs/architecture.md +7 -0
- package/docs/configuration.md +19 -0
- package/docs/evaluator-plus.md +9 -0
- package/docs/operating-guide.it.md +10 -0
- package/docs/operating-guide.md +10 -0
- package/docs/quality-contract.md +119 -0
- package/docs/verification.md +17 -0
- package/package.json +2 -2
- package/runtime/lib/quality-baseline.mjs +205 -0
- package/runtime/triad-evaluator-validate.mjs +261 -0
- package/runtime/triad-verify.mjs +19 -5
- package/schemas/evaluator-plus-result.schema.json +16 -0
- package/schemas/quality-baseline.schema.json +42 -0
- package/schemas/verification-evidence.schema.json +1 -1
- package/skills/triad-loop-bootstrap/SKILL.md +8 -0
- package/skills/triad-loop-bootstrap/assets/loop-template/handoff-report.template.md +13 -0
- package/skills/triad-loop-bootstrap/assets/loop-template/quality-baseline.json +15 -0
- package/skills/triad-loop-bootstrap/assets/loop-template/run-state.yaml +7 -0
- package/skills/triad-loop-bootstrap/assets/loop-template/runtime/assignments/assignment.template.json +2 -0
- package/skills/triad-loop-bootstrap/assets/project.yaml +5 -0
- package/skills/triad-loop-evaluator/SKILL.md +12 -2
- package/skills/triad-loop-orchestrator/SKILL.md +51 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,15 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 1.8.0 — 2026-09-12
|
|
4
|
+
|
|
5
|
+
- Add an optional Immutable Quality Contract with a deterministic Quality
|
|
6
|
+
Baseline fingerprint and fail-closed source-drift detection.
|
|
7
|
+
- Add phase-scoped `product_quality` and `delivery_closure` criteria,
|
|
8
|
+
criterion-level Evaluator+ validation, and a deterministic aggregate
|
|
9
|
+
verdict.
|
|
10
|
+
- Add deterministic Evaluator+/Delivery control-plane validation while
|
|
11
|
+
preserving legacy Triad+ 1.7 compatibility.
|
|
12
|
+
|
|
3
13
|
## 1.7.0 — 2026-09-10
|
|
4
14
|
|
|
5
15
|
- Add an optional deterministic importer for one BMAD `ready-for-dev` Story at
|
package/README.md
CHANGED
|
@@ -107,6 +107,15 @@ The Orchestrator first shows the feature cards, then delegates the bounded work.
|
|
|
107
107
|
If a tutorial step is unclear, see the [OpenCode guide](docs/runtimes.md#opencode)
|
|
108
108
|
and [troubleshooting](docs/troubleshooting.md).
|
|
109
109
|
|
|
110
|
+
## Optional immutable quality target
|
|
111
|
+
|
|
112
|
+
An initialized project may bind an immutable JSON Quality Baseline through
|
|
113
|
+
`project.quality_contract`. Triad fingerprints the canonical manifest, verifies
|
|
114
|
+
all declared sources before running costly gates, and fails closed on malformed
|
|
115
|
+
contracts or source drift. `product_quality` criteria are evaluated one by one
|
|
116
|
+
by a fresh Evaluator+; `delivery_closure` criteria remain in the delivery gate.
|
|
117
|
+
Projects without this opt-in continue to use the legacy PRD baseline path.
|
|
118
|
+
|
|
110
119
|
## Optional BMAD Story import
|
|
111
120
|
|
|
112
121
|
When BMAD planning has already produced a Story with `status: ready-for-dev`,
|
|
@@ -16,3 +16,8 @@ Return `candidate_wins`, `bar_wins`, or `indeterminate`; a `bar_wins` decision
|
|
|
16
16
|
has exactly one evidence-based largest gap. Do not edit source, change loop
|
|
17
17
|
state, commit, push, or review delivery, request repair, or reopen the Triad
|
|
18
18
|
run.
|
|
19
|
+
|
|
20
|
+
For an immutable Quality Contract, use the supplied baseline fingerprint and
|
|
21
|
+
evaluate each `product_quality` criterion exactly once. `delivery_closure`
|
|
22
|
+
criteria are excluded from this fresh packet and belong to delivery closure;
|
|
23
|
+
the control plane validates the final aggregate.
|
|
@@ -23,3 +23,8 @@ Return `candidate_wins`, `bar_wins`, or `indeterminate`. A `bar_wins` result has
|
|
|
23
23
|
exactly one evidence-based largest gap with one bounded repair scope. Do not edit
|
|
24
24
|
product source, change state, commit, push, or approve delivery; do not request
|
|
25
25
|
repair or reopen the Triad run.
|
|
26
|
+
|
|
27
|
+
For an immutable Quality Contract, report the supplied baseline fingerprint and
|
|
28
|
+
evaluate every supplied `product_quality` criterion once. `delivery_closure`
|
|
29
|
+
criteria are not part of this packet and are checked by delivery closure. The
|
|
30
|
+
control plane, not the model, determines the deterministic aggregate verdict.
|
|
@@ -24,3 +24,8 @@ out-of-packet control record to justify a verdict. Return `PASS`, `FAIL`, or
|
|
|
24
24
|
Do not edit source, change Triad queue/state, assign work, commit, push, publish,
|
|
25
25
|
or start repair. Your verdict is post-run information. It never reopens Triad,
|
|
26
26
|
requests repair, or changes an already closed result.
|
|
27
|
+
|
|
28
|
+
When an immutable Quality Contract is present, report its fingerprint and one
|
|
29
|
+
result for every supplied `product_quality` criterion. Do not receive or judge
|
|
30
|
+
`delivery_closure` criteria; the control plane validates the deterministic
|
|
31
|
+
aggregate rather than trusting a model-selected overall verdict.
|
|
@@ -51,3 +51,8 @@ post-run report at the absolute `<control-workspace>/artifacts/evaluator-plus/`
|
|
|
51
51
|
path supplied by the Orchestrator; never infer it relative to a product worktree.
|
|
52
52
|
`FAIL` or `INDETERMINATE` is information for a new
|
|
53
53
|
owner-requested run, never an automatic repair and never reopens Triad.
|
|
54
|
+
|
|
55
|
+
If an immutable Quality Contract is configured, pass only its fingerprint and
|
|
56
|
+
`product_quality` criteria to the fresh Evaluator+. Exclude `delivery_closure`
|
|
57
|
+
criteria, which are recorded by delivery closure. Return one result per product
|
|
58
|
+
criterion; the shared control plane validates coverage and aggregate verdict.
|
|
@@ -36,3 +36,9 @@ result being assessed.
|
|
|
36
36
|
Return `PASS`, `FAIL`, or `INDETERMINATE` with concise evidence. Do not edit
|
|
37
37
|
source, make delivery decisions, commit, push, or change workflow state; do not
|
|
38
38
|
request repair. A verdict never reopens Triad or starts repair.
|
|
39
|
+
|
|
40
|
+
If the packet includes an immutable Quality Contract, use its fingerprint and
|
|
41
|
+
evaluate exactly the supplied `product_quality` criteria once each. Never
|
|
42
|
+
evaluate `delivery_closure` criteria; delivery closure records those separately.
|
|
43
|
+
The control plane validates the deterministic aggregate, so do not invent a
|
|
44
|
+
different overall verdict.
|
package/docs/architecture.md
CHANGED
|
@@ -22,3 +22,10 @@ runtime-specific branches to Core installer or verification logic.
|
|
|
22
22
|
|
|
23
23
|
Evaluator+ is outside the Core: it runs after a Reviewer-approved result, receives
|
|
24
24
|
a deliberately limited fresh packet, and cannot reopen the completed run.
|
|
25
|
+
|
|
26
|
+
An optional immutable Quality Contract extends this existing control-plane
|
|
27
|
+
baseline without creating a second control plane. Its JSON manifest binds
|
|
28
|
+
approved intent sources and phase-scoped criteria to one canonical fingerprint.
|
|
29
|
+
`product_quality` criteria travel in the fresh Evaluator+ packet;
|
|
30
|
+
`delivery_closure` criteria are checked only during delivery closure. Projects
|
|
31
|
+
without the contract keep the legacy PRD baseline behavior.
|
package/docs/configuration.md
CHANGED
|
@@ -47,3 +47,22 @@ optional gate becomes required for that card, and an absent or empty list keeps
|
|
|
47
47
|
the legacy gate behavior. Selected IDs are validated before Developer dispatch
|
|
48
48
|
and are bound to the assignment; Triad does not attach visual or other
|
|
49
49
|
domain-specific meaning to a gate ID.
|
|
50
|
+
|
|
51
|
+
## Optional immutable Quality Contract
|
|
52
|
+
|
|
53
|
+
New or upgraded control workspaces may set:
|
|
54
|
+
|
|
55
|
+
```yaml
|
|
56
|
+
project:
|
|
57
|
+
quality_contract:
|
|
58
|
+
baseline: artifacts/quality-baseline.json
|
|
59
|
+
fingerprint: <sha256 of the canonical manifest>
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
The JSON manifest must contain at least one hashed, project-relative source and
|
|
63
|
+
may contain criteria scoped only to `product_quality` or `delivery_closure`.
|
|
64
|
+
The fingerprint is deterministic and independent of JSON whitespace or object
|
|
65
|
+
key order. A configured contract is validated before expensive verification
|
|
66
|
+
gates; malformed manifests and source drift fail closed. Omitting the section
|
|
67
|
+
preserves the 1.7 legacy PRD baseline behavior. A rebaseline is a new manifest
|
|
68
|
+
revision and explicit owner event, never an in-place edit.
|
package/docs/evaluator-plus.md
CHANGED
|
@@ -12,3 +12,12 @@ attempt history unless the owner explicitly requires it. It reports `PASS`,
|
|
|
12
12
|
|
|
13
13
|
An Evaluator+ `FAIL` does not reopen Triad, change the approved state, or start
|
|
14
14
|
repair. An owner or a later Orchestrator may use it as input to a new run.
|
|
15
|
+
|
|
16
|
+
When a project declares an immutable Quality Contract, the packet also carries
|
|
17
|
+
the approved Quality Baseline fingerprint and exactly one result for each
|
|
18
|
+
`product_quality` criterion. `delivery_closure` criteria are deliberately
|
|
19
|
+
excluded and are evaluated by delivery closure. The control plane validates
|
|
20
|
+
fingerprint, candidate binding, criterion coverage, and the deterministic
|
|
21
|
+
aggregate (`FAIL` over `INDETERMINATE` over `PASS`) before recording the report.
|
|
22
|
+
Formatting a manifest or changing a bound source invalidates the contract; it
|
|
23
|
+
never starts a product retry.
|
|
@@ -51,6 +51,16 @@ selezionati possono essere saltati. Lista assente o vuota mantiene il
|
|
|
51
51
|
comportamento legacy. Triad collega e registra gli ID senza attribuire loro un
|
|
52
52
|
significato visuale o di altro dominio.
|
|
53
53
|
|
|
54
|
+
## Target di qualità immutabile
|
|
55
|
+
|
|
56
|
+
Un progetto può optare per `project.quality_contract`, che collega al run un
|
|
57
|
+
fingerprint canonico del Quality Baseline JSON e verifica le source hashate prima
|
|
58
|
+
del dispatch e dei gate costosi. Drift o manifest invalido fermano il run come
|
|
59
|
+
contesto invalido senza consumare il budget di retry del prodotto. I criteri
|
|
60
|
+
`product_quality` vengono inviati al nuovo Evaluator+; quelli
|
|
61
|
+
`delivery_closure` sono verificati separatamente prima di `delivered`. Senza
|
|
62
|
+
questo opt-in i progetti legacy restano invariati.
|
|
63
|
+
|
|
54
64
|
## Retry e scope del candidato
|
|
55
65
|
|
|
56
66
|
Gli attempt sono record storici di esecuzione. I nuovi workspace separano e
|
package/docs/operating-guide.md
CHANGED
|
@@ -51,6 +51,16 @@ become required for that card, and unselected optional gates may be skipped. An
|
|
|
51
51
|
absent or empty list preserves the legacy behavior. Triad binds and records the
|
|
52
52
|
IDs but does not attach visual or other domain-specific meaning to them.
|
|
53
53
|
|
|
54
|
+
## Immutable quality target
|
|
55
|
+
|
|
56
|
+
When a project opts in with `project.quality_contract`, the control plane binds a
|
|
57
|
+
canonical JSON Quality Baseline fingerprint and verifies its hashed sources before
|
|
58
|
+
dispatch or expensive gates. A source drift or invalid manifest stops the run as
|
|
59
|
+
context invalid, without consuming product retry budget. Product-quality criteria
|
|
60
|
+
are sent to the fresh Evaluator+ one at a time; delivery-closure criteria are
|
|
61
|
+
checked separately before `delivered`. The contract is optional, and legacy
|
|
62
|
+
projects without it are unchanged.
|
|
63
|
+
|
|
54
64
|
## Retry accounting and candidate scope
|
|
55
65
|
|
|
56
66
|
Attempts are historical execution records. New workspaces separately bound
|
|
@@ -0,0 +1,119 @@
|
|
|
1
|
+
# Immutable Quality Contract
|
|
2
|
+
|
|
3
|
+
Triad+ 1.8 optionally binds a run to an owner-approved, machine-readable
|
|
4
|
+
Quality Baseline. It is an additional target record, not a second control
|
|
5
|
+
plane and not a replacement for the PRD, card baseline, or candidate
|
|
6
|
+
fingerprint.
|
|
7
|
+
|
|
8
|
+
```text
|
|
9
|
+
Quality Baseline fingerprint = what the run was meant to satisfy
|
|
10
|
+
Repository/card baseline = where implementation started
|
|
11
|
+
Candidate fingerprint = what implementation produced
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
## Manifest
|
|
15
|
+
|
|
16
|
+
The manifest is JSON and is normally stored at
|
|
17
|
+
`artifacts/quality-baseline.json`:
|
|
18
|
+
|
|
19
|
+
```json
|
|
20
|
+
{
|
|
21
|
+
"schema_version": 1,
|
|
22
|
+
"id": "my-project-quality-baseline",
|
|
23
|
+
"revision": 1,
|
|
24
|
+
"sources": [
|
|
25
|
+
{
|
|
26
|
+
"id": "prd",
|
|
27
|
+
"role": "intent",
|
|
28
|
+
"path": "artifacts/prd.md",
|
|
29
|
+
"sha256": "<64-hex-digest>"
|
|
30
|
+
}
|
|
31
|
+
],
|
|
32
|
+
"criteria": [
|
|
33
|
+
{ "id": "QB-001", "scope": "product_quality", "requirement": "..." },
|
|
34
|
+
{ "id": "QB-010", "scope": "delivery_closure", "requirement": "..." }
|
|
35
|
+
],
|
|
36
|
+
"fingerprint": "<sha256-of-canonical-manifest-without-fingerprint>"
|
|
37
|
+
}
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
The manifest requires a positive revision, at least one source, unique source
|
|
41
|
+
and criterion IDs, project-relative source paths, SHA-256 for every source, and
|
|
42
|
+
non-empty requirements. Version 1 has exactly two criterion scopes:
|
|
43
|
+
`product_quality` and `delivery_closure`.
|
|
44
|
+
|
|
45
|
+
The fingerprint is SHA-256 over canonical JSON with object keys sorted
|
|
46
|
+
recursively, array order preserved, and the `fingerprint` field excluded from
|
|
47
|
+
the payload. Whitespace and object formatting therefore do not change it.
|
|
48
|
+
|
|
49
|
+
## Binding and drift
|
|
50
|
+
|
|
51
|
+
When `project.quality_contract` is present, the Orchestrator binds both the
|
|
52
|
+
manifest path and its fingerprint in every active Developer assignment:
|
|
53
|
+
|
|
54
|
+
```json
|
|
55
|
+
{
|
|
56
|
+
"quality_baseline_path": "artifacts/quality-baseline.json",
|
|
57
|
+
"expected_quality_baseline_fingerprint": "<sha256>"
|
|
58
|
+
}
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
`triad-verify` validates the manifest and all source hashes before expensive
|
|
62
|
+
gates. A malformed manifest or fingerprint mismatch is
|
|
63
|
+
`quality_baseline_invalid`; a valid manifest whose declared sources or
|
|
64
|
+
assignment fingerprint no longer match is `quality_baseline_drift`. Both are
|
|
65
|
+
`invalid_context`: no Developer dispatch, expensive gates, or retry budget.
|
|
66
|
+
|
|
67
|
+
Projects without `quality_contract` retain the legacy PRD-only path. A
|
|
68
|
+
rebaseline is never an in-place edit: create a new revision/fingerprint and
|
|
69
|
+
record an explicit lineage event. Historical evidence is not silently
|
|
70
|
+
reinterpreted.
|
|
71
|
+
|
|
72
|
+
## Phase ownership
|
|
73
|
+
|
|
74
|
+
`product_quality` criteria are included in the fresh, blind Evaluator+ packet.
|
|
75
|
+
The packet carries the baseline fingerprint, final candidate fingerprint,
|
|
76
|
+
criteria, approved source material, and bounded verifier evidence. It never
|
|
77
|
+
includes delivery criteria, queue state, handoff state, or attempt history.
|
|
78
|
+
|
|
79
|
+
Evaluator+ returns one result per product criterion. The control plane validates
|
|
80
|
+
coverage, uniqueness, scope, candidate/baseline fingerprints, and the overall
|
|
81
|
+
verdict. Aggregation is deterministic:
|
|
82
|
+
|
|
83
|
+
```text
|
|
84
|
+
any FAIL -> FAIL
|
|
85
|
+
else any INDETERMINATE -> INDETERMINATE
|
|
86
|
+
else -> PASS
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
`delivery_closure` criteria are evaluated separately during delivery closure
|
|
90
|
+
and recorded with criterion ID, verdict, and evidence references. A run is not
|
|
91
|
+
`delivered` when any configured delivery criterion is `FAIL` or
|
|
92
|
+
`INDETERMINATE`. Quality Bar evaluation is not a required-gate replacement,
|
|
93
|
+
and an Evaluator+ failure never repairs or reopens Triad automatically.
|
|
94
|
+
|
|
95
|
+
## Explicit control-plane validation
|
|
96
|
+
|
|
97
|
+
The installed runtime exposes deterministic commands for the two phase
|
|
98
|
+
boundaries. Run the baseline preflight before dispatch, then run the phase
|
|
99
|
+
validators from the control workspace so the baseline is reloaded from disk:
|
|
100
|
+
|
|
101
|
+
```bash
|
|
102
|
+
node .triad-runtime/triad-evaluator-validate.mjs --mode baseline \
|
|
103
|
+
--project /absolute/path/to/control \
|
|
104
|
+
--baseline artifacts/quality-baseline.json
|
|
105
|
+
|
|
106
|
+
node .triad-runtime/triad-evaluator-validate.mjs --mode evaluator \
|
|
107
|
+
--project /absolute/path/to/control \
|
|
108
|
+
--baseline artifacts/quality-baseline.json \
|
|
109
|
+
--result artifacts/evaluator-plus/evaluation.json \
|
|
110
|
+
--expected-candidate-fingerprint <final-candidate-fingerprint>
|
|
111
|
+
|
|
112
|
+
node .triad-runtime/triad-evaluator-validate.mjs --mode delivery \
|
|
113
|
+
--project /absolute/path/to/control \
|
|
114
|
+
--baseline artifacts/quality-baseline.json \
|
|
115
|
+
--result artifacts/delivery-closure.json
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
Each command emits one machine-readable JSON result and exits non-zero for an
|
|
119
|
+
invalid contract, source drift, stale candidate binding, or invalid result.
|
package/docs/verification.md
CHANGED
|
@@ -17,6 +17,23 @@ An agent-reported claim is not the same as verification evidence. A Developer ca
|
|
|
17
17
|
report the commands it ran; `triad-verify` independently observes declared
|
|
18
18
|
required `control-plane` gates and writes atomic evidence.
|
|
19
19
|
|
|
20
|
+
## Immutable Quality Contract
|
|
21
|
+
|
|
22
|
+
Projects may opt into `project.quality_contract` with a project-relative JSON
|
|
23
|
+
manifest and its expected SHA-256 fingerprint. The shared
|
|
24
|
+
`runtime/lib/quality-baseline.mjs` loader canonicalizes the manifest (excluding
|
|
25
|
+
its self-declared `fingerprint`), validates source paths, IDs, scopes, and hashes,
|
|
26
|
+
then verifies every bound source before any expensive gate runs. A malformed,
|
|
27
|
+
missing, or mismatched contract is `invalid_context`; a valid manifest whose
|
|
28
|
+
bound source content has changed is `quality_baseline_drift`. Both fail closed:
|
|
29
|
+
no Developer dispatch, gate execution, or retry budget consumption is allowed.
|
|
30
|
+
|
|
31
|
+
The verifier records `baseline.quality_baseline_fingerprint` when configured and
|
|
32
|
+
records `null` for legacy projects. The Quality Contract is distinct from the
|
|
33
|
+
repository/card baseline and the candidate fingerprint: it says what the run is
|
|
34
|
+
trying to satisfy, not which Git commit was checked out or what the candidate
|
|
35
|
+
changed.
|
|
36
|
+
|
|
20
37
|
Before gates run, the verifier validates the active assignment, PRD/card/gate
|
|
21
38
|
hashes, worktree, expected branch, and candidate fingerprint. It records the
|
|
22
39
|
assignment ID/hash and run ID, executes deterministic commands with a bounded
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "triad-plus",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.8.0",
|
|
4
4
|
"description": "A lightweight, evidence-backed engineering loop for coding agents.",
|
|
5
5
|
"license": "Apache-2.0",
|
|
6
6
|
"type": "module",
|
|
@@ -28,7 +28,7 @@
|
|
|
28
28
|
"node": ">=20"
|
|
29
29
|
},
|
|
30
30
|
"scripts": {
|
|
31
|
-
"test": "node tests/runtime-forward-test.mjs && node tests/retry-scope-contracts-test.mjs && node tests/cli-install-test.mjs && node tests/codex-liveness-test.mjs && node tests/bmad-story-importer-test.mjs",
|
|
31
|
+
"test": "node tests/runtime-forward-test.mjs && node tests/retry-scope-contracts-test.mjs && node tests/cli-install-test.mjs && node tests/codex-liveness-test.mjs && node tests/bmad-story-importer-test.mjs && node tests/immutable-quality-contract-test.mjs",
|
|
32
32
|
"pack:check": "npm pack --dry-run"
|
|
33
33
|
},
|
|
34
34
|
"repository": {
|
|
@@ -0,0 +1,205 @@
|
|
|
1
|
+
import { createHash } from "node:crypto";
|
|
2
|
+
import { readFile, realpath } from "node:fs/promises";
|
|
3
|
+
import path from "node:path";
|
|
4
|
+
|
|
5
|
+
const SHA256 = /^[a-f0-9]{64}$/i;
|
|
6
|
+
const QUALITY_SCOPES = new Set(["product_quality", "delivery_closure"]);
|
|
7
|
+
|
|
8
|
+
function qualityError(code, message) {
|
|
9
|
+
const error = new Error(message);
|
|
10
|
+
error.code = code;
|
|
11
|
+
return error;
|
|
12
|
+
}
|
|
13
|
+
|
|
14
|
+
function objectLike(value) {
|
|
15
|
+
return value !== null && typeof value === "object" && !Array.isArray(value);
|
|
16
|
+
}
|
|
17
|
+
|
|
18
|
+
/**
|
|
19
|
+
* Return a JSON-compatible value with object keys sorted recursively. Arrays
|
|
20
|
+
* intentionally retain their declared order: source and criterion order is
|
|
21
|
+
* part of the authored contract, while object formatting is not.
|
|
22
|
+
*/
|
|
23
|
+
export function canonicalizeQualityValue(value) {
|
|
24
|
+
if (Array.isArray(value)) return value.map(canonicalizeQualityValue);
|
|
25
|
+
if (!objectLike(value)) return value;
|
|
26
|
+
return Object.fromEntries(
|
|
27
|
+
Object.keys(value)
|
|
28
|
+
.sort()
|
|
29
|
+
.map((key) => [key, canonicalizeQualityValue(value[key])])
|
|
30
|
+
);
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
export function qualityBaselinePayload(manifest) {
|
|
34
|
+
if (!objectLike(manifest)) return manifest;
|
|
35
|
+
const { fingerprint: _fingerprint, ...payload } = manifest;
|
|
36
|
+
return canonicalizeQualityValue(payload);
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
export function qualityBaselineFingerprint(manifest) {
|
|
40
|
+
const canonical = JSON.stringify(qualityBaselinePayload(manifest));
|
|
41
|
+
return createHash("sha256").update(canonical).digest("hex");
|
|
42
|
+
}
|
|
43
|
+
|
|
44
|
+
function projectRelativePath(root, value, label) {
|
|
45
|
+
if (typeof value !== "string" || !value.trim() || path.isAbsolute(value)) {
|
|
46
|
+
throw qualityError("quality_baseline_invalid", `${label} must be a non-empty project-relative path`);
|
|
47
|
+
}
|
|
48
|
+
const normalized = value.replaceAll("\\", "/");
|
|
49
|
+
const posix = path.posix.normalize(normalized);
|
|
50
|
+
if (posix === "." || posix === ".." || posix.startsWith("../") || normalized.split("/").includes("..")) {
|
|
51
|
+
throw qualityError("quality_baseline_invalid", `${label} escapes the project root`);
|
|
52
|
+
}
|
|
53
|
+
const resolved = path.resolve(root, normalized);
|
|
54
|
+
if (!resolved.startsWith(`${root}${path.sep}`)) {
|
|
55
|
+
throw qualityError("quality_baseline_invalid", `${label} escapes the project root`);
|
|
56
|
+
}
|
|
57
|
+
return resolved;
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
function validateSha(value, label) {
|
|
61
|
+
if (typeof value !== "string" || !SHA256.test(value)) {
|
|
62
|
+
throw qualityError("quality_baseline_invalid", `${label} must be a SHA-256 hex digest`);
|
|
63
|
+
}
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
function rejectUnknownProperties(value, allowed, label) {
|
|
67
|
+
for (const key of Object.keys(value)) {
|
|
68
|
+
if (!allowed.has(key)) throw qualityError("quality_baseline_invalid", `${label} contains unknown property: ${key}`);
|
|
69
|
+
}
|
|
70
|
+
}
|
|
71
|
+
|
|
72
|
+
/**
|
|
73
|
+
* Validate the manifest shape and its self-declared fingerprint without
|
|
74
|
+
* touching source files. This is useful to callers that only need to inspect
|
|
75
|
+
* the immutable contract before resolving it against a project.
|
|
76
|
+
*/
|
|
77
|
+
export function validateQualityBaselineManifest(manifest) {
|
|
78
|
+
if (!objectLike(manifest)) throw qualityError("quality_baseline_invalid", "quality baseline must be a JSON object");
|
|
79
|
+
rejectUnknownProperties(manifest, new Set(["schema_version", "id", "revision", "sources", "criteria", "fingerprint"]), "quality baseline manifest");
|
|
80
|
+
if (manifest.schema_version !== 1) throw qualityError("quality_baseline_invalid", "quality baseline schema_version must be 1");
|
|
81
|
+
if (typeof manifest.id !== "string" || !manifest.id.trim()) throw qualityError("quality_baseline_invalid", "quality baseline id must be non-empty");
|
|
82
|
+
if (!Number.isInteger(manifest.revision) || manifest.revision < 1) throw qualityError("quality_baseline_invalid", "quality baseline revision must be a positive integer");
|
|
83
|
+
if (!Array.isArray(manifest.sources) || manifest.sources.length === 0) throw qualityError("quality_baseline_invalid", "quality baseline must declare at least one source");
|
|
84
|
+
if (!Array.isArray(manifest.criteria)) throw qualityError("quality_baseline_invalid", "quality baseline criteria must be an array");
|
|
85
|
+
|
|
86
|
+
const sourceIds = new Set();
|
|
87
|
+
for (const source of manifest.sources) {
|
|
88
|
+
if (!objectLike(source)) throw qualityError("quality_baseline_invalid", "quality baseline sources must contain objects");
|
|
89
|
+
rejectUnknownProperties(source, new Set(["id", "role", "path", "sha256"]), "quality baseline source");
|
|
90
|
+
if (typeof source.id !== "string" || !source.id.trim()) throw qualityError("quality_baseline_invalid", "quality baseline source id must be non-empty");
|
|
91
|
+
if (sourceIds.has(source.id)) throw qualityError("quality_baseline_invalid", `duplicate quality baseline source id: ${source.id}`);
|
|
92
|
+
sourceIds.add(source.id);
|
|
93
|
+
if (typeof source.role !== "string" || !source.role.trim()) throw qualityError("quality_baseline_invalid", `quality baseline source role is missing: ${source.id}`);
|
|
94
|
+
if (typeof source.path !== "string" || !source.path.trim() || path.isAbsolute(source.path)) throw qualityError("quality_baseline_invalid", `quality baseline source path must be project-relative: ${source.id}`);
|
|
95
|
+
const normalizedSourcePath = path.posix.normalize(source.path.replaceAll("\\", "/"));
|
|
96
|
+
if (normalizedSourcePath === "." || normalizedSourcePath === ".." || normalizedSourcePath.startsWith("../") || source.path.replaceAll("\\", "/").split("/").includes("..")) throw qualityError("quality_baseline_invalid", `quality baseline source path escapes the project root: ${source.id}`);
|
|
97
|
+
validateSha(source.sha256, `quality baseline source ${source.id}`);
|
|
98
|
+
}
|
|
99
|
+
|
|
100
|
+
const criterionIds = new Set();
|
|
101
|
+
for (const criterion of manifest.criteria) {
|
|
102
|
+
if (!objectLike(criterion)) throw qualityError("quality_baseline_invalid", "quality baseline criteria must contain objects");
|
|
103
|
+
rejectUnknownProperties(criterion, new Set(["id", "scope", "requirement"]), "quality baseline criterion");
|
|
104
|
+
if (typeof criterion.id !== "string" || !criterion.id.trim()) throw qualityError("quality_baseline_invalid", "quality baseline criterion id must be non-empty");
|
|
105
|
+
if (criterionIds.has(criterion.id)) throw qualityError("quality_baseline_invalid", `duplicate quality baseline criterion id: ${criterion.id}`);
|
|
106
|
+
criterionIds.add(criterion.id);
|
|
107
|
+
if (!QUALITY_SCOPES.has(criterion.scope)) throw qualityError("quality_baseline_invalid", `invalid quality baseline criterion scope: ${criterion.id}`);
|
|
108
|
+
if (typeof criterion.requirement !== "string" || !criterion.requirement.trim()) throw qualityError("quality_baseline_invalid", `quality baseline criterion requirement must be non-empty: ${criterion.id}`);
|
|
109
|
+
}
|
|
110
|
+
|
|
111
|
+
validateSha(manifest.fingerprint, "quality baseline fingerprint");
|
|
112
|
+
const calculatedFingerprint = qualityBaselineFingerprint(manifest);
|
|
113
|
+
if (manifest.fingerprint.toLowerCase() !== calculatedFingerprint) {
|
|
114
|
+
throw qualityError("quality_baseline_invalid", "quality baseline declared fingerprint does not match its canonical content");
|
|
115
|
+
}
|
|
116
|
+
return { manifest, fingerprint: calculatedFingerprint };
|
|
117
|
+
}
|
|
118
|
+
|
|
119
|
+
async function verifySources(manifest, root) {
|
|
120
|
+
const verifiedSources = [];
|
|
121
|
+
for (const source of manifest.sources) {
|
|
122
|
+
const sourcePath = projectRelativePath(root, source.path, `quality baseline source ${source.id}`);
|
|
123
|
+
let resolvedSource;
|
|
124
|
+
try {
|
|
125
|
+
resolvedSource = await realpath(sourcePath);
|
|
126
|
+
} catch {
|
|
127
|
+
throw qualityError("quality_baseline_invalid", `quality baseline source is missing: ${source.path}`);
|
|
128
|
+
}
|
|
129
|
+
if (!resolvedSource.startsWith(`${root}${path.sep}`)) {
|
|
130
|
+
throw qualityError("quality_baseline_invalid", `quality baseline source escapes the project root: ${source.path}`);
|
|
131
|
+
}
|
|
132
|
+
let content;
|
|
133
|
+
try {
|
|
134
|
+
content = await readFile(resolvedSource);
|
|
135
|
+
} catch {
|
|
136
|
+
throw qualityError("quality_baseline_invalid", `quality baseline source cannot be read: ${source.path}`);
|
|
137
|
+
}
|
|
138
|
+
const actual = createHash("sha256").update(content).digest("hex");
|
|
139
|
+
if (actual !== source.sha256.toLowerCase()) {
|
|
140
|
+
throw qualityError("quality_baseline_drift", `quality baseline source hash mismatch: ${source.path}`);
|
|
141
|
+
}
|
|
142
|
+
verifiedSources.push({ id: source.id, path: source.path, sha256: actual });
|
|
143
|
+
}
|
|
144
|
+
return verifiedSources;
|
|
145
|
+
}
|
|
146
|
+
|
|
147
|
+
/**
|
|
148
|
+
* Load, validate, fingerprint, and resolve the immutable quality contract
|
|
149
|
+
* against source files under the project root.
|
|
150
|
+
*/
|
|
151
|
+
export async function loadQualityBaseline(manifestPath, { projectRoot = process.cwd(), expectedFingerprint = null } = {}) {
|
|
152
|
+
const root = await realpath(projectRoot);
|
|
153
|
+
const resolvedManifestPath = projectRelativePath(root, manifestPath, "quality baseline path");
|
|
154
|
+
let source;
|
|
155
|
+
try {
|
|
156
|
+
source = await readFile(resolvedManifestPath, "utf8");
|
|
157
|
+
} catch {
|
|
158
|
+
throw qualityError("quality_baseline_invalid", `quality baseline manifest is missing: ${manifestPath}`);
|
|
159
|
+
}
|
|
160
|
+
let manifest;
|
|
161
|
+
try {
|
|
162
|
+
manifest = JSON.parse(source);
|
|
163
|
+
} catch (error) {
|
|
164
|
+
throw qualityError("quality_baseline_invalid", `quality baseline manifest is not valid JSON: ${error.message}`);
|
|
165
|
+
}
|
|
166
|
+
const validated = validateQualityBaselineManifest(manifest);
|
|
167
|
+
if (expectedFingerprint !== null && expectedFingerprint !== undefined) {
|
|
168
|
+
validateSha(expectedFingerprint, "expected quality baseline fingerprint");
|
|
169
|
+
if (expectedFingerprint.toLowerCase() !== validated.fingerprint) {
|
|
170
|
+
throw qualityError("quality_baseline_drift", "quality baseline fingerprint does not match the assignment binding");
|
|
171
|
+
}
|
|
172
|
+
}
|
|
173
|
+
const sources = await verifySources(manifest, root);
|
|
174
|
+
return {
|
|
175
|
+
manifest,
|
|
176
|
+
path: path.relative(root, resolvedManifestPath),
|
|
177
|
+
fingerprint: validated.fingerprint,
|
|
178
|
+
sources,
|
|
179
|
+
criteria: manifest.criteria,
|
|
180
|
+
};
|
|
181
|
+
}
|
|
182
|
+
|
|
183
|
+
/** Resolve the optional assignment binding while preserving legacy mode. */
|
|
184
|
+
export async function resolveQualityContract(assignment, projectRoot) {
|
|
185
|
+
const manifestPath = assignment?.quality_baseline_path;
|
|
186
|
+
const expectedFingerprint = assignment?.expected_quality_baseline_fingerprint;
|
|
187
|
+
const pathPresent = typeof manifestPath === "string" && manifestPath.trim();
|
|
188
|
+
const fingerprintPresent = typeof expectedFingerprint === "string" && expectedFingerprint.trim();
|
|
189
|
+
if (!pathPresent && !fingerprintPresent) return null;
|
|
190
|
+
if (!pathPresent || !fingerprintPresent) {
|
|
191
|
+
throw qualityError("quality_baseline_invalid", "quality contract requires quality_baseline_path and expected_quality_baseline_fingerprint together");
|
|
192
|
+
}
|
|
193
|
+
return loadQualityBaseline(manifestPath, { projectRoot, expectedFingerprint });
|
|
194
|
+
}
|
|
195
|
+
|
|
196
|
+
export function qualityCriteriaByScope(manifestOrContract) {
|
|
197
|
+
const manifest = manifestOrContract?.manifest ?? manifestOrContract;
|
|
198
|
+
if (!objectLike(manifest) || !Array.isArray(manifest.criteria)) throw qualityError("quality_baseline_invalid", "quality baseline criteria are unavailable");
|
|
199
|
+
return {
|
|
200
|
+
product_quality: manifest.criteria.filter((criterion) => criterion.scope === "product_quality"),
|
|
201
|
+
delivery_closure: manifest.criteria.filter((criterion) => criterion.scope === "delivery_closure"),
|
|
202
|
+
};
|
|
203
|
+
}
|
|
204
|
+
|
|
205
|
+
export { QUALITY_SCOPES };
|
|
@@ -0,0 +1,261 @@
|
|
|
1
|
+
import { loadQualityBaseline, qualityCriteriaByScope, validateQualityBaselineManifest } from "./lib/quality-baseline.mjs";
|
|
2
|
+
import { readFile, realpath } from "node:fs/promises";
|
|
3
|
+
import path from "node:path";
|
|
4
|
+
import { fileURLToPath } from "node:url";
|
|
5
|
+
|
|
6
|
+
const VERDICTS = new Set(["PASS", "FAIL", "INDETERMINATE"]);
|
|
7
|
+
const SHA256 = /^[a-f0-9]{64}$/i;
|
|
8
|
+
const DATE_TIME = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?(?:Z|[+-]\d{2}:\d{2})$/;
|
|
9
|
+
|
|
10
|
+
function evaluatorError(code, message) {
|
|
11
|
+
const error = new Error(message);
|
|
12
|
+
error.code = code;
|
|
13
|
+
return error;
|
|
14
|
+
}
|
|
15
|
+
|
|
16
|
+
function rejectUnknownProperties(value, allowed, label) {
|
|
17
|
+
for (const key of Object.keys(value)) {
|
|
18
|
+
if (!allowed.has(key)) throw evaluatorError("evaluator_result_invalid", `${label} contains unknown property: ${key}`);
|
|
19
|
+
}
|
|
20
|
+
}
|
|
21
|
+
|
|
22
|
+
function requireBaseResult(result) {
|
|
23
|
+
if (!result || typeof result !== "object" || Array.isArray(result)) throw evaluatorError("evaluator_result_invalid", "Evaluator+ result must be an object");
|
|
24
|
+
rejectUnknownProperties(result, new Set(["schema_version", "feature_id", "candidate_fingerprint", "quality_baseline_fingerprint", "verdict", "summary", "evidence_refs", "criteria", "created_at"]), "Evaluator+ result");
|
|
25
|
+
if (result.schema_version !== 1) throw evaluatorError("evaluator_result_invalid", "Evaluator+ schema_version must be 1");
|
|
26
|
+
if (typeof result.feature_id !== "string" || !result.feature_id.trim()) throw evaluatorError("evaluator_result_invalid", "Evaluator+ feature_id must be non-empty");
|
|
27
|
+
if (typeof result.candidate_fingerprint !== "string" || !result.candidate_fingerprint.trim()) throw evaluatorError("evaluator_result_invalid", "Evaluator+ candidate_fingerprint must be non-empty");
|
|
28
|
+
if (result.quality_baseline_fingerprint !== undefined && (typeof result.quality_baseline_fingerprint !== "string" || !SHA256.test(result.quality_baseline_fingerprint))) throw evaluatorError("evaluator_result_invalid", "Evaluator+ quality_baseline_fingerprint must be a SHA-256 hex digest");
|
|
29
|
+
if (!VERDICTS.has(result.verdict)) throw evaluatorError("evaluator_result_invalid", "Evaluator+ verdict must be PASS, FAIL, or INDETERMINATE");
|
|
30
|
+
if (typeof result.summary !== "string" || !result.summary.trim()) throw evaluatorError("evaluator_result_invalid", "Evaluator+ summary must be non-empty");
|
|
31
|
+
if (!Array.isArray(result.evidence_refs) || result.evidence_refs.some((ref) => typeof ref !== "string")) throw evaluatorError("evaluator_result_invalid", "Evaluator+ evidence_refs must be an array of strings");
|
|
32
|
+
if (typeof result.created_at !== "string" || !DATE_TIME.test(result.created_at) || Number.isNaN(Date.parse(result.created_at))) throw evaluatorError("evaluator_result_invalid", "Evaluator+ created_at must be an RFC 3339 date-time");
|
|
33
|
+
}
|
|
34
|
+
|
|
35
|
+
export function aggregateEvaluatorVerdict(criteria) {
|
|
36
|
+
if (!Array.isArray(criteria)) throw evaluatorError("evaluator_result_invalid", "Evaluator+ criteria must be an array");
|
|
37
|
+
if (criteria.some((criterion) => criterion.verdict === "FAIL")) return "FAIL";
|
|
38
|
+
if (criteria.some((criterion) => criterion.verdict === "INDETERMINATE")) return "INDETERMINATE";
|
|
39
|
+
return "PASS";
|
|
40
|
+
}
|
|
41
|
+
|
|
42
|
+
function validateCriterionShape(criterion) {
|
|
43
|
+
if (!criterion || typeof criterion !== "object" || Array.isArray(criterion)) throw evaluatorError("evaluator_result_invalid", "Evaluator+ criterion entries must be objects");
|
|
44
|
+
rejectUnknownProperties(criterion, new Set(["id", "scope", "verdict", "summary", "evidence_refs"]), "Evaluator+ criterion");
|
|
45
|
+
if (typeof criterion.id !== "string" || !criterion.id.trim()) throw evaluatorError("evaluator_result_invalid", "Evaluator+ criterion id must be non-empty");
|
|
46
|
+
if (criterion.scope !== "product_quality") throw evaluatorError("evaluator_result_invalid", `Evaluator+ criterion ${criterion.id} is not product_quality`);
|
|
47
|
+
if (!VERDICTS.has(criterion.verdict)) throw evaluatorError("evaluator_result_invalid", `Evaluator+ criterion ${criterion.id} has an invalid verdict`);
|
|
48
|
+
if (typeof criterion.summary !== "string" || !criterion.summary.trim()) throw evaluatorError("evaluator_result_invalid", `Evaluator+ criterion ${criterion.id} summary must be non-empty`);
|
|
49
|
+
if (!Array.isArray(criterion.evidence_refs) || criterion.evidence_refs.some((ref) => typeof ref !== "string")) throw evaluatorError("evaluator_result_invalid", `Evaluator+ criterion ${criterion.id} evidence_refs must be an array of strings`);
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
/**
|
|
53
|
+
* Validate an Evaluator+ result. Without a quality baseline the v1.7 legacy
|
|
54
|
+
* result contract remains valid. With a baseline, every product-quality
|
|
55
|
+
* criterion must be covered exactly once and the aggregate is deterministic.
|
|
56
|
+
*/
|
|
57
|
+
export function validateEvaluatorResult(result, { qualityBaseline = null, expectedCandidateFingerprint = null } = {}) {
|
|
58
|
+
requireBaseResult(result);
|
|
59
|
+
if (!qualityBaseline) {
|
|
60
|
+
if (result.criteria !== undefined) {
|
|
61
|
+
if (!Array.isArray(result.criteria)) throw evaluatorError("evaluator_result_invalid", "Evaluator+ criteria must be an array");
|
|
62
|
+
for (const criterion of result.criteria) validateCriterionShape(criterion);
|
|
63
|
+
}
|
|
64
|
+
return { valid: true, legacy: true, verdict: result.verdict, result };
|
|
65
|
+
}
|
|
66
|
+
|
|
67
|
+
const baseline = qualityBaseline.manifest ?? qualityBaseline;
|
|
68
|
+
const validatedBaseline = validateQualityBaselineManifest(baseline);
|
|
69
|
+
const expectedBaselineFingerprint = validatedBaseline.fingerprint;
|
|
70
|
+
if (qualityBaseline.fingerprint !== undefined && qualityBaseline.fingerprint !== expectedBaselineFingerprint) {
|
|
71
|
+
throw evaluatorError("evaluator_quality_baseline_mismatch", "loaded Quality Baseline fingerprint does not match its canonical content");
|
|
72
|
+
}
|
|
73
|
+
if (result.quality_baseline_fingerprint !== expectedBaselineFingerprint) {
|
|
74
|
+
throw evaluatorError("evaluator_quality_baseline_mismatch", "Evaluator+ result quality baseline fingerprint does not match the approved contract");
|
|
75
|
+
}
|
|
76
|
+
if (typeof expectedCandidateFingerprint !== "string" || !expectedCandidateFingerprint.trim()) {
|
|
77
|
+
throw evaluatorError("evaluator_candidate_binding_missing", "Quality Contract Evaluator+ validation requires an expected candidate fingerprint");
|
|
78
|
+
}
|
|
79
|
+
if (result.candidate_fingerprint !== expectedCandidateFingerprint) {
|
|
80
|
+
throw evaluatorError("evaluator_candidate_fingerprint_mismatch", "Evaluator+ result candidate fingerprint does not match the approved candidate");
|
|
81
|
+
}
|
|
82
|
+
|
|
83
|
+
const { product_quality: expectedCriteria, delivery_closure: deliveryCriteria } = qualityCriteriaByScope(baseline);
|
|
84
|
+
if (!Array.isArray(result.criteria)) throw evaluatorError("evaluator_result_invalid", "Quality Contract Evaluator+ result must include criteria");
|
|
85
|
+
const expectedById = new Map(expectedCriteria.map((criterion) => [criterion.id, criterion]));
|
|
86
|
+
const deliveryIds = new Set(deliveryCriteria.map((criterion) => criterion.id));
|
|
87
|
+
const seen = new Set();
|
|
88
|
+
for (const criterion of result.criteria) {
|
|
89
|
+
validateCriterionShape(criterion);
|
|
90
|
+
if (seen.has(criterion.id)) throw evaluatorError("evaluator_result_invalid", `duplicate Evaluator+ criterion: ${criterion.id}`);
|
|
91
|
+
seen.add(criterion.id);
|
|
92
|
+
if (deliveryIds.has(criterion.id)) throw evaluatorError("evaluator_result_invalid", `delivery_closure criterion was sent to Evaluator+: ${criterion.id}`);
|
|
93
|
+
if (!expectedById.has(criterion.id)) throw evaluatorError("evaluator_result_invalid", `unexpected product_quality criterion: ${criterion.id}`);
|
|
94
|
+
}
|
|
95
|
+
for (const criterion of expectedCriteria) {
|
|
96
|
+
if (!seen.has(criterion.id)) throw evaluatorError("evaluator_result_invalid", `missing product_quality criterion: ${criterion.id}`);
|
|
97
|
+
}
|
|
98
|
+
const aggregate = aggregateEvaluatorVerdict(result.criteria);
|
|
99
|
+
if (result.verdict !== aggregate) throw evaluatorError("evaluator_result_invalid", `Evaluator+ aggregate verdict must be ${aggregate}`);
|
|
100
|
+
return {
|
|
101
|
+
valid: true,
|
|
102
|
+
legacy: false,
|
|
103
|
+
verdict: aggregate,
|
|
104
|
+
quality_baseline_fingerprint: expectedBaselineFingerprint,
|
|
105
|
+
product_quality_criteria: result.criteria,
|
|
106
|
+
excluded_delivery_closure_ids: deliveryCriteria.map((criterion) => criterion.id),
|
|
107
|
+
result,
|
|
108
|
+
};
|
|
109
|
+
}
|
|
110
|
+
|
|
111
|
+
function validateClosureCriterion(criterion, expectedScope = "delivery_closure") {
|
|
112
|
+
if (!criterion || typeof criterion !== "object" || Array.isArray(criterion)) throw evaluatorError("delivery_closure_invalid", "delivery closure criteria must be objects");
|
|
113
|
+
const allowed = new Set(["id", "scope", "verdict", "summary", "evidence_refs"]);
|
|
114
|
+
for (const key of Object.keys(criterion)) if (!allowed.has(key)) throw evaluatorError("delivery_closure_invalid", `delivery closure criterion contains unknown property: ${key}`);
|
|
115
|
+
if (typeof criterion.id !== "string" || !criterion.id.trim()) throw evaluatorError("delivery_closure_invalid", "delivery closure criterion id must be non-empty");
|
|
116
|
+
if (criterion.scope !== expectedScope) throw evaluatorError("delivery_closure_invalid", `delivery closure criterion ${criterion.id} has the wrong scope`);
|
|
117
|
+
if (!VERDICTS.has(criterion.verdict)) throw evaluatorError("delivery_closure_invalid", `delivery closure criterion ${criterion.id} has an invalid verdict`);
|
|
118
|
+
if (typeof criterion.summary !== "string" || !criterion.summary.trim()) throw evaluatorError("delivery_closure_invalid", `delivery closure criterion ${criterion.id} summary must be non-empty`);
|
|
119
|
+
if (!Array.isArray(criterion.evidence_refs) || criterion.evidence_refs.some((ref) => typeof ref !== "string")) throw evaluatorError("delivery_closure_invalid", `delivery closure criterion ${criterion.id} evidence_refs must be an array of strings`);
|
|
120
|
+
}
|
|
121
|
+
|
|
122
|
+
/** Validate the separate delivery-closure criterion record. */
|
|
123
|
+
export function validateDeliveryClosureCriteria(qualityBaseline, results) {
|
|
124
|
+
const baseline = qualityBaseline?.manifest ?? qualityBaseline;
|
|
125
|
+
const validatedBaseline = validateQualityBaselineManifest(baseline);
|
|
126
|
+
const expected = qualityCriteriaByScope(baseline).delivery_closure;
|
|
127
|
+
if (!Array.isArray(results)) throw evaluatorError("delivery_closure_invalid", "delivery closure results must be an array");
|
|
128
|
+
const expectedIds = new Set(expected.map((criterion) => criterion.id));
|
|
129
|
+
const seen = new Set();
|
|
130
|
+
for (const result of results) {
|
|
131
|
+
validateClosureCriterion(result);
|
|
132
|
+
if (seen.has(result.id)) throw evaluatorError("delivery_closure_invalid", `duplicate delivery closure criterion: ${result.id}`);
|
|
133
|
+
seen.add(result.id);
|
|
134
|
+
if (!expectedIds.has(result.id)) throw evaluatorError("delivery_closure_invalid", `unexpected delivery closure criterion: ${result.id}`);
|
|
135
|
+
}
|
|
136
|
+
for (const criterion of expected) {
|
|
137
|
+
if (!seen.has(criterion.id)) throw evaluatorError("delivery_closure_invalid", `missing delivery closure criterion: ${criterion.id}`);
|
|
138
|
+
}
|
|
139
|
+
const verdict = aggregateEvaluatorVerdict(results);
|
|
140
|
+
if (qualityBaseline?.fingerprint !== undefined && qualityBaseline.fingerprint !== validatedBaseline.fingerprint) {
|
|
141
|
+
throw evaluatorError("delivery_closure_invalid", "loaded Quality Baseline fingerprint does not match its canonical content");
|
|
142
|
+
}
|
|
143
|
+
return {
|
|
144
|
+
valid: true,
|
|
145
|
+
verdict,
|
|
146
|
+
can_deliver: verdict === "PASS",
|
|
147
|
+
quality_baseline_fingerprint: validatedBaseline.fingerprint,
|
|
148
|
+
criteria: results,
|
|
149
|
+
};
|
|
150
|
+
}
|
|
151
|
+
|
|
152
|
+
export { VERDICTS };
|
|
153
|
+
|
|
154
|
+
function cliError(code, message) {
|
|
155
|
+
const error = new Error(message);
|
|
156
|
+
error.code = code;
|
|
157
|
+
return error;
|
|
158
|
+
}
|
|
159
|
+
|
|
160
|
+
function parseArgs(argv) {
|
|
161
|
+
const args = { mode: null, project: process.cwd(), baseline: null, result: null, expectedCandidateFingerprint: null };
|
|
162
|
+
for (let index = 0; index < argv.length; index += 1) {
|
|
163
|
+
const flag = argv[index];
|
|
164
|
+
if (flag === "--help" || flag === "-h") {
|
|
165
|
+
process.stdout.write("Usage: node runtime/triad-evaluator-validate.mjs --mode <baseline|evaluator|delivery> --baseline <path> [--result <path>] [--project <root>] [--expected-candidate-fingerprint <sha256>]\n");
|
|
166
|
+
return null;
|
|
167
|
+
}
|
|
168
|
+
const key = {
|
|
169
|
+
"--mode": "mode",
|
|
170
|
+
"--project": "project",
|
|
171
|
+
"--baseline": "baseline",
|
|
172
|
+
"--result": "result",
|
|
173
|
+
"--expected-candidate-fingerprint": "expectedCandidateFingerprint",
|
|
174
|
+
}[flag];
|
|
175
|
+
if (!key) throw cliError("quality_contract_cli_invalid", `unknown option: ${flag}`);
|
|
176
|
+
const value = argv[index + 1];
|
|
177
|
+
if (!value || value.startsWith("--")) throw cliError("quality_contract_cli_invalid", `${flag} requires a value`);
|
|
178
|
+
args[key] = value;
|
|
179
|
+
index += 1;
|
|
180
|
+
}
|
|
181
|
+
if (!args.mode || !["baseline", "evaluator", "delivery"].includes(args.mode)) throw cliError("quality_contract_cli_invalid", "--mode must be baseline, evaluator, or delivery");
|
|
182
|
+
if (args.mode !== "baseline" && !args.result) throw cliError("quality_contract_cli_invalid", "--result is required");
|
|
183
|
+
if ((args.mode === "baseline" || args.mode === "delivery") && !args.baseline) throw cliError("quality_contract_cli_invalid", `--baseline is required for ${args.mode} validation`);
|
|
184
|
+
return args;
|
|
185
|
+
}
|
|
186
|
+
|
|
187
|
+
async function readJsonFile(root, value, label) {
|
|
188
|
+
const target = path.isAbsolute(value) ? value : path.resolve(root, value);
|
|
189
|
+
let source;
|
|
190
|
+
try {
|
|
191
|
+
source = await readFile(target, "utf8");
|
|
192
|
+
} catch (error) {
|
|
193
|
+
throw cliError("quality_contract_cli_invalid", `${label} cannot be read: ${error.message}`);
|
|
194
|
+
}
|
|
195
|
+
try {
|
|
196
|
+
return JSON.parse(source);
|
|
197
|
+
} catch (error) {
|
|
198
|
+
throw cliError("quality_contract_cli_invalid", `${label} is not valid JSON: ${error.message}`);
|
|
199
|
+
}
|
|
200
|
+
}
|
|
201
|
+
|
|
202
|
+
/**
|
|
203
|
+
* Explicit control-plane entry point. It reloads the baseline from disk on
|
|
204
|
+
* every invocation so Evaluator+ and delivery closure cannot rely on a stale
|
|
205
|
+
* manifest/source check performed earlier in the run.
|
|
206
|
+
*/
|
|
207
|
+
async function runCli() {
|
|
208
|
+
const args = parseArgs(process.argv.slice(2));
|
|
209
|
+
if (!args) return;
|
|
210
|
+
const projectRoot = await realpath(args.project);
|
|
211
|
+
const qualityBaseline = args.baseline
|
|
212
|
+
? await loadQualityBaseline(args.baseline, { projectRoot })
|
|
213
|
+
: null;
|
|
214
|
+
if (args.mode === "baseline") {
|
|
215
|
+
process.stdout.write(`${JSON.stringify({
|
|
216
|
+
valid: true,
|
|
217
|
+
mode: "baseline",
|
|
218
|
+
quality_baseline_fingerprint: qualityBaseline.fingerprint,
|
|
219
|
+
sources: qualityBaseline.sources,
|
|
220
|
+
})}\n`);
|
|
221
|
+
return;
|
|
222
|
+
}
|
|
223
|
+
if (args.mode === "evaluator") {
|
|
224
|
+
const result = await readJsonFile(projectRoot, args.result, "Evaluator+ result");
|
|
225
|
+
if (qualityBaseline && (!args.expectedCandidateFingerprint || !SHA256.test(args.expectedCandidateFingerprint))) {
|
|
226
|
+
throw cliError("evaluator_candidate_binding_missing", "Quality Contract Evaluator+ validation requires --expected-candidate-fingerprint");
|
|
227
|
+
}
|
|
228
|
+
const validated = validateEvaluatorResult(result, {
|
|
229
|
+
qualityBaseline,
|
|
230
|
+
expectedCandidateFingerprint: qualityBaseline ? args.expectedCandidateFingerprint : null,
|
|
231
|
+
});
|
|
232
|
+
process.stdout.write(`${JSON.stringify({
|
|
233
|
+
valid: true,
|
|
234
|
+
mode: "evaluator",
|
|
235
|
+
legacy: validated.legacy,
|
|
236
|
+
verdict: validated.verdict,
|
|
237
|
+
quality_baseline_fingerprint: validated.quality_baseline_fingerprint ?? null,
|
|
238
|
+
candidate_fingerprint: result.candidate_fingerprint,
|
|
239
|
+
product_quality_criteria: validated.product_quality_criteria ?? null,
|
|
240
|
+
})}\n`);
|
|
241
|
+
return;
|
|
242
|
+
}
|
|
243
|
+
|
|
244
|
+
const results = await readJsonFile(projectRoot, args.result, "delivery closure results");
|
|
245
|
+
const validated = validateDeliveryClosureCriteria(qualityBaseline, results);
|
|
246
|
+
process.stdout.write(`${JSON.stringify({
|
|
247
|
+
valid: true,
|
|
248
|
+
mode: "delivery",
|
|
249
|
+
verdict: validated.verdict,
|
|
250
|
+
can_deliver: validated.can_deliver,
|
|
251
|
+
quality_baseline_fingerprint: validated.quality_baseline_fingerprint,
|
|
252
|
+
criteria: validated.criteria,
|
|
253
|
+
})}\n`);
|
|
254
|
+
}
|
|
255
|
+
|
|
256
|
+
if (process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
|
|
257
|
+
runCli().catch((error) => {
|
|
258
|
+
process.stdout.write(`${JSON.stringify({ valid: false, error: { code: error.code ?? "quality_contract_cli_invalid", message: error.message } })}\n`);
|
|
259
|
+
process.exitCode = 2;
|
|
260
|
+
});
|
|
261
|
+
}
|
package/runtime/triad-verify.mjs
CHANGED
|
@@ -6,6 +6,7 @@ import { fileURLToPath } from "node:url";
|
|
|
6
6
|
import { writeAtomicJson } from "./lib/evidence.mjs";
|
|
7
7
|
import { calculateCandidateFingerprint, collectCandidateChanges, worktreeBranch } from "./lib/fingerprint.mjs";
|
|
8
8
|
import { executeGates, gateSelectionEvidence, loadTrustedGates, resolveGateSelection } from "./lib/gates.mjs";
|
|
9
|
+
import { resolveQualityContract } from "./lib/quality-baseline.mjs";
|
|
9
10
|
import { evaluateScopeContract, parseScopeContract } from "./lib/scope-contract.mjs";
|
|
10
11
|
|
|
11
12
|
const argv = process.argv.slice(2);
|
|
@@ -24,6 +25,10 @@ function sha256(value) {
|
|
|
24
25
|
return createHash("sha256").update(value).digest("hex");
|
|
25
26
|
}
|
|
26
27
|
|
|
28
|
+
function evidenceFingerprint(value) {
|
|
29
|
+
return typeof value === "string" && /^[a-f0-9]{64}$/i.test(value) ? value : null;
|
|
30
|
+
}
|
|
31
|
+
|
|
27
32
|
async function sha256File(value) {
|
|
28
33
|
return sha256(await readFile(value));
|
|
29
34
|
}
|
|
@@ -65,7 +70,7 @@ async function resolveAssignment(projectRoot, trigger, explicitAssignment) {
|
|
|
65
70
|
return { assignmentPath, assignment: JSON.parse(source), assignmentHash: sha256(source) };
|
|
66
71
|
}
|
|
67
72
|
|
|
68
|
-
async function buildInvalidEvidence({ runId, trigger, assignment, reason, outputPath, failureCode = "verification_context_invalid", gateSelection = null }) {
|
|
73
|
+
async function buildInvalidEvidence({ runId, trigger, assignment, reason, outputPath, failureCode = "verification_context_invalid", gateSelection = null, qualityBaselineFingerprint = null }) {
|
|
69
74
|
const evidence = {
|
|
70
75
|
schema_version: 1,
|
|
71
76
|
run_id: runId,
|
|
@@ -77,6 +82,7 @@ async function buildInvalidEvidence({ runId, trigger, assignment, reason, output
|
|
|
77
82
|
baseline: {
|
|
78
83
|
prd_sha256: assignment?.expected_prd_sha256 ?? null,
|
|
79
84
|
card_sha256: assignment?.expected_card_sha256 ?? null,
|
|
85
|
+
quality_baseline_fingerprint: evidenceFingerprint(qualityBaselineFingerprint ?? assignment?.expected_quality_baseline_fingerprint),
|
|
80
86
|
git_head: null,
|
|
81
87
|
candidate_fingerprint: null,
|
|
82
88
|
},
|
|
@@ -147,6 +153,7 @@ async function main() {
|
|
|
147
153
|
let assignmentPath;
|
|
148
154
|
let outputPath;
|
|
149
155
|
let gateSelection = null;
|
|
156
|
+
let qualityBaseline = null;
|
|
150
157
|
try {
|
|
151
158
|
let assignmentHash;
|
|
152
159
|
({ assignmentPath, assignment, assignmentHash } = await resolveAssignment(projectRoot, trigger, option("--assignment")));
|
|
@@ -166,6 +173,10 @@ async function main() {
|
|
|
166
173
|
const cardPath = path.resolve(projectRoot, assignment.card_path);
|
|
167
174
|
await access(prdPath);
|
|
168
175
|
await access(cardPath);
|
|
176
|
+
// Resolve the optional immutable Quality Contract before any baseline or
|
|
177
|
+
// expensive-gate check so a bound source drift keeps its precise failure
|
|
178
|
+
// classification (including when the source is the PRD itself).
|
|
179
|
+
qualityBaseline = await resolveQualityContract(assignment, projectRoot);
|
|
169
180
|
if ((await sha256File(prdPath)) !== assignment.expected_prd_sha256) throw new Error("PRD baseline hash mismatch");
|
|
170
181
|
if ((await sha256File(cardPath)) !== assignment.expected_card_sha256) throw new Error("feature card hash mismatch");
|
|
171
182
|
const repositorySkills = await validateRepositorySkills(assignment.required_repository_skills, worktree);
|
|
@@ -203,6 +214,7 @@ async function main() {
|
|
|
203
214
|
baseline: {
|
|
204
215
|
prd_sha256: assignment.expected_prd_sha256,
|
|
205
216
|
card_sha256: assignment.expected_card_sha256,
|
|
217
|
+
quality_baseline_fingerprint: qualityBaseline?.fingerprint ?? null,
|
|
206
218
|
gates_sha256: null,
|
|
207
219
|
git_head: before.git_head,
|
|
208
220
|
candidate_fingerprint: before.value,
|
|
@@ -237,9 +249,10 @@ async function main() {
|
|
|
237
249
|
trigger,
|
|
238
250
|
assignment_ref: path.relative(projectRoot, assignmentPath),
|
|
239
251
|
baseline: {
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
252
|
+
prd_sha256: assignment.expected_prd_sha256,
|
|
253
|
+
card_sha256: assignment.expected_card_sha256,
|
|
254
|
+
quality_baseline_fingerprint: qualityBaseline?.fingerprint ?? null,
|
|
255
|
+
gates_sha256: trusted.actualHash,
|
|
243
256
|
git_head: before.git_head,
|
|
244
257
|
candidate_fingerprint: before.value,
|
|
245
258
|
branch,
|
|
@@ -269,7 +282,8 @@ async function main() {
|
|
|
269
282
|
reason: error.message,
|
|
270
283
|
outputPath,
|
|
271
284
|
failureCode: error.code ?? "verification_context_invalid",
|
|
272
|
-
gateSelection
|
|
285
|
+
gateSelection,
|
|
286
|
+
qualityBaselineFingerprint: qualityBaseline?.fingerprint ?? null
|
|
273
287
|
});
|
|
274
288
|
process.stdout.write(`${JSON.stringify({ run_id: runId, status: evidence.status, evidence: outputPath ?? null })}\n`);
|
|
275
289
|
process.exitCode = 3;
|
|
@@ -8,9 +8,25 @@
|
|
|
8
8
|
"schema_version": { "const": 1 },
|
|
9
9
|
"feature_id": { "type": "string", "minLength": 1 },
|
|
10
10
|
"candidate_fingerprint": { "type": "string", "minLength": 1 },
|
|
11
|
+
"quality_baseline_fingerprint": { "type": "string", "pattern": "^[a-fA-F0-9]{64}$" },
|
|
11
12
|
"verdict": { "enum": ["PASS", "FAIL", "INDETERMINATE"] },
|
|
12
13
|
"summary": { "type": "string", "minLength": 1 },
|
|
13
14
|
"evidence_refs": { "type": "array", "items": { "type": "string" } },
|
|
15
|
+
"criteria": {
|
|
16
|
+
"type": "array",
|
|
17
|
+
"items": {
|
|
18
|
+
"type": "object",
|
|
19
|
+
"required": ["id", "scope", "verdict", "summary", "evidence_refs"],
|
|
20
|
+
"properties": {
|
|
21
|
+
"id": { "type": "string", "minLength": 1 },
|
|
22
|
+
"scope": { "const": "product_quality" },
|
|
23
|
+
"verdict": { "enum": ["PASS", "FAIL", "INDETERMINATE"] },
|
|
24
|
+
"summary": { "type": "string", "minLength": 1 },
|
|
25
|
+
"evidence_refs": { "type": "array", "items": { "type": "string" } }
|
|
26
|
+
},
|
|
27
|
+
"additionalProperties": false
|
|
28
|
+
}
|
|
29
|
+
},
|
|
14
30
|
"created_at": { "type": "string", "format": "date-time" }
|
|
15
31
|
},
|
|
16
32
|
"additionalProperties": false
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
3
|
+
"$id": "https://triad.engineering/schemas/quality-baseline.schema.json",
|
|
4
|
+
"title": "Triad immutable Quality Baseline",
|
|
5
|
+
"type": "object",
|
|
6
|
+
"additionalProperties": false,
|
|
7
|
+
"required": ["schema_version", "id", "revision", "sources", "criteria", "fingerprint"],
|
|
8
|
+
"properties": {
|
|
9
|
+
"schema_version": { "const": 1 },
|
|
10
|
+
"id": { "type": "string", "minLength": 1 },
|
|
11
|
+
"revision": { "type": "integer", "minimum": 1 },
|
|
12
|
+
"sources": {
|
|
13
|
+
"type": "array",
|
|
14
|
+
"minItems": 1,
|
|
15
|
+
"items": {
|
|
16
|
+
"type": "object",
|
|
17
|
+
"additionalProperties": false,
|
|
18
|
+
"required": ["id", "role", "path", "sha256"],
|
|
19
|
+
"properties": {
|
|
20
|
+
"id": { "type": "string", "minLength": 1 },
|
|
21
|
+
"role": { "type": "string", "minLength": 1 },
|
|
22
|
+
"path": { "type": "string", "minLength": 1, "pattern": "^(?![\\\\/])(?!.*(?:^|[\\\\/])\\.\\.(?:[\\\\/]|$)).+" },
|
|
23
|
+
"sha256": { "type": "string", "pattern": "^[a-fA-F0-9]{64}$" }
|
|
24
|
+
}
|
|
25
|
+
}
|
|
26
|
+
},
|
|
27
|
+
"criteria": {
|
|
28
|
+
"type": "array",
|
|
29
|
+
"items": {
|
|
30
|
+
"type": "object",
|
|
31
|
+
"additionalProperties": false,
|
|
32
|
+
"required": ["id", "scope", "requirement"],
|
|
33
|
+
"properties": {
|
|
34
|
+
"id": { "type": "string", "minLength": 1 },
|
|
35
|
+
"scope": { "enum": ["product_quality", "delivery_closure"] },
|
|
36
|
+
"requirement": { "type": "string", "minLength": 1 }
|
|
37
|
+
}
|
|
38
|
+
}
|
|
39
|
+
},
|
|
40
|
+
"fingerprint": { "type": "string", "pattern": "^[a-fA-F0-9]{64}$" }
|
|
41
|
+
}
|
|
42
|
+
}
|
|
@@ -12,7 +12,7 @@
|
|
|
12
12
|
"assignment_id": { "type": ["string", "null"] },
|
|
13
13
|
"assignment_sha256": { "type": ["string", "null"] },
|
|
14
14
|
"trigger": { "type": "object", "required": ["event", "agent_id", "agent_type"], "properties": { "event": { "type": "string" }, "agent_id": { "type": ["string", "null"] }, "agent_type": { "type": ["string", "null"] } } },
|
|
15
|
-
"baseline": { "type": "object", "required": ["prd_sha256", "card_sha256", "git_head", "candidate_fingerprint"], "properties": { "prd_sha256": { "type": ["string", "null"] }, "card_sha256": { "type": ["string", "null"] }, "git_head": { "type": ["string", "null"] }, "candidate_fingerprint": { "type": ["string", "null"] }, "branch": { "type": ["string", "null"] } } },
|
|
15
|
+
"baseline": { "type": "object", "required": ["prd_sha256", "card_sha256", "git_head", "candidate_fingerprint"], "properties": { "prd_sha256": { "type": ["string", "null"] }, "card_sha256": { "type": ["string", "null"] }, "quality_baseline_fingerprint": { "type": ["string", "null"], "pattern": "^[a-fA-F0-9]{64}$" }, "git_head": { "type": ["string", "null"] }, "candidate_fingerprint": { "type": ["string", "null"] }, "branch": { "type": ["string", "null"] } } },
|
|
16
16
|
"scope": {
|
|
17
17
|
"type": "object",
|
|
18
18
|
"properties": {
|
|
@@ -20,6 +20,14 @@ conditions, runnable quality gates, practical-test need, and integration need.
|
|
|
20
20
|
snapshot, revision when available, and SHA-256.
|
|
21
21
|
4. Copy `assets/loop-template/` to `.loop/`, then create bounded feature cards
|
|
22
22
|
under `features/` and a complete `feature-plan.md`.
|
|
23
|
+
When an owner-approved Quality Bar or additional intent/architecture source
|
|
24
|
+
exists, copy `assets/loop-template/quality-baseline.json`, fill only the
|
|
25
|
+
declared source hashes and criteria, compute its canonical fingerprint, and
|
|
26
|
+
bind `project.quality_contract.baseline` plus `fingerprint`. Use
|
|
27
|
+
`criteria: []` when no machine-readable Quality Bar was supplied; never
|
|
28
|
+
invent criteria. The manifest is immutable for the run and a requirement
|
|
29
|
+
change creates a new revision/rebaseline event rather than editing it in
|
|
30
|
+
place.
|
|
23
31
|
5. Replace every gate placeholder. Gate executors in v1 are only
|
|
24
32
|
`control-plane`; remove a non-applicable gate with a recorded reason instead
|
|
25
33
|
of declaring manual or MCP execution.
|
|
@@ -22,6 +22,19 @@
|
|
|
22
22
|
|
|
23
23
|
- Evidence/review chain: `<verification/review/evaluator references>`
|
|
24
24
|
|
|
25
|
+
## Quality Contract closure
|
|
26
|
+
|
|
27
|
+
- Quality Baseline: `<path, revision, fingerprint or legacy PRD-only mode>`
|
|
28
|
+
- Product-quality Evaluator+ verdict: `<PASS|FAIL|INDETERMINATE|not configured>`
|
|
29
|
+
|
|
30
|
+
| Criterion | Scope | Verdict | Evidence references |
|
|
31
|
+
| --- | --- | --- | --- |
|
|
32
|
+
| `<delivery criterion ID>` | `delivery_closure` | `<PASS|FAIL|INDETERMINATE>` | `<refs>` |
|
|
33
|
+
|
|
34
|
+
Delivery is `delivered` only when every configured `delivery_closure` criterion
|
|
35
|
+
is recorded as `PASS`. Product-quality criteria are evaluated by the fresh
|
|
36
|
+
Evaluator+ packet and are never silently evaluated during delivery closure.
|
|
37
|
+
|
|
25
38
|
## Local-worktree integration
|
|
26
39
|
|
|
27
40
|
| Card | Consumer worktree | Provider worktree | Branch/commit map | Setup command | Data/event/callback gate |
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
{
|
|
2
|
+
"schema_version": 1,
|
|
3
|
+
"id": "REPLACE_ME_QUALITY_BASELINE_ID",
|
|
4
|
+
"revision": 1,
|
|
5
|
+
"sources": [
|
|
6
|
+
{
|
|
7
|
+
"id": "prd",
|
|
8
|
+
"role": "intent",
|
|
9
|
+
"path": "artifacts/prd.md",
|
|
10
|
+
"sha256": "REPLACE_ME_SHA256"
|
|
11
|
+
}
|
|
12
|
+
],
|
|
13
|
+
"criteria": [],
|
|
14
|
+
"fingerprint": "REPLACE_ME_SHA256"
|
|
15
|
+
}
|
|
@@ -7,9 +7,16 @@ delivery:
|
|
|
7
7
|
handoff: null
|
|
8
8
|
branches: []
|
|
9
9
|
evaluator_report: null
|
|
10
|
+
quality_criteria: [] # delivery_closure results; product_quality stays in Evaluator+
|
|
10
11
|
delivered_at: null
|
|
11
12
|
owner_message: null
|
|
12
13
|
|
|
14
|
+
quality_contract:
|
|
15
|
+
baseline: null
|
|
16
|
+
fingerprint: null
|
|
17
|
+
revision: null
|
|
18
|
+
rebaseline_events: []
|
|
19
|
+
|
|
13
20
|
feature_plan:
|
|
14
21
|
path: feature-plan.md
|
|
15
22
|
status: declared
|
|
@@ -16,6 +16,8 @@
|
|
|
16
16
|
"expected_prd_sha256": "REPLACE_ME_SHA256",
|
|
17
17
|
"expected_card_sha256": "REPLACE_ME_SHA256",
|
|
18
18
|
"expected_gates_sha256": "REPLACE_ME_SHA256",
|
|
19
|
+
"quality_baseline_path": null,
|
|
20
|
+
"expected_quality_baseline_fingerprint": null,
|
|
19
21
|
"required_gate_ids": [],
|
|
20
22
|
"scope_contract": null,
|
|
21
23
|
"required_repository_skills": [
|
|
@@ -12,6 +12,11 @@ project:
|
|
|
12
12
|
snapshot: artifacts/prd.md
|
|
13
13
|
sha256: null
|
|
14
14
|
source_revision: null
|
|
15
|
+
quality_contract:
|
|
16
|
+
# Optional immutable target for v1.8 projects. Leave both values null for
|
|
17
|
+
# legacy PRD-only behavior; bootstrap fills them only from declared sources.
|
|
18
|
+
baseline: null # artifacts/quality-baseline.json
|
|
19
|
+
fingerprint: null
|
|
15
20
|
loop: .loop
|
|
16
21
|
control_plane:
|
|
17
22
|
verification: optional
|
|
@@ -31,13 +31,23 @@ values in the evaluation report. A missing or mismatched binding makes the
|
|
|
31
31
|
evaluation `INDETERMINATE`; it never starts repair or changes the closed Triad
|
|
32
32
|
result.
|
|
33
33
|
|
|
34
|
-
|
|
35
|
-
|
|
34
|
+
When the approved packet declares an immutable Quality Contract, receive its
|
|
35
|
+
fingerprint and only the criteria whose scope is `product_quality`. Do not
|
|
36
|
+
receive or evaluate `delivery_closure` criteria; those belong to the delivery
|
|
37
|
+
gate. Return a report conforming to `schemas/evaluator-plus-result.schema.json`
|
|
38
|
+
and store it separately under `artifacts/evaluator-plus/<evaluation-id>.json`:
|
|
36
39
|
|
|
37
40
|
- `PASS`: the final artifact meets the supplied target;
|
|
38
41
|
- `FAIL`: the target is not met, with direct evidence/references;
|
|
39
42
|
- `INDETERMINATE`: the artifact or target cannot be observed reliably.
|
|
40
43
|
|
|
44
|
+
For a Quality Contract result, include `quality_baseline_fingerprint` and one
|
|
45
|
+
criterion result for every product-quality criterion, exactly once. Use only
|
|
46
|
+
`PASS`, `FAIL`, or `INDETERMINATE` per criterion. The overall `verdict` is
|
|
47
|
+
deterministically checked by `runtime/triad-evaluator-validate.mjs`: any FAIL
|
|
48
|
+
wins, otherwise any INDETERMINATE wins, otherwise PASS. A mismatch or missing
|
|
49
|
+
criterion is invalid evidence, not a product verdict.
|
|
50
|
+
|
|
41
51
|
Include concise rationale, direct evidence references, and confidence. Do not
|
|
42
52
|
edit source, change the Triad queue/state, commit, push, approve delivery, or
|
|
43
53
|
request automatic repair, and never reopen the Triad run. A `FAIL` leaves the
|
|
@@ -52,6 +52,14 @@ presentation for this invocation, do not repeat it.
|
|
|
52
52
|
that exact card baseline for every rework attempt: preserved candidate changes
|
|
53
53
|
are normal during rework. Do not create the first scope-bound assignment from
|
|
54
54
|
unattributable dirty product changes; record `invalid_context` instead.
|
|
55
|
+
If `project.quality_contract` is configured, load and validate its
|
|
56
|
+
project-relative JSON manifest before dispatch. Verify the canonical
|
|
57
|
+
fingerprint and every bound source SHA-256, then bind
|
|
58
|
+
`quality_baseline_path` and `expected_quality_baseline_fingerprint` on the
|
|
59
|
+
assignment. A `quality_baseline_invalid` or `quality_baseline_drift`
|
|
60
|
+
result is `invalid_context`: do not dispatch the Developer, run expensive
|
|
61
|
+
gates, or consume retry/remediation budget. Legacy projects without the
|
|
62
|
+
quality contract retain PRD-only behavior.
|
|
55
63
|
2. Choose one dependency-approved `ready` card, mark it `in_progress`, append an
|
|
56
64
|
attempt, and create an active assignment before delegating. Before each
|
|
57
65
|
delegation, publish an owner-facing activation notice that attributes the
|
|
@@ -80,6 +88,9 @@ presentation for this invocation, do not repeat it.
|
|
|
80
88
|
catalog and fails closed if a stale assignment names a missing or invalid
|
|
81
89
|
gate. It records the selection mode, card IDs, effective IDs, and required
|
|
82
90
|
IDs in the verification evidence.
|
|
91
|
+
When a Quality Contract is bound, `triad-verify` performs that baseline
|
|
92
|
+
preflight before the expensive gates and records its fingerprint in
|
|
93
|
+
`evidence.baseline.quality_baseline_fingerprint`.
|
|
83
94
|
5. A passing verifier result is **environment-derived evidence**. Move only then
|
|
84
95
|
to `in_review`. Missing, stale, failed, timed-out, invalid-context, or
|
|
85
96
|
invalidated evidence never advances the card.
|
|
@@ -196,3 +207,43 @@ or change the already closed Triad run. A per-run `--evaluator` or
|
|
|
196
207
|
`--no-evaluator` request may override the configuration when the host exposes it.
|
|
197
208
|
Before dispatching it, publish the corresponding attributed Evaluator+
|
|
198
209
|
activation notice.
|
|
210
|
+
|
|
211
|
+
When `project.quality_contract` is configured, the approved packet additionally
|
|
212
|
+
binds the immutable Quality Baseline fingerprint and includes only criteria whose
|
|
213
|
+
scope is `product_quality`. Never send `delivery_closure` criteria, queue state,
|
|
214
|
+
handoff state, or attempt history to Evaluator+. Validate the returned report
|
|
215
|
+
with a fresh source-integrity preflight from the control workspace first:
|
|
216
|
+
|
|
217
|
+
```bash
|
|
218
|
+
node .triad-runtime/triad-evaluator-validate.mjs --mode baseline \
|
|
219
|
+
--project /absolute/path/to/control-workspace \
|
|
220
|
+
--baseline artifacts/quality-baseline.json
|
|
221
|
+
```
|
|
222
|
+
|
|
223
|
+
Only after that command returns JSON `valid: true` may Evaluator+ be dispatched.
|
|
224
|
+
Then validate the returned report with this exact command (substituting the real
|
|
225
|
+
paths/fingerprint):
|
|
226
|
+
|
|
227
|
+
```bash
|
|
228
|
+
node .triad-runtime/triad-evaluator-validate.mjs --mode evaluator \
|
|
229
|
+
--project /absolute/path/to/control-workspace \
|
|
230
|
+
--baseline artifacts/quality-baseline.json \
|
|
231
|
+
--result artifacts/evaluator-plus/evaluation.json \
|
|
232
|
+
--expected-candidate-fingerprint <final-candidate-fingerprint>
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
The command reloads the baseline and every bound source immediately before
|
|
236
|
+
dispatch; a non-zero result is a closed validation failure. The control-plane
|
|
237
|
+
aggregate (FAIL over INDETERMINATE over PASS) is authoritative rather than an
|
|
238
|
+
LLM-supplied summary. Immediately before delivery closure, run this second exact
|
|
239
|
+
command with the delivery result array:
|
|
240
|
+
|
|
241
|
+
```bash
|
|
242
|
+
node .triad-runtime/triad-evaluator-validate.mjs --mode delivery \
|
|
243
|
+
--project /absolute/path/to/control-workspace \
|
|
244
|
+
--baseline artifacts/quality-baseline.json \
|
|
245
|
+
--result artifacts/delivery-closure.json
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
Record each delivery criterion and its evidence reference in the run/handoff,
|
|
249
|
+
and declare delivery only when every required delivery criterion is `PASS`.
|