ccqa 1.35.0 → 1.36.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -1
- package/dist/bin/ccqa.mjs +25 -7
- package/dist/package.json +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -206,12 +206,15 @@ end-to-end on its own:
|
|
|
206
206
|
spec, choose deterministic or live mode, record, run to green.
|
|
207
207
|
- **ccqa-rerecord** — bring a flagged or failing test case back to green:
|
|
208
208
|
read the hub's finding, decide what went stale, repair that, re-record.
|
|
209
|
+
- **ccqa-resolve** — clear everything the hub is holding for a person: read
|
|
210
|
+
the verdict, order the rows, route each to a product fix, a re-recording or
|
|
211
|
+
an environment repair.
|
|
209
212
|
|
|
210
213
|
Install them with the [skills CLI](https://github.com/vercel-labs/skills)
|
|
211
214
|
into a consuming project (or `-g` for all projects):
|
|
212
215
|
|
|
213
216
|
```bash
|
|
214
|
-
npx skills add <this-repo> --skill ccqa-record --skill ccqa-rerecord
|
|
217
|
+
npx skills add <this-repo> --skill ccqa-record --skill ccqa-rerecord --skill ccqa-resolve
|
|
215
218
|
```
|
|
216
219
|
|
|
217
220
|
## Documentation
|
package/dist/bin/ccqa.mjs
CHANGED
|
@@ -16142,7 +16142,9 @@ Ask whether the **intent** the step describes still exists in the product:
|
|
|
16142
16142
|
- The intent exists, but the string or selector the spec names is gone or renamed → **TEST_DRIFT**. Cite where the replacement lives.
|
|
16143
16143
|
- The intent itself is gone, or deliberately different → **SPEC_CHANGE**. Cite the source that shows the new shape.
|
|
16144
16144
|
|
|
16145
|
-
A renamed button is TEST_DRIFT. A button that no longer exists because the flow was replaced is SPEC_CHANGE. If the source shows a rename you can point at, prefer TEST_DRIFT
|
|
16145
|
+
A renamed button is TEST_DRIFT. A button that no longer exists because the flow was replaced is SPEC_CHANGE. If the source shows a rename you can point at, prefer TEST_DRIFT.
|
|
16146
|
+
|
|
16147
|
+
SPEC_CHANGE is the more expensive answer — it sends a human to rewrite or retire the spec — so it takes the *stronger* evidence, not the weaker. Failing to find where the intent went is not a finding; that is UNKNOWN. Claim SPEC_CHANGE only when you can point at the source that shows the new shape, or at where the implementation would sit if it still existed.
|
|
16146
16148
|
|
|
16147
16149
|
## Which surface drifted
|
|
16148
16150
|
|
|
@@ -16175,6 +16177,8 @@ sit nearby.
|
|
|
16175
16177
|
|
|
16176
16178
|
- **No drift is a claim, not a default.** Make it after picking the concrete strings from *every* surface you were given — the spec's \`expected\` and the generated code's selectors alike — and finding each of them in the source. Clearing the test case because one surface checked out is the most common way to miss a real finding. If you never looked, the honest answer is UNKNOWN.
|
|
16177
16179
|
- **A finding needs a citation.** Every TEST_DRIFT and SPEC_CHANGE must carry at least one \`evidence\` entry with a real \`file\`, and a line where you can give one. A label with no citation is a guess wearing a verdict's clothes — answer UNKNOWN instead.
|
|
16180
|
+
- **A citation must apply to the case at hand.** Finding the string is not the end of it — read what encloses the line before you cite it. A line inside a guard, behind an early \`return\`, or in a branch this spec's steps never enter says nothing about this spec. Name the conditions that must hold for that line to run, and check the spec puts the product in them. A citation that only proves the line exists is not evidence.
|
|
16181
|
+
- **A comment is not the code.** Comments in the product's source say what someone intended, and they rarely restate the conditions they sit under. A line reading "this is not supported", sitting inside a guarded branch, is true only inside that branch. Cite the control flow you traced, not the sentence you found.
|
|
16178
16182
|
- **Do not report style.** Wording you would have phrased differently is not drift. Report only what would make a replay fail, or what asks about something the product no longer does.
|
|
16179
16183
|
- \`confidence\` is about the label: how sure you are it is the right one, not how bad the finding is.
|
|
16180
16184
|
|
|
@@ -16184,6 +16188,7 @@ sit nearby.
|
|
|
16184
16188
|
2. \`Grep\` the source for them, at the page, component or handler the step is about.
|
|
16185
16189
|
3. For \`include\` steps, confirm the block exists under \`.ccqa/blocks/<name>/spec.yaml\` and that every \`params\` key is declared on it.
|
|
16186
16190
|
4. When a string is missing, look for what replaced it before concluding. Where it went is what decides the label.
|
|
16191
|
+
5. Before citing any line, read the block that encloses it. Which conditions must hold for it to run, and does the spec put the product in those conditions? A line that only runs in a case the spec never enters proves nothing about the spec.
|
|
16187
16192
|
|
|
16188
16193
|
${guidance.userPromptBlock ?? ""}${guidance.customPromptBlock ?? ""}## Output (STRICT)
|
|
16189
16194
|
|
|
@@ -16541,7 +16546,7 @@ function determineExitCode(results, threshold) {
|
|
|
16541
16546
|
//#endregion
|
|
16542
16547
|
//#region src/drift/to-report.ts
|
|
16543
16548
|
/** Tracks the drift prompt's own version — the two must never drift apart. */
|
|
16544
|
-
const DRIFT_REPORT_PROMPT_VERSION = "
|
|
16549
|
+
const DRIFT_REPORT_PROMPT_VERSION = "7";
|
|
16545
16550
|
/**
|
|
16546
16551
|
* Spec-level status under the given threshold, mirroring determineExitCode's
|
|
16547
16552
|
* per-spec logic (exit-code.ts) but scoped to a single SpecResult.
|
|
@@ -19414,9 +19419,11 @@ function computeRerun(input) {
|
|
|
19414
19419
|
const runMoved = specMovedSince(spec.changedAt, coords.lastRun?.deployedSha ?? null, coords.lastRun?.at ?? "", deployTimes);
|
|
19415
19420
|
if (auditMoved && audit.audit !== "due") audit = { audit: "due" };
|
|
19416
19421
|
if (runMoved && execution.execution === "passed") execution = { execution: "stale" };
|
|
19422
|
+
const manualState = readAttestation(attestations.specs[spec.key], spec, coords.lastRed, range, log, deployTimes);
|
|
19423
|
+
const lapsedOverRed = manualState?.kind === "lapsed" && manualState.because !== "newerRed";
|
|
19424
|
+
if (execution.execution === "failed" && lapsedOverRed) execution = { execution: "stale" };
|
|
19417
19425
|
const held = heldBy(locks, spec.key, now);
|
|
19418
19426
|
let verdict = decide(audit.audit, execution.execution, held);
|
|
19419
|
-
const manualState = readAttestation(attestations.specs[spec.key], spec, coords.lastRed, range, log, deployTimes);
|
|
19420
19427
|
if (manualState?.kind === "covers" && !held && verdict !== "verified") verdict = "manuallyVerified";
|
|
19421
19428
|
out[spec.key] = {
|
|
19422
19429
|
verdict,
|
|
@@ -19525,8 +19532,10 @@ function auditState(drift, key, range) {
|
|
|
19525
19532
|
* buckets advance from the same terminal-run trigger, so the run that wrote
|
|
19526
19533
|
* `run` wrote exactly one of `green` or `red`.
|
|
19527
19534
|
*
|
|
19528
|
-
* `failed` outranks `stale
|
|
19529
|
-
* deployed since, and re-running it teaches nothing until someone repairs
|
|
19535
|
+
* `failed` outranks `stale` here: a red result is current information whatever
|
|
19536
|
+
* has deployed since, and re-running it teaches nothing until someone repairs
|
|
19537
|
+
* it. Its caller retires a red a lapsed attestation already answered
|
|
19538
|
+
* (ADR-0020) — this function does not know about attestations.
|
|
19530
19539
|
*/
|
|
19531
19540
|
function executionState(coords, since) {
|
|
19532
19541
|
const { lastRun, lastRed } = coords;
|
|
@@ -24010,6 +24019,13 @@ const CLIENT_JS = `
|
|
|
24010
24019
|
return null;
|
|
24011
24020
|
}
|
|
24012
24021
|
|
|
24022
|
+
// Did the last run end red? Read off the ledger the same way the execution
|
|
24023
|
+
// axis reads it, because a red a lapsed attestation retired ships as stale
|
|
24024
|
+
// and the sites that need the failure itself must still find it (ADR-0020).
|
|
24025
|
+
function lastFailed(rr) {
|
|
24026
|
+
return !!(rr && rr.lastRun && rr.lastRed && rr.lastRed.runId === rr.lastRun.runId);
|
|
24027
|
+
}
|
|
24028
|
+
|
|
24013
24029
|
// The execution axis as one state. "What did it say last time" and "is that
|
|
24014
24030
|
// still true" are two faces of one question, so the row answers it once and
|
|
24015
24031
|
// the detail panel keeps the coordinates.
|
|
@@ -24540,7 +24556,9 @@ const CLIENT_JS = `
|
|
|
24540
24556
|
}
|
|
24541
24557
|
// Only once the audit has actually cleared the spec: while it is still
|
|
24542
24558
|
// due, nobody knows yet whether the environment was the only thing wrong.
|
|
24543
|
-
|
|
24559
|
+
// Read off the ledger, not the axis: an environment red is demoted to
|
|
24560
|
+
// stale so it re-runs (ADR-0020), and the offer has to survive that.
|
|
24561
|
+
if (rr.audit === "clean" && lastFailed(rr) && rr.lastRed.label === "ENVIRONMENT") {
|
|
24544
24562
|
return buildOverrideOffer("environment", feature, spec);
|
|
24545
24563
|
}
|
|
24546
24564
|
return null;
|
|
@@ -24580,7 +24598,7 @@ const CLIENT_JS = `
|
|
|
24580
24598
|
// report loads — and the whole content if it never arrives.
|
|
24581
24599
|
function reasonFindingSource(rr, driftEntry) {
|
|
24582
24600
|
if (auditOpen(rr) && driftEntry && driftEntry.runId && driftEntry.label) return driftEntry;
|
|
24583
|
-
if (rr
|
|
24601
|
+
if (lastFailed(rr) && rr.lastRed.runId && rr.lastRed.label) return rr.lastRed;
|
|
24584
24602
|
return null;
|
|
24585
24603
|
}
|
|
24586
24604
|
|
package/dist/package.json
CHANGED