pi-aia-asf 0.17.4 → 0.17.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +87 -4
- package/lib/activation.mjs +0 -2
- package/lib/extension/commands/session.ts +1 -1
- package/lib/extension/evidence.ts +24 -0
- package/lib/extension/handlers/tool-call.ts +25 -8
- package/lib/extension/handlers/tool-result.ts +7 -3
- package/lib/extension/run.ts +2 -1
- package/lib/extension/runtime.ts +9 -0
- package/lib/gates/blockers.d.mts +2 -3
- package/lib/gates/blockers.mjs +34 -51
- package/lib/gates/ledger.d.mts +3 -14
- package/lib/gates/ledger.mjs +0 -94
- package/lib/gates/refusals.d.mts +12 -0
- package/lib/gates/refusals.mjs +66 -0
- package/lib/gates/suite-runs.d.mts +31 -0
- package/lib/gates/suite-runs.mjs +119 -0
- package/package.json +1 -1
- package/skills/aia-asf/references/06i-completeness.md +1 -1
- package/skills/aia-asf/references/07-release.md +8 -0
package/CHANGELOG.md
CHANGED
|
@@ -7,13 +7,96 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
-
## [0.17.
|
|
10
|
+
## [0.17.6] - 2026-11-09
|
|
11
11
|
|
|
12
|
-
###
|
|
12
|
+
### Fixed — a gate that crashes is not a gate that passed
|
|
13
|
+
|
|
14
|
+
Both tool handlers were fenced with `catch { return undefined }`, which had two consequences of
|
|
15
|
+
exactly the kind ASF bans:
|
|
16
|
+
|
|
17
|
+
- A failure inside the **suite recorder** lost the run silently, after which the publish and
|
|
18
|
+
completion floors demanded a run that could never register — an unclosable gap with no visible
|
|
19
|
+
cause.
|
|
20
|
+
- Because pi's own semantics for a crashing `tool_call` handler are *block*, returning "allow"
|
|
21
|
+
overrode them: anything that made the earlier code throw — a config read, the activation
|
|
22
|
+
observer — disabled the **irreversibility floor** for that call.
|
|
23
|
+
|
|
24
|
+
Now:
|
|
25
|
+
|
|
26
|
+
- `reportGateError(gate, e)` records a `gate-error` ledger entry and tells the agent once, as a
|
|
27
|
+
steer: *"That is not a pass — the check failed to execute."* Guidance, never a stop.
|
|
28
|
+
- When the gate dies **before** the blockers ran, `failSafeBlock(input)` re-runs just the rules
|
|
29
|
+
whose `failSafe` is `block`, from the raw tool input alone, with no context to fail on. A secret
|
|
30
|
+
in a tracked file is still refused when the gate itself is broken; ordinary work is never
|
|
31
|
+
stopped by an internal error.
|
|
32
|
+
- A fail-safe rule that *itself* throws refuses ("could not run") rather than reporting nothing
|
|
33
|
+
found.
|
|
13
34
|
|
|
14
|
-
-
|
|
35
|
+
Three assertions pin this, each mutation-tested: removing the report, the fail-closed call, or the
|
|
36
|
+
crash-refusal each fails a test.
|
|
15
37
|
|
|
16
|
-
|
|
38
|
+
- `lib/gates/blockers.d.mts` still declared `RULE_TEXT`, `refusalStage` and `__resetRefusals` after
|
|
39
|
+
they moved to `lib/gates/refusals.mjs` in 0.17.5 — a `.ts` could import them, typecheck, and get
|
|
40
|
+
`undefined` at runtime. The declarations now match the modules.
|
|
41
|
+
|
|
42
|
+
### Removed
|
|
43
|
+
|
|
44
|
+
- Dead export `SOURCE_EXTENSIONS` from `lib/activation.mjs` (no importer).
|
|
45
|
+
|
|
46
|
+
### Documented
|
|
47
|
+
|
|
48
|
+
- **The loaded copy is not the shipped copy** (07-release.md step 7 + hygiene rule): `pi` loads
|
|
49
|
+
extensions once at process start, so a session open across a release keeps enforcing the previous
|
|
50
|
+
version and quoting text that no longer exists in the source. Confirm which build is loaded — a
|
|
51
|
+
string unique to the new release, and the process start time — before concluding a released gate
|
|
52
|
+
is broken. That is the difference between "the fix did not work" and "the fix is not running", and
|
|
53
|
+
it cost hours chasing a phantom bug in 0.17.5.
|
|
54
|
+
- `06i-completeness.md` pointed its drift example at `lib/gates/ledger.mjs` / `deriveSuiteRun()`;
|
|
55
|
+
those moved to `lib/gates/suite-runs.mjs` in 0.17.5, and the doc — which is about not trusting
|
|
56
|
+
stale references — was itself stale.
|
|
57
|
+
|
|
58
|
+
## [0.17.5] - 2026-11-08
|
|
59
|
+
|
|
60
|
+
### Fixed — the publish floor could be satisfied by a green that proved nothing
|
|
61
|
+
|
|
62
|
+
`push-without-recorded-green` asked for "a recorded full-suite green run in this
|
|
63
|
+
session" and accepted **any** green: a run from before the last edit, or a green
|
|
64
|
+
from earlier in the session that a red had since overtaken. It also disagreed
|
|
65
|
+
with the completion floor, which already required a passing run *started after
|
|
66
|
+
the last source change* — so the two ASF floors had different ideas about what
|
|
67
|
+
"green" means, and the weaker one guarded the only irreversible action.
|
|
68
|
+
|
|
69
|
+
The floor is now the same rule in both places: the **last** full run, green, and
|
|
70
|
+
started after the last source mutation. Passive mode keeps it — "no steering"
|
|
71
|
+
suspends guidance, not the irreversibility floor. The refusal now names which
|
|
72
|
+
condition failed (`no full-suite run recorded` / `the last full suite run exited
|
|
73
|
+
1` / `the last full-suite green predates the most recent source change`), so the
|
|
74
|
+
autonomous exit is the specific action that closes it.
|
|
75
|
+
|
|
76
|
+
New assertions drive the real `tool_call`/`tool_result` pair rather than a
|
|
77
|
+
hand-built context, which is what pins the *recorder*: a floor whose demanded
|
|
78
|
+
action does not actually close it is the 2026-11-07 bug class, and the previous
|
|
79
|
+
tests could not see that path at all. Six mutations of the new guards (accept a
|
|
80
|
+
red, accept a stale green, take the oldest full run, drop the mutation clock,
|
|
81
|
+
never record the run, never set the clock) each fail at least one test.
|
|
82
|
+
|
|
83
|
+
### Changed — two modules split to stay inside the limit ASF enforces
|
|
84
|
+
|
|
85
|
+
`lib/gates/ledger.mjs` (327 lines) and `lib/gates/blockers.mjs` (301) crossed the
|
|
86
|
+
300-line file limit, so the two halves that changed for different reasons were
|
|
87
|
+
separated: `lib/gates/suite-runs.mjs` (suite-run recognition and the green-run
|
|
88
|
+
rule) and `lib/gates/refusals.mjs` (the refusal ladder and its per-rule text).
|
|
89
|
+
Both are imported explicitly — no re-export shim with two ways to reach one
|
|
90
|
+
function.
|
|
91
|
+
|
|
92
|
+
### Removed — `deriveHealthRun`
|
|
93
|
+
|
|
94
|
+
It recognised a health run by matching `/asf health` in a command's text, which
|
|
95
|
+
is the class of bug that produced the endless loop of 2026-11-07: a human slash
|
|
96
|
+
command the agent has no tool for. The `asf_health` tool appends the `health-run`
|
|
97
|
+
ledger entry directly, so nothing called it and nothing tested it.
|
|
98
|
+
|
|
99
|
+
## [0.17.4] - 2026-11-07
|
|
17
100
|
|
|
18
101
|
### Changed — ASF now passes its own health gate, and the gate measures the right tree
|
|
19
102
|
|
package/lib/activation.mjs
CHANGED
|
@@ -9,7 +9,7 @@ import type { AsfPhase, AsfScale, AsfState } from "../types.ts";
|
|
|
9
9
|
import { projectName, runRoot } from "../paths.ts";
|
|
10
10
|
import { loadState, saveState } from "../state.ts";
|
|
11
11
|
import { newRunId } from "../run.ts";
|
|
12
|
-
import { __resetRefusals } from "../../gates/
|
|
12
|
+
import { __resetRefusals } from "../../gates/refusals.mjs";
|
|
13
13
|
|
|
14
14
|
export interface PhaseChange {
|
|
15
15
|
phase: AsfPhase;
|
|
@@ -27,6 +27,30 @@ export function queueSteer(text: string) {
|
|
|
27
27
|
pendingSteers.push(text);
|
|
28
28
|
}
|
|
29
29
|
|
|
30
|
+
/**
|
|
31
|
+
* An internal gate error must never be able to look like a pass.
|
|
32
|
+
*
|
|
33
|
+
* Every ASF handler is fenced, because an observer must not break the work it
|
|
34
|
+
* watches. Fencing *without* reporting is exactly how the 2026-11-07 loop became
|
|
35
|
+
* possible: a gate demanded something, the code that would have shown why
|
|
36
|
+
* swallowed its own error, and the agent was steered forever toward an action
|
|
37
|
+
* that could not register. So a swallowed error is (a) recorded as evidence and
|
|
38
|
+
* (b) told to the agent once, as a steer — guidance, never a stop.
|
|
39
|
+
*
|
|
40
|
+
* The settle gate has a stronger channel (it returns the entry inline, so the
|
|
41
|
+
* agent sees it in the same turn); the tool handlers do not, so they use this.
|
|
42
|
+
*/
|
|
43
|
+
export function reportGateError(gate: string, e: unknown) {
|
|
44
|
+
const message = String((e as Error | null)?.message || e || "unknown");
|
|
45
|
+
appendEvidence({ kind: "gate-error", gate, error: message.slice(0, 300) });
|
|
46
|
+
const cls = (e as { code?: string } | null)?.code || (e as Error | null)?.name || "unknown";
|
|
47
|
+
queueSteer(
|
|
48
|
+
`ASF: the ${gate} hit an internal error (${cls}: ${message.slice(0, 140)}) and did not run. ` +
|
|
49
|
+
`That is not a pass — the check failed to execute. Re-issue the action; if it repeats, ` +
|
|
50
|
+
`this is an ASF bug to report, not a rule to satisfy.`,
|
|
51
|
+
);
|
|
52
|
+
}
|
|
53
|
+
|
|
30
54
|
|
|
31
55
|
export function appendEvidence(entry: Record<string, unknown>) {
|
|
32
56
|
const res = ledger.append(entry);
|
|
@@ -15,13 +15,13 @@
|
|
|
15
15
|
|
|
16
16
|
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
17
17
|
import { classifyToolCall, activationDecision } from "../../activation.mjs";
|
|
18
|
-
import { evaluateBlockers, BLOCKERS } from "../../gates/blockers.mjs";
|
|
19
|
-
import { deriveSuiteRun } from "../../gates/
|
|
18
|
+
import { evaluateBlockers, BLOCKERS, failSafeBlock } from "../../gates/blockers.mjs";
|
|
19
|
+
import { deriveSuiteRun } from "../../gates/suite-runs.mjs";
|
|
20
20
|
import { readProjectConfig } from "../../gates/config.mjs";
|
|
21
21
|
import { projectName, runRoot } from "../paths.ts";
|
|
22
22
|
import { loadState, saveState } from "../state.ts";
|
|
23
23
|
import { openAutoRun, gitDirty, numberConfig } from "../run.ts";
|
|
24
|
-
import { appendEvidence, queueSteer } from "../evidence.ts";
|
|
24
|
+
import { appendEvidence, queueSteer, reportGateError } from "../evidence.ts";
|
|
25
25
|
import { gateStats, pendingSuiteRuns, suiteRuns, recordVerdict } from "../gate-state.ts";
|
|
26
26
|
import { authority } from "../gate-state.ts";
|
|
27
27
|
import { runtime, BLIND_EDIT_NUDGE_AFTER } from "../runtime.ts";
|
|
@@ -47,6 +47,11 @@ function nudgeBlindEdits(file: Awaited<ReturnType<typeof loadState>>, path: stri
|
|
|
47
47
|
async function observeActivation(toolName: string, input: Input): Promise<void> {
|
|
48
48
|
const call = classifyToolCall(toolName, input);
|
|
49
49
|
if (!call.mutates || !call.source) return;
|
|
50
|
+
// The mutation clock the publish floor compares the suite runs against. Set for
|
|
51
|
+
// EVERY source mutation, a passive run included: "no steering" is not "no floor"
|
|
52
|
+
// — the irreversibility floor holds in passive mode, so it must not be able to
|
|
53
|
+
// publish on a green that predates the edit.
|
|
54
|
+
runtime.lastMutationAt = Date.now();
|
|
50
55
|
const project = projectName();
|
|
51
56
|
const file = await loadState(project);
|
|
52
57
|
const decision = activationDecision({ call, activeRun: file.current, passiveRunId: file.current?.passiveRunId });
|
|
@@ -207,7 +212,15 @@ export function registerToolCallHandler(pi: ExtensionAPI): void {
|
|
|
207
212
|
|
|
208
213
|
const config = readProjectConfig(runRoot());
|
|
209
214
|
const res = evaluateBlockers(toolName, input, {
|
|
210
|
-
ctx: {
|
|
215
|
+
ctx: {
|
|
216
|
+
suiteRuns,
|
|
217
|
+
dirty: () => gitDirty(),
|
|
218
|
+
impactStated: runtime.impactStated,
|
|
219
|
+
trackedPaths: undefined,
|
|
220
|
+
// The publish floor is the settle floor: the LAST full run, green, and
|
|
221
|
+
// started after the last source change — not "any green this session".
|
|
222
|
+
lastMutationAt: runtime.lastMutationAt,
|
|
223
|
+
},
|
|
211
224
|
enabled: blockerSwitches(config),
|
|
212
225
|
});
|
|
213
226
|
emitVerdicts(res);
|
|
@@ -217,10 +230,14 @@ export function registerToolCallHandler(pi: ExtensionAPI): void {
|
|
|
217
230
|
if (park) await parkRun(park);
|
|
218
231
|
if (!res.block) return undefined;
|
|
219
232
|
return await handleBlock({ pi, res: res as Refusal["res"], park, config });
|
|
220
|
-
} catch {
|
|
221
|
-
// An observer must never break the thing it observes
|
|
222
|
-
// the
|
|
223
|
-
|
|
233
|
+
} catch (e) {
|
|
234
|
+
// An observer must never break the thing it observes — so this does not
|
|
235
|
+
// stop the work. But it must not look like a clean pass either, and it must
|
|
236
|
+
// not hand anyone a way past the irreversibility floor: an error raised
|
|
237
|
+
// before the blockers ran is re-checked against the rules that guard
|
|
238
|
+
// irreversible actions, from the raw tool input alone.
|
|
239
|
+
reportGateError("tool_call gate", e);
|
|
240
|
+
return failSafeBlock((event as { input?: Input } | null)?.input || {});
|
|
224
241
|
}
|
|
225
242
|
});
|
|
226
243
|
}
|
|
@@ -8,8 +8,8 @@
|
|
|
8
8
|
*/
|
|
9
9
|
|
|
10
10
|
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
11
|
-
import { deriveSuiteRun } from "../../gates/
|
|
12
|
-
import { appendEvidence } from "../evidence.ts";
|
|
11
|
+
import { deriveSuiteRun } from "../../gates/suite-runs.mjs";
|
|
12
|
+
import { appendEvidence, reportGateError } from "../evidence.ts";
|
|
13
13
|
import { pendingSuiteRuns, suiteRuns } from "../gate-state.ts";
|
|
14
14
|
import { runtime } from "../runtime.ts";
|
|
15
15
|
|
|
@@ -52,7 +52,11 @@ export function registerToolResultHandler(pi: ExtensionAPI): void {
|
|
|
52
52
|
pendingSuiteRuns.delete(id);
|
|
53
53
|
completeSuiteRun(ev, pending);
|
|
54
54
|
return undefined;
|
|
55
|
-
} catch {
|
|
55
|
+
} catch (e) {
|
|
56
|
+
// Swallowing this one silently is how the publish floor becomes an
|
|
57
|
+
// unclosable gap: the run happened, the recorder failed, and the gate then
|
|
58
|
+
// demands a run that can never register.
|
|
59
|
+
reportGateError("suite recorder", e);
|
|
56
60
|
return undefined;
|
|
57
61
|
}
|
|
58
62
|
});
|
package/lib/extension/run.ts
CHANGED
|
@@ -14,7 +14,7 @@ import { appendEvidence } from "./evidence.ts";
|
|
|
14
14
|
import { loadState, saveState } from "./state.ts";
|
|
15
15
|
import { projectName } from "./paths.ts";
|
|
16
16
|
import { runtime } from "./runtime.ts";
|
|
17
|
-
import { __resetRefusals } from "../gates/
|
|
17
|
+
import { __resetRefusals } from "../gates/refusals.mjs";
|
|
18
18
|
import type { AsfState } from "./types.ts";
|
|
19
19
|
|
|
20
20
|
export function newRunId(): string {
|
|
@@ -74,6 +74,7 @@ export async function openAutoRun(trigger: string, path: string | null): Promise
|
|
|
74
74
|
__resetRefusals();
|
|
75
75
|
runtime.blindMutations = 0;
|
|
76
76
|
runtime.blindNudged = false;
|
|
77
|
+
runtime.lastMutationAt = Date.now();
|
|
77
78
|
appendEvidence({ kind: "run-open", runId: state.runId, auto: true, trigger, project });
|
|
78
79
|
runtime.pendingCard = { text: cardFor(state), reason: "activation" };
|
|
79
80
|
return state;
|
package/lib/extension/runtime.ts
CHANGED
|
@@ -28,6 +28,14 @@ export interface RuntimeState {
|
|
|
28
28
|
/** P6: source mutations since the last test run in this run. */
|
|
29
29
|
blindMutations: number;
|
|
30
30
|
blindNudged: boolean;
|
|
31
|
+
/**
|
|
32
|
+
* In-process mirror of the active run's `lastMutationAt`, kept so an exit
|
|
33
|
+
* floor can ask "did a test run start after the last edit?" without a file
|
|
34
|
+
* read on every tool call. Session-scoped on purpose, exactly like the suite
|
|
35
|
+
* runs it is compared against: after a restart there are no recorded runs and
|
|
36
|
+
* no mutations, so the two clocks can only disagree about the past.
|
|
37
|
+
*/
|
|
38
|
+
lastMutationAt: number;
|
|
31
39
|
passiveHeartbeatWritten: boolean;
|
|
32
40
|
lastScaleWasSmall: boolean;
|
|
33
41
|
/** Re-anchoring is count-limited per run. */
|
|
@@ -44,6 +52,7 @@ export const runtime: RuntimeState = {
|
|
|
44
52
|
lastReanchorAt: 0,
|
|
45
53
|
blindMutations: 0,
|
|
46
54
|
blindNudged: false,
|
|
55
|
+
lastMutationAt: 0,
|
|
47
56
|
passiveHeartbeatWritten: false,
|
|
48
57
|
lastScaleWasSmall: false,
|
|
49
58
|
reanchorsThisRun: 0,
|
package/lib/gates/blockers.d.mts
CHANGED
|
@@ -34,9 +34,8 @@ export function evaluateBlockers(
|
|
|
34
34
|
verdicts: BlockerVerdict[];
|
|
35
35
|
steers?: Array<{ gate: string; reason: string }>;
|
|
36
36
|
};
|
|
37
|
-
|
|
38
|
-
export function
|
|
39
|
-
export function __resetRefusals(): void;
|
|
37
|
+
/** Re-run the fail-safe-block rules from the raw input alone, when the gate itself could not run. */
|
|
38
|
+
export function failSafeBlock(input: Record<string, unknown>): { block: true; reason: string } | undefined;
|
|
40
39
|
export function blockSecretWrite(input: Record<string, unknown>, ctx?: Record<string, unknown>): BlockerVerdict;
|
|
41
40
|
export function blockTestSubset(input: Record<string, unknown>, ctx?: Record<string, unknown>): BlockerVerdict;
|
|
42
41
|
export function blockPushWithoutGreen(input: Record<string, unknown>, ctx?: Record<string, unknown>): BlockerVerdict;
|
package/lib/gates/blockers.mjs
CHANGED
|
@@ -20,6 +20,11 @@
|
|
|
20
20
|
* rule and wrong for the convenience rules, so `failSafe` is explicit below.
|
|
21
21
|
*/
|
|
22
22
|
|
|
23
|
+
// The green-run rule lives with the ledger that records the runs, so the exit
|
|
24
|
+
// floors and the recorder cannot drift apart.
|
|
25
|
+
import { greenFullRun } from "./suite-runs.mjs";
|
|
26
|
+
import { refusalStage, RULE_TEXT } from "./refusals.mjs";
|
|
27
|
+
|
|
23
28
|
// ─── (a) secret written to a tracked path ──────────────────────────────────
|
|
24
29
|
//
|
|
25
30
|
// No self-service bypass: the autonomous exit (do not write the secret) costs
|
|
@@ -100,15 +105,17 @@ export function blockPushWithoutGreen(input, ctx = {}) {
|
|
|
100
105
|
if (!command || !PUSH_CMD.test(command)) return { fired: false, scanned: 0 };
|
|
101
106
|
if (ctx.dryRun || /--dry-run\b/.test(command)) return { fired: false, scanned: 1, detail: "dry run" };
|
|
102
107
|
|
|
103
|
-
|
|
104
|
-
|
|
108
|
+
// The LAST full run, green, and started after the last source change. A green
|
|
109
|
+
// that any earlier run left behind is not evidence about the code being pushed.
|
|
110
|
+
const green = greenFullRun(ctx.suiteRuns, ctx.lastMutationAt);
|
|
111
|
+
if (green.ok) return { fired: false, scanned: 1, detail: "last full suite run green, after the last edit" };
|
|
105
112
|
|
|
106
113
|
return {
|
|
107
114
|
fired: true,
|
|
108
115
|
scanned: 1,
|
|
109
116
|
rule: "push-without-recorded-green",
|
|
110
117
|
reason:
|
|
111
|
-
`refusing to push/publish:
|
|
118
|
+
`refusing to push/publish: ${green.why}. ` +
|
|
112
119
|
`Autonomous exit: run the full suite, and once it exits 0 this is unblocked. ` +
|
|
113
120
|
`The gate never moves — the suite does.`,
|
|
114
121
|
};
|
|
@@ -134,54 +141,6 @@ export function blockDestructiveGitDirty(input, ctx = {}) {
|
|
|
134
141
|
};
|
|
135
142
|
}
|
|
136
143
|
|
|
137
|
-
// ─── The refusal ladder (P3) ───────────────────────────────────────────────
|
|
138
|
-
//
|
|
139
|
-
// 53% of the refusals the agent ever received (9 of 17) were it walking into the
|
|
140
|
-
// same wall again — `secret-in-tracked-file` fired 3 of its 4 times as a repeat,
|
|
141
|
-
// all on test files with realistic-looking placeholder keys. A remedy in a
|
|
142
|
-
// message that is re-issued identically is not made to stick, so the ladder
|
|
143
|
-
// changes what the agent is *told* before it ever changes what it is *asked*:
|
|
144
|
-
//
|
|
145
|
-
// 1st fire → refuse + autonomous exit (the text already carried this)
|
|
146
|
-
// 2nd same → refuse + the rule text itself, so the why and the sanctioned
|
|
147
|
-
// alternative reach the context instead of just the veto
|
|
148
|
-
// 3rd same → park for the PO: insisting three times on an irreversible action
|
|
149
|
-
// with no safe path is exactly the park case, not a nag
|
|
150
|
-
|
|
151
|
-
/** Normative text per refusal, injected on the second identical fire. */
|
|
152
|
-
export const RULE_TEXT = {
|
|
153
|
-
"secret-in-tracked-file":
|
|
154
|
-
"Secrets never go into tracked files, including tests. The sanctioned form is " +
|
|
155
|
-
"a fixture constant that cannot be a real key (sk-test-not-a-real-key), an " +
|
|
156
|
-
"environment variable name read at runtime, or a git-ignored .env. A key " +
|
|
157
|
-
"shape that a scanner can match is a key shape that a scanner will match.",
|
|
158
|
-
"destructive-git-dirty-tree":
|
|
159
|
-
"A destructive git command against a dirty tree discards work that exists " +
|
|
160
|
-
"nowhere else. Commit it, or stash it with a message, then re-run the " +
|
|
161
|
-
"destructive step. The gate is not protecting the tree from you; it is " +
|
|
162
|
-
"protecting the only copy of the change.",
|
|
163
|
-
"push-without-recorded-green":
|
|
164
|
-
"Nothing is published or pushed without a recorded full-suite green run in " +
|
|
165
|
-
"this session. The gate never moves — the suite does. Run the full suite; " +
|
|
166
|
-
"when it exits 0 this is unblocked, and the published artifact is then " +
|
|
167
|
-
"verified from the registry rather than assumed.",
|
|
168
|
-
};
|
|
169
|
-
|
|
170
|
-
const refusals = new Map();
|
|
171
|
-
|
|
172
|
-
/** Which fire of this identical refusal is this? 1 = first time. */
|
|
173
|
-
export function refusalStage(gate, fingerprint) {
|
|
174
|
-
const key = `${gate}|${String(fingerprint || "").slice(0, 200)}`;
|
|
175
|
-
const n = (refusals.get(key) || 0) + 1;
|
|
176
|
-
refusals.set(key, n);
|
|
177
|
-
return n;
|
|
178
|
-
}
|
|
179
|
-
|
|
180
|
-
/** Test hook, and the same reset a new run performs. */
|
|
181
|
-
export function __resetRefusals() {
|
|
182
|
-
refusals.clear();
|
|
183
|
-
}
|
|
184
|
-
|
|
185
144
|
// ─── Registry ──────────────────────────────────────────────────────────────
|
|
186
145
|
|
|
187
146
|
/**
|
|
@@ -201,6 +160,30 @@ export const BLOCKERS = [
|
|
|
201
160
|
{ name: "destructive-git-dirty-tree", run: blockDestructiveGitDirty, failSafe: "allow", switch: "gate.block.destructiveGit", criticality: "medium", bypassable: true, mode: "block" },
|
|
202
161
|
];
|
|
203
162
|
|
|
163
|
+
/**
|
|
164
|
+
* The floor that must not fail open.
|
|
165
|
+
*
|
|
166
|
+
* A fenced handler that returns "allow" on its own internal error hands anyone
|
|
167
|
+
* who can make the *other* code throw a way past the irreversibility floor — the
|
|
168
|
+
* secret rule would be disabled by a bug in a config read. So when the gate
|
|
169
|
+
* could not run, the rules whose `failSafe` is `block` are re-run from the raw
|
|
170
|
+
* tool input alone, with no context to fail on; and if even one of those throws,
|
|
171
|
+
* that is a refusal, not a pass.
|
|
172
|
+
*/
|
|
173
|
+
export function failSafeBlock(input) {
|
|
174
|
+
for (const b of BLOCKERS) {
|
|
175
|
+
if (b.failSafe !== "block") continue;
|
|
176
|
+
let res;
|
|
177
|
+
try {
|
|
178
|
+
res = b.run(input, {});
|
|
179
|
+
} catch {
|
|
180
|
+
res = { fired: true, reason: `${b.name} could not run, and it guards an irreversible action.` };
|
|
181
|
+
}
|
|
182
|
+
if (res?.fired) return { block: true, reason: `${b.name}: ${res.reason || "refused"}` };
|
|
183
|
+
}
|
|
184
|
+
return undefined;
|
|
185
|
+
}
|
|
186
|
+
|
|
204
187
|
function str(input, keys) {
|
|
205
188
|
if (!input || typeof input !== "object") return null;
|
|
206
189
|
for (const k of keys) {
|
package/lib/gates/ledger.d.mts
CHANGED
|
@@ -3,20 +3,15 @@
|
|
|
3
3
|
* JavaScript on purpose; this file is the boundary index.ts may rely on, so a
|
|
4
4
|
* field that is not declared here is a type error at the call site rather than
|
|
5
5
|
* a silent `any`.
|
|
6
|
+
*
|
|
7
|
+
* Suite-run recognition lives in `suite-runs.d.mjs` (split out in 0.17.5): the
|
|
8
|
+
* ledger stores entries, the observer decides what a tool call proved.
|
|
6
9
|
*/
|
|
7
10
|
export interface LedgerEntry {
|
|
8
11
|
kind: string;
|
|
9
12
|
at?: string;
|
|
10
13
|
[field: string]: unknown;
|
|
11
14
|
}
|
|
12
|
-
export interface SuiteRunEntry extends LedgerEntry {
|
|
13
|
-
kind: "suite-run";
|
|
14
|
-
command: string;
|
|
15
|
-
exitCode: number;
|
|
16
|
-
durationMs: number;
|
|
17
|
-
testCount: number | "unknown";
|
|
18
|
-
scope: "full" | "subset";
|
|
19
|
-
}
|
|
20
15
|
export interface Ledger {
|
|
21
16
|
entries: LedgerEntry[];
|
|
22
17
|
rejected: string[];
|
|
@@ -27,12 +22,6 @@ export interface Ledger {
|
|
|
27
22
|
}
|
|
28
23
|
export function createLedger(): Ledger;
|
|
29
24
|
export function validateEntry(entry: unknown): string | null;
|
|
30
|
-
export function deriveSuiteRun(
|
|
31
|
-
command: unknown,
|
|
32
|
-
outcome?: { exitCode?: number; durationMs?: number; output?: string },
|
|
33
|
-
): SuiteRunEntry | null;
|
|
34
|
-
export function isTestCommand(command: unknown): boolean;
|
|
35
|
-
export function deriveHealthRun(command: unknown, opts?: { verdict?: string; scanned?: number; level?: string }): LedgerEntry | null;
|
|
36
25
|
export function sessionFacts(
|
|
37
26
|
ledger: Ledger,
|
|
38
27
|
opts?: { scale?: "small" | "standard" | "major"; webTouched?: boolean },
|
package/lib/gates/ledger.mjs
CHANGED
|
@@ -125,100 +125,6 @@ export function createLedger() {
|
|
|
125
125
|
};
|
|
126
126
|
}
|
|
127
127
|
|
|
128
|
-
// ─── Derivation from observed tool traffic ─────────────────────────────────
|
|
129
|
-
//
|
|
130
|
-
// These functions turn "the extension watched a tool run" into an entry. They
|
|
131
|
-
// are the only way entries get created, which is the whole point.
|
|
132
|
-
|
|
133
|
-
/**
|
|
134
|
-
* Recognise a full test-suite run from a bash invocation and its outcome.
|
|
135
|
-
* Returns null when the command is not a test run — an unrecognised command is
|
|
136
|
-
* not evidence, and is not silently counted as one either.
|
|
137
|
-
*/
|
|
138
|
-
/**
|
|
139
|
-
* Test-runner recognition. The first version matched `npm test`/vitest/jest/
|
|
140
|
-
* pytest/go/cargo/make only; on the real traces that missed 64% of test
|
|
141
|
-
* invocations (custom runners: `bash test/run-all.sh`, `node test-runner.js`,
|
|
142
|
-
* `timeout 300 node tests/e2e/x.test.js`). Under the completion floor a missed
|
|
143
|
-
* runner is a false "no test ran" steer, so the net is wide and the subset rule
|
|
144
|
-
* is explicit: a run naming one test file is a subset, a runner script is full.
|
|
145
|
-
*/
|
|
146
|
-
const TEST_RUNNER = new RegExp(
|
|
147
|
-
[
|
|
148
|
-
String.raw`(^|[\s&;|(])(npm|pnpm|yarn|bun)\s+(run\s+)?test(:[\w-]+)?\b`,
|
|
149
|
-
String.raw`\b(vitest|jest|pytest|mocha|ava|tap|rspec|phpunit|minitest|nextest)\b`,
|
|
150
|
-
String.raw`\b(go|cargo|deno|bun|dotnet|mvn|gradle|gradlew|mix|swift|zig|make|just|tox|nox)\s+test\b`,
|
|
151
|
-
String.raw`\bpython3?\s+-m\s+(pytest|unittest)\b`,
|
|
152
|
-
String.raw`\bnpx\s+(playwright|vitest|jest|mocha)\s+test\b|\bnpx\s+(vitest|jest|mocha)\b`,
|
|
153
|
-
String.raw`\bnode\s+--test\b`,
|
|
154
|
-
// runner scripts: run-all.sh, test-runner.js, run_tests.py, scripts/test.sh
|
|
155
|
-
String.raw`(^|[\s/])(run[-_]?all|test[-_]?runner|run[-_]?tests?|tests?)\.(sh|js|mjs|cjs|ts|py)\b`,
|
|
156
|
-
// a test file executed directly: node tests/x.test.js, tsx foo.spec.ts
|
|
157
|
-
String.raw`\b(node|tsx|ts-node|bun|deno\s+run|python3?)\s+[^\s|;&]*\.(test|spec)\.(js|mjs|cjs|ts|tsx|py)\b`,
|
|
158
|
-
String.raw`\b(node|tsx|bun|python3?)\s+[^\s|;&]*(^|/)(tests?|__tests__|spec)/[^\s|;&]+\.(js|mjs|cjs|ts|py)\b`,
|
|
159
|
-
].join("|"),
|
|
160
|
-
);
|
|
161
|
-
/** A lint/typecheck/format is a check, never a test run. */
|
|
162
|
-
const CHECK_ONLY_CMD = /^\s*(npm\s+run\s+)?(lint|typecheck|check|format|prettier|eslint|tsc|biome)\b/;
|
|
163
|
-
|
|
164
|
-
export function isTestCommand(command) {
|
|
165
|
-
if (typeof command !== "string") return false;
|
|
166
|
-
if (CHECK_ONLY_CMD.test(command) && !/\btest\b/.test(command)) return false;
|
|
167
|
-
return TEST_RUNNER.test(command);
|
|
168
|
-
}
|
|
169
|
-
|
|
170
|
-
export function deriveSuiteRun(command, { exitCode = 0, durationMs = 0, output = "" } = {}) {
|
|
171
|
-
if (typeof command !== "string") return null;
|
|
172
|
-
if (!isTestCommand(command)) return null;
|
|
173
|
-
const directFile = /\b(node|tsx|ts-node|bun|deno\s+run|python3?)\s+[^\s|;&]*(\.(test|spec)\.(js|mjs|cjs|ts|tsx|py)|(^|\/)(tests?|__tests__|spec)\/[^\s|;&]+\.(js|mjs|cjs|ts|py))\b/.test(command)
|
|
174
|
-
&& !/(run[-_]?all|test[-_]?runner|run[-_]?tests?)\./.test(command);
|
|
175
|
-
const isSubset = directFile || /--\s*(grep|testPathPattern|filter)\b|-t\s+\S+|\brun\s+\S+\.spec\b|--to-grep|-k\s+\S+|--testNamePattern/.test(command);
|
|
176
|
-
const parsed = parseTestCounts(output);
|
|
177
|
-
return {
|
|
178
|
-
kind: "suite-run",
|
|
179
|
-
command: command.slice(0, 400),
|
|
180
|
-
exitCode,
|
|
181
|
-
durationMs,
|
|
182
|
-
// Runner output differs (vitest, jest, pytest, go test). Counts are recorded
|
|
183
|
-
// only where they can actually be read — never inferred, never 0 by default.
|
|
184
|
-
testCount: parsed == null ? "unknown" : parsed,
|
|
185
|
-
scope: isSubset ? "subset" : "full",
|
|
186
|
-
impactStated: false, // filled by the caller when the impact area was declared
|
|
187
|
-
};
|
|
188
|
-
}
|
|
189
|
-
|
|
190
|
-
function parseTestCounts(output) {
|
|
191
|
-
if (!output || typeof output !== "string") return null;
|
|
192
|
-
const patterns = [
|
|
193
|
-
/\bTests:\s*(\d+)\s+passed\b/, // jest
|
|
194
|
-
/(\d+)\s+tests?\s+passed/, // vitest / pytest style
|
|
195
|
-
/(\d+)\s+passing\b/, // mocha
|
|
196
|
-
/\bok\s+\.\.\.\s+(\d+)\s+tests?/, // go test
|
|
197
|
-
];
|
|
198
|
-
for (const re of patterns) {
|
|
199
|
-
const m = output.match(re);
|
|
200
|
-
if (m) return Number(m[1]);
|
|
201
|
-
}
|
|
202
|
-
return null;
|
|
203
|
-
}
|
|
204
|
-
|
|
205
|
-
/** Recognise an `/asf health` invocation from the command text. */
|
|
206
|
-
export function deriveHealthRun(command, { verdict, scanned, level } = {}) {
|
|
207
|
-
if (typeof command !== "string" || !/asf\s+health\b/.test(command)) return null;
|
|
208
|
-
return {
|
|
209
|
-
kind: "health-run",
|
|
210
|
-
level: level || guessLevel(command),
|
|
211
|
-
verdict: verdict || "unknown",
|
|
212
|
-
scanned: typeof scanned === "number" ? scanned : 0,
|
|
213
|
-
};
|
|
214
|
-
}
|
|
215
|
-
|
|
216
|
-
function guessLevel(command) {
|
|
217
|
-
if (/--major\b/.test(command)) return "major";
|
|
218
|
-
if (/--small\b/.test(command)) return "small";
|
|
219
|
-
return "standard";
|
|
220
|
-
}
|
|
221
|
-
|
|
222
128
|
// ─── What the completion gate may conclude ─────────────────────────────────
|
|
223
129
|
|
|
224
130
|
/**
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Type surface for the refusal ladder (split out of `blockers.mjs` in 0.17.5).
|
|
3
|
+
*/
|
|
4
|
+
|
|
5
|
+
/** Normative text per refusal, injected on the second identical fire. */
|
|
6
|
+
export const RULE_TEXT: Record<string, string>;
|
|
7
|
+
|
|
8
|
+
/** Which fire of this identical refusal is this? 1 = first time. */
|
|
9
|
+
export function refusalStage(gate: string, fingerprint: unknown): number;
|
|
10
|
+
|
|
11
|
+
/** Test hook, and the same reset a new run performs. */
|
|
12
|
+
export function __resetRefusals(): void;
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The refusal ladder (P3).
|
|
3
|
+
*
|
|
4
|
+
* 53% of the refusals the agent ever received (9 of 17) were it walking into the
|
|
5
|
+
* same wall again — `secret-in-tracked-file` fired 3 of its 4 times as a repeat,
|
|
6
|
+
* all on test files with realistic-looking placeholder keys. A remedy in a
|
|
7
|
+
* message that is re-issued identically is not made to stick, so the ladder
|
|
8
|
+
* changes what the agent is *told* before it ever changes what it is *asked*:
|
|
9
|
+
*
|
|
10
|
+
* 1st fire → refuse + autonomous exit (the text already carried this)
|
|
11
|
+
* 2nd same → refuse + the rule text itself, so the why and the sanctioned
|
|
12
|
+
* alternative reach the context instead of just the veto
|
|
13
|
+
* 3rd same → park for the PO: insisting three times on an irreversible action
|
|
14
|
+
* with no safe path is exactly the park case, not a nag
|
|
15
|
+
*
|
|
16
|
+
* Split out of `blockers.mjs` (0.17.5): the blockers decide, this decides what
|
|
17
|
+
* the agent is told when it walks into one twice.
|
|
18
|
+
*/
|
|
19
|
+
|
|
20
|
+
// ─── The refusal ladder (P3) ───────────────────────────────────────────────
|
|
21
|
+
//
|
|
22
|
+
// 53% of the refusals the agent ever received (9 of 17) were it walking into the
|
|
23
|
+
// same wall again — `secret-in-tracked-file` fired 3 of its 4 times as a repeat,
|
|
24
|
+
// all on test files with realistic-looking placeholder keys. A remedy in a
|
|
25
|
+
// message that is re-issued identically is not made to stick, so the ladder
|
|
26
|
+
// changes what the agent is *told* before it ever changes what it is *asked*:
|
|
27
|
+
//
|
|
28
|
+
// 1st fire → refuse + autonomous exit (the text already carried this)
|
|
29
|
+
// 2nd same → refuse + the rule text itself, so the why and the sanctioned
|
|
30
|
+
// alternative reach the context instead of just the veto
|
|
31
|
+
// 3rd same → park for the PO: insisting three times on an irreversible action
|
|
32
|
+
// with no safe path is exactly the park case, not a nag
|
|
33
|
+
|
|
34
|
+
/** Normative text per refusal, injected on the second identical fire. */
|
|
35
|
+
export const RULE_TEXT = {
|
|
36
|
+
"secret-in-tracked-file":
|
|
37
|
+
"Secrets never go into tracked files, including tests. The sanctioned form is " +
|
|
38
|
+
"a fixture constant that cannot be a real key (sk-test-not-a-real-key), an " +
|
|
39
|
+
"environment variable name read at runtime, or a git-ignored .env. A key " +
|
|
40
|
+
"shape that a scanner can match is a key shape that a scanner will match.",
|
|
41
|
+
"destructive-git-dirty-tree":
|
|
42
|
+
"A destructive git command against a dirty tree discards work that exists " +
|
|
43
|
+
"nowhere else. Commit it, or stash it with a message, then re-run the " +
|
|
44
|
+
"destructive step. The gate is not protecting the tree from you; it is " +
|
|
45
|
+
"protecting the only copy of the change.",
|
|
46
|
+
"push-without-recorded-green":
|
|
47
|
+
"Nothing is published or pushed without a recorded full-suite green run in " +
|
|
48
|
+
"this session. The gate never moves — the suite does. Run the full suite; " +
|
|
49
|
+
"when it exits 0 this is unblocked, and the published artifact is then " +
|
|
50
|
+
"verified from the registry rather than assumed.",
|
|
51
|
+
};
|
|
52
|
+
|
|
53
|
+
const refusals = new Map();
|
|
54
|
+
|
|
55
|
+
/** Which fire of this identical refusal is this? 1 = first time. */
|
|
56
|
+
export function refusalStage(gate, fingerprint) {
|
|
57
|
+
const key = `${gate}|${String(fingerprint || "").slice(0, 200)}`;
|
|
58
|
+
const n = (refusals.get(key) || 0) + 1;
|
|
59
|
+
refusals.set(key, n);
|
|
60
|
+
return n;
|
|
61
|
+
}
|
|
62
|
+
|
|
63
|
+
/** Test hook, and the same reset a new run performs. */
|
|
64
|
+
export function __resetRefusals() {
|
|
65
|
+
refusals.clear();
|
|
66
|
+
}
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Type surface for suite-run recognition (the observer half of the ledger, split
|
|
3
|
+
* out of `ledger.mjs` in 0.17.5). The implementation is plain ESM JavaScript;
|
|
4
|
+
* this is what index.ts and the handlers may rely on.
|
|
5
|
+
*/
|
|
6
|
+
export interface SuiteRunEntry {
|
|
7
|
+
kind: "suite-run";
|
|
8
|
+
command: string;
|
|
9
|
+
exitCode: number;
|
|
10
|
+
durationMs: number;
|
|
11
|
+
testCount: number | "unknown";
|
|
12
|
+
scope: "full" | "subset";
|
|
13
|
+
impactStated: boolean;
|
|
14
|
+
at?: number;
|
|
15
|
+
[field: string]: unknown;
|
|
16
|
+
}
|
|
17
|
+
|
|
18
|
+
export function isTestCommand(command: unknown): boolean;
|
|
19
|
+
export function deriveSuiteRun(
|
|
20
|
+
command: unknown,
|
|
21
|
+
outcome?: { exitCode?: number; durationMs?: number; output?: string },
|
|
22
|
+
): SuiteRunEntry | null;
|
|
23
|
+
|
|
24
|
+
/** The newest entry that was a FULL run (subset runs never satisfy an exit floor). */
|
|
25
|
+
export function lastFullRun(suiteRuns?: SuiteRunEntry[]): SuiteRunEntry | null;
|
|
26
|
+
|
|
27
|
+
/** The green an exit floor accepts, or why it does not. */
|
|
28
|
+
export function greenFullRun(
|
|
29
|
+
suiteRuns?: SuiteRunEntry[],
|
|
30
|
+
since?: number,
|
|
31
|
+
): { run: SuiteRunEntry | null; ok: boolean; why: string };
|
|
@@ -0,0 +1,119 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Suite runs and health runs, derived from observed tool traffic.
|
|
3
|
+
*
|
|
4
|
+
* A suite run is a fact only because the extension *saw* the command and the
|
|
5
|
+
* exit code — never because the agent said it ran the tests. This module owns
|
|
6
|
+
* the recognition (what counts as a runner, what counts as a subset) and the
|
|
7
|
+
* one question the exit floors ask: is the LAST full run green, and did it
|
|
8
|
+
* start after the last source change?
|
|
9
|
+
*
|
|
10
|
+
* Split out of `ledger.mjs` (0.17.5) because the ledger is the entry store and
|
|
11
|
+
* this is the observer; they changed for different reasons and together they
|
|
12
|
+
* outgrew the file limit ASF enforces on everyone else.
|
|
13
|
+
*/
|
|
14
|
+
|
|
15
|
+
// ─── Derivation from observed tool traffic ─────────────────────────────────
|
|
16
|
+
//
|
|
17
|
+
// These functions turn "the extension watched a tool run" into an entry. They
|
|
18
|
+
// are the only way entries get created, which is the whole point.
|
|
19
|
+
|
|
20
|
+
/**
|
|
21
|
+
* Recognise a full test-suite run from a bash invocation and its outcome.
|
|
22
|
+
* Returns null when the command is not a test run — an unrecognised command is
|
|
23
|
+
* not evidence, and is not silently counted as one either.
|
|
24
|
+
*/
|
|
25
|
+
/**
|
|
26
|
+
* Test-runner recognition. The first version matched `npm test`/vitest/jest/
|
|
27
|
+
* pytest/go/cargo/make only; on the real traces that missed 64% of test
|
|
28
|
+
* invocations (custom runners: `bash test/run-all.sh`, `node test-runner.js`,
|
|
29
|
+
* `timeout 300 node tests/e2e/x.test.js`). Under the completion floor a missed
|
|
30
|
+
* runner is a false "no test ran" steer, so the net is wide and the subset rule
|
|
31
|
+
* is explicit: a run naming one test file is a subset, a runner script is full.
|
|
32
|
+
*/
|
|
33
|
+
const TEST_RUNNER = new RegExp(
|
|
34
|
+
[
|
|
35
|
+
String.raw`(^|[\s&;|(])(npm|pnpm|yarn|bun)\s+(run\s+)?test(:[\w-]+)?\b`,
|
|
36
|
+
String.raw`\b(vitest|jest|pytest|mocha|ava|tap|rspec|phpunit|minitest|nextest)\b`,
|
|
37
|
+
String.raw`\b(go|cargo|deno|bun|dotnet|mvn|gradle|gradlew|mix|swift|zig|make|just|tox|nox)\s+test\b`,
|
|
38
|
+
String.raw`\bpython3?\s+-m\s+(pytest|unittest)\b`,
|
|
39
|
+
String.raw`\bnpx\s+(playwright|vitest|jest|mocha)\s+test\b|\bnpx\s+(vitest|jest|mocha)\b`,
|
|
40
|
+
String.raw`\bnode\s+--test\b`,
|
|
41
|
+
// runner scripts: run-all.sh, test-runner.js, run_tests.py, scripts/test.sh
|
|
42
|
+
String.raw`(^|[\s/])(run[-_]?all|test[-_]?runner|run[-_]?tests?|tests?)\.(sh|js|mjs|cjs|ts|py)\b`,
|
|
43
|
+
// a test file executed directly: node tests/x.test.js, tsx foo.spec.ts
|
|
44
|
+
String.raw`\b(node|tsx|ts-node|bun|deno\s+run|python3?)\s+[^\s|;&]*\.(test|spec)\.(js|mjs|cjs|ts|tsx|py)\b`,
|
|
45
|
+
String.raw`\b(node|tsx|bun|python3?)\s+[^\s|;&]*(^|/)(tests?|__tests__|spec)/[^\s|;&]+\.(js|mjs|cjs|ts|py)\b`,
|
|
46
|
+
].join("|"),
|
|
47
|
+
);
|
|
48
|
+
/** A lint/typecheck/format is a check, never a test run. */
|
|
49
|
+
const CHECK_ONLY_CMD = /^\s*(npm\s+run\s+)?(lint|typecheck|check|format|prettier|eslint|tsc|biome)\b/;
|
|
50
|
+
|
|
51
|
+
export function isTestCommand(command) {
|
|
52
|
+
if (typeof command !== "string") return false;
|
|
53
|
+
if (CHECK_ONLY_CMD.test(command) && !/\btest\b/.test(command)) return false;
|
|
54
|
+
return TEST_RUNNER.test(command);
|
|
55
|
+
}
|
|
56
|
+
|
|
57
|
+
export function deriveSuiteRun(command, { exitCode = 0, durationMs = 0, output = "" } = {}) {
|
|
58
|
+
if (typeof command !== "string") return null;
|
|
59
|
+
if (!isTestCommand(command)) return null;
|
|
60
|
+
const directFile = /\b(node|tsx|ts-node|bun|deno\s+run|python3?)\s+[^\s|;&]*(\.(test|spec)\.(js|mjs|cjs|ts|tsx|py)|(^|\/)(tests?|__tests__|spec)\/[^\s|;&]+\.(js|mjs|cjs|ts|py))\b/.test(command)
|
|
61
|
+
&& !/(run[-_]?all|test[-_]?runner|run[-_]?tests?)\./.test(command);
|
|
62
|
+
const isSubset = directFile || /--\s*(grep|testPathPattern|filter)\b|-t\s+\S+|\brun\s+\S+\.spec\b|--to-grep|-k\s+\S+|--testNamePattern/.test(command);
|
|
63
|
+
const parsed = parseTestCounts(output);
|
|
64
|
+
return {
|
|
65
|
+
kind: "suite-run",
|
|
66
|
+
command: command.slice(0, 400),
|
|
67
|
+
exitCode,
|
|
68
|
+
durationMs,
|
|
69
|
+
// Runner output differs (vitest, jest, pytest, go test). Counts are recorded
|
|
70
|
+
// only where they can actually be read — never inferred, never 0 by default.
|
|
71
|
+
testCount: parsed == null ? "unknown" : parsed,
|
|
72
|
+
scope: isSubset ? "subset" : "full",
|
|
73
|
+
impactStated: false, // filled by the caller when the impact area was declared
|
|
74
|
+
};
|
|
75
|
+
}
|
|
76
|
+
|
|
77
|
+
/**
|
|
78
|
+
* The suite runs an exit floor reads.
|
|
79
|
+
*
|
|
80
|
+
* `some(green)` is not a floor: it lets a session publish on a green from
|
|
81
|
+
* before the last edit, and it lets a green from earlier in the session outrank
|
|
82
|
+
* a red that came after it. The floor is the LAST full run, and it must have
|
|
83
|
+
* started after the last source mutation — the same rule the settle floor uses
|
|
84
|
+
* (`run.ts`), so the two cannot disagree about what "green" means.
|
|
85
|
+
*/
|
|
86
|
+
export function lastFullRun(suiteRuns = []) {
|
|
87
|
+
if (!Array.isArray(suiteRuns)) return null;
|
|
88
|
+
for (let i = suiteRuns.length - 1; i >= 0; i--) {
|
|
89
|
+
const r = suiteRuns[i];
|
|
90
|
+
if (r && r.scope === "full") return r;
|
|
91
|
+
}
|
|
92
|
+
return null;
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
/** The green that opens an exit floor, or null with the reason it is not green. */
|
|
96
|
+
export function greenFullRun(suiteRuns = [], since = 0) {
|
|
97
|
+
const run = lastFullRun(suiteRuns);
|
|
98
|
+
if (!run) return { run: null, ok: false, why: "no full-suite run recorded in this session" };
|
|
99
|
+
if (run.exitCode !== 0) return { run, ok: false, why: `the last full suite run exited ${run.exitCode}` };
|
|
100
|
+
if ((run.at ?? 0) < (Number(since) || 0)) {
|
|
101
|
+
return { run, ok: false, why: "the last full-suite green predates the most recent source change" };
|
|
102
|
+
}
|
|
103
|
+
return { run, ok: true, why: "" };
|
|
104
|
+
}
|
|
105
|
+
|
|
106
|
+
function parseTestCounts(output) {
|
|
107
|
+
if (!output || typeof output !== "string") return null;
|
|
108
|
+
const patterns = [
|
|
109
|
+
/\bTests:\s*(\d+)\s+passed\b/, // jest
|
|
110
|
+
/(\d+)\s+tests?\s+passed/, // vitest / pytest style
|
|
111
|
+
/(\d+)\s+passing\b/, // mocha
|
|
112
|
+
/\bok\s+\.\.\.\s+(\d+)\s+tests?/, // go test
|
|
113
|
+
];
|
|
114
|
+
for (const re of patterns) {
|
|
115
|
+
const m = output.match(re);
|
|
116
|
+
if (m) return Number(m[1]);
|
|
117
|
+
}
|
|
118
|
+
return null;
|
|
119
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-aia-asf",
|
|
3
|
-
"version": "0.17.
|
|
3
|
+
"version": "0.17.6",
|
|
4
4
|
"description": "Ai Applied Agentic Software Factory — codifies the full software development flow: intake, research, spec capture, adversarial analysis, planning with approval gates, test-first implementation, and release. Requires pi-vigilant, pi-smart-web-search, pi-smart-fetch, and pi-aia-browser.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"pi-package",
|
|
@@ -60,7 +60,7 @@ goes in the final message in plain words — never discovered by the user throug
|
|
|
60
60
|
## Your own artifacts are part of the deliverable (task-scoped drift)
|
|
61
61
|
|
|
62
62
|
The plan and the spec tree are artifacts too, and they make checkable claims: a PLAN.md
|
|
63
|
-
that names `lib/gates/
|
|
63
|
+
that names `lib/gates/suite-runs.mjs` or `deriveSuiteRun()` is asserting those exist. When the
|
|
64
64
|
code has moved and the artifact has not, the delivery report describes something that is
|
|
65
65
|
not there — which is the same defect as a stub, just further from the user's eye.
|
|
66
66
|
|
|
@@ -63,6 +63,13 @@ in the absence of a user definition.
|
|
|
63
63
|
- **Deployed app:** smoke-test in **production** (health endpoint, key user journey) — a deploy that breaks prod is not a successful release.
|
|
64
64
|
- **Not applicable** (internal-only experiment, no runtime surface): say so explicitly in the report (06b Rule 11).
|
|
65
65
|
- If the published artifact is broken: **fix forward** (new patch), never pretend the release succeeded.
|
|
66
|
+
- **The copy you are testing may not be the copy you shipped.** `pi` loads extensions once at
|
|
67
|
+
process start: a session that was already open keeps enforcing the *previous* version's
|
|
68
|
+
gates no matter what you publish, and its messages quote text that no longer exists in the
|
|
69
|
+
source. Before concluding a released gate is broken, confirm which build is loaded — compare
|
|
70
|
+
a string unique to the new release against the behaviour you saw, and check the process
|
|
71
|
+
start time (`ps -o lstart -p <pid>`). `/reload`, or a new session, is what picks up the new
|
|
72
|
+
code. This is the difference between "the fix did not work" and "the fix is not running".
|
|
66
73
|
|
|
67
74
|
## CI/CD offer
|
|
68
75
|
|
|
@@ -79,3 +86,4 @@ After a successful release (or for a new project), **offer** to set up a CI/CD p
|
|
|
79
86
|
- The version in package.json, the git tag, and the npm release must match
|
|
80
87
|
- If publishing fails partway, fix forward; never publish a second version with the same number
|
|
81
88
|
- **Test on publish / in production, if applicable — always** (step 7)
|
|
89
|
+
- **Never diagnose a gate from a session that predates the release** — confirm the loaded build first (step 7)
|