pi-aia-asf 0.17.10 → 0.17.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +79 -6
- package/index.ts +2 -0
- package/lib/code-health/config.mjs +9 -4
- package/lib/code-health/tools.d.mts +21 -0
- package/lib/extension/commands/inspect.ts +2 -2
- package/lib/extension/commands/verify.ts +2 -1
- package/lib/extension/run.ts +39 -2
- package/lib/extension/tools/evidence-check.ts +134 -0
- package/lib/extension/tools/health.ts +5 -5
- package/lib/gates/evidence-coverage.d.mts +19 -0
- package/lib/gates/evidence-coverage.mjs +124 -0
- package/lib/home.mjs +1 -0
- package/package.json +1 -1
- package/skills/aia-asf/references/06e-code-health.md +31 -2
package/CHANGELOG.md
CHANGED
|
@@ -7,13 +7,86 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
-
## [0.17.
|
|
11
|
-
|
|
12
|
-
###
|
|
10
|
+
## [0.17.12] - 2026-11-10
|
|
11
|
+
|
|
12
|
+
### Fixed — the health gate measured the wrong project
|
|
13
|
+
|
|
14
|
+
`runRoot()` is the git root of the session's cwd. That is right for where run state
|
|
15
|
+
lives and wrong for what to measure, and a session launched from a home directory that
|
|
16
|
+
is itself a git repo (common: `pi` started from $HOME with checkouts underneath) had
|
|
17
|
+
every health gate measure the parent. It did: `asf_health` reported 363 model findings
|
|
18
|
+
from an unrelated project checked out under $HOME, and eslint, madge and jscpd each
|
|
19
|
+
timed out at 120s scanning a tree nobody was working in — a verdict of NOT healthy on a
|
|
20
|
+
repository that is, in fact, healthy.
|
|
21
|
+
|
|
22
|
+
A gate that measures the wrong directory is worse than no gate: it blocks on findings
|
|
23
|
+
in code the run must not touch, or waves through the one project that is actually
|
|
24
|
+
unhealthy. `measureRoot()` now answers from what the run already knows — the git root of
|
|
25
|
+
the most recently touched file — and is used by the three paths that measure
|
|
26
|
+
(`asf_health`, `/asf health`, `/asf verify` Gate 8). It is deliberately **not** used to
|
|
27
|
+
locate state: a root that moved mid-run would orphan the run's own ledger, so
|
|
28
|
+
`runRoot()` still follows the session for that.
|
|
29
|
+
|
|
30
|
+
A touched path outside the workspace (a scratch dir, an unrelated checkout) does not
|
|
31
|
+
move the measurement either — the session root is the safe answer. `test/test-measure-root.mjs`
|
|
32
|
+
pins both on throwaway repos rather than this machine's $HOME, and both guards are
|
|
33
|
+
mutation-caught.
|
|
34
|
+
|
|
35
|
+
## [0.17.11] - 2026-11-10
|
|
36
|
+
|
|
37
|
+
### Added — the coverage invariant is checkable from the installed package
|
|
38
|
+
|
|
39
|
+
`test/test-evidence-kinds.mjs` asserts that every evidence kind a gate requires has a
|
|
40
|
+
producer, but `test/` is not shipped — so a broken or half-updated install reproduced
|
|
41
|
+
the one bug that suite exists to catch, and the session inside it had no way to find
|
|
42
|
+
out. It could only be steered, at the same gap, forever. Three callers now ask the one
|
|
43
|
+
question, against one definition in `lib/gates/evidence-coverage.mjs`:
|
|
44
|
+
|
|
45
|
+
- `asf_evidence_check {kind?}` — the agent's own check. An unproducible **required**
|
|
46
|
+
kind is reported as a broken install rather than as work to keep attempting, because
|
|
47
|
+
no amount of effort records evidence nothing in the package can write.
|
|
48
|
+
- `scripts/verify-published.mjs` — a fourth release section, run against the copy npm
|
|
49
|
+
just installed. A tarball missing a producer now fails the release instead of
|
|
50
|
+
shipping the loop to every user while the repository stayed green. A section that
|
|
51
|
+
throws is a failed check, never a silent pass.
|
|
52
|
+
- `test/test-evidence-kinds.mjs` — asserts the answer, and asserts the shipped module
|
|
53
|
+
derives the same required set an independent derivation in the test does.
|
|
54
|
+
|
|
55
|
+
### Fixed — the coverage checker could not say "no"
|
|
56
|
+
|
|
57
|
+
Writing the negatives proved it: five mutations of the new module initially passed the
|
|
58
|
+
suite, because every assertion asked only whether a producer was **found**. A checker
|
|
59
|
+
whose regex matches everything, or that counts the ledger as the producer of its own
|
|
60
|
+
kinds, is invisible to a suite like that — and `requiredOrphans: []` was invisible too.
|
|
61
|
+
It is now tested against fixture packages where the answer is known to be no: a kind
|
|
62
|
+
nothing writes, and a kind only the ledger mentions. The tool's two paths no session
|
|
63
|
+
hits on a good day (broken install, check that threw) were untested for the same reason
|
|
64
|
+
and are now driven directly.
|
|
65
|
+
|
|
66
|
+
### Changed — the code-health config lives under `.asf/`
|
|
67
|
+
|
|
68
|
+
`.asf/code-health.json` is now read first, with the root `.asf-code-health.json` still
|
|
69
|
+
honoured so a project configured before the move does not wake up on defaults after an
|
|
70
|
+
upgrade. This was the last ASF-authored file still written to the project root, and
|
|
71
|
+
"everything ASF authors lives under `.asf/`" is one decision with one table of names:
|
|
72
|
+
the path now comes from `FILES` in `lib/home.mjs` rather than a literal in the reader,
|
|
73
|
+
which is how a file ends up written in one place and read from another.
|
|
74
|
+
|
|
75
|
+
### Fixed — the shipped CHANGELOG said nothing
|
|
76
|
+
|
|
77
|
+
The 0.17.10 section was the release script's untouched placeholder, twice over: an
|
|
78
|
+
empty `### Added` above a second heading for the same version. `CHANGELOG.md` ships in
|
|
79
|
+
the tarball, so that is what a user reads. `test/test-antipatterns.mjs` now asserts the
|
|
80
|
+
shipped version has a section, that it is not the placeholder, that it has real prose,
|
|
81
|
+
and that no version is listed twice — making "Vague commits or CHANGELOG placeholders"
|
|
82
|
+
an enforced anti-pattern rather than a listed one.
|
|
83
|
+
|
|
84
|
+
Also: `test/harness.mjs` passes a jiti loader to suites, since a `.mjs` suite cannot
|
|
85
|
+
import ASF's own `.ts` modules and each suite spinning up its own loader is how two
|
|
86
|
+
different loaders start disagreeing; `lib/code-health/tools.d.mts` added (an imported
|
|
87
|
+
`.mjs` used from `.ts` needs a declaration or typecheck fails).
|
|
13
88
|
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
## [0.18.0] - 2026-11-10
|
|
89
|
+
## [0.17.10] - 2026-11-10
|
|
17
90
|
|
|
18
91
|
### Added — the browser evidence the completion gate demands can now be produced
|
|
19
92
|
|
package/index.ts
CHANGED
|
@@ -19,6 +19,7 @@ import { setPiRef } from "./lib/extension/pi-ref.ts";
|
|
|
19
19
|
import { registerHealthTool } from "./lib/extension/tools/health.ts";
|
|
20
20
|
import { registerDecisionsTool } from "./lib/extension/tools/decisions.ts";
|
|
21
21
|
import { registerBrowserTools } from "./lib/extension/tools/browser.ts";
|
|
22
|
+
import { registerEvidenceCheckTool } from "./lib/extension/tools/evidence-check.ts";
|
|
22
23
|
import { registerCommands } from "./lib/extension/commands/index.ts";
|
|
23
24
|
import { registerToolCallHandler } from "./lib/extension/handlers/tool-call.ts";
|
|
24
25
|
import { registerToolResultHandler } from "./lib/extension/handlers/tool-result.ts";
|
|
@@ -32,6 +33,7 @@ export default function register(pi: ExtensionAPI): void {
|
|
|
32
33
|
registerHealthTool(pi);
|
|
33
34
|
registerDecisionsTool(pi);
|
|
34
35
|
registerBrowserTools(pi);
|
|
36
|
+
registerEvidenceCheckTool(pi);
|
|
35
37
|
registerCommands(pi);
|
|
36
38
|
registerToolCallHandler(pi);
|
|
37
39
|
registerToolResultHandler(pi);
|
|
@@ -203,15 +203,20 @@ export function normalizeConfig(raw) {
|
|
|
203
203
|
}
|
|
204
204
|
|
|
205
205
|
/**
|
|
206
|
-
* Load the project config:
|
|
207
|
-
* `
|
|
206
|
+
* Load the project config: `.asf/code-health.json`, falling back to the root
|
|
207
|
+
* `.asf-code-health.json` that older releases wrote.
|
|
208
|
+
*
|
|
209
|
+
* The path comes from the home table rather than a literal here, because "everything
|
|
210
|
+
* ASF authors lives under `.asf/`" is one decision with one table of names — a second
|
|
211
|
+
* place that hardcodes a location is how a file ends up written in one place and read
|
|
212
|
+
* from another. `readFile` is injectable for tests.
|
|
208
213
|
*/
|
|
209
214
|
export async function loadConfig(projectRoot, readFile = null) {
|
|
210
215
|
const fs = readFile || (await import("node:fs/promises")).readFile;
|
|
216
|
+
const { readPath } = await import("../home.mjs");
|
|
211
217
|
let raw = null;
|
|
212
218
|
try {
|
|
213
|
-
|
|
214
|
-
raw = JSON.parse(text);
|
|
219
|
+
raw = JSON.parse(await fs(readPath(projectRoot, "healthConfig"), "utf-8"));
|
|
215
220
|
} catch {
|
|
216
221
|
raw = null; // no file, or unreadable/invalid → defaults
|
|
217
222
|
}
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
/** Where ASF is installed — the base for resolving its own bundled toolchain. */
|
|
2
|
+
export const ASF_ROOT: string;
|
|
3
|
+
|
|
4
|
+
export interface ResolvedTools {
|
|
5
|
+
eslint?: string;
|
|
6
|
+
sonarjs?: string;
|
|
7
|
+
tsParser?: string;
|
|
8
|
+
jscpd?: string;
|
|
9
|
+
madge?: string;
|
|
10
|
+
depcruise?: string;
|
|
11
|
+
/** Where each tool was found: "asf" | "project" | "path" | undefined when missing. */
|
|
12
|
+
sources: Record<string, string | undefined>;
|
|
13
|
+
missing: string[];
|
|
14
|
+
[key: string]: unknown;
|
|
15
|
+
}
|
|
16
|
+
|
|
17
|
+
/** Resolve the health toolchain ASF-first: ASF's install, then the project, then PATH. */
|
|
18
|
+
export function resolveTools(projectRoot: string, opts?: { asfRoot?: string }): ResolvedTools;
|
|
19
|
+
|
|
20
|
+
/** The required tools that could not be found; empty means the gate can measure. */
|
|
21
|
+
export function missingTools(tools: ResolvedTools): string[];
|
|
@@ -7,7 +7,7 @@ import { projectName, runRoot } from "../paths.ts";
|
|
|
7
7
|
import { loadState } from "../state.ts";
|
|
8
8
|
import { ledger, authority, gateStats } from "../gate-state.ts";
|
|
9
9
|
import { runtime } from "../runtime.ts";
|
|
10
|
-
import { cardFor, collectMeasurements, extensionDir } from "../run.ts";
|
|
10
|
+
import { cardFor, collectMeasurements, extensionDir, measureRoot } from "../run.ts";
|
|
11
11
|
import { renderLedger } from "../../gates/ledger.mjs";
|
|
12
12
|
import { renderAuthority } from "../../gates/authority.mjs";
|
|
13
13
|
import { renderCompletionGate } from "../../gates/settle.mjs";
|
|
@@ -41,7 +41,7 @@ export async function cmdStatus(): Promise<string> {
|
|
|
41
41
|
export async function cmdHealth(args: string[]): Promise<string> {
|
|
42
42
|
const { runHealth } = await import("../../code-health/index.mjs");
|
|
43
43
|
const { recordHealthVerdict, recordLine } = await import("../health-record.ts");
|
|
44
|
-
const res = await runHealth(
|
|
44
|
+
const res = await runHealth(await measureRoot(), args.slice(1));
|
|
45
45
|
// The command records what it measured. Before this, only the asf_health TOOL
|
|
46
46
|
// recorded: a human could run /asf health, read HEALTHY, and leave the
|
|
47
47
|
// completion gate's health gap open with no trace that anything was measured.
|
|
@@ -6,6 +6,7 @@
|
|
|
6
6
|
*/
|
|
7
7
|
|
|
8
8
|
import { projectName, runRoot } from "../paths.ts";
|
|
9
|
+
import { measureRoot } from "../run.ts";
|
|
9
10
|
import { loadState } from "../state.ts";
|
|
10
11
|
import { loadProjectSpecs, validateTrace } from "../specs.ts";
|
|
11
12
|
import { QA_CHECKLIST, type SpecItem } from "../types.ts";
|
|
@@ -56,7 +57,7 @@ async function traceabilitySection(): Promise<string[]> {
|
|
|
56
57
|
async function healthSection(): Promise<string[]> {
|
|
57
58
|
const { runHealth } = await import("../../code-health/index.mjs");
|
|
58
59
|
const { recordHealthVerdict, recordLine } = await import("../health-record.ts");
|
|
59
|
-
const health = await runHealth(
|
|
60
|
+
const health = await runHealth(await measureRoot(), []);
|
|
60
61
|
// Same rule as the tool and the slash command: a measurement that is produced
|
|
61
62
|
// is a measurement that is recorded.
|
|
62
63
|
const rec = recordHealthVerdict(health, { by: "verify" });
|
package/lib/extension/run.ts
CHANGED
|
@@ -4,16 +4,16 @@
|
|
|
4
4
|
*/
|
|
5
5
|
|
|
6
6
|
import { execFileSync } from "node:child_process";
|
|
7
|
+
import { runRoot, projectName } from "./paths.ts";
|
|
7
8
|
import { fileURLToPath } from "node:url";
|
|
8
9
|
import { dirname } from "node:path";
|
|
9
|
-
import {
|
|
10
|
+
import { projectRoot as projectRootOf } from "../project.mjs";
|
|
10
11
|
import { ensureHome } from "../home.mjs";
|
|
11
12
|
import { cpus, loadavg } from "node:os";
|
|
12
13
|
import { renderRunCard, cardSummary, CARD_CEILING_BYTES } from "../run-card.mjs";
|
|
13
14
|
import { ledger, suiteRuns, gateStats } from "./gate-state.ts";
|
|
14
15
|
import { appendEvidence } from "./evidence.ts";
|
|
15
16
|
import { loadState, saveState } from "./state.ts";
|
|
16
|
-
import { projectName } from "./paths.ts";
|
|
17
17
|
import { runtime } from "./runtime.ts";
|
|
18
18
|
import { __resetRefusals } from "../gates/refusals.mjs";
|
|
19
19
|
import { cardIndex, registerAt } from "../decisions/consult.mjs";
|
|
@@ -118,6 +118,43 @@ export async function openAutoRun(trigger: string, path: string | null): Promise
|
|
|
118
118
|
return state;
|
|
119
119
|
}
|
|
120
120
|
|
|
121
|
+
/**
|
|
122
|
+
* The project the gates should MEASURE.
|
|
123
|
+
*
|
|
124
|
+
* `runRoot()` is the git root of the session's cwd, which is right for where state
|
|
125
|
+
* lives and wrong for what to measure: a session started in a parent repository
|
|
126
|
+
* (a home directory that is itself a git repo is the common case) would have every
|
|
127
|
+
* gate measure that parent. It did — `asf_health` reported 363 model findings from an
|
|
128
|
+
* unrelated project checked out underneath $HOME, and every measurement tool then
|
|
129
|
+
* timed out scanning a tree nobody was working in. A gate that measures the wrong
|
|
130
|
+
* directory is worse than no gate: it blocks on findings in code the run must not
|
|
131
|
+
* touch, or waves through the one project that is actually unhealthy.
|
|
132
|
+
*
|
|
133
|
+
* The run already knows what it changed, so the answer is taken from there: the git
|
|
134
|
+
* root of the most recently touched file wins. It is deliberately NOT used to locate
|
|
135
|
+
* state — a root that moved mid-run would orphan the run's own ledger — only to
|
|
136
|
+
* decide what a measurement looks at.
|
|
137
|
+
*/
|
|
138
|
+
export async function measureRoot(): Promise<string> {
|
|
139
|
+
const fallback = runRoot();
|
|
140
|
+
try {
|
|
141
|
+
const state = await loadState(projectName());
|
|
142
|
+
const touched = (state?.current?.filesTouched || []) as string[];
|
|
143
|
+
for (let i = touched.length - 1; i >= 0; i--) {
|
|
144
|
+
const f = touched[i];
|
|
145
|
+
if (!f || !f.includes("/")) continue;
|
|
146
|
+
const root = projectRootOf(dirname(f));
|
|
147
|
+
// Only a root that CONTAINS the session root counts as a better answer: a
|
|
148
|
+
// stray touched path outside the workspace must not drag the measurement to
|
|
149
|
+
// someone's scratch directory.
|
|
150
|
+
if (root && (root === fallback || root.startsWith(fallback + "/"))) return root;
|
|
151
|
+
}
|
|
152
|
+
} catch {
|
|
153
|
+
/* measurement falls back to the session root; a broken guess is worse than none */
|
|
154
|
+
}
|
|
155
|
+
return fallback;
|
|
156
|
+
}
|
|
157
|
+
|
|
121
158
|
/** Where this extension is installed — the declaration file ships beside it. */
|
|
122
159
|
|
|
123
160
|
export function extensionDir(): string {
|
|
@@ -0,0 +1,134 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* asf_evidence_check — can this install produce what its gates demand?
|
|
3
|
+
*
|
|
4
|
+
* The invariant "every evidence kind a gate requires has a producer" is the difference
|
|
5
|
+
* between a warranted loop and the unwarranted one of 2026-11-07: a run steered
|
|
6
|
+
* forever at evidence its own hands cannot record. The repository asserts it in
|
|
7
|
+
* `test/test-evidence-kinds.mjs`, but `test/` is not shipped, so a session with a
|
|
8
|
+
* broken or half-updated install had no way to find out — it could only be steered,
|
|
9
|
+
* and steered, and steered, at a gap that was never closable from inside.
|
|
10
|
+
*
|
|
11
|
+
* This is that way. It reads the installed package's own files (never the user's
|
|
12
|
+
* project) and answers one question, so "the gate is stuck on something impossible"
|
|
13
|
+
* becomes a checkable fact instead of a guess.
|
|
14
|
+
*/
|
|
15
|
+
|
|
16
|
+
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
17
|
+
import { Type } from "typebox";
|
|
18
|
+
|
|
19
|
+
interface Coverage {
|
|
20
|
+
kinds: string[];
|
|
21
|
+
required: string[];
|
|
22
|
+
producers: Record<string, string[]>;
|
|
23
|
+
orphans: string[];
|
|
24
|
+
requiredOrphans: string[];
|
|
25
|
+
files: number;
|
|
26
|
+
}
|
|
27
|
+
|
|
28
|
+
/** One details shape for every branch: a union whose branches disagree about which
|
|
29
|
+
* fields exist makes the host read `error` as required and reject the tool. */
|
|
30
|
+
export function details() {
|
|
31
|
+
return {
|
|
32
|
+
action: "evidence-check",
|
|
33
|
+
ok: false,
|
|
34
|
+
error: "",
|
|
35
|
+
kind: "",
|
|
36
|
+
required: false,
|
|
37
|
+
producers: [] as string[],
|
|
38
|
+
kinds: 0,
|
|
39
|
+
orphans: [] as string[],
|
|
40
|
+
requiredOrphans: [] as string[],
|
|
41
|
+
requiredList: [] as string[],
|
|
42
|
+
};
|
|
43
|
+
}
|
|
44
|
+
|
|
45
|
+
async function coverage(): Promise<{ coverage: Coverage; gap: { gap: string; remedy: string } | null; root: string }> {
|
|
46
|
+
const { ASF_ROOT } = await import("../../code-health/tools.mjs");
|
|
47
|
+
const { ENTRY_KINDS } = await import("../../gates/ledger.mjs");
|
|
48
|
+
const { evidenceCoverage, coverageGap } = await import("../../gates/evidence-coverage.mjs");
|
|
49
|
+
const cov = evidenceCoverage(ASF_ROOT, ENTRY_KINDS) as Coverage;
|
|
50
|
+
return { coverage: cov, gap: coverageGap(cov) as { gap: string; remedy: string } | null, root: String(ASF_ROOT) };
|
|
51
|
+
}
|
|
52
|
+
|
|
53
|
+
/** The answer about one kind: who records it, or the honest fact that nobody does. */
|
|
54
|
+
export function reportKind(cov: Coverage, kind: string) {
|
|
55
|
+
const d = details();
|
|
56
|
+
d.kind = kind;
|
|
57
|
+
if (!cov.kinds.includes(kind)) {
|
|
58
|
+
return {
|
|
59
|
+
content: [{ type: "text" as const, text: `${kind} is not a ledger kind this install knows (${cov.kinds.join(", ")})` }],
|
|
60
|
+
details: d,
|
|
61
|
+
};
|
|
62
|
+
}
|
|
63
|
+
const producers = cov.producers[kind] || [];
|
|
64
|
+
const required = cov.required.includes(kind);
|
|
65
|
+
const text = producers.length
|
|
66
|
+
? `${kind}: produced by ${producers.join(", ")}${required ? " (required by a gate)" : ""}`
|
|
67
|
+
: `${kind}: NOTHING in the installed package records it${required ? " — and a gate requires it, so no run can close on it" : " (nothing requires it either: dead weight)"}`;
|
|
68
|
+
return {
|
|
69
|
+
content: [{ type: "text" as const, text }],
|
|
70
|
+
details: { ...d, ok: producers.length > 0, producers, required },
|
|
71
|
+
};
|
|
72
|
+
}
|
|
73
|
+
|
|
74
|
+
/** The whole picture, said plainly: sound, or broken in a way no effort can fix. */
|
|
75
|
+
export function reportAll(cov: Coverage, gap: { gap: string; remedy: string } | null, root: string) {
|
|
76
|
+
const d = details();
|
|
77
|
+
const text = gap
|
|
78
|
+
? `ASF install at ${root}: ${gap.gap}\n${gap.remedy}\n\n${cov.kinds.length} kinds over ${cov.files} files; required by gates: ${cov.required.join(", ")}`
|
|
79
|
+
: `ASF install at ${root} is sound: ${cov.kinds.length} evidence kinds over ${cov.files} files, each with a producer; ${cov.required.length} of them are required by a gate (${cov.required.join(", ")}).`;
|
|
80
|
+
return {
|
|
81
|
+
content: [{ type: "text" as const, text }],
|
|
82
|
+
details: {
|
|
83
|
+
...d,
|
|
84
|
+
ok: !gap,
|
|
85
|
+
kinds: cov.kinds.length,
|
|
86
|
+
required: cov.required.length > 0,
|
|
87
|
+
requiredList: cov.required,
|
|
88
|
+
orphans: cov.orphans,
|
|
89
|
+
requiredOrphans: cov.requiredOrphans,
|
|
90
|
+
},
|
|
91
|
+
};
|
|
92
|
+
}
|
|
93
|
+
|
|
94
|
+
/**
|
|
95
|
+
* A check that could not run is not a check that passed. This is its own function so
|
|
96
|
+
* the claim is testable: a mutation that made the catch branch report `ok: true` was
|
|
97
|
+
* invisible while nothing ever drove the failure path.
|
|
98
|
+
*/
|
|
99
|
+
export function reportFailure(error: string) {
|
|
100
|
+
return {
|
|
101
|
+
content: [{ type: "text" as const, text: `evidence coverage could not be checked: ${error}` }],
|
|
102
|
+
details: { ...details(), error },
|
|
103
|
+
};
|
|
104
|
+
}
|
|
105
|
+
|
|
106
|
+
export function registerEvidenceCheckTool(pi: ExtensionAPI): void {
|
|
107
|
+
pi.registerTool({
|
|
108
|
+
name: "asf_evidence_check",
|
|
109
|
+
label: "ASF Evidence Coverage",
|
|
110
|
+
description:
|
|
111
|
+
"Check whether the installed ASF can actually produce every kind of evidence its gates demand. " +
|
|
112
|
+
"Call this when a run keeps being steered at the same evidence gap and you cannot close it — an " +
|
|
113
|
+
"unproducible kind is a broken install, not unfinished work, and no amount of effort closes it. " +
|
|
114
|
+
"Reads ASF's own installed files, never the user's project.",
|
|
115
|
+
promptSnippet: "Check that every gate-required evidence kind has a producer in this install",
|
|
116
|
+
parameters: Type.Object({
|
|
117
|
+
kind: Type.Optional(
|
|
118
|
+
Type.String({
|
|
119
|
+
description: "Report only this evidence kind (e.g. 'browser-verify'); omit for the whole picture.",
|
|
120
|
+
}),
|
|
121
|
+
),
|
|
122
|
+
}),
|
|
123
|
+
|
|
124
|
+
async execute(_toolCallId: string, params: { kind?: unknown }) {
|
|
125
|
+
try {
|
|
126
|
+
const { coverage: cov, gap, root } = await coverage();
|
|
127
|
+
const kind = typeof params.kind === "string" ? params.kind.trim() : "";
|
|
128
|
+
return kind ? reportKind(cov, kind) : reportAll(cov, gap, root);
|
|
129
|
+
} catch (e) {
|
|
130
|
+
return reportFailure(String((e as Error)?.message || e));
|
|
131
|
+
}
|
|
132
|
+
},
|
|
133
|
+
});
|
|
134
|
+
}
|
|
@@ -27,12 +27,12 @@ interface HealthResult {
|
|
|
27
27
|
|
|
28
28
|
async function runHealthGate(scope: string): Promise<HealthResult> {
|
|
29
29
|
const { runHealth } = await import("../../code-health/index.mjs");
|
|
30
|
-
const {
|
|
30
|
+
const { measureRoot } = await import("../run.ts");
|
|
31
31
|
const args = scope ? [`--scope=${scope}`] : [];
|
|
32
|
-
// Measure the RUN
|
|
33
|
-
//
|
|
34
|
-
//
|
|
35
|
-
return (await runHealth(
|
|
32
|
+
// Measure the project the RUN is about, not process.cwd() and not the session
|
|
33
|
+
// cwd's git root: a session started in $HOME (which is itself a repo here) would
|
|
34
|
+
// otherwise measure every checkout underneath it.
|
|
35
|
+
return (await runHealth(await measureRoot(), args)) as HealthResult;
|
|
36
36
|
}
|
|
37
37
|
|
|
38
38
|
/**
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
export interface EvidenceCoverage {
|
|
2
|
+
kinds: string[];
|
|
3
|
+
required: string[];
|
|
4
|
+
producers: Record<string, string[]>;
|
|
5
|
+
orphans: string[];
|
|
6
|
+
requiredOrphans: string[];
|
|
7
|
+
files: number;
|
|
8
|
+
}
|
|
9
|
+
|
|
10
|
+
export interface CoverageGap {
|
|
11
|
+
gap: string;
|
|
12
|
+
remedy: string;
|
|
13
|
+
}
|
|
14
|
+
|
|
15
|
+
/** Does every ledger kind have a producer in the package at `root`? */
|
|
16
|
+
export function evidenceCoverage(root: string, entryKinds: string[]): EvidenceCoverage;
|
|
17
|
+
|
|
18
|
+
/** The coverage result as a completion gap, or null when the install is sound. */
|
|
19
|
+
export function coverageGap(coverage: EvidenceCoverage): CoverageGap | null;
|
|
@@ -0,0 +1,124 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Does every piece of evidence a gate demands have something that can produce it?
|
|
3
|
+
*
|
|
4
|
+
* Three tickets in a row were one bug wearing different clothes: the health gate
|
|
5
|
+
* demanded a verdict only a pi slash command could produce; the completion gate
|
|
6
|
+
* demanded browser verification no registered tool could record; the delivery report
|
|
7
|
+
* rendered sections whose kinds nothing ever wrote. Each time a run was steered at a
|
|
8
|
+
* gap its own hands could not close — the unwarranted loop the process rules forbid.
|
|
9
|
+
*
|
|
10
|
+
* The invariant is stated here, once, as a pure function over a package root, because
|
|
11
|
+
* three different callers need to ask the same question and a second definition would
|
|
12
|
+
* drift from the first:
|
|
13
|
+
*
|
|
14
|
+
* - `test/test-evidence-kinds.mjs` asserts it against the repository;
|
|
15
|
+
* - `scripts/verify-published.mjs` asserts it against the copy npm just installed
|
|
16
|
+
* (the test directory is not shipped, so this is the only place the installed
|
|
17
|
+
* artifact's coverage can be checked);
|
|
18
|
+
* - `asf_evidence_check` answers it at runtime, so a session that suspects a broken
|
|
19
|
+
* install can ask instead of guessing.
|
|
20
|
+
*
|
|
21
|
+
* `root` is the package directory, not a project: this is a question about ASF's own
|
|
22
|
+
* shipped code, and asking it about a user's repository would be meaningless.
|
|
23
|
+
*/
|
|
24
|
+
|
|
25
|
+
import fs from "node:fs";
|
|
26
|
+
import path from "node:path";
|
|
27
|
+
|
|
28
|
+
/** Files that could create a ledger entry. `.d.mts` are declarations, not code. */
|
|
29
|
+
function sourceFiles(root) {
|
|
30
|
+
const out = [];
|
|
31
|
+
const walk = (dir) => {
|
|
32
|
+
let entries;
|
|
33
|
+
try {
|
|
34
|
+
entries = fs.readdirSync(path.join(root, dir), { withFileTypes: true });
|
|
35
|
+
} catch {
|
|
36
|
+
return;
|
|
37
|
+
}
|
|
38
|
+
for (const e of entries) {
|
|
39
|
+
const rel = dir ? `${dir}/${e.name}` : e.name;
|
|
40
|
+
if (e.isDirectory()) walk(rel);
|
|
41
|
+
else if (/\.(ts|mjs)$/.test(e.name) && !e.name.endsWith(".d.mts")) out.push(rel);
|
|
42
|
+
}
|
|
43
|
+
};
|
|
44
|
+
walk("lib");
|
|
45
|
+
if (fs.existsSync(path.join(root, "index.ts"))) out.push("index.ts");
|
|
46
|
+
return out;
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
/**
|
|
50
|
+
* The kinds a gate actually blocks on.
|
|
51
|
+
*
|
|
52
|
+
* Derived from the two files that make completion decisions, not from a list someone
|
|
53
|
+
* maintains: a hand-kept list rots the first time a gate starts requiring a kind and
|
|
54
|
+
* nobody updates the list next door.
|
|
55
|
+
*/
|
|
56
|
+
function requiredKinds(root, entryKinds) {
|
|
57
|
+
let text = "";
|
|
58
|
+
for (const rel of ["lib/gates/ledger.mjs", "lib/gates/settle.mjs"]) {
|
|
59
|
+
try {
|
|
60
|
+
text += fs.readFileSync(path.join(root, rel), "utf8") + "\n";
|
|
61
|
+
} catch {
|
|
62
|
+
/* a missing gate file is reported by the shipped-file check, not here */
|
|
63
|
+
}
|
|
64
|
+
}
|
|
65
|
+
const required = new Set(["suite-run", "health-run"]);
|
|
66
|
+
for (const m of text.matchAll(/(?:has|kind ===)\s*\(\s*"([a-z-]+)"/g)) required.add(m[1]);
|
|
67
|
+
for (const m of text.matchAll(/"([a-z-]+)"/g)) {
|
|
68
|
+
const k = m[1];
|
|
69
|
+
if (entryKinds.includes(k) && new RegExp(`(has|missing|latest|kind ===)[^\\n]*"${k}"`).test(text)) required.add(k);
|
|
70
|
+
}
|
|
71
|
+
return required;
|
|
72
|
+
}
|
|
73
|
+
|
|
74
|
+
/**
|
|
75
|
+
* @param {string} root the ASF package directory (contains lib/ and index.ts)
|
|
76
|
+
* @param {string[]} entryKinds ENTRY_KINDS from the ledger — passed in rather than
|
|
77
|
+
* imported so this stays a question about files, and so a
|
|
78
|
+
* caller can ask it of any version's package directory
|
|
79
|
+
* @returns {{orphans: string[], requiredOrphans: string[], producers: Record<string,string[]>,
|
|
80
|
+
* required: string[], kinds: string[], files: number}}
|
|
81
|
+
*/
|
|
82
|
+
export function evidenceCoverage(root, entryKinds) {
|
|
83
|
+
// ledger.mjs only defines, validates and renders its own kinds, so counting it as a
|
|
84
|
+
// producer would let the ledger certify itself.
|
|
85
|
+
const files = sourceFiles(root).filter((f) => !f.endsWith("gates/ledger.mjs"));
|
|
86
|
+
const sources = files.map((f) => ({ f, src: fs.readFileSync(path.join(root, f), "utf8") }));
|
|
87
|
+
const producers = {};
|
|
88
|
+
for (const kind of entryKinds) {
|
|
89
|
+
producers[kind] = sources.filter(({ src }) => new RegExp(`kind:\\s*"${kind}"`).test(src)).map(({ f }) => f);
|
|
90
|
+
}
|
|
91
|
+
const orphans = entryKinds.filter((k) => producers[k].length === 0);
|
|
92
|
+
const required = [...requiredKinds(root, entryKinds)];
|
|
93
|
+
return {
|
|
94
|
+
kinds: [...entryKinds],
|
|
95
|
+
required,
|
|
96
|
+
producers,
|
|
97
|
+
orphans,
|
|
98
|
+
requiredOrphans: required.filter((k) => !entryKinds.includes(k) || producers[k].length === 0),
|
|
99
|
+
files: files.length,
|
|
100
|
+
};
|
|
101
|
+
}
|
|
102
|
+
|
|
103
|
+
/**
|
|
104
|
+
* A coverage result expressed as a gap, or null when the install is sound.
|
|
105
|
+
*
|
|
106
|
+
* An orphan a gate requires is the serious case: the run is steered at evidence it has
|
|
107
|
+
* no hands to produce, which is the loop this module exists to prevent. An orphan
|
|
108
|
+
* nothing requires is dead weight — reported, never blocking.
|
|
109
|
+
*/
|
|
110
|
+
export function coverageGap(coverage) {
|
|
111
|
+
if (coverage.requiredOrphans.length) {
|
|
112
|
+
return {
|
|
113
|
+
gap: `the completion gate requires evidence of kind ${coverage.requiredOrphans.join(", ")}, but nothing in the installed package can record it`,
|
|
114
|
+
remedy: "that evidence can never be produced, so no run can close on it — this is a broken ASF install, not unfinished work",
|
|
115
|
+
};
|
|
116
|
+
}
|
|
117
|
+
if (coverage.orphans.length) {
|
|
118
|
+
return {
|
|
119
|
+
gap: `ledger kinds ${coverage.orphans.join(", ")} have no producer in the installed package`,
|
|
120
|
+
remedy: "reported, not blocking: nothing requires them, so they are dead weight rather than an unclosable gap",
|
|
121
|
+
};
|
|
122
|
+
}
|
|
123
|
+
return null;
|
|
124
|
+
}
|
package/lib/home.mjs
CHANGED
|
@@ -41,6 +41,7 @@ export const GITIGNORE_BODY = "# ASF telemetry: append-only, machine-local, neve
|
|
|
41
41
|
export const FILES = {
|
|
42
42
|
config: { home: "config.json", legacy: ".asf-config.json" },
|
|
43
43
|
baseline: { home: "code-health-baseline.json", legacy: ".asf-code-health-baseline.json" },
|
|
44
|
+
healthConfig: { home: "code-health.json", legacy: ".asf-code-health.json" },
|
|
44
45
|
plan: { home: "PLAN.md", legacy: "PLAN.md" },
|
|
45
46
|
};
|
|
46
47
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-aia-asf",
|
|
3
|
-
"version": "0.17.
|
|
3
|
+
"version": "0.17.12",
|
|
4
4
|
"description": "Ai Applied Agentic Software Factory — codifies the full software development flow: intake, research, spec capture, adversarial analysis, planning with approval gates, test-first implementation, and release. Requires pi-vigilant, pi-smart-web-search, pi-smart-fetch, and pi-aia-browser.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"pi-package",
|
|
@@ -166,5 +166,34 @@ installs anything.
|
|
|
166
166
|
3. **The gate is a floor, not a ceiling.** Passing thresholds does not mean
|
|
167
167
|
the code is good — 06c rules still apply.
|
|
168
168
|
4. **Never disable the gate to ship.** If a metric is genuinely inapplicable,
|
|
169
|
-
disable that metric in `.asf
|
|
170
|
-
|
|
169
|
+
disable that metric in `.asf/code-health.json` (the old root
|
|
170
|
+
`.asf-code-health.json` is still read, so projects configured before the move
|
|
171
|
+
keep their thresholds) with a comment in the commit message, not the whole gate.
|
|
172
|
+
|
|
173
|
+
## When the gate itself is the problem: `asf_evidence_check`
|
|
174
|
+
|
|
175
|
+
Every rule above has the same shape: the gate demands evidence, and there is an agent
|
|
176
|
+
action that produces it. When that symmetry breaks, the run does not fail — it is
|
|
177
|
+
steered, and steered, and steered, at a gap its own hands cannot close. That is the
|
|
178
|
+
2026-11-07 loop, and it has reappeared three times in different clothes: a health
|
|
179
|
+
verdict only a slash command could record, browser verification no tool could write, a
|
|
180
|
+
delivery report rendering sections nothing ever produced.
|
|
181
|
+
|
|
182
|
+
So the symmetry is now checked rather than hoped for. `asf_evidence_check` reads
|
|
183
|
+
ASF's own installed files and answers one question: does every evidence kind a gate
|
|
184
|
+
requires have something in the package that can record it?
|
|
185
|
+
|
|
186
|
+
```
|
|
187
|
+
asf_evidence_check # whole picture: sound, or broken and how
|
|
188
|
+
asf_evidence_check { kind: "browser-verify" } # who records this kind, or nobody
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
**If it reports an unproducible required kind, stop trying to close the gap.** That is
|
|
192
|
+
a broken install, not unfinished work — no amount of effort produces evidence nothing
|
|
193
|
+
in the package can write. Say so, and say what it named.
|
|
194
|
+
|
|
195
|
+
The same invariant is asserted against the repository in
|
|
196
|
+
`test/test-evidence-kinds.mjs`, and against the copy npm actually serves in
|
|
197
|
+
`scripts/verify-published.mjs` — the test directory is not shipped, so without that
|
|
198
|
+
last one a broken tarball would reproduce the loop for every user while the repo
|
|
199
|
+
stayed green.
|