@vantageos/mcp-doctor 0.1.0 → 0.2.0

package/CHANGELOG.md

@@ -0,0 +1,36 @@
+# Changelog
+
+All notable changes to `@vantageos/mcp-doctor` are documented here.
+This project adheres to [Semantic Versioning](https://semver.org/).
+
+## 0.2.0 — 2026-08-16
+
+The engine gains a live-session contract and an honest exit code; a second
+doctor can now consume it as a dependency rather than copy it.
+
+### Added
+- **Live-session contract** (additive, no existing detector changed): `Session`
+  (`listTools` / `discover` / `callTool`), `LiveDetector` (`runLive(session)`),
+  `isLiveDetector`, and the async runner `runDoctorAsync(target, { session?, detectorList? })`.
+  A control that must call a running server is now expressible. A session that is
+  unreachable, times out, or refuses returns `could-not-judge` with a reason —
+  never a pass, never a violation. All four states are exported.
+- **`computeExitCode`** and a result-driven process exit in the CLI.
+
+### Changed — observable contract
+- **Exit code.** The CLI previously exited `0` on every run. It now derives the
+  code from the results: **`1`** if any result is `non-conforming` (precedence),
+  else **`2`** if any result is `could-not-judge`, else **`0`**.
+
+  **What exit `0` means, exactly:** no violation and no abstention was *reached*.
+  It is **not** a certificate of conformance — the same `0` is produced by a tree
+  the doctor cannot read at all, and by one whose rules are all `absent` (nothing
+  of that kind present to judge). A consumer must read `0` as "nothing raised the
+  code", not as "conformant". A judged-conformant demonstration tree is tracked
+  as follow-up; until then the `0` pole is guarded by the unit suite, not by a
+  fixture.
+
+### Notes
+- The fourth state `could-not-judge` (a detector that could not read what it was
+  pointed at, with a required reason) is distinct from `absent`; the exit code
+  now lets a machine caller tell an abstention from a clean run.

package/README.md

@@ -59,6 +59,10 @@ Each detector maps one standard item and is proven **bipolar** on real material:
 | **D5** server/discover | a `server/discover` handler is registered (absence-based, with a positive control) | present / absent |
 | **D6** mcp-apps ui-key | MCP Apps UI key is the GA nested `_meta.ui.resourceUri` — not flat `ui/resourceUri`, not community `io.modelcontextprotocol/ui` | GA-nested / flat-deprecated / community-nonGA / none |
 | **D7** protocol-kit | `@vantageos/mcp-protocol-kit` listed in `package.json` **and actually called** (builder invoked, not merely declared) | listed+called / listed-not-called / hand-rolled |
+| **D15** cache-coherence | Server Rule 15 — every tool-set mutation (RegisteredTool `.enable/.disable/.update/.remove`, `unregisterTool`) is correlated with a `notifications/tools/list_changed` emission | present / absent / non-conforming / could-not-judge |
+| **D16** version-observable | Server Rule 16 — `server/discover` (or an HTTP `/health` route) exposes a `version` field sourced from `package.json`, reachable pre-auth | present / absent / non-conforming / could-not-judge |
+
+The registered detector set is the source of truth (`src/detectors/index.ts`); the table above is illustrative, not the count. **Adding a detector is how a friction becomes durable — see [CONTRIBUTING.md](./CONTRIBUTING.md) for the loop.**
 
 ## Positioning
 

package/dist/doctor.d.ts

@@ -1,14 +1,206 @@
 #!/usr/bin/env node
-type State = "present" | "absent" | "non-conforming";
+interface SourceLine {
+    n: number;
+    raw: string;
+    /** true when the line is a comment-only line (JSDoc star, slash-slash, block open/close). */
+    comment: boolean;
+}
+interface SourceFile {
+    path: string;
+    rel: string;
+    lines: SourceLine[];
+}
+interface Tree {
+    root: string;
+    files: SourceFile[];
+    packageJson: {
+        path: string;
+        json: Record<string, unknown>;
+    } | null;
+}
+
+type State = "present" | "absent" | "non-conforming" | "could-not-judge";
 interface DetectorResult {
     id: string;
     /** display anchor(s) file:line — the site the verdict is about. */
     anchor: string;
     state: State;
     verdict: string;
+    /**
+     * Machine-readable reason the detector could not decide. REQUIRED whenever
+     * `state === "could-not-judge"`: a could-not-judge with no reason is a shrug,
+     * and the runner refuses it. Ignored for the three decisive states.
+     */
+    reason?: string;
+}
+/** A single tool as reported by the RUNNING server over the MCP `tools/list` call. */
+interface RuntimeTool {
+    /** The tool name the server actually advertises at runtime (post-startup). */
+    name: string;
+    /**
+     * Number of eval cases covering this tool, resolved by the runtime builder
+     * from the target's `evals/evals.json` (matched by tool name). Optional
+     * because the eval file may be absent; a tool present at runtime with no
+     * eval file has `evalCaseCount` unset, which D8 reads as zero coverage.
+     */
+    evalCaseCount?: number;
+}
+/**
+ * What the RUNNING target server exposes, gathered by launching it and speaking
+ * MCP over stdio (`initialize` -> `initialized` -> `tools/list`). This is the
+ * surface the static scan cannot see: names computed at startup, tools shipped
+ * only as built `dist` with no readable source.
+ *
+ * `launchError` is set (and `tools` is empty) whenever the server could not be
+ * started or introspected — a spawn failure, a handshake failure, a timeout.
+ * A dynamic detector reads a set `launchError` as `could-not-judge` (naming the
+ * failure), NEVER as a pass and NEVER as absent.
+ */
+interface RuntimeContext {
+    /**
+     * The target root the runtime was built from. Carried so a dynamic detector
+     * can locate static co-artifacts (e.g. D8's `evals/evals.json`) without a
+     * second argument — the decided `runDynamic(runtime)` signature stays single.
+     */
+    root: string;
+    tools: RuntimeTool[];
+    launchError?: string;
 }
+interface Detector {
+    id: string;
+    /** Static verdict from the source tree. REQUIRED — static is never optional. */
+    run(tree: Tree): DetectorResult;
+    /**
+     * Optional dynamic verdict from the RUNNING server. A detector that can judge
+     * something only visible at runtime (a computed tool name, the real tool set)
+     * implements this; detectors without it run static-only in every mode.
+     */
+    runDynamic?(runtime: RuntimeContext): DetectorResult;
+}
+/**
+ * A minimal handle to a RUNNING MCP session — a live client the consumer has
+ * already connected (`initialize` -> `initialized`). Where `RuntimeContext` is
+ * a one-shot SNAPSHOT (launch the server, read `tools/list` once), a `Session`
+ * is an INTERACTIVE handle a detector drives: it can call tools and read served
+ * responses at will. This is the surface the ~21 MCP + 4 backend *dynamique*
+ * rules need — the ones a static tree read and a single tool-list snapshot both
+ * cannot express (served-response conformance, pagination, auth challenge).
+ *
+ * Kept deliberately small and extensible: three verbs — `listTools` (the served
+ * `tools/list` payload), `discover` (the served discovery / description payload),
+ * `callTool` (one tool invoked, its served response or auth challenge read).
+ * These reach the *dynamique* rules expressible by an argument-less list read, a
+ * discovery read, and a single single-identity tool call — e.g. S-2 / S-4 served
+ * responses and C-19 served descriptions.
+ *
+ * What these three verbs deliberately do NOT yet reach — named so the surface is
+ * not credited in advance for a rule it cannot express:
+ *  - S-12 (pagination): `listTools()` takes no cursor, so a page cannot be
+ *    requested — needs a cursor argument.
+ *  - S-3 (`CacheableResult` on the five list/read surfaces): needs the four
+ *    non-tool list/read surfaces, not only `tools/list`.
+ *  - S-6 (tools/list coherence): needs an emitted `list_changed` notification to
+ *    observe across an emission, not a single snapshot.
+ *  - S-5 (OAuth 2.1 resource server) and backend R-6 / R-11: need the same call made
+ *    under varied credentials / as two principals — `callTool` carries no
+ *    identity.
+ * A rule that needs one of these adds its verb here in one additive edit; the
+ * surface stays honest about what it can read today, and nothing is pre-invented
+ * for a rule that does not yet read it.
+ *
+ * Every method is async and MAY reject (connection refused, timeout, protocol
+ * error). A `LiveDetector` reading a rejecting session returns `could-not-judge`
+ * naming the failure — never a pass, never a violation — exactly as a dynamic
+ * detector reads a set `launchError`.
+ */
+interface Session {
+    /** The served `tools/list` payload (post-startup, as the client receives it). */
+    listTools(): Promise<unknown>;
+    /** The served discovery / description payload the client reads at connect. */
+    discover(): Promise<unknown>;
+    /** Invoke one tool and read the served response (or the auth challenge it returns). */
+    callTool(name: string, args: unknown): Promise<unknown>;
+}
+/**
+ * A detector whose verdict requires CALLING a running server — a live session,
+ * not a static tree and not a one-shot snapshot. It is a SEPARATE contract from
+ * `Detector`: the async runner routes an object to `runLive` iff it carries one,
+ * and to `run(tree)` otherwise. No existing `Detector` is touched or migrated.
+ */
+interface LiveDetector {
+    id: string;
+    /**
+     * Live verdict from a running session. REQUIRED for a live detector. It owns
+     * the abstain contract: a session it cannot read (absent, throwing, timing
+     * out before anything is read) yields `could-not-judge` with a non-empty
+     * `reason` — never a pass, never a violation, both directions.
+     */
+    runLive(session: Session): Promise<DetectorResult>;
+}
+/**
+ * Guard: an object is a `LiveDetector` iff it exposes a `runLive` function. The
+ * domain of "kinds of detector" is exactly two (tree `run`, live `runLive`);
+ * this guard covers it, and the async runner routes on it.
+ */
+declare function isLiveDetector(d: Detector | LiveDetector): d is LiveDetector;
 
-declare function runDoctor(target: string): DetectorResult[];
+type RunMode = "static" | "dynamic" | "both";
+interface RunDoctorOptions {
+    mode?: RunMode;
+}
+/**
+ * Run every detector against the target.
+ *
+ * Modes (default `static`):
+ *  - `static`  — the default; every detector's `run(tree)`. Behaviour and
+ *    output are IDENTICAL to before dynamic mode existed.
+ *  - `dynamic` — the running server is introspected once; a dynamic-capable
+ *    detector runs `runDynamic(runtime)`, every other detector runs `run(tree)`.
+ *  - `both`    — static for all detectors PLUS dynamic for the dynamic-capable;
+ *    when a rule's static and dynamic `state` DIFFER, a distinct disagreement
+ *    result is emitted naming BOTH states and BOTH anchors. The modes never
+ *    disagree silently.
+ *
+ * The runner still fails LOUD on its own coverage: a detector that throws aborts
+ * the run; a could-not-judge with no reason is refused; every detector produces
+ * at least one well-formed result.
+ *
+ * `detectorList` is injectable for tests; production uses the full registry.
+ */
+declare function runDoctor(target: string, detectorList?: Detector[], opts?: RunDoctorOptions): DetectorResult[];
+/**
+ * Async runner — the ADDITIVE extension for *dynamique* rules that must CALL a
+ * running server. It is a superset of the sync `runDoctor` static path, never a
+ * replacement:
+ *  - a tree `Detector` runs via `run(tree)`, byte-identical to `runDoctor`;
+ *  - a `LiveDetector` runs via `await runLive(session)`;
+ *  - a live detector with no session, or whose session throws/times out/refuses
+ *    before reading anything, yields `could-not-judge` with a non-empty reason
+ *    (never a pass, never a violation, both directions).
+ *
+ * The fail-loud discipline is identical: a tree detector that throws aborts the
+ * run naming the detector; a could-not-judge with no reason is refused; the
+ * coverage-count check asserts exactly one result per detector.
+ *
+ * The sync `runDoctor`, every registered detector, and `scan.ts` are untouched
+ * — a tree-only `runDoctorAsync(target)` returns exactly what `runDoctor(target)`
+ * returns.
+ */
+declare function runDoctorAsync(target: string, opts?: {
+    session?: Session;
+    detectorList?: (Detector | LiveDetector)[];
+}): Promise<DetectorResult[]>;
 declare function renderTable(surface: string, results: DetectorResult[]): string;
+/**
+ * The process exit code a run's results carry, three-way with precedence:
+ *   1 — a VIOLATION: at least one detector is `non-conforming` (something IS
+ *       wrong). Takes precedence over an abstention.
+ *   2 — an ABSTENTION with zero violations: at least one `could-not-judge`
+ *       ("I did not look" must NOT reach a caller as a clean 0).
+ *   0 — fully judged and conformant: nothing non-conforming, nothing abstained.
+ * The rendered table already draws this distinction; this only lets a machine
+ * reading the exit code (CI, the foreign-bite probes) see what the table says.
+ */
+declare function computeExitCode(results: DetectorResult[]): number;
 
-export { renderTable, runDoctor };
+export { type LiveDetector, type RunDoctorOptions, type RunMode, type Session, computeExitCode, isLiveDetector, renderTable, runDoctor, runDoctorAsync };