mcp-vitals 0.1.0

package/CHANGELOG.md

@@ -0,0 +1,33 @@
+# Changelog
+
+All notable changes to this project are documented here. The format is based on
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project
+adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.1.0] — 2026-06-20
+
+Initial release.
+
+### Added
+
+- `bench` — latency-benchmark a tool or no-arg probe: warmup + N iterations (or
+  `--duration`), closed-model concurrency (`-c`) or open-model arrival rate
+  (`--rps`), reporting min/mean/p50/p90/p95/p99/max/stddev, cold-start,
+  throughput, and error rate. Inline SLA gating via `--fail-on 'p95<200ms'`.
+- `check` — run a committed `mcp-vitals.{yaml,yml,json}` assertion suite
+  (capability presence, inputSchema validity, latency SLAs) with JUnit/JSON
+  reporters and distinct exit codes — the CI gate.
+- `inspect` — list tools/resources/prompts and validate every tool's
+  `inputSchema` is valid JSON Schema.
+- `call` — invoke one tool with timing, `--raw` and `--expect-error` modes.
+- `ping` — handshake/initialize latency and liveness, single or distribution.
+- Transports: stdio, Streamable HTTP, and SSE (with automatic HTTP→SSE
+  fallback), header-based auth, and stdio env injection.
+- `--json` single-object output on every command, with strict stdout/stderr
+  discipline so `| jq` always works.
+- Published JSON Schema for `mcp-vitals.yaml` editor IntelliSense.
+
+[Unreleased]: https://github.com/shaxzodbek-uzb/mcp-vitals/compare/v0.1.0...HEAD
+[0.1.0]: https://github.com/shaxzodbek-uzb/mcp-vitals/releases/tag/v0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Shaxzodbek Sobirov / Blaze
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,222 @@
+# mcp-vitals
+
+> **Vital signs for your MCP server.** Inspect capabilities, **benchmark tool-call latency (p50/p95/p99)**, and **assert health in CI** — the `ab` / `k6` / `pytest` for [Model Context Protocol](https://modelcontextprotocol.io) servers. Non-interactive, scriptable, no LLM key required.
+
+[![CI](https://github.com/shaxzodbek-uzb/mcp-vitals/actions/workflows/ci.yml/badge.svg)](https://github.com/shaxzodbek-uzb/mcp-vitals/actions/workflows/ci.yml)
+[![npm](https://img.shields.io/npm/v/mcp-vitals.svg)](https://www.npmjs.com/package/mcp-vitals)
+[![license](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+
+```bash
+# Benchmark a tool's latency — no install, no API key
+npx mcp-vitals bench --tool search --args '{"q":"hello"}' npx your-mcp-server
+```
+
+```
+Target: search (tool)
+Server: npx your-mcp-server via stdio
+Load:   50 iters, concurrency 1, warmup 1
+
+Cold start  41.8 ms
+
+   min    mean     p50     p90     p95     p99     max  stddev
+ 6.1 ms  8.0 ms  7.6 ms  9.9 ms  11.4 ms  18.7 ms  22.0 ms  2.4 ms
+
+50 completed · 124.6 req/s · 0 errors
+```
+
+---
+
+## Why mcp-vitals?
+
+There are great tools for *exploring* an MCP server interactively. There is nothing for **measuring** one and **gating CI** on the result.
+
+| | [Inspector](https://github.com/modelcontextprotocol/inspector) | [mcpjam](https://github.com/MCPJam/inspector) | [mcp-probe](https://github.com/conikeec/mcp-probe) | **mcp-vitals** |
+|---|:---:|:---:|:---:|:---:|
+| Inspect tools / resources / prompts | ✅ | ✅ | ✅ | ✅ |
+| One-shot tool call | ✅ | ✅ | ✅ | ✅ |
+| **Latency percentiles (p50/p95/p99)** | ❌ | ❌ | ❌ | **✅** |
+| **Concurrency / throughput load** | ❌ | ❌ | ❌ | **✅** |
+| **Gate a PR on a latency budget** | ❌ | ❌ | ❌ | **✅** |
+| LLM-free (no API key) | ✅ | ❌ | ✅ | ✅ |
+| `npx`-installable | ✅ | ✅ | cargo | ✅ |
+| JUnit + JSON reporters | ❌ | ✅ | partial | ✅ |
+
+mcp-vitals measures the **real `tools/call` round-trip** with `process.hrtime` — pure protocol latency, no model inference mixed in — and lets you commit a `mcp-vitals.yaml` that fails the build when a tool goes missing, ships an invalid schema, or blows its latency budget.
+
+## Install
+
+```bash
+npx mcp-vitals --help        # zero-install
+npm i -g mcp-vitals          # or install globally
+```
+
+Requires Node.js ≥ 20.
+
+## Commands
+
+mcp-vitals connects to any MCP server over **stdio** (a launch command), **Streamable HTTP**, or **SSE** (`--url`).
+
+> Put mcp-vitals options *before* the server command: `mcp-vitals bench --tool x -n 100 npx your-server`.
+
+### `bench` — latency benchmark (the headline)
+
+```bash
+# 200 iterations, 5 warmup, fail the command if p95 exceeds 200ms
+mcp-vitals bench --tool search --args '{"q":"test"}' \
+  -n 200 -w 5 --fail-on 'p95<200ms' --fail-on 'errorRate<=0' \
+  npx your-mcp-server
+
+# Load test: keep 10 calls in flight
+mcp-vitals bench --tool search -c 10 -n 500 npx your-mcp-server
+
+# Baseline raw transport overhead with a no-arg probe
+mcp-vitals bench --probe listTools npx your-mcp-server
+```
+
+Reports `min / mean / p50 / p90 / p95 / p99 / max / stddev`, cold-start, throughput, and error rate. `--json` emits the full distribution for dashboards.
+
+### `check` — the CI gate
+
+Commit a `mcp-vitals.yaml` next to your server and run one line in CI:
+
+```bash
+mcp-vitals check --junit report.xml
+```
+
+```
+Capabilities
+CHECK        EXPECTED       ACTUAL   STATUS
+tools/search tools present  present  PASS
+
+Latency SLAs
+CHECK            EXPECTED         ACTUAL    STATUS
+search/p95       p95 <= 200 ms    182 ms    PASS
+search/errorRate errorRate <= 0%  0.0%      PASS
+
+3 passed, 0 failed, 0 skipped in 4.21s
+```
+
+See [the assertions file format](#assertions-file-mcp-vitalsyaml) below.
+
+### `inspect` — discover & validate
+
+```bash
+mcp-vitals inspect npx your-mcp-server          # capabilities + tools/resources/prompts
+mcp-vitals inspect --json npx your-mcp-server    # machine-readable
+```
+
+Lists everything the server exposes and **validates every tool's `inputSchema`** is valid JSON Schema (exit 2 on any invalid).
+
+### `call` — invoke one tool
+
+```bash
+mcp-vitals call --tool search --args '{"q":"hello"}' npx your-mcp-server
+mcp-vitals call --tool search --args @query.json --raw npx your-mcp-server | jq
+```
+
+### `ping` — handshake latency / liveness
+
+```bash
+mcp-vitals ping npx your-mcp-server              # connected in 142 ms
+mcp-vitals ping -n 20 npx your-mcp-server        # distribution over 20 handshakes
+```
+
+## Transports & auth
+
+```bash
+# stdio (default): the trailing command launches the server
+mcp-vitals inspect node dist/server.js
+
+# Streamable HTTP / SSE (auto-falls back to SSE)
+mcp-vitals inspect --url https://api.example.com/mcp \
+  --header "Authorization: Bearer $TOKEN"
+
+# inject env into a stdio child (allowlisted by default)
+mcp-vitals bench --tool search --env API_KEY=$API_KEY npx your-mcp-server
+```
+
+## Assertions file (`mcp-vitals.yaml`)
+
+Auto-discovered as `mcp-vitals.{yaml,yml,json}` in the working directory. `${VAR}` values expand from the environment, so secrets stay out of the file.
+
+```yaml
+$schema: https://unpkg.com/mcp-vitals/schema/assertions.schema.json
+
+server:
+  command: node
+  args: ["dist/server.js"]
+  # url: https://api.example.com/mcp
+  # headers: { Authorization: "Bearer ${MCP_TOKEN}" }
+
+defaults:
+  iterations: 100
+  warmup: 3
+
+expect:
+  tools: [search, fetch-url]
+  schemasValid: true        # every listed tool's inputSchema must be valid
+
+latency:
+  - id: search-fast
+    tool: search
+    args: { q: "ping" }
+    p95: 200ms              # bare numbers are ms; '1.5s' also works
+    p99: 500ms
+    errorRate: 0            # 0 = no tool errors allowed
+
+  - id: handshake
+    probe: listTools
+    p95: 100ms
+```
+
+Add it to CI — see [`examples/github-actions.yml`](examples/github-actions.yml).
+
+## Exit codes
+
+mcp-vitals uses **distinct exit codes** so a pipeline can branch on the failure class:
+
+| Code | Meaning |
+|:---:|---|
+| `0` | success — all assertions passed |
+| `2` | **health/assertion failed** — SLA exceeded, tool errored, or invalid schema (the "red build") |
+| `3` | **connection failed** — couldn't connect / handshake (infra, distinct from a bad server) |
+| `4` | usage error — bad/conflicting flags or malformed `--args` |
+| `5` | tool error — a single `call` returned `isError` (without `--expect-error`) |
+| `6` | config error — assertions file missing / unparseable / invalid |
+
+## JSON & scripting
+
+Every command supports `--json` for a single, self-contained object on stdout (everything else goes to stderr, so `| jq` always works):
+
+```bash
+mcp-vitals bench --tool search -n 100 --json npx your-mcp-server | jq '.warm.p95'
+```
+
+## Library use
+
+```ts
+import { Connection, runBench, computeStats } from 'mcp-vitals';
+
+const conn = await Connection.connect({
+  command: 'npx', args: ['your-mcp-server'], headers: {}, env: {},
+  inheritEnv: false, connectTimeoutMs: 10_000, requestTimeoutMs: 10_000,
+});
+const result = await runBench(conn, {
+  targetKind: 'tool', targetName: 'search', args: { q: 'hi' },
+  iterations: 100, warmup: 3, concurrency: 1, keepAlive: true,
+}, 10_000);
+console.log(result.warm?.p95);
+await conn.close();
+```
+
+## Notes on benchmarking
+
+Latency numbers are only as stable as the host. Run benchmarks on a quiet machine, always keep a warmup (cold-start is reported separately), and watch `stddev` to spot a noisy environment. mcp-vitals' own test suite gates on **relative** ordering, not absolute milliseconds — a good practice for your CI thresholds too (set budgets with headroom).
+
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md). Issues and PRs welcome.
+
+## License
+
+[MIT](LICENSE) © Shaxzodbek Sobirov / Blaze

package/bin/mcp-vitals.js

@@ -0,0 +1,12 @@
+#!/usr/bin/env node
+import { main } from '../dist/cli.js';
+
+main(process.argv)
+  .then((code) => {
+    process.exitCode = code;
+  })
+  .catch((err) => {
+    // Last-resort guard; commands map their own errors to exit codes.
+    process.stderr.write(`mcp-vitals: ${err?.stack ?? err}\n`);
+    process.exitCode = 1;
+  });

package/dist/args.d.ts

@@ -0,0 +1,9 @@
+/**
+ * Resolve a `--args` value into a parsed JSON object.
+ * - undefined / empty  => `{}`
+ * - `-`                => read JSON from stdin
+ * - `@path`            => read JSON from a file
+ * - otherwise          => inline JSON
+ * Malformed JSON throws a UsageError (exit 4).
+ */
+export declare function resolveArgs(input: string | undefined): Promise<unknown>;

package/dist/args.js

@@ -0,0 +1,50 @@
+import { readFile } from 'node:fs/promises';
+import { UsageError } from './errors.js';
+async function readStdin() {
+    const chunks = [];
+    for await (const chunk of process.stdin) {
+        chunks.push(chunk);
+    }
+    return Buffer.concat(chunks).toString('utf8');
+}
+/**
+ * Resolve a `--args` value into a parsed JSON object.
+ * - undefined / empty  => `{}`
+ * - `-`                => read JSON from stdin
+ * - `@path`            => read JSON from a file
+ * - otherwise          => inline JSON
+ * Malformed JSON throws a UsageError (exit 4).
+ */
+export async function resolveArgs(input) {
+    if (input === undefined || input === '')
+        return {};
+    let text;
+    let source;
+    if (input === '-') {
+        text = await readStdin();
+        source = 'stdin';
+    }
+    else if (input.startsWith('@')) {
+        const path = input.slice(1);
+        try {
+            text = await readFile(path, 'utf8');
+        }
+        catch (err) {
+            throw new UsageError(`cannot read --args file "${path}": ${err.message}`);
+        }
+        source = `file "${path}"`;
+    }
+    else {
+        text = input;
+        source = 'inline JSON';
+    }
+    const trimmed = text.trim();
+    if (trimmed === '')
+        return {};
+    try {
+        return JSON.parse(trimmed);
+    }
+    catch (err) {
+        throw new UsageError(`invalid JSON from ${source}: ${err.message}`);
+    }
+}

package/dist/assertions/loader.d.ts

@@ -0,0 +1,9 @@
+import type { AssertionsConfig } from '../types.js';
+/** Find the first mcp-vitals config in `cwd`, or undefined. */
+export declare function discoverConfig(cwd?: string): string | undefined;
+/** Expand ${VAR} from process.env in every string value, recursively. */
+export declare function expandEnv<T>(value: T): T;
+export declare function loadConfig(explicitPath?: string): Promise<{
+    path: string;
+    config: AssertionsConfig;
+}>;

package/dist/assertions/loader.js

@@ -0,0 +1,75 @@
+import { readFile } from 'node:fs/promises';
+import { existsSync } from 'node:fs';
+import { resolve } from 'node:path';
+import { parse as parseYaml } from 'yaml';
+import { ConfigError } from '../errors.js';
+import { validateData } from '../schema.js';
+import { ASSERTIONS_SCHEMA } from './schema.js';
+const DISCOVERY_ORDER = ['mcp-vitals.yaml', 'mcp-vitals.yml', 'mcp-vitals.json'];
+/** Find the first mcp-vitals config in `cwd`, or undefined. */
+export function discoverConfig(cwd = process.cwd()) {
+    for (const name of DISCOVERY_ORDER) {
+        const path = resolve(cwd, name);
+        if (existsSync(path))
+            return path;
+    }
+    return undefined;
+}
+/** Expand ${VAR} from process.env in every string value, recursively. */
+export function expandEnv(value) {
+    if (typeof value === 'string') {
+        return value.replace(/\$\{([A-Za-z_][A-Za-z0-9_]*)\}/g, (_, name) => {
+            return process.env[name] ?? '';
+        });
+    }
+    if (Array.isArray(value)) {
+        return value.map((v) => expandEnv(v));
+    }
+    if (value !== null && typeof value === 'object') {
+        const out = {};
+        for (const [k, v] of Object.entries(value))
+            out[k] = expandEnv(v);
+        return out;
+    }
+    return value;
+}
+export async function loadConfig(explicitPath) {
+    const path = explicitPath ? resolve(explicitPath) : discoverConfig();
+    if (!path) {
+        throw new ConfigError('no assertions file found (looked for mcp-vitals.yaml/yml/json) — pass one with -c <path>');
+    }
+    if (!existsSync(path)) {
+        throw new ConfigError(`assertions file not found: ${path}`);
+    }
+    let text;
+    try {
+        text = await readFile(path, 'utf8');
+    }
+    catch (err) {
+        throw new ConfigError(`cannot read ${path}: ${err.message}`);
+    }
+    let data;
+    try {
+        data = parseYaml(text); // YAML is a superset of JSON, so this handles both.
+    }
+    catch (err) {
+        throw new ConfigError(`cannot parse ${path}: ${err.message}`);
+    }
+    const expanded = expandEnv(data);
+    const check = validateData(ASSERTIONS_SCHEMA, expanded);
+    if (!check.valid) {
+        const first = check.errors[0];
+        const detail = first ? `${first.path} ${first.message}` : 'schema validation failed';
+        throw new ConfigError(`invalid assertions file ${path}: ${detail}`);
+    }
+    const config = expanded;
+    if (!config.server.command && !config.server.url) {
+        throw new ConfigError(`invalid assertions file ${path}: server must set "command" or "url"`);
+    }
+    for (const a of config.latency ?? []) {
+        if (!a.tool && !a.probe) {
+            throw new ConfigError(`invalid assertions file ${path}: latency assertion "${a.id}" must set "tool" or "probe"`);
+        }
+    }
+    return { path, config };
+}

package/dist/assertions/run.d.ts

@@ -0,0 +1,19 @@
+import type { Connection } from '../mcpClient.js';
+import type { AssertionsConfig, CheckRow, CheckSummary, LatencyAssertion } from '../types.js';
+export interface RunOptions {
+    noLatency: boolean;
+    only: string[];
+    skip: string[];
+    bail: boolean;
+    iterations?: number;
+    warmup?: number;
+    concurrency?: number;
+    timeoutMs: number;
+}
+declare const LATENCY_METRICS: readonly ["p50", "p90", "p95", "p99", "max", "mean", "errorRate"];
+type LatencyMetric = (typeof LATENCY_METRICS)[number];
+export declare function runChecks(conn: Connection, config: AssertionsConfig, options: RunOptions): Promise<{
+    rows: CheckRow[];
+    summary: CheckSummary;
+}>;
+export type { LatencyAssertion, LatencyMetric };

package/dist/assertions/run.js

@@ -0,0 +1,154 @@
+import { runBench } from '../bench/engine.js';
+import { evaluate } from '../thresholds.js';
+import { parseBound } from '../thresholds.js';
+import { validateJsonSchema } from '../schema.js';
+import { matchesGlob } from '../glob.js';
+import { formatMs, formatPct } from '../renderers/table.js';
+const LATENCY_METRICS = ['p50', 'p90', 'p95', 'p99', 'max', 'mean', 'errorRate'];
+function included(id, only, skip) {
+    const passOnly = only.length === 0 || only.some((g) => matchesGlob(id, g));
+    const blocked = skip.length > 0 && skip.some((g) => matchesGlob(id, g));
+    return passOnly && !blocked;
+}
+function displayBound(metric, bound) {
+    return metric === 'errorRate' ? formatPct(bound) : formatMs(bound);
+}
+function displayActual(metric, actual) {
+    if (actual === null)
+        return '—';
+    return metric === 'errorRate' ? formatPct(actual) : formatMs(actual);
+}
+export async function runChecks(conn, config, options) {
+    const start = process.hrtime.bigint();
+    const rows = [];
+    let bailed = false;
+    const add = (row) => {
+        rows.push(row);
+        if (options.bail && row.status === 'fail')
+            bailed = true;
+    };
+    const tools = await conn.listTools();
+    const toolByName = new Map(tools.map((t) => [t.name, t]));
+    const resources = await conn.listResources();
+    const prompts = await conn.listPrompts();
+    const resourceNames = new Set(resources.map((r) => r.name).filter((n) => !!n));
+    const promptNames = new Set(prompts.map((p) => p.name));
+    const expect = config.expect ?? {};
+    // ---- presence suite ----
+    const presence = [
+        ['tools', expect.tools ?? [], (n) => toolByName.has(n)],
+        ['resources', expect.resources ?? [], (n) => resourceNames.has(n)],
+        ['prompts', expect.prompts ?? [], (n) => promptNames.has(n)],
+    ];
+    for (const [kind, names, has] of presence) {
+        for (const name of names) {
+            if (bailed)
+                break;
+            const id = `${kind}/${name}`;
+            if (!included(id, options.only, options.skip)) {
+                add({ id, kind: 'presence', target: name, expected: 'present', actual: 'skipped', status: 'skip' });
+                continue;
+            }
+            const ok = has(name);
+            add({
+                id,
+                kind: 'presence',
+                target: name,
+                expected: `${kind} present`,
+                actual: ok ? 'present' : 'MISSING',
+                status: ok ? 'pass' : 'fail',
+            });
+        }
+    }
+    // ---- schema suite ----
+    if (expect.schemasValid && !bailed) {
+        const targets = (expect.tools && expect.tools.length > 0 ? expect.tools : tools.map((t) => t.name));
+        for (const name of targets) {
+            if (bailed)
+                break;
+            const id = `schema/${name}`;
+            if (!included(id, options.only, options.skip)) {
+                add({ id, kind: 'schema', target: name, expected: 'valid', actual: 'skipped', status: 'skip' });
+                continue;
+            }
+            const tool = toolByName.get(name);
+            if (!tool) {
+                add({ id, kind: 'schema', target: name, expected: 'valid inputSchema', actual: 'tool missing', status: 'fail' });
+                continue;
+            }
+            const res = validateJsonSchema(tool.inputSchema);
+            add({
+                id,
+                kind: 'schema',
+                target: name,
+                expected: 'valid inputSchema',
+                actual: res.valid ? 'valid' : (res.errors[0]?.message ?? 'invalid'),
+                status: res.valid ? 'pass' : 'fail',
+            });
+        }
+    }
+    // ---- latency suite ----
+    for (const assertion of config.latency ?? []) {
+        if (bailed)
+            break;
+        const thresholds = LATENCY_METRICS.filter((m) => assertion[m] !== undefined);
+        if (thresholds.length === 0)
+            continue;
+        const targetName = assertion.tool ?? assertion.probe ?? 'listTools';
+        const targetKind = assertion.tool ? 'tool' : 'probe';
+        if (options.noLatency) {
+            for (const metric of thresholds) {
+                const id = `${assertion.id}/${metric}`;
+                add({ id, kind: 'latency', target: targetName, expected: 'latency SLA', actual: 'skipped', status: 'skip' });
+            }
+            continue;
+        }
+        // Determine which thresholds are actually selected before running the bench.
+        const selected = thresholds.filter((m) => included(`${assertion.id}/${m}`, options.only, options.skip));
+        if (selected.length === 0) {
+            for (const metric of thresholds) {
+                const id = `${assertion.id}/${metric}`;
+                add({ id, kind: 'latency', target: targetName, expected: 'latency SLA', actual: 'skipped', status: 'skip' });
+            }
+            continue;
+        }
+        const benchConfig = {
+            targetKind,
+            targetName,
+            args: assertion.args ?? {},
+            iterations: options.iterations ?? assertion.iterations ?? config.defaults?.iterations ?? 50,
+            warmup: options.warmup ?? assertion.warmup ?? config.defaults?.warmup ?? 1,
+            concurrency: options.concurrency ?? assertion.concurrency ?? config.defaults?.concurrency ?? 1,
+            keepAlive: true,
+        };
+        const result = await runBench(conn, benchConfig, options.timeoutMs);
+        for (const metric of thresholds) {
+            const id = `${assertion.id}/${metric}`;
+            if (!selected.includes(metric)) {
+                add({ id, kind: 'latency', target: targetName, expected: 'latency SLA', actual: 'skipped', status: 'skip' });
+                continue;
+            }
+            const bound = parseBound(metric, assertion[metric]);
+            const outcome = evaluate({ metric, op: '<=', bound }, result.warm, result.throughput);
+            const status = outcome.pass ? 'pass' : 'fail';
+            add({
+                id,
+                kind: 'latency',
+                target: `${targetName} ${metric}`,
+                expected: `${metric} <= ${displayBound(metric, bound)}`,
+                actual: displayActual(metric, outcome.actual),
+                status,
+            });
+            if (bailed)
+                break;
+        }
+    }
+    const durationMs = Number(process.hrtime.bigint() - start) / 1e6;
+    const summary = {
+        passed: rows.filter((r) => r.status === 'pass').length,
+        failed: rows.filter((r) => r.status === 'fail').length,
+        skipped: rows.filter((r) => r.status === 'skip').length,
+        durationMs,
+    };
+    return { rows, summary };
+}