@progen-labs/spectra 0.1.3 → 0.3.0

package/README.md

@@ -14,6 +14,9 @@ and per-component render/change-detection time; source files too, where the fram
 functions. A [Progen Labs](https://github.com/progen-labs) project — open source, non-profit, no
 telemetry, no account required.
 
+**Documentation:** the full guide, with annotated screenshots of every screen, is at
+[progen-labs.github.io/website](https://progen-labs.github.io/website/#spectra/docs/introduction).
+
 ## Install
 
 ```bash
@@ -52,15 +55,24 @@ Then open the visual dashboard — ranked culprits, React re-renders, long tasks
 **PDF export** and **commit-to-commit compare**:
 
 ```bash
-npx spectra dashboard
+npx spectra dashboard --open
 ```
 
+![The Spectra dashboard: the top culprit of a recording, burn() in src/heavy.js, with its time and a suggested fix](docs/dashboard.webp)
+
+A [tour of the dashboard](https://progen-labs.github.io/website/#spectra/docs/dashboard), with every part
+labelled, is in the docs.
+
 ## Commands
 
 ```
-spectra performance [options]   Golden-path evaluation (default)
-spectra dashboard [--port N]    Open the dashboard UI
-spectra --help                  Full option list
+spectra performance [options]            Golden-path evaluation (default)
+spectra performance --quick              One page-load profile (about 15 s)
+spectra compare <before> <after>         Did it get faster? Noise-aware before/after
+spectra reports                          List saved reports
+spectra mcp [--project <dir>]            MCP server for AI tools (stdio)
+spectra dashboard [--port N] [--open]    Open the dashboard UI
+spectra --help                           Full option list
 spectra --version
 ```
 
@@ -74,7 +86,147 @@ spectra --version
 | `--boot-timeout <s>` | How long to wait for the dev server (default 60) |
 | `--headed` | Show the browser window |
 | `--sign-in <url>` | Sign-in URL template, e.g. `http://host/login/?redirect={url}` |
+| `--quick` | One page-load profile instead of the golden-path suite: the fast loop |
+| `--runs <n>` | Record n times and report the median (with `--quick` or `--mode`) |
 | `--mode live\|load` | Classic single-page profile instead of the golden-path suite |
+| `--json` | Print a machine-readable result on stdout (tables go to stderr) |
+| `--out <file>` | Also write that result to a file (an input for `compare`) |
+| `--config <file>` | Budgets file (default: `spectra.config.json` in your project) |
+
+## Measure, change, verify
+
+Performance numbers are noisy, so Spectra never trusts a single recording when you ask it whether a change
+helped. Record a few runs before and after, then compare:
+
+```bash
+npx spectra performance --quick --runs 5 --out before.json
+# …make your change…
+npx spectra performance --quick --runs 5 --out after.json
+npx spectra compare before.json after.json
+```
+
+```
+Spectra compare — IMPROVED (confidence: medium)
+  ✓ JavaScript CPU time: 1.52 s → 92.1 ms (−94%)
+  ✓ Row (src/Row.jsx) — render time: 1.44 s → 7.4 ms (−99%)
+
+Findings: 3 resolved · 2 new · 10 still present
+  resolved  [critical] Row is slow to render: 1444ms total, 0.2ms per render (7200×)
+```
+
+Each side is reduced to its median, and a difference only counts when it is larger than the run-to-run
+spread. A side can be a `--out` result file, a report id from `spectra reports`, a report file, or several
+ids joined by commas.
+
+## Budgets and CI
+
+Put limits in `spectra.config.json` at your project root:
+
+```json
+{
+  "$schema": "./node_modules/@progen-labs/spectra/schema/config.schema.json",
+  "budgets": { "tbtMs": 200, "lcpMs": 2500, "cls": 0.1, "criticalFindings": 0 }
+}
+```
+
+Every `spectra performance` run checks them: the median for `--quick`/`--mode`, every journey for the
+golden-path suite. Metrics a recording did not measure are skipped, never failed. Available budgets:
+`tbtMs`, `lcpMs`, `fcpMs`, `cls`, `inpMs`, `ttfbMs`, `longTaskCount`, `maxLongTaskMs`, `scriptingMs`,
+`forcedLayoutCount`, `domNodes`, `heapUsedMb`, `transferKb`, `criticalFindings`, `findingCount`.
+
+Exit codes make it a CI gate:
+
+| Code | Meaning |
+| --- | --- |
+| `0` | Done, and within budget |
+| `1` | Could not run (no Chrome, server down, bad config) |
+| `2` | Ran, but a budget was exceeded, or `compare --fail-on-regression` found a regression |
+
+### GitHub Action
+
+The repository is also a GitHub Action. On every pull request it profiles the change and its base commit on the
+same runner, compares them beyond run-to-run noise, enforces your budgets, and writes the result to the job summary
+and (optionally) a PR comment:
+
+```yaml
+name: Performance
+on: pull_request
+permissions:
+  contents: read
+  pull-requests: write   # only for comment: true
+jobs:
+  spectra:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with: { node-version: 22 }
+      - run: npm ci
+      - uses: progen-labs/spectra@v1
+        with:
+          compare-to-base: true
+          fail-on-regression: true
+          comment: true
+```
+
+Inputs include `url`, `command`, `mode` (`quick` or `golden-paths`), `runs`, `config`, `fail-on-budget` and
+`working-directory`; outputs are `result-file`, `verdict`, `over-budget` and `summary-file`. Full reports are
+uploaded as an artifact. See the [Action guide](https://progen-labs.github.io/website/#spectra/docs/github-action).
+
+## Machine-readable output
+
+`--json` prints a small, versioned result on stdout and sends every human-readable line to stderr, so it can
+be piped into `jq`, an editor or an AI agent. Each result has `schemaVersion` and a `kind` (`profile`,
+`evaluation`, `compare` or `reports`) and points at the full reports on disk rather than embedding them.
+Findings carry the file, line, cost in ms and a suggested fix. The format is described by a JSON Schema
+shipped with the package (`schema/result.schema.json`); additive changes keep `schemaVersion`, breaking
+changes bump it. Saved reports carry their own `schemaVersion` too.
+
+## Use it from your AI assistant
+
+Spectra is an [MCP](https://modelcontextprotocol.io) server too, so an AI coding assistant can profile a page,
+fix what it finds, and prove each fix with before/after measurements.
+
+**Claude Code**, with the plugin (adds an *improve-performance* skill and `/spectra:audit`):
+
+```
+/plugin marketplace add progen-labs/spectra
+/plugin install spectra@progen-labs
+```
+
+**Any MCP client** (Claude Code without the plugin, Claude Desktop, Cursor, VS Code, Windsurf…): run
+`npx -y @progen-labs/spectra mcp` as a stdio server (Spectra 0.3.0 or later). For example:
+
+```bash
+claude mcp add spectra -- npx -y @progen-labs/spectra mcp
+```
+
+```json
+{ "mcpServers": { "spectra": { "command": "npx", "args": ["-y", "@progen-labs/spectra", "mcp", "--project", "/path/to/your/app"] } } }
+```
+
+Tools: `profile_page`, `verify_change` (re-profile and compare against a baseline), `compare_profiles`,
+`run_golden_paths`, `list_reports` and `get_report`. The server profiles pages on this machine only (start it with
+`--allow-remote` to lift that), runs one browser at a time, and stops Chrome and any dev server it started when a
+call is cancelled. See the [MCP guide](https://progen-labs.github.io/website/#spectra/docs/mcp-server).
+
+## In VS Code
+
+The Spectra extension (`vscode/`) puts results where you work: every finding in the **Problems panel** at the file
+and line responsible, **CodeLens** above the costliest components and functions
+(`⚡ Spectra: Row · 7,200 renders · 1.44 s rendering`), headline numbers in the **status bar**, **Verify against
+baseline** after a change, golden paths, and the dashboard inside the editor. Results that the CLI or an AI
+assistant writes to `.spectra/results` appear automatically, and on VS Code 1.101+ it offers Spectra's MCP server to
+agent mode. It runs your project's Spectra (0.2.0 or later) or `npx`.
+
+Until it is listed on the VS Code Marketplace and Open VSX, build and install it from this repository:
+
+```bash
+cd vscode && npx @vscode/vsce package --no-dependencies
+code --install-extension spectra-0.1.0.vsix
+```
+
+See the [VS Code guide](https://progen-labs.github.io/website/#spectra/docs/vscode).
 
 ## Safe by default
 
@@ -97,6 +249,19 @@ that cost the most main-thread time. See the
 [deep-dive docs](https://github.com/progen-labs/spectra/blob/main/DOCS.md) for how capture, correlation
 and report generation work.
 
+## Documentation
+
+- **[The guide](https://progen-labs.github.io/website/#spectra/docs/introduction)**: every feature, with
+  annotated screenshots. Start with
+  [Quick start](https://progen-labs.github.io/website/#spectra/docs/quick-start),
+  [Compare before and after](https://progen-labs.github.io/website/#spectra/docs/compare),
+  [Budgets and CI](https://progen-labs.github.io/website/#spectra/docs/budgets),
+  [JSON output](https://progen-labs.github.io/website/#spectra/docs/json-output),
+  [Claude Code](https://progen-labs.github.io/website/#spectra/docs/claude-code), the
+  [Findings reference](https://progen-labs.github.io/website/#spectra/docs/findings) and
+  [Troubleshooting](https://progen-labs.github.io/website/#spectra/docs/troubleshooting).
+- **[DOCS.md](DOCS.md)**: the in-repo manual, the technical deep dive, and how to contribute.
+
 ## License
 
 MIT © Progen Labs

package/bin/spectra.js

@@ -3,6 +3,9 @@
  * Spectra CLI — a Progen Labs project.
  *
  *   spectra performance [--url <page>] [profiler options]   profile a frontend
+ *   spectra compare <before> <after>                         before vs after, noise-aware
+ *   spectra reports                                          list saved reports
+ *   spectra mcp [--project <dir>]                            MCP server for AI tools (stdio)
  *   spectra dashboard                                        open the dashboard UI
  *   spectra --help                                           show this help
  *   spectra --version                                        print the version
@@ -12,7 +15,7 @@
  * root is used for source lookup, unless the matching PERF_* env var overrides it.
  */
 import { spawn } from 'node:child_process';
-import { readFileSync } from 'node:fs';
+import { existsSync, readFileSync } from 'node:fs';
 import path from 'node:path';
 import { fileURLToPath } from 'node:url';
 import { bootDevServer } from '../server/explore/boot.js';
@@ -33,6 +36,13 @@ Spectra ${pkg.version} — frontend performance profiler (Progen Labs)
 
 Usage:
   spectra performance [options]   Profile a frontend build and report the culprits
+  spectra compare <before> <after> [options]
+                                  Compare two profiles; a change counts only when
+                                  it is bigger than the noise between runs
+  spectra reports [--json]        List saved reports (ids for compare / dashboard)
+  spectra mcp [--project <dir>] [--allow-remote]
+                                  Run as an MCP server (stdio) for Claude Code,
+                                  Cursor, VS Code and other AI tools
   spectra dashboard               Open the Spectra dashboard UI
   spectra --help, -h              Show this help
   spectra --version, -v           Print the version
@@ -52,20 +62,58 @@ performance options:
   --headed              Show the browser window
   --sign-in <url>       Sign-in URL template, e.g. "http://host/login/?redirect={url}"
 
-classic single-page profile (opt in with --mode or --scenario):
+quick single-page profile (opt in with --quick, --mode or --scenario):
+  --quick               One page-load profile (about 15 s): the fast loop
+  --runs <n>            Record n times and report the median (with --quick/--mode)
   --mode live|load      Live session or page-load recording
   --duration <seconds>  Live recording length (default 8)
   --throttle <n>        CPU throttle multiplier (default 1)
   --scenario <file.mjs> Script a live session: export default async page => {…}
   --save-trace          Keep the raw Chrome trace alongside the report
 
+output and checks (any performance run):
+  --json                Print a machine-readable result on stdout (progress and
+                        tables go to stderr)
+  --out <file>          Also write that result to a file (input for compare)
+  --config <file>       Budgets file (default: spectra.config.json in the project)
+
+compare options:
+  <before> <after>      Each: a report id, a report file, a --out result file,
+                        or several ids joined by commas
+  --json, --out <file>  As above
+  --fail-on-regression  Exit with code 2 when a key metric got worse
+
+Budgets (spectra.config.json in your project root):
+  { "budgets": { "tbtMs": 200, "lcpMs": 2500, "cls": 0.1, "criticalFindings": 0 } }
+
+Exit codes: 0 done · 1 could not run · 2 over budget / regression
+
 Examples:
   spectra performance                              # self-boot + golden-path suite
   spectra performance --paths 8                    # cap the suite at 8 journeys
   spectra performance --url http://localhost:3000/ # evaluate a running server
-  spectra performance --mode load                  # classic single page-load profile
+  spectra performance --quick --runs 5 --out before.json
+  spectra performance --quick --runs 5 --out after.json
+  spectra compare before.json after.json           # did the change help?
+  spectra performance --json | jq .summary         # pipe it anywhere
 `;
 
+// What a terminate signal must clean up: the running script and a dev server
+// Spectra started. Without this, stopping Spectra from outside (an editor or
+// an AI client cancelling a run) would leave the dev server running.
+const active = { child: null, server: null };
+let stopping = false;
+for (const signal of ['SIGINT', 'SIGTERM']) {
+  process.on(signal, () => {
+    if (stopping) return;
+    stopping = true;
+    active.child?.kill(signal);
+    Promise.resolve(active.server?.stop())
+      .catch(() => {})
+      .finally(() => process.exit(signal === 'SIGINT' ? 130 : 143));
+  });
+}
+
 function runNodeAsync(scriptRelPath, args, extraEnv = {}) {
   return new Promise((resolve, reject) => {
     const child = spawn(
@@ -77,7 +125,11 @@ function runNodeAsync(scriptRelPath, args, extraEnv = {}) {
         env: { ...process.env, ...extraEnv },
       },
     );
-    child.on('exit', code => resolve(code ?? 0));
+    active.child = child;
+    child.on('exit', code => {
+      if (active.child === child) active.child = null;
+      resolve(code ?? 0);
+    });
     child.on('error', reject);
   });
 }
@@ -91,6 +143,19 @@ function takeOption(args, name) {
   return { value, rest: copy };
 }
 
+/**
+ * The scripts run with the package as their working directory, so path
+ * options the user typed relative to their own directory are made absolute
+ * here, before they cross over.
+ */
+function absolutize(args, names) {
+  return args.map((a, i) =>
+    i > 0 && names.includes(args[i - 1].replace(/^--/, '')) && args[i - 1].startsWith('--') && !a.startsWith('--')
+      ? path.resolve(process.cwd(), a)
+      : a,
+  );
+}
+
 /**
  * Default the profiled project and the reports location to the user's current
  * directory, so an installed Spectra profiles the host project and writes
@@ -114,27 +179,44 @@ async function runProfile() {
     return 0;
   }
   const env = projectEnv();
+  let args = absolutize(rest, ['scenario', 'out', 'config']);
+
+  // --quick is shorthand for a single page-load profile.
+  if (args.includes('--quick')) {
+    const mode = takeOption(args, 'mode').value;
+    if ((mode && mode !== 'load') || args.includes('--scenario')) {
+      console.error('spectra: --quick is a page-load profile; it cannot be combined with --mode live or --scenario.');
+      return 1;
+    }
+    args = args.filter(a => a !== '--quick');
+    if (!mode) args.push('--mode', 'load');
+  }
 
   // Golden-path evaluation is the default. --mode/--scenario opt into the
   // classic single-page profile instead.
-  const classic = rest.includes('--mode') || rest.includes('--scenario');
+  const classic = args.includes('--mode') || args.includes('--scenario');
   const script = classic ? 'server/cli.js' : 'server/explore/evaluate-cli.js';
+  if (!classic && args.includes('--runs')) {
+    console.error(
+      'spectra: --runs works with single-page profiles (--quick, --mode or --scenario); the golden-path suite records each journey once.',
+    );
+    return 1;
+  }
 
   // An explicit --url means the server is already running: run against it.
-  if (rest.includes('--url')) {
-    return runNodeAsync(script, rest, env);
+  if (args.includes('--url')) {
+    return runNodeAsync(script, args, env);
   }
 
   // No --url: self-boot the target project's own dev server, run, stop.
   const cwd = process.cwd();
-  const { value: command, rest: afterCmd } = takeOption(rest, 'command');
+  const { value: command, rest: afterCmd } = takeOption(args, 'command');
   const { value: bootTimeout, rest: profileArgs } = takeOption(afterCmd, 'boot-timeout');
   const fw = detectFramework(cwd);
-  const label =
-    fw.framework !== 'unknown' ? `${fw.framework}/${fw.bundler}` : fw.bundler;
+  const label = [fw.framework, fw.bundler].filter(x => x && x !== 'unknown').join('/');
   // Status goes to stderr so it never mixes with the report on stdout.
   console.error(
-    `spectra: no --url given — starting the ${label} dev server in ${cwd} …`,
+    `spectra: no --url given — starting the ${label ? `${label} ` : ''}dev server in ${cwd} …`,
   );
 
   let server;
@@ -148,6 +230,7 @@ async function runProfile() {
     console.error(`spectra: ${err.message}`);
     return 1;
   }
+  active.server = server;
   console.error(`spectra: dev server ready at ${server.url} — profiling …\n`);
 
   let code = 1;
@@ -158,12 +241,23 @@ async function runProfile() {
       env,
     );
   } finally {
+    active.server = null;
     await server.stop();
     console.error('\nspectra: stopped the dev server.');
   }
   return code;
 }
 
+function runCompare() {
+  // Sides may be files relative to the user's directory (ids pass through).
+  const fixToken = t =>
+    t.endsWith('.json') || existsSync(path.resolve(process.cwd(), t)) ? path.resolve(process.cwd(), t) : t;
+  const args = absolutize(rest, ['out']).map((a, i, all) =>
+    a.startsWith('--') || all[i - 1] === '--out' ? a : a.split(',').map(fixToken).join(','),
+  );
+  return runNodeAsync('server/compare-cli.js', args, projectEnv());
+}
+
 function runDashboard() {
   // Standalone server: serves the prebuilt dashboard and the backend API, so an
   // installed Spectra needs no dev toolchain. projectEnv() points it at this
@@ -177,6 +271,7 @@ function runDashboard() {
       env: { ...process.env, ...projectEnv() },
     },
   );
+  active.child = child;
   child.on('exit', code => process.exit(code ?? 0));
   child.on('error', err => {
     console.error(`spectra: failed to start the dashboard — ${err.message}`);
@@ -194,6 +289,26 @@ switch (command) {
         process.exit(1);
       });
     break;
+  case 'compare':
+  case 'reports': {
+    const run = command === 'compare' ? runCompare() : runNodeAsync('server/reports-cli.js', rest, projectEnv());
+    run
+      .then(code => process.exit(code ?? 0))
+      .catch(err => {
+        console.error(`spectra: ${err.message}`);
+        process.exit(1);
+      });
+    break;
+  }
+  case 'mcp':
+    // In-process: stdin/stdout are the protocol channel, and the working
+    // directory stays the user's (the project, unless --project or the
+    // client's roots say otherwise).
+    import('../server/mcp.js').catch(err => {
+      console.error(`spectra: ${err.message}`);
+      process.exit(1);
+    });
+    break;
   case 'dashboard':
   case 'ui':
     runDashboard();