@progen-labs/spectra 0.1.3 → 0.2.0

package/README.md

@@ -14,6 +14,9 @@ and per-component render/change-detection time; source files too, where the fram
 functions. A [Progen Labs](https://github.com/progen-labs) project — open source, non-profit, no
 telemetry, no account required.
 
+**Documentation:** the full guide, with annotated screenshots of every screen, is at
+[progen-labs.github.io/website](https://progen-labs.github.io/website/#spectra/docs/introduction).
+
 ## Install
 
 ```bash
@@ -52,15 +55,23 @@ Then open the visual dashboard — ranked culprits, React re-renders, long tasks
 **PDF export** and **commit-to-commit compare**:
 
 ```bash
-npx spectra dashboard
+npx spectra dashboard --open
 ```
 
+![The Spectra dashboard: the top culprit of a recording, burn() in src/heavy.js, with its time and a suggested fix](docs/dashboard.webp)
+
+A [tour of the dashboard](https://progen-labs.github.io/website/#spectra/docs/dashboard), with every part
+labelled, is in the docs.
+
 ## Commands
 
 ```
-spectra performance [options]   Golden-path evaluation (default)
-spectra dashboard [--port N]    Open the dashboard UI
-spectra --help                  Full option list
+spectra performance [options]            Golden-path evaluation (default)
+spectra performance --quick              One page-load profile (about 15 s)
+spectra compare <before> <after>         Did it get faster? Noise-aware before/after
+spectra reports                          List saved reports
+spectra dashboard [--port N] [--open]    Open the dashboard UI
+spectra --help                           Full option list
 spectra --version
 ```
 
@@ -74,7 +85,70 @@ spectra --version
 | `--boot-timeout <s>` | How long to wait for the dev server (default 60) |
 | `--headed` | Show the browser window |
 | `--sign-in <url>` | Sign-in URL template, e.g. `http://host/login/?redirect={url}` |
+| `--quick` | One page-load profile instead of the golden-path suite: the fast loop |
+| `--runs <n>` | Record n times and report the median (with `--quick` or `--mode`) |
 | `--mode live\|load` | Classic single-page profile instead of the golden-path suite |
+| `--json` | Print a machine-readable result on stdout (tables go to stderr) |
+| `--out <file>` | Also write that result to a file (an input for `compare`) |
+| `--config <file>` | Budgets file (default: `spectra.config.json` in your project) |
+
+## Measure, change, verify
+
+Performance numbers are noisy, so Spectra never trusts a single recording when you ask it whether a change
+helped. Record a few runs before and after, then compare:
+
+```bash
+npx spectra performance --quick --runs 5 --out before.json
+# …make your change…
+npx spectra performance --quick --runs 5 --out after.json
+npx spectra compare before.json after.json
+```
+
+```
+Spectra compare — IMPROVED (confidence: medium)
+  ✓ JavaScript CPU time: 1.52 s → 92.1 ms (−94%)
+  ✓ Row (src/Row.jsx) — render time: 1.44 s → 7.4 ms (−99%)
+
+Findings: 3 resolved · 2 new · 10 still present
+  resolved  [critical] Row is slow to render: 1444ms total, 0.2ms per render (7200×)
+```
+
+Each side is reduced to its median, and a difference only counts when it is larger than the run-to-run
+spread. A side can be a `--out` result file, a report id from `spectra reports`, a report file, or several
+ids joined by commas.
+
+## Budgets and CI
+
+Put limits in `spectra.config.json` at your project root:
+
+```json
+{
+  "$schema": "./node_modules/@progen-labs/spectra/schema/config.schema.json",
+  "budgets": { "tbtMs": 200, "lcpMs": 2500, "cls": 0.1, "criticalFindings": 0 }
+}
+```
+
+Every `spectra performance` run checks them: the median for `--quick`/`--mode`, every journey for the
+golden-path suite. Metrics a recording did not measure are skipped, never failed. Available budgets:
+`tbtMs`, `lcpMs`, `fcpMs`, `cls`, `inpMs`, `ttfbMs`, `longTaskCount`, `maxLongTaskMs`, `scriptingMs`,
+`forcedLayoutCount`, `domNodes`, `heapUsedMb`, `transferKb`, `criticalFindings`, `findingCount`.
+
+Exit codes make it a CI gate:
+
+| Code | Meaning |
+| --- | --- |
+| `0` | Done, and within budget |
+| `1` | Could not run (no Chrome, server down, bad config) |
+| `2` | Ran, but a budget was exceeded, or `compare --fail-on-regression` found a regression |
+
+## Machine-readable output
+
+`--json` prints a small, versioned result on stdout and sends every human-readable line to stderr, so it can
+be piped into `jq`, an editor or an AI agent. Each result has `schemaVersion` and a `kind` (`profile`,
+`evaluation`, `compare` or `reports`) and points at the full reports on disk rather than embedding them.
+Findings carry the file, line, cost in ms and a suggested fix. The format is described by a JSON Schema
+shipped with the package (`schema/result.schema.json`); additive changes keep `schemaVersion`, breaking
+changes bump it. Saved reports carry their own `schemaVersion` too.
 
 ## Safe by default
 
@@ -97,6 +171,18 @@ that cost the most main-thread time. See the
 [deep-dive docs](https://github.com/progen-labs/spectra/blob/main/DOCS.md) for how capture, correlation
 and report generation work.
 
+## Documentation
+
+- **[The guide](https://progen-labs.github.io/website/#spectra/docs/introduction)**: every feature, with
+  annotated screenshots. Start with
+  [Quick start](https://progen-labs.github.io/website/#spectra/docs/quick-start),
+  [Compare before and after](https://progen-labs.github.io/website/#spectra/docs/compare),
+  [Budgets and CI](https://progen-labs.github.io/website/#spectra/docs/budgets),
+  [JSON output](https://progen-labs.github.io/website/#spectra/docs/json-output), the
+  [Findings reference](https://progen-labs.github.io/website/#spectra/docs/findings) and
+  [Troubleshooting](https://progen-labs.github.io/website/#spectra/docs/troubleshooting).
+- **[DOCS.md](DOCS.md)**: the in-repo manual, the technical deep dive, and how to contribute.
+
 ## License
 
 MIT © Progen Labs

package/bin/spectra.js

@@ -3,6 +3,8 @@
  * Spectra CLI — a Progen Labs project.
  *
  *   spectra performance [--url <page>] [profiler options]   profile a frontend
+ *   spectra compare <before> <after>                         before vs after, noise-aware
+ *   spectra reports                                          list saved reports
  *   spectra dashboard                                        open the dashboard UI
  *   spectra --help                                           show this help
  *   spectra --version                                        print the version
@@ -12,7 +14,7 @@
  * root is used for source lookup, unless the matching PERF_* env var overrides it.
  */
 import { spawn } from 'node:child_process';
-import { readFileSync } from 'node:fs';
+import { existsSync, readFileSync } from 'node:fs';
 import path from 'node:path';
 import { fileURLToPath } from 'node:url';
 import { bootDevServer } from '../server/explore/boot.js';
@@ -33,6 +35,10 @@ Spectra ${pkg.version} — frontend performance profiler (Progen Labs)
 
 Usage:
   spectra performance [options]   Profile a frontend build and report the culprits
+  spectra compare <before> <after> [options]
+                                  Compare two profiles; a change counts only when
+                                  it is bigger than the noise between runs
+  spectra reports [--json]        List saved reports (ids for compare / dashboard)
   spectra dashboard               Open the Spectra dashboard UI
   spectra --help, -h              Show this help
   spectra --version, -v           Print the version
@@ -52,18 +58,40 @@ performance options:
   --headed              Show the browser window
   --sign-in <url>       Sign-in URL template, e.g. "http://host/login/?redirect={url}"
 
-classic single-page profile (opt in with --mode or --scenario):
+quick single-page profile (opt in with --quick, --mode or --scenario):
+  --quick               One page-load profile (about 15 s): the fast loop
+  --runs <n>            Record n times and report the median (with --quick/--mode)
   --mode live|load      Live session or page-load recording
   --duration <seconds>  Live recording length (default 8)
   --throttle <n>        CPU throttle multiplier (default 1)
   --scenario <file.mjs> Script a live session: export default async page => {…}
   --save-trace          Keep the raw Chrome trace alongside the report
 
+output and checks (any performance run):
+  --json                Print a machine-readable result on stdout (progress and
+                        tables go to stderr)
+  --out <file>          Also write that result to a file (input for compare)
+  --config <file>       Budgets file (default: spectra.config.json in the project)
+
+compare options:
+  <before> <after>      Each: a report id, a report file, a --out result file,
+                        or several ids joined by commas
+  --json, --out <file>  As above
+  --fail-on-regression  Exit with code 2 when a key metric got worse
+
+Budgets (spectra.config.json in your project root):
+  { "budgets": { "tbtMs": 200, "lcpMs": 2500, "cls": 0.1, "criticalFindings": 0 } }
+
+Exit codes: 0 done · 1 could not run · 2 over budget / regression
+
 Examples:
   spectra performance                              # self-boot + golden-path suite
   spectra performance --paths 8                    # cap the suite at 8 journeys
   spectra performance --url http://localhost:3000/ # evaluate a running server
-  spectra performance --mode load                  # classic single page-load profile
+  spectra performance --quick --runs 5 --out before.json
+  spectra performance --quick --runs 5 --out after.json
+  spectra compare before.json after.json           # did the change help?
+  spectra performance --json | jq .summary         # pipe it anywhere
 `;
 
 function runNodeAsync(scriptRelPath, args, extraEnv = {}) {
@@ -91,6 +119,19 @@ function takeOption(args, name) {
   return { value, rest: copy };
 }
 
+/**
+ * The scripts run with the package as their working directory, so path
+ * options the user typed relative to their own directory are made absolute
+ * here, before they cross over.
+ */
+function absolutize(args, names) {
+  return args.map((a, i) =>
+    i > 0 && names.includes(args[i - 1].replace(/^--/, '')) && args[i - 1].startsWith('--') && !a.startsWith('--')
+      ? path.resolve(process.cwd(), a)
+      : a,
+  );
+}
+
 /**
  * Default the profiled project and the reports location to the user's current
  * directory, so an installed Spectra profiles the host project and writes
@@ -114,20 +155,38 @@ async function runProfile() {
     return 0;
   }
   const env = projectEnv();
+  let args = absolutize(rest, ['scenario', 'out', 'config']);
+
+  // --quick is shorthand for a single page-load profile.
+  if (args.includes('--quick')) {
+    const mode = takeOption(args, 'mode').value;
+    if ((mode && mode !== 'load') || args.includes('--scenario')) {
+      console.error('spectra: --quick is a page-load profile; it cannot be combined with --mode live or --scenario.');
+      return 1;
+    }
+    args = args.filter(a => a !== '--quick');
+    if (!mode) args.push('--mode', 'load');
+  }
 
   // Golden-path evaluation is the default. --mode/--scenario opt into the
   // classic single-page profile instead.
-  const classic = rest.includes('--mode') || rest.includes('--scenario');
+  const classic = args.includes('--mode') || args.includes('--scenario');
   const script = classic ? 'server/cli.js' : 'server/explore/evaluate-cli.js';
+  if (!classic && args.includes('--runs')) {
+    console.error(
+      'spectra: --runs works with single-page profiles (--quick, --mode or --scenario); the golden-path suite records each journey once.',
+    );
+    return 1;
+  }
 
   // An explicit --url means the server is already running: run against it.
-  if (rest.includes('--url')) {
-    return runNodeAsync(script, rest, env);
+  if (args.includes('--url')) {
+    return runNodeAsync(script, args, env);
   }
 
   // No --url: self-boot the target project's own dev server, run, stop.
   const cwd = process.cwd();
-  const { value: command, rest: afterCmd } = takeOption(rest, 'command');
+  const { value: command, rest: afterCmd } = takeOption(args, 'command');
   const { value: bootTimeout, rest: profileArgs } = takeOption(afterCmd, 'boot-timeout');
   const fw = detectFramework(cwd);
   const label =
@@ -164,6 +223,16 @@ async function runProfile() {
   return code;
 }
 
+function runCompare() {
+  // Sides may be files relative to the user's directory (ids pass through).
+  const fixToken = t =>
+    t.endsWith('.json') || existsSync(path.resolve(process.cwd(), t)) ? path.resolve(process.cwd(), t) : t;
+  const args = absolutize(rest, ['out']).map((a, i, all) =>
+    a.startsWith('--') || all[i - 1] === '--out' ? a : a.split(',').map(fixToken).join(','),
+  );
+  return runNodeAsync('server/compare-cli.js', args, projectEnv());
+}
+
 function runDashboard() {
   // Standalone server: serves the prebuilt dashboard and the backend API, so an
   // installed Spectra needs no dev toolchain. projectEnv() points it at this
@@ -194,6 +263,17 @@ switch (command) {
         process.exit(1);
       });
     break;
+  case 'compare':
+  case 'reports': {
+    const run = command === 'compare' ? runCompare() : runNodeAsync('server/reports-cli.js', rest, projectEnv());
+    run
+      .then(code => process.exit(code ?? 0))
+      .catch(err => {
+        console.error(`spectra: ${err.message}`);
+        process.exit(1);
+      });
+    break;
+  }
   case 'dashboard':
   case 'ui':
     runDashboard();