@scope-profiler/plotly 0.2.0 → 0.4.0

package/CHANGELOG.md

@@ -0,0 +1,61 @@
+# Changelog
+
+## Unreleased
+
+- Validate row fields, format versions, call ancestry, and duplicate cells with
+  actionable errors. Preserve nested layout defaults and isolate theme presets.
+- Use stable colors; add `createColorRegistry()` and scoped `createFigureBuilder()`.
+- Add time-series variability bands/error bars and absolute/percent comparison
+  deltas, including diagnostics for missing or zero baselines.
+- Add `customdata.identity`, `getPointIdentity()`, linked legend groups, and
+  `disposeFigure()`. Custom data now uses objects with numeric metric keys;
+  consumers accessing raw custom data should migrate to `getPointIdentity()`.
+- Preserve unequal density-bin boundaries, reject overlaps, and retain gaps.
+- Detect cycles after Sankey aggregation, prefer measured edge weights, expose
+  omitted/inferred edges, and hide traces with no usable links.
+- Ship typed payloads and a discriminated `PlotData` union with consumer checks.
+- Add browser rendering/interaction tests, JavaScript coverage enforcement,
+  deterministic benchmark configuration, and bundled-asset synchronization.
+
+## 0.4.0
+
+### Fixed
+
+- Region, file and series names are escaped before they reach a hover label. A
+  name containing `%{...}` rewrote every hover template that mentioned it, and
+  one containing markup injected it into the label.
+- An empty payload no longer discards the `annotations` a caller passed through
+  `options.layout`, and hides the axes behind the "No data to display." message
+  instead of leaving an empty grid that reads as a broken chart.
+- `buildDensityFigure` places each cell at its own bin centre. A single bin
+  width taken from the first point misplaced the second run's cells whenever
+  two runs of different length were binned into the same number of bins.
+- `buildRooflineFigure` draws its ceiling in the theme's text colour rather than
+  a fixed near-black, which was invisible on the dark theme, and leaves room at
+  the top of the figure for the title it is alone in carrying.
+- `buildDurationsFigure` accepts a colour keyed by the bare rank behind a
+  `rank N` series, which it previously ignored in favour of the default cycle.
+- `buildRegionSummaryFigure` and `buildComparisonFigure` reject an unknown
+  `metric` instead of drawing a full chart of nulls.
+
+### Changed
+
+- `buildDurationsFigure` and `buildRankHeatmapFigure` order their categories by
+  pooled cost rather than by the order the exporter happened to walk, which
+  interleaved two runs with different region sets unpredictably.
+- `buildGanttFigure` puts a lane index on each bar and the lane names on the
+  axis (`tickvals`/`ticktext`) instead of repeating the lane string per
+  interval, which a large trace felt. `layout.yaxis.categoryarray` is gone;
+  read `layout.yaxis.ticktext` for the lane order.
+
+### Added
+
+- `updateFigure(plotly, element, figure, config)` redraws through Plotly's
+  `react`, keeping the viewer's zoom and pan across a rebuild.
+- `SUMMARY_METRICS`, the short metric names the region_statistics builders
+  accept, so a page can offer the list rather than guess it.
+- `laneBy`, `updateFigure`, `SUMMARY_METRICS` and `PlotlyLike.react`/`purge` are
+  declared in `index.d.ts`, which is now type-checked in CI, as is a
+  compile-only usage fixture. `package.json` gained a top-level `types` entry
+  so classic `node` module resolution finds the declarations at all.
+- ESLint and Prettier configuration, and a `lint` script, gated in CI.

package/README.md

@@ -1,5 +1,103 @@
 # @scope-profiler/plotly
 
+## Validation, updates, and linked charts
+
+Builders validate consumed records, including finite numbers and interval order.
+Errors identify the array, row index, and field. `validatePlotData()` also checks
+the envelope; unknown additional fields remain compatible. Present format
+versions must be positive integers. Duplicate heatmap/bar cells and call IDs
+are errors; aggregate repeated measurements before building a figure.
+
+`createColorRegistry(names, overrides)` returns an immutable name-to-color map.
+Default colors are also stable across filtering and row ordering. Use explicit
+colors when more categories need distinct hues than the eight-color palette.
+
+```js
+import {
+  createFigureBuilder,
+  createColorRegistry,
+  getPointIdentity,
+  updateFigure,
+  disposeFigure,
+} from "@scope-profiler/plotly";
+
+const build = createFigureBuilder({
+  theme: "dark",
+  colors: createColorRegistry(["solve", "assemble"]),
+  layout: { uirevision: "profile-123" },
+});
+await updateFigure(Plotly, element, build(payload));
+element.on("plotly_click", ({ points }) => {
+  const identity = getPointIdentity(points[0]);
+  // identity: { region, file, rank, call_id }; unavailable fields are null.
+  if (identity) console.log(identity.region);
+});
+// On unmount:
+disposeFigure(Plotly, element);
+```
+
+`createFigureBuilder()` snapshots defaults without modifying global theme state.
+Nested layout objects merge, while arrays replace. Keep `layout.uirevision`
+constant to retain zoom during `updateFigure()`; change it to reset the view.
+The fallback to `newPlot()` for bundles without `react()` cannot preserve zoom.
+`disposeFigure()` requires a bundle with `purge()`.
+
+Data-bearing traces expose `customdata.identity`; existing numeric hover
+fields are available as numeric keys on the same object. Reference/ideal lines
+have no selection identity. Imbalance means and time-series variability traces
+share their measured series' legend group.
+
+## Variability and comparisons
+
+`buildDurationTimeseriesFigure(payload, { variability: "band" })` displays a
+min/max band; `"error"` displays asymmetric error bars. Missing bounds produce
+gaps, and the default `"none"` retains the mean curve alone.
+
+`buildComparisonFigure(payload, { comparison: "absolute" | "percent" })`
+subtracts the first selected run from the second. `files` selects exactly two
+runs, `metric` selects a statistic, and `sortBy: "regression"` (default) sorts
+largest increases first; `"name"` sorts alphabetically. Percentage changes use
+the first run as denominator. Missing measurements and zero percentage
+baselines produce null values and `figure.diagnostics`, never invented zeros.
+The default `"side-by-side"` mode retains grouped bars. Higher values are not
+necessarily worse for every metric; interpret the sign for your chosen metric.
+
+## Graph and density semantics
+
+Flame figures reject ancestry cycles. Sankey figures remove edges that would
+introduce a cycle after region aggregation and report them in
+`figure.diagnostics`. Compact graph edges prefer `edge[valueKey]`, then
+`edge.value`. A child total is used only when the original graph has exactly
+one incoming non-self edge, with an explicit inference diagnostic. Unattributable
+weights are omitted with a diagnostic. Zero measured weights stay zero.
+
+Density figures with unequal grids use separate lane traces and exact bin
+edges; gaps remain empty and overlapping bins are rejected. Raw-seconds traces
+share one color range across lanes.
+
+## Development checks
+
+```sh
+npm ci
+npm run lint
+npm run check:types
+npm run test:coverage # Node 24 (CI runtime)
+npx playwright install chromium
+npm run test:browser
+npm run sync:asset
+```
+
+The browser suite renders exporter fixtures in light/dark themes and checks
+hover text, zoom retention, teardown, and a deterministic large timeline. It
+writes screenshots and build/render/heap measurements to `test-results`.
+The broad timing limits detect catastrophic regressions, not small speedups.
+
+For measured optimization, run the repository benchmark workflow using
+`packages/plotly/benchmarks/benchmark.toml`. It performs warmups and five runs,
+records medians and variation, and runs the package tests as correctness checks.
+`npm run benchmark` also prints per-build timings and process memory for the
+fixed 8,192-cell workload. Keep optimizations only after comparison says `keep`.
+
 Pure, framework-neutral Plotly figure builders for JSON written by
 `scope-profiler export plot-data --format json`. The package does not import
 Plotly; applications choose their own Plotly bundle.
@@ -14,7 +112,9 @@ npm install @scope-profiler/plotly plotly.js-dist-min
 import Plotly from "plotly.js-dist-min";
 import { buildGanttFigure, renderFigure } from "@scope-profiler/plotly";
 
-const payload = await fetch("/figures/gantt_data.json").then((response) => response.json());
+const payload = await fetch("/figures/gantt_data.json").then((response) =>
+  response.json(),
+);
 const figure = buildGanttFigure(payload);
 await renderFigure(Plotly, document.querySelector("#gantt"), figure);
 ```
@@ -27,10 +127,20 @@ produced it, so `buildFigure` can pick the builder for you:
 ```js
 import { buildFigure, renderFigure } from "@scope-profiler/plotly";
 
-const payload = await fetch("/figures/rank_heatmap_data.json").then((response) => response.json());
-await renderFigure(Plotly, document.querySelector("#chart"), buildFigure(payload));
+const payload = await fetch("/figures/rank_heatmap_data.json").then(
+  (response) => response.json(),
+);
+await renderFigure(
+  Plotly,
+  document.querySelector("#chart"),
+  buildFigure(payload),
+);
 ```
 
+`validatePlotData(payload, options)` performs the same checks on its own and
+returns the resolved kind, for a page that wants to report a bad file rather
+than catch a build.
+
 `buildFigure` rejects a document that is not `scope-profiler-plot-data` and one
 whose `format_version` is newer than this package supports. JSON written before
 scope-profiler stamped that envelope on every kind still works: the kind is then
@@ -41,13 +151,61 @@ inferred from the payload shape, and `{ plot: "gantt" }` settles it by hand.
 `buildGanttFigure`, `buildFlameFigure`, `buildCallgraphFigure`,
 `buildDensityFigure`, `buildDurationsFigure`, `buildDurationTimeseriesFigure`,
 `buildHistogramFigure`, `buildRankHeatmapFigure`, `buildImbalanceFigure`,
-`buildRegionSummaryFigure`, `buildLikwidFigure`, `buildSpeedupFigure`,
-`buildWeakScalingFigure` and `buildScalingEfficiencyFigure` each take
-`(payload, options)` and return a plain `{ data, layout }` figure.
+`buildRegionSummaryFigure`, `buildComparisonFigure`, `buildLikwidFigure`,
+`buildSpeedupFigure`, `buildWeakScalingFigure`,
+`buildScalingEfficiencyFigure` and `buildWeakScalingEfficiencyFigure` each
+take `(payload, options)` and return a plain `{ data, layout }` figure.
 
 Common options: `colors` (region or series name to color), `filterRegion(name,
-row)` to drop rows, `layout` to merge into the generated layout, and `metric`
-where a payload carries several.
+row)` to drop rows, `layout` to merge into the generated layout, `metric` where
+a payload carries several, and `theme` (see below).
+
+`filterRegion` always receives the name being filtered first, but its second
+argument is the record the builder is walking, and that differs by figure: a
+row of the payload's own array for most of them, a node or a call for
+`buildCallgraphFigure`, and the region's statistics object for
+`buildRegionSummaryFigure` and `buildComparisonFigure`.
+
+`buildGanttFigure` also takes `laneBy`: the default `"region"` gives each
+region and rank its own lane, which is the only way a nested profile stays
+legible; `"rank"` gives the compact one-row-per-rank view, which suits a flat
+profile compared across many ranks.
+
+The region_statistics builders take `metric` as either a short name or the
+stored field: `SUMMARY_METRICS` maps `avg`, `min`, `max`, `total`, `first`,
+`last`, `std` and `count` to the fields a document stores them under, and a
+name in neither spelling is rejected rather than drawn as an empty chart.
+
+The two efficiency curves store their y column under the same name, so which
+reading a payload gets comes from the document's own `plot` field rather than
+its rows. `buildScalingEfficiencyFigure` is for a _strong_-scaling study, where
+the problem size is fixed and the ideal is a speedup proportional to the rank
+count; `buildWeakScalingEfficiencyFigure` is for one that grows the problem
+with the machine, where the ideal is constant runtime.
+
+## Rendering and redrawing
+
+`renderFigure(plotly, element, figure, config)` draws a figure with any
+Plotly-compatible bundle. `updateFigure(...)` takes the same arguments but
+redraws into an element that already holds a plot, using Plotly's `react`, so
+the viewer's zoom and pan survive. Use it for anything that rebuilds a figure
+the viewer is already looking at -- a theme toggle, a changed filter, a new
+metric -- and keep `renderFigure` for the first draw.
+
+```js
+setTheme(darkMode ? "dark" : "light");
+await updateFigure(Plotly, element, buildFigure(payload));
+```
+
+## Themes
+
+Chrome -- text, gridlines, hover surface, the dashed ideal lines -- comes from
+a theme. The default `auto` sets no text colour and uses a half-transparent
+grey grid that reads on any background, so a figure inherits the host page.
+Pass `theme: "light" | "dark"`, or your own token object, to a builder, or call
+`setTheme(...)` once to change the default for every later build. Plotly bakes
+colours into the layout, so a theme change means rebuilding the figures. The
+categorical series palette is not themed: those hues read on both backgrounds.
 
 ## More than one run in a payload
 

package/package.json

@@ -1,21 +1,64 @@
 {
   "name": "@scope-profiler/plotly",
-  "version": "0.2.0",
+  "version": "0.4.0",
   "description": "Framework-neutral Plotly figure builders for scope-profiler plot-data JSON.",
   "license": "MIT",
   "repository": {
     "type": "git",
-    "url": "git+https://github.com/max-models/scope-profiler.git"
+    "url": "git+https://github.com/max-models/scope-profiler.git",
+    "directory": "packages/plotly"
   },
   "type": "module",
+  "sideEffects": false,
+  "types": "./src/index.d.ts",
   "exports": {
     ".": {
       "types": "./src/index.d.ts",
       "default": "./src/index.js"
+    },
+    "./dashboard": {
+      "types": "./src/dashboard.d.ts",
+      "default": "./src/dashboard.js"
     }
   },
-  "files": ["src", "README.md", "LICENSE.txt"],
-  "publishConfig": { "access": "public" },
-  "scripts": { "test": "node --test" },
-  "keywords": ["scope-profiler", "plotly", "profiling", "gantt", "flamegraph", "callgraph", "hpc", "mpi"]
+  "engines": {
+    "node": ">=20.11"
+  },
+  "files": [
+    "src",
+    "README.md",
+    "CHANGELOG.md",
+    "LICENSE.txt"
+  ],
+  "publishConfig": {
+    "access": "public"
+  },
+  "scripts": {
+    "test": "node --test test/*.test.js",
+    "lint": "eslint src test && prettier --check src test",
+    "format": "prettier --write src test",
+    "check:types": "tsc --noEmit -p tsconfig.json",
+    "test:browser": "playwright test",
+    "test:coverage": "node --test --experimental-test-coverage --test-coverage-include=src/index.js --test-coverage-lines=99 test/*.test.js",
+    "benchmark": "node benchmarks/build.mjs",
+    "sync:asset": "node scripts/sync-asset.js"
+  },
+  "devDependencies": {
+    "@playwright/test": "1.63.0",
+    "eslint": "^9.13.0",
+    "plotly.js-dist-min": "4.1.0",
+    "plotly3": "npm:plotly.js-dist-min@3.7.0",
+    "prettier": "^3.3.3",
+    "typescript": "^5.6.0"
+  },
+  "keywords": [
+    "scope-profiler",
+    "plotly",
+    "profiling",
+    "gantt",
+    "flamegraph",
+    "callgraph",
+    "hpc",
+    "mpi"
+  ]
 }

package/src/dashboard.d.ts

@@ -0,0 +1,51 @@
+import type {
+  BuildOptions,
+  Figure,
+  PlotlyLike,
+  ThemeName,
+  ThemeTokens,
+} from "./index.js";
+
+export function parseRegionFilter(text: string | null | undefined): string[];
+export function matchesRegionFilter(
+  region: string | null | undefined,
+  terms: string[],
+): boolean;
+export function regionFilter(
+  text: string | null | undefined,
+): ((region: string) => boolean) | undefined;
+export function documentTheme(): ThemeName;
+
+export interface FigureRegistryOptions {
+  /** A theme, or a function returning one, read at each render. */
+  theme?: ThemeName | ThemeTokens | (() => ThemeName | ThemeTokens);
+  /** Plotly config merged into every render. */
+  config?: object;
+  /** Figure builder, defaulting to `buildFigure`. */
+  build?: (payload: object, options?: BuildOptions) => Figure;
+}
+
+export interface RenderOptions extends BuildOptions {
+  /** A filter box's contents, turned into `filterRegion` for this figure. */
+  regionFilter?: string;
+  /** Builder for this figure only, overriding the registry's. */
+  build?: (payload: object, options?: BuildOptions) => Figure;
+}
+
+export interface FigureRegistry {
+  render(
+    container: Element | string | null | undefined,
+    payload: object,
+    options?: RenderOptions,
+  ): Promise<unknown> | undefined;
+  refresh(): Promise<unknown[]>;
+  forget(container: Element | string): boolean;
+  clear(): void;
+  readonly size: number;
+  watch(target?: EventTarget, event?: string): () => void;
+}
+
+export function createFigureRegistry(
+  plotly: PlotlyLike,
+  options?: FigureRegistryOptions,
+): FigureRegistry;

package/src/dashboard.js

@@ -0,0 +1,177 @@
+/** Browser glue for a page that renders scope-profiler figures.
+ *
+ * `index.js` is deliberately framework-neutral and pure: it turns a plot-data
+ * document into `{ data, layout }` and touches nothing else. A page built on
+ * it still needs two things every time, and both were being written from
+ * scratch by each deployment:
+ *
+ * * **Somewhere to keep the figures.** Plotly bakes colours into the layout,
+ *   so a theme toggle means rebuilding every figure on the page -- which means
+ *   remembering which containers hold one and what each was built from.
+ * * **A filter syntax.** `filterRegion` takes a predicate, and should: a
+ *   library has no business deciding how a user spells a filter. But every
+ *   dashboard grows the same comma-separated substring box, so here is one.
+ *
+ * This module is a separate entry point because it reaches for `document` and
+ * `window`; importing `@scope-profiler/plotly` never does.
+ */
+
+import { buildFigure, renderFigure, setTheme, updateFigure } from "./index.js";
+
+/** Split a filter box's contents into terms.
+ *
+ * A filter is a comma-separated list, each term matched case-insensitively as
+ * a substring of the region name. Empty terms -- a trailing comma while
+ * someone is still typing -- are dropped.
+ */
+export function parseRegionFilter(text) {
+  return String(text ?? "")
+    .split(",")
+    .map((term) => term.trim().toLowerCase())
+    .filter(Boolean);
+}
+
+/** Whether a region name matches any of `terms`.
+ *
+ * A leading "^" anchors a term to the start of the name, which is how a
+ * dashboard tells a group of regions apart from the regions that merely
+ * mention it: "prop:" also matches "setup prop: X", "^prop:" does not.
+ */
+export function matchesRegionFilter(region, terms) {
+  const name = String(region ?? "").toLowerCase();
+  return terms.some((term) =>
+    term.startsWith("^") ? name.startsWith(term.slice(1)) : name.includes(term),
+  );
+}
+
+/** A `filterRegion` predicate for a filter box, or undefined when it is empty.
+ *
+ * Undefined rather than a predicate that accepts everything: an empty box
+ * means every region, and passing no filter at all is both faster and what
+ * the builders document.
+ */
+export function regionFilter(text) {
+  const terms = parseRegionFilter(text);
+  return terms.length
+    ? (region) => matchesRegionFilter(region, terms)
+    : undefined;
+}
+
+/** The theme the host page is in, as `setTheme` spells it.
+ *
+ * Reads `data-theme` from the document element, the attribute a theme toggle
+ * conventionally stamps there. With neither value set the page is following
+ * the system preference, which "auto" already handles by committing to no
+ * text colour.
+ */
+export function documentTheme() {
+  if (typeof document === "undefined") return "auto";
+  const theme = document.documentElement?.dataset?.theme;
+  return theme === "dark" || theme === "light" ? theme : "auto";
+}
+
+/** Track the figures on a page so they can all be rebuilt at once.
+ *
+ * Each container remembers the payload and options it was last drawn from, so
+ * `refresh()` can rebuild it under whatever the theme now is. A container that
+ * has already been drawn is redrawn through `updateFigure`, which keeps the
+ * viewer's zoom and pan across a theme toggle or a changed filter.
+ *
+ * ```js
+ * const figures = createFigureRegistry(Plotly);
+ * figures.watch();                       // rebuild on a "themechanged" event
+ * await figures.render(el, payload, { regionFilter: box.value });
+ * ```
+ *
+ * @param plotly A Plotly-compatible bundle.
+ * @param options.theme A theme, or a function returning one, read at each
+ *   render. Defaults to `documentTheme`.
+ * @param options.config Plotly config, merged into every render.
+ * @param options.build Figure builder, defaulting to `buildFigure`. Override
+ *   for a figure no plot kind names, such as `buildComparisonFigure`.
+ */
+export function createFigureRegistry(plotly, options = {}) {
+  const {
+    theme = documentTheme,
+    config = {},
+    build: defaultBuild = buildFigure,
+  } = options;
+  // The spec lives here rather than on the container, so a page cannot be
+  // left holding a detached node through a property nobody remembers setting.
+  const specs = new Map();
+  const currentTheme = () => (typeof theme === "function" ? theme() : theme);
+
+  // A container that has left the document is dead weight: the page swapped
+  // it out, and redrawing into it paints nothing. Anything without the DOM
+  // property -- a test double, a server-side stub -- counts as live.
+  const isLive = (container) => container?.isConnected !== false;
+
+  async function draw(container, spec) {
+    const { payload, options: buildOptions = {}, drawn } = spec;
+    const build = buildOptions.build ?? defaultBuild;
+    const { build: _build, regionFilter: filterText, ...rest } = buildOptions;
+    const figure = build(payload, {
+      theme: currentTheme(),
+      ...(filterText != null ? { filterRegion: regionFilter(filterText) } : {}),
+      ...rest,
+    });
+    // First draw builds the plot; later ones react into it, so a theme toggle
+    // does not reset the view the reader had scrolled to.
+    const paint = drawn ? updateFigure : renderFigure;
+    spec.drawn = true;
+    return paint(plotly, container, figure, config);
+  }
+
+  return {
+    /** Draw `payload` into `container` and remember how, for `refresh()`. */
+    render(container, payload, buildOptions = {}) {
+      if (!container) return undefined;
+      const spec = specs.get(container) ?? {};
+      Object.assign(spec, { payload, options: buildOptions });
+      specs.set(container, spec);
+      return draw(container, spec);
+    },
+    /** Rebuild every live figure under the current theme. */
+    refresh() {
+      setTheme(currentTheme());
+      const drawing = [];
+      for (const [container, spec] of specs) {
+        if (!isLive(container)) specs.delete(container);
+        else drawing.push(draw(container, spec));
+      }
+      return Promise.all(drawing);
+    },
+    /** Stop tracking one container, and tear its plot down if it has one.
+     *
+     * Dropping the spec alone leaves Plotly's own event handlers attached to
+     * a node the page is finished with, so purge as well where the bundle
+     * offers it.
+     */
+    forget(container) {
+      const tracked = specs.get(container);
+      if (tracked?.drawn && typeof plotly?.purge === "function")
+        plotly.purge(container);
+      return specs.delete(container);
+    },
+    /** Stop tracking every container, tearing down each plot. */
+    clear() {
+      for (const container of [...specs.keys()]) this.forget(container);
+    },
+    /** Figures currently tracked, live or not. */
+    get size() {
+      return specs.size;
+    },
+    /**
+     * Rebuild on an event, and return the function that stops listening.
+     * Defaults to `window` and the "themechanged" event a toggle can dispatch.
+     */
+    watch(target, event = "themechanged") {
+      const source =
+        target ?? (typeof window === "undefined" ? undefined : window);
+      if (!source?.addEventListener) return () => {};
+      const listener = () => this.refresh();
+      source.addEventListener(event, listener);
+      return () => source.removeEventListener(event, listener);
+    },
+  };
+}