@toclocoinc/lattice-grid 1.51.0 → 1.53.0

package/docs/api-detail.html

@@ -437,7 +437,7 @@
 <div class="shell">
   <aside class="rail">
     <p class="rail__brand">Lattice Grid</p>
-    <p class="rail__sub">Developer guide · v1.51.0</p>
+    <p class="rail__sub">Developer guide · v1.53.0</p>
     <nav>
       <div class="rail__group">
         <span class="rail__label">Start here</span>
@@ -3396,11 +3396,126 @@ createGrid(right, {
       (<code>idle</code>, or a debounce), and prefer fewer, larger sources over many small ones where
       the shape of your data allows it.</p>
       <p><strong>Not supported alongside a union.</strong> <code>crossFilter</code> has no single
-      target once there is more than one parent; <code>profile</code> reduces one grid's own columns.
-      Both are refused with a warning rather than guessed at, and the top-level <code>follow</code> is
-      ignored in favour of each source's own.</p>
+      target once there is more than one parent; <code>profile</code> and <code>statistics</code>
+      reduce one grid's own columns. All three are refused with a warning rather than guessed at, and
+      the top-level <code>follow</code> is ignored in favour of each source's own.</p>
     </div>
 
+    <h2 id="derived-statistics">The relational statistics, as rows</h2>
+    <p class="lead-in">
+      One grid is the data; a second <em>is</em> the analysis of it. <code>statistics</code> projects
+      the figures that need two or more columns &mdash; or a second grid &mdash; into rows you can
+      sort, filter, chart and export like any others.
+    </p>
+    <p><strong>You probably do not need it for a single-column statistic.</strong> Those already have
+      a route: a derived <code>select</code> reduces a group with any kernel the totals row uses, and
+      that table is a superset of the statistics one. <code>select: { p95: { of: 'amount', fn: 'p95'
+      } }</code> works today, and so do <code>median</code>, <code>stddev</code>, <code>gini</code>,
+      <code>iqr</code>, <code>entropy</code>, <code>trimmedMean</code> and the rest.
+      <code>statistics</code> is for what <code>select</code> structurally cannot reach.</p>
+    <div class="example">
+      <p class="example__label">Which columns move together</p>
+      <pre><code>source: {
+  mode: <span class="str">'derived'</span>,
+  from: trades,
+  statistics: { fn: <span class="str">'correlation'</span>, columns: [<span class="str">'price'</span>, <span class="str">'volume'</span>, <span class="str">'spread'</span>] },
+  <span class="cmt">// -&gt; one row per PAIR: { a, b, coefficient, n }</span>
+}</code></pre>
+    </div>
+    <p><strong>Three statistics in this release.</strong> Each has one row shape, and the shape is the
+      contract:</p>
+    <ul>
+      <li><code>{ fn: 'correlation', columns, orient? }</code> &mdash; Pearson's r across N columns.
+        <code>orient: 'pairs'</code> (the default) gives <strong>one row per unordered pair</strong>,
+        <code>{ a, b, coefficient, n }</code>. Long form by default because that is what
+        a grid sorts, filters and charts well &mdash; "the three most correlated pairs" is then a
+        sort and a <code>limit</code> on the derived grid. Only the upper triangle is emitted: r is
+        symmetric, so <code>(a,b)</code> and <code>(b,a)</code> are one finding, and a column against
+        itself is 1 by definition. <code>orient: 'matrix'</code> gives the classic square instead,
+        one row per column with a field per other column, for a heat map.</li>
+      <li><code>{ fn: 'series', of, by, periodsPerYear? }</code> &mdash; <strong>one row per
+        metric</strong>, <code>{ metric, value, n }</code>. Note <em>per metric</em>, not
+        per point: <code>grid.statistics.series</code> returns a summary &mdash; <code>n</code>,
+        <code>first</code>, <code>last</code>, <code>change</code>, <code>changePercent</code>,
+        <code>volatility</code>, <code>annualisedVolatility</code>, <code>growth</code>,
+        <code>maxDrawdown</code>, <code>maxDrawdownFrom</code>, <code>maxDrawdownTo</code>,
+        <code>autocorrelation</code>, <code>upDays</code>, <code>downDays</code> &mdash; and not a
+        value per row. The shape is the one <code>profile</code>'s <code>orient: 'metrics'</code>
+        already emits, deliberately, rather than a third convention for the same idea.
+        <code>by</code> is required and never guessed, because kernels see rows in arrival order and
+        that is not the grid's sort.</li>
+      <li><code>{ fn: 'datasetVsDataset', with, columns? }</code> &mdash; <strong>one row per compared
+        column</strong>, largest difference first: <code>{ column, measure, magnitude, distance,
+        direction, nA, nB, reliable, unmatched }</code>. Both sides are read over their
+        <em>filtered</em> rows, and <strong>the peer is watched</strong> &mdash; an edit or a filter
+        on it re-derives the comparison, because a comparison whose other side has moved is wrong
+        rather than merely late. A column present on only one side cannot be compared and is still
+        reported, with a null <code>magnitude</code> and <code>unmatched</code> set to
+        <code>'A'</code> or <code>'B'</code>, so you see that it was skipped and why.</li>
+    </ul>
+    <p><strong>Every row says how much it saw.</strong> <code>n</code> is the rows the figure
+      covered, and it is on the row because a derived statistic travels: a coefficient exported to
+      CSV or bound to a chart has left every bit of its context behind, and &ldquo;r = 0.98 over
+      eleven rows&rdquo; is a different claim from the same number over eleven thousand.</p>
+    <p><strong>What the row does <em>not</em> tell you: whether the source was windowed.</strong> A
+      statistic over a source holding fewer rows than match its filters is computed on the loaded
+      window rather than the whole set. The grid detects that from the <em>source's</em> own
+      counters and says so in a console warning
+      (<code>[lattice] correlation on &hellip; computed over N of M matching rows</code>) &mdash; and
+      that remains the signal to watch. A derived source cannot reach those counters: a grid's public
+      <code>rows.matchCount()</code> reports the <em>loaded</em> matches, so on a bounded stream
+      evicted to 200 of 2,000 rows it returns 200 and agrees exactly with <code>rows.count()</code>.
+      Rather than ship a flag that could never be true, no such flag is emitted; <code>n</code> says
+      what the figure actually saw and nothing more is claimed.</p>
+    <p><strong>A terminal producer, not a pipeline stage.</strong> A correlation is one row per pair,
+      a series summary one row per metric, a comparison one row per column &mdash; none of which is
+      one row per group, so there is no position in
+      <code>unnest &rarr; where &rarr; bucket &rarr; groupBy &rarr; select &rarr; sort &rarr;
+      limit</code> for <code>statistics</code> to occupy. It replaces the pipeline, exactly as
+      <code>profile</code> does. Those keys are now <strong>ignored with a warning that names
+      them</strong> rather than discarded in silence, for both producers. Sort, filter or limit the
+      derived grid itself, or chain a second derived grid whose <code>from</code> is this one.
+      <code>profile</code> and <code>statistics</code> are mutually exclusive and declaring both is
+      refused, by name, when the source is built.</p>
+
+    <h3 id="derived-producer-cost">What a terminal producer costs</h3>
+    <p><strong>No terminal producer patches incrementally &mdash; know this before you point one at a
+      live feed.</strong> The grouped pipeline maintains its grouping across changes: an edit that
+      names the rows it touched re-reduces only the groups those rows belong to. A producer has no
+      grouping to maintain, so <em>every</em> change on the parent re-derives its whole output. This
+      has always been true of <code>profile</code> and was not previously written down; it is written
+      down here now, and it applies to <code>statistics</code> in the same way.</p>
+    <p>Measured on a synthetic 200,000-row grid, five changes after a warm-up, one derived grid
+      attached (machine-dependent &mdash; run <code>node bench/derived-producers.mjs</code> against
+      your own shape):</p>
+    <table>
+      <thead><tr><th>Derived shape</th><th>Per change at 200k rows</th></tr></thead>
+      <tbody>
+        <tr><td>grouped, no <code>where</code> &mdash; <em>the patching pipeline</em></td><td>~1&nbsp;ms</td></tr>
+        <tr><td><code>profile</code>, one column</td><td>~20&nbsp;ms</td></tr>
+        <tr><td><code>statistics</code> <code>correlation</code>, 2 columns (1 pair)</td><td>~5&nbsp;ms</td></tr>
+        <tr><td><code>statistics</code> <code>correlation</code>, 6 columns (15 pairs)</td><td>~32&nbsp;ms</td></tr>
+        <tr><td><code>statistics</code> <code>series</code></td><td>~23&nbsp;ms</td></tr>
+        <tr><td><code>statistics</code> <code>datasetVsDataset</code>, two grids</td><td>~36&nbsp;ms</td></tr>
+        <tr><td><code>where</code>, no <code>groupBy</code> &mdash; <em>a pipeline shape that also never patches</em></td><td>~700&nbsp;ms</td></tr>
+      </tbody>
+    </table>
+    <p><strong>Correlation is quadratic in its column count.</strong> It scans the rows once per pair,
+      so N columns cost N&middot;(N&minus;1)/2 passes: six columns is fifteen passes, twenty columns is
+      a hundred and ninety. Correlate the columns you mean rather than every numeric column you
+      have.</p>
+    <p><strong>The costs of several panels add up.</strong> Every derived grid attached to a parent
+      re-derives on the same change, so three analysis panels over one grid cost the sum of the
+      three, not the largest. This is BACKLOG-0001044's known gap &mdash; a hidden derived grid still
+      does full read and compute work &mdash; with a larger constant behind it; a hidden analysis tab
+      recomputing a correlation matrix on every tick is exactly that cost.</p>
+    <p><strong>The escape hatch is <code>refresh</code>, and it already exists.</strong>
+      <code>'idle'</code> is the default and coalesces a burst of changes into one derivation on the
+      next frame; a <strong>number</strong> is a debounce in milliseconds; <code>'live'</code>
+      derives on every change and is the one to avoid for an expensive analysis over a ticking feed;
+      <code>'manual'</code> stops automatic derivation entirely, leaving the host to drive the
+      source. Below a few tens of thousands of rows none of this matters.</p>
+
     <h2 id="cross-filter">Cross-filtering</h2>
     <p class="lead-in">
       A derived panel can filter the grid it summarises. Click a region in the summary and the
@@ -6876,7 +6991,8 @@ grid.import.apply(preview);</code></pre>
           <tr><td class="name">createKPI</td><td class="desc">A KPI / stat-tile view (module <code>kpi</code>) of a dataset as a panel of aggregate tiles — each tile a <code>sum</code>, <code>avg</code>, <code>min</code>, <code>max</code>, <code>count</code>, <code>countDistinct</code> or a <code>custom</code> reducer over the routed rows, with an optional <code>filter</code> predicate, number <code>format</code> (<code>number</code>/<code>currency</code>/<code>percent</code>/<code>compact</code>), a <code>baseline</code> for a delta, semantic threshold bands (<code>thresholds</code> with two cut points and a direction, or an explicit <code>bands</code> list, giving a <code>good</code>/<code>warn</code>/<code>critical</code> status kept separate from any accent), and an optional <code>sparkline</code> series. It is a dataset viewer like any other: it consumes data through the same keyed-diff <code>rows.apply({ add, update, remove })</code> contract a grid exposes, so a Data Router can <code>attach(value, kpi)</code> and drive a KPI panel beside a grid, a kanban and a chart off one feed. Updates are incremental — a delta adjusts each tile's running accumulator by only the rows it carries (an add contributes, a remove reverses, an update reverses-then-contributes), the two bounded exceptions being a <code>min</code>/<code>max</code> whose current extreme is removed (a rescan of that tile's own value multiset) and a <code>custom</code> reducer (recomputed over the filtered store, an arbitrary function having no inverse). Each tile is a labelled <code>&lt;figure&gt;</code>, focusable and keyboard-activatable, its value announced, the sparkline respecting <code>prefers-reduced-motion</code>; a <code>tile:click</code> event (also from the keyboard) lets a host drill down or filter a routed grid. Pass <code>null</code> as the element for a headless panel that computes the same tile model without a DOM. Not a dashboard layout engine (that is the parked dashboard generator) and no charting beyond the minimal sparkline (that is the charts module).</td></tr>
           <tr><td class="name">createAI</td><td class="desc">Create an AI narrative / insights controller (module <code>ai</code>) over a live grid. It produces a short, plain-language narrative of the grid's <em>computed</em> figures — a per-KPI / per-chart / per-column <code>explain</code>, or an <code>insights</code> panel over the current filtered view. The grid makes no AI call of its own: <code>createAI</code> imports no provider SDK, holds no key, and makes no network request; it calls one host callback, <code>ask({ system, messages, prompt, tools?, schema?, signal })</code> — your model, your key, your privacy decision — the same philosophy as the data adapters. Two grounding paths feed one guard: where the provider offers tool-use the model is given a curated read-only tool set (<code>getSchema</code>, <code>getProfile</code>, <code>getStatistics</code>, <code>getForecast</code>, <code>runQuery</code>) and the grid's own engine computes what it asks for; otherwise the module builds a facts packet from <code>grid.statistics</code>/the profile/forecasts/view counts and passes it in the prompt. Every figure in the narrative is reconciled against the values the engine produced this render — an ungrounded number is stripped before display (the number-reconciliation guard), so a hallucinated figure never reaches the user. The prompt is constrained to narrate-only; the layer is read-only and never mutates data. <code>redact</code> (a column id, a list, or a predicate) and <code>maxRows</code> bound what the module hands <code>ask()</code>, and the module sends nothing itself; an <code>ask()</code> error surfaces a friendly message and the grid stays fully usable, AI being additive rather than load-bearing. Complementary to <code>grid.ai</code> (the intent/plan skill layer): pass no <code>ask</code> and it adopts the grid's configured <code>ai.ask</code>, running the facts-packet path over it. Pass a headless grid for a DOM-free narrative; <code>facts(target)</code> returns the exact grounded packet without calling <code>ask()</code>. UMD global <code>LatticeGridAI</code>.</td></tr>
                     <tr><td class="name">createTabs</td><td class="desc">Create a tabbed grid (module <code>tabs</code>): a <code>role="tablist"</code> strip above a stack of <code>role="tabpanel"</code> regions, each hosting its own, independently-configured <code>createGrid</code> instance — "configure each tab as per a normal grid" rather than one grid whose state is swapped (<code>ColumnModel#applyState</code> only repositions/hides/resizes existing columns by id; it carries no field, type or row data, so a state-swap only works when every tab shares one schema). <code>createGrid</code> is injected (<code>createTabs(el, { createGrid, tabs })</code>), the same pattern the React/Vue/Svelte adapters use, so the module imports no engine code and adds nothing to a page that does not load it. A tab that names <code>from: '&lt;tabId&gt;'</code> gets a <code>source: { mode: 'derived', from: &lt;the parent tab’s live grid&gt;, where, group, join, … }</code> wired for it automatically — reusing the shipped derived-source mechanism rather than a new config-inheritance one — and activating a derived tab materialises its whole ancestor chain first; a cyclic <code>from</code> graph is refused (naming the exact cycle) when <code>createTabs</code> is called, not at first click. A tab’s grid mounts on first activation and then stays alive, hidden, so its scroll/selection/filters/sort/grouping/expansion — and an open cell/row editor, left exactly as it was, uncommitted and undiscarded — survive a switch natively; <code>destroy()</code> tears every mounted tab down. The strip is a real tablist with <code>aria-selected</code>, a roving <code>tabindex</code>, and manual-activation keyboard handling (arrows/Home/End move focus, Enter/Space or a click activates). Events: <code>tab:changed</code>, a cancellable <code>beforeTabChange</code> paired with <code>tabChange:cancelled</code>. UMD global <code>LatticeGridTabs</code>.</td></tr>
-<tr><td class="name">createDevtools</td><td class="desc">Mount the devtools panel against a grid, including its accessibility checks.</td></tr>
+          <tr><td class="name">createLayout</td><td class="desc">Create a reconfigurable dashboard layout (module <code>layout</code>): a cell grid inside an element, and a set of windows on it that a user can move, resize and close by drag <em>or</em> by keyboard &mdash; the surface a customer would otherwise reach for GridStack to get. It is <strong>payload-agnostic</strong>: a window body is a <code>div</code> with an <code>id</code> that the module creates, sizes and never reads, so it imports no engine code at all (not even <code>createGrid</code>) and its own code is 11,202 bytes gzipped, measured against a 62,206-byte fixed bundle floor. <code>columns</code>/<code>rows</code> divide the element; <code>overflowX</code> and <code>overflowY</code> are <em>independent</em> axes, each <code>'static'</code> (tracks divide the container with <code>minmax(0, 1fr)</code>) or <code>'scroll'</code> (tracks take a fixed <code>columnWidth</code>/<code>rowHeight</code> and the canvas extends past the viewport, so a column keeps the size it asked for &mdash; measured: shrinking a 600px host to 300px leaves a 200px column at 200px). Spacing takes a real CSS length: a number of pixels, or <code>'200px'</code>, <code>'25%'</code>, <code>'1fr'</code>, <code>'2rem'</code>; anything else is refused by name and replaced by the default, because the value reaches an inline style. Windows are placed by 1-based <code>xPos</code>/<code>yPos</code>/<code>xSize</code>/<code>ySize</code>, or auto-placed in the first free cell; <code>chrome</code> defaults on, and <code>closable</code>/<code>movable</code>/<code>resizable</code> all default <em>off</em>, so a fixed dashboard is fixed without opting out. <code>compact: 'vertical'</code> pushes displaced windows down then pulls them up (<code>window:moved</code> carries both <code>to</code> and <code>landed</code>); <code>'none'</code> keeps every window where it is put. <code>closable</code>/<code>movable</code>/<code>resizable</code> each also take a <em>layout-level</em> default of the same name, which a window's own boolean overrides, and <code>setInteractive(true|false|{movable, resizable, closable})</code> changes that default at runtime &mdash; the &ldquo;Edit layout&rdquo; button &mdash; without destroying the layout or any payload in it, with <code>getInteractive()</code> reading it back. Locking always wins and unlocking never overrides an opt-out: <code>setInteractive(false)</code> pins a window that declared <code>movable: true</code>, while <code>setInteractive(true)</code> leaves a window that declared <code>movable: false</code> pinned. The config key and the method are deliberately different: <code>movable: false</code> in the config states the <em>default</em> for windows that declare nothing and takes nothing away from one that opted in, whereas <code>setInteractive(false)</code> is an <em>active lock</em>. <code>getInteractive()</code> is three-valued &mdash; <code>undefined</code> for unset, <code>true</code>, or <code>false</code> for a lock &mdash; and a key carrying <code>undefined</code> is treated as absent, so <code>setInteractive(getInteractive())</code> is a no-op in every state. It moves both halves of the enforcement, the rendered handles <em>and</em> the pointer and keyboard gesture checks, and fires no event because a mode is not an arrangement &mdash; <code>getLayout()</code> neither carries it nor restores it. A locked layout is not a read-only dashboard: what is inside a window is configured with that payload's own settings. Keyboard parity with the drag: a focusable handle per window running the kanban board's grab/move/drop/cancel model, with a polite live region announcing grabbed, every tentative position, dropped, cancelled and reverted. It owns exactly <strong>one</strong> <code>ResizeObserver</code> for the whole layout, over two targets, and tells payloads their new content box through <code>window:resized</code> &mdash; it never calls into a payload, because it cannot know what one is. Events: <code>window:moved</code>, <code>window:resized</code>, <code>window:closed</code>, <code>layout:changed</code>, the cancellable <code>beforeWindowMove</code>/<code>beforeWindowResize</code>/<code>beforeWindowClose</code> and their <code>*:cancelled</code> pairs; drag progress is not emitted per frame. <code>getLayout()</code>/<code>setLayout()</code> round-trip the arrangement as plain JSON, and <code>getState()</code>/<code>setState()</code> are the versioned pair. Closing a window does <strong>not</strong> destroy its payload &mdash; the container is handed back on <code>window:closed</code> and the host owns that lifecycle. UMD global <code>LatticeGridLayout</code>.</td></tr>
+          <tr><td class="name">createDevtools</td><td class="desc">Mount the devtools panel against a grid, including its accessibility checks.</td></tr>
           <tr><td class="name">createGantt</td><td class="desc">Create a project-planning Gantt controller (module <code>gantt</code>) over a task list and a dependency list. A CPM engine (<code>computeSchedule</code>) computes each task's early/late start and finish, its slack and the zero-float critical path, honouring the four link types (<code>LINK_TYPES</code>: FS/SS/FF/SF) with lag, and recomputes on every edit — emitting <code>schedule</code> or, on a dependency cycle or bad input, <code>error</code> (a code from <code>SCHEDULE_ERROR</code>). Milestones are zero-duration points; summary (WBS) tasks are derived from their children (earliest start, latest finish, weighted progress) rather than scheduled; <code>findViolations</code> flags a task placed earlier than its predecessors allow, and <code>toISODate</code> maps an engine day-number back to a calendar date.</td></tr>
           <tr><td class="name">createLatticeGridElement</td><td class="desc">Build the element class without registering it, for a custom registry.</td></tr>
           <tr><td class="name">createMessages</td><td class="desc">Build a message catalogue. A partial set lays over the built-in British English one.</td></tr>