@toclocoinc/lattice-grid 1.16.0 → 1.18.0

package/docs/api-detail.html

@@ -437,7 +437,7 @@
 <div class="shell">
   <aside class="rail">
     <p class="rail__brand">Lattice Grid</p>
-    <p class="rail__sub">Developer guide · v1.16.0</p>
+    <p class="rail__sub">Developer guide · v1.18.0</p>
     <nav>
       <div class="rail__group">
         <span class="rail__label">Start here</span>
@@ -1368,6 +1368,17 @@ dataTypes: {
       is <code>1.00 MB</code> rather than <code>1,000 kB</code>.
     </p>
 
+    <p class="lead-in">
+      <code>compound: ['ft', 'in']</code> renders one stored number across an ordered subset of the
+      system's units &mdash; a length as <code>5 ft 11 in</code>, a duration as <code>1 h 23 m</code>.
+      It is display and parse only: the stored value stays a single base-unit number, so sorting,
+      filtering, grouping and totals are the same arithmetic as any other unit column. The units are
+      sorted largest to smallest, the smallest carries the remainder, and parsing sums the parts, so
+      the display round-trips through a paste. The mid-value <em>editor</em> &mdash; keystroke
+      roll-over between feet and inches, caret behaviour at a rung boundary &mdash; is a separate,
+      later piece of work; this is the read-and-paste half.
+    </p>
+
     <div class="why">
       <p><strong>Angles wrap, so their mean is replaced.</strong> The average of 359° and 1° is 0°,
       and the arithmetic answer (180°) is a confident, plausible number pointing in exactly the
@@ -1961,12 +1972,14 @@ grid.destroy();
     </p>
 
     <p class="lead-in">
-      On a memory source the grand total is <strong>maintained incrementally</strong>: a cell
-      update moves the running value by the difference rather than re-reducing the column, so a
-      totals row costs the same on a million rows as on a thousand. This applies to
-      <code>sum</code>, <code>avg</code>, <code>countValues</code>, <code>min</code> and
-      <code>max</code> on numeric columns. Everything else re-reduces, and so does the
-      incremental path itself whenever it cannot reach the right answer:
+      On a memory source totals are <strong>maintained incrementally</strong>: a cell update moves
+      the running value by the difference rather than re-reducing the column, so a totals row costs
+      the same on a million rows as on a thousand. This applies to both the grand total and each
+      group subtotal, for <code>sum</code>, <code>avg</code>, <code>countValues</code>,
+      <code>min</code> and <code>max</code> on numeric columns. A cell edit that moves a row between
+      groups is subtracted from its old group and added to its new one; only the affected groups are
+      touched. Everything else re-reduces, and so does the incremental path itself whenever it cannot
+      reach the right answer:
     </p>
     <div class="table-wrap">
       <table>
@@ -1979,7 +1992,7 @@ grid.destroy();
           <tr><td class="name">Adding or removing rows</td><td class="desc">Re-reduced once, then incremental again.</td></tr>
           <tr><td class="name">Filtering, sorting or grouping</td><td class="desc">Re-reduced once, because the rows contributing to the total have changed.</td></tr>
           <tr><td class="name">Totals above ~1e15</td><td class="desc">Re-reduced. Past that magnitude a small change no longer moves a 64-bit float, and a running total would silently stop tracking the data.</td></tr>
-          <tr><td class="name">Group totals</td><td class="desc">Re-reduced per group on every change. Only the grand total is incremental.</td></tr>
+          <tr><td class="name">Group subtotals</td><td class="desc">Maintained incrementally the same way the grand total is: an in-group edit applies the difference, and a cross-group move subtracts from the old group and adds to the new. A <code>min</code>/<code>max</code> move that leaves a group's current extreme reseeds only that group. A custom group total, or a statistical reduction, keeps the full per-group pass.</td></tr>
         </tbody>
       </table>
     </div>
@@ -2073,10 +2086,12 @@ pivot: { groupTotals: 'after', totalsLabel: 'All regions' }</code></pre>
       <pre><code>totalOnlyChangedColumns: <span class="kw">true</span></code></pre>
     </div>
     <p class="lead-in">
-      Group totals re-reduce every totalled column on every change, including columns the change
-      did not touch. Switching this on reduces only the columns whose values actually moved, and
-      an update that rewrites a field with the value it already held reduces nothing at all,
-      which is what a feed resending unchanged fields looks like.
+      By default the total stage considers every totalled column on every change, even columns the
+      change did not touch. Switching this on considers only the columns whose values actually moved,
+      and an update that rewrites a field with the value it already held reduces nothing at all,
+      which is what a feed resending unchanged fields looks like. This narrows <em>which</em> columns
+      are looked at; the incremental grand total and group subtotals described above narrow
+      <em>how</em> each one is brought up to date, and the two compound.
     </p>
     <p class="lead-in">
       On a million rows in seven groups, a single-cell update with four totalled columns:
@@ -2547,6 +2562,49 @@ grid.destroy();
 <span class="kw">return</span> offered.includes('sum') ? <span class="num">6</span> : <span class="num">0</span>;   <span class="cmt">// sum is offered on a number column</span></code></pre>
     </div>
 
+    <div class="why">
+      <p><strong>The group subtotals and the grand total can reduce differently.</strong> By default
+      one <code>total</code> drives both, and that is unchanged. When a column needs, say, an
+      <em>average per group</em> under a <em>sum of everything</em>, set the two independently with
+      the <code>scope</code> option: <code>setTotal(id, 'avg', { scope: 'group' })</code> and
+      <code>setTotal(id, 'sum', { scope: 'grand' })</code>. A scope with no override falls back to
+      <code>total</code>, and passing no <code>scope</code> sets the shared <code>total</code> and
+      clears both overrides — so the one-property behaviour is exactly what it was. The same split is
+      declarable on a column as <code>groupTotal</code> / <code>grandTotal</code>, and it persists in
+      saved views alongside <code>total</code>.</p>
+    </div>
+    <div class="example">
+      <p class="example__label">Group average under a grand sum, executed</p>
+      <pre data-run="js" data-expect="avg groups under sum grand" data-covers="method:columns config:columns"><code><span class="kw">const</span> { createHeadlessGrid } = <span class="kw">await</span> import('../packages/core/src/index.js');
+
+<span class="kw">const</span> grid = createHeadlessGrid({
+  columns: [{ field: 'amount', type: 'number', total: 'sum' }, { field: 'region' }],
+  rows: [
+    { id: '1', amount: 2, region: 'N' }, { id: '2', amount: 6, region: 'N' },
+    { id: '3', amount: 4, region: 'S' }, { id: '4', amount: 8, region: 'S' },
+  ],
+  rowKey: 'id',
+  grandTotalRow: <span class="kw">true</span>,
+});
+grid.columns.group(['region']);
+
+<span class="cmt">// Group subtotals average within each region; the grand total sums everything.</span>
+grid.columns.setTotal('amount', 'avg', { scope: 'group' });
+grid.columns.setTotal('amount', 'sum', { scope: 'grand' });
+
+<span class="cmt">// The first group subtotal (region N: (2+6)/2 = 4) and the grand row (sum = 20).</span>
+<span class="kw">let</span> groupSubtotal = <span class="kw">null</span>;
+<span class="kw">for</span> (<span class="kw">let</span> i = <span class="num">0</span>; i &lt; grid.rows.count(); i++) {
+  <span class="kw">const</span> r = grid.rows.get(i);
+  <span class="kw">if</span> (r &amp;&amp; r.group &amp;&amp; !r.grandTotal &amp;&amp; groupSubtotal === <span class="kw">null</span>) groupSubtotal = r.totals.amount;
+}
+<span class="kw">const</span> grand = grid.rows.get(grid.rows.count() - <span class="num">1</span>).totals.amount;
+
+grid.destroy();
+<span class="kw">return</span> (groupSubtotal === <span class="num">4</span> &amp;&amp; grand === <span class="num">20</span>)
+  ? 'avg groups under sum grand' : `group ${groupSubtotal}, grand ${grand}`;</code></pre>
+    </div>
+
     <h2 id="sticky-group-headings">Sticky group headings</h2>
     <p class="lead-in">
       Scrolling inside a group keeps that group's headings pinned above the rows, so the rows on
@@ -2554,9 +2612,9 @@ grid.destroy();
     </p>
 
     <div class="example">
-      <p class="example__label">On by default; turn it off or change the cap</p>
-      <pre><code>createGrid(element, { stickyGroupHeaders: <span class="kw">false</span> });   <span class="cmt">// off</span>
-createGrid(element, { stickyGroupHeaders: 3 });       <span class="cmt">// stack up to three</span></code></pre>
+      <p class="example__label">Off by default; opt in or set the cap</p>
+      <pre><code>createGrid(element, { stickyGroupHeaders: <span class="kw">true</span> });    <span class="cmt">// on, up to two</span>
+createGrid(element, { stickyGroupHeaders: 3 });       <span class="cmt">// on, up to three</span></code></pre>
     </div>
 
     <div class="why">
@@ -2608,7 +2666,8 @@ createGrid(element, { stickyGroupHeaders: 3 });       <span class="cmt">// stack
       <table>
         <thead><tr><th>Option</th><th>Type</th><th>Description</th></tr></thead>
         <tbody>
-          <tr><td class="name">retainSource</td><td class="type">boolean</td><td class="desc">Default <code>true</code>. Set <code>false</code> to keep only the packed columns and reconstruct a plain row object from them on demand. <code>rows.data()</code> then returns freshly reconstructed objects — a new object each call — so <code>row === sourceObject</code> and a custom renderer reading <code>row.sourceObject</code> no longer hold, and equality becomes value-based. Cell values are identical either way, so <code>get()</code>, <code>byKey()</code>, <code>value()</code> and <code>values()</code> are unaffected.</td></tr>
+          <tr><td class="name">retainSource</td><td class="type">boolean</td><td class="desc">Default <code>true</code>. Set <code>false</code> to keep only the packed columns and reconstruct a plain row object from them on demand. <code>rows.data()</code> then returns freshly reconstructed objects — a new object each call — so <code>row === sourceObject</code> and a custom renderer reading <code>row.sourceObject</code> no longer hold, and equality becomes value-based. Cell values are identical either way, so <code>get()</code>, <code>byKey()</code>, <code>value()</code> and <code>values()</code> are unaffected. On its own this drops only the store's reference array, not the objects: the source layer still holds them.</td></tr>
+          <tr><td class="name">dropSourceRows</td><td class="type">boolean</td><td class="desc">Default <code>false</code>. Set <code>true</code> to release the caller's row objects from the <em>source layer</em> and the grid config once the store is built, so the packed columns are the only resident copy. This is where the large reduction lives — roughly an order of magnitude at a million rows — because the caller's objects are the dominant term. Implies <code>retainSource: false</code> (the store must reconstruct), so it carries the same identity tradeoff. An impure computed column — a shadow or a rank/positional column — is never stored, so it cannot be served from the columns: it warns once and drops out rather than answering wrongly, and <code>rows.move()</code> is refused. Do not enable it on a grid that sorts, filters, groups or totals on such a column, or that reorders rows in place.</td></tr>
         </tbody>
       </table>
     </div>
@@ -2618,14 +2677,17 @@ createGrid(element, { stickyGroupHeaders: 3 });       <span class="cmt">// stack
   columns,
   rowKey: 'id',
   rows,
-  ingest: { retainSource: <span class="kw">false</span> },
+  <span class="cmt">// Release the caller's objects entirely; the packed columns are the sole copy.</span>
+  ingest: { dropSourceRows: <span class="kw">true</span> },
 });</code></pre>
     </div>
     <div class="why">
-      <p>The saving is the store's own copy of the object references, not the objects
-      themselves: whoever handed the grid its rows still owns them. The footprint drops
-      materially only when the grid becomes the sole holder of the data. Reach for this when
-      you can let go of the source array and do not depend on caller identity through
+      <p>With <code>retainSource: false</code> alone the saving is only the store's own copy of the
+      object references, not the objects themselves: the source layer and the grid config still
+      hold the array, so whoever handed the grid its rows keeps them alive. <code>dropSourceRows</code>
+      releases those references too, so once the caller lets go the objects can be collected and the
+      grid becomes the sole holder of the data. Reach for it on a large, read-mostly grid where you
+      can let go of the source array and do not depend on caller identity through
       <code>rows.data()</code>.</p>
     </div>
 
@@ -2746,7 +2808,8 @@ createGrid(el, { columns, rowKey: 'id', source });
 plan.pushed;      <span class="cmt">// the query the adapter was given</span>
 plan.residual;    <span class="cmt">// { filters, sort, quick } the grid applied after</span>
 plan.unpushed;    <span class="cmt">// ['filter'], the parts that stayed behind</span>
-plan.needsAll;    <span class="cmt">// true when the whole result had to be fetched</span></code></pre>
+plan.needsAll;    <span class="cmt">// true when the whole result had to be fetched</span>
+plan.full;        <span class="cmt">// true when fullDataset forced it, not just residual work</span></code></pre>
     </div>
     <p>
       The grid also warns once, naming the predicate that could not be pushed, because the fix is
@@ -2788,6 +2851,105 @@ createGrid(el, {
       with no server involved.</p>
     </div>
 
+    <h3 id="fulldataset">Whole-dataset statistics over a remote source</h3>
+    <p class="lead-in">
+      A windowed source reduces a total or statistic over the rows it has loaded, not the whole
+      matching set — a footer median of the 200 rows on screen, which is wrong and looks right.
+      <code>fullDataset</code> makes the source hold the entire matching set client-side so those
+      figures are computed over everything.
+    </p>
+    <div class="example">
+      <p class="example__label">Correct footer figures over a REST or DuckDB source</p>
+      <pre><code><span class="kw">const</span> source = createPushdownSource({
+  adapter: restAdapter({ url: '/api/trades' }),
+  fullDataset: {
+    enabled: <span class="kw">true</span>,            <span class="cmt">// hold the whole matching set, once per query</span>
+    maxRows: 1_000_000,        <span class="cmt">// refuse (visible error) past this</span>
+    maxBytesEstimate: 512 * 1024 * 1024,
+  },
+});</code></pre>
+    </div>
+    <div class="why">
+      <p>It is <strong>off by default</strong> and reuses the same whole-result path that residual
+      work already takes: the flag ORs into <code>needsAll</code>, so once the set is held it is
+      ordinary in-memory data and the grid's existing total and statistics kernels reduce over all of
+      it, with no per-stat change. When it is active and within the limits, the whole matching set is
+      covered, so the windowed-statistic warning (BACKLOG-0000731) stays silent — the figure is now
+      honestly whole-dataset.</p>
+      <p><strong>It is refused loudly, never truncated.</strong> A matching set past
+      <code>maxRows</code> or <code>maxBytesEstimate</code> is thrown and surfaced as a
+      <code>source:error</code> with no rows shown, rather than held as a fraction and presented as
+      the whole. A fraction shown as the whole is exactly the silent wrong answer this feature
+      exists to remove, so it is never how the feature fails. For a <code>restAdapter</code>, which
+      cannot compute, this is the only route to a correct whole-dataset statistic.</p>
+    </div>
+
+    <h3 id="allowpartialresults">Refusing a partial result over residual work</h3>
+    <p class="lead-in">
+      When residual work has to run in the browser, the source asks the adapter for the whole matching
+      set and pages from what it holds. An adapter that answers with a <em>page</em> of that result —
+      it paged when told not to — leaves the client-side filter or sort running over the wrong rows:
+      the ones that belong on page one may be in the fraction never fetched. A page presented as the
+      full filtered set is a wrong answer, not a slow one.
+    </p>
+    <div class="example">
+      <p class="example__label">Refused by default; opt in only when you knowingly accept it</p>
+      <pre><code><span class="cmt">// Default: a shortfall under residual work throws, surfaced as source:error.</span>
+<span class="kw">const</span> strict = createPushdownSource({ adapter });
+
+<span class="cmt">// Knowing escape hatch: keep the old warn-once-and-proceed behaviour.</span>
+<span class="kw">const</span> lax = createPushdownSource({ adapter, allowPartialResults: <span class="kw">true</span> });</code></pre>
+    </div>
+    <div class="why">
+      <p><strong>Off by default, refused loudly.</strong> Filtering or sorting a fraction of the result
+      does not lose rows quietly, it returns the <em>wrong</em> rows, so the default throws rather than
+      warns — <code>nothing fails silently</code> (§8) means a wrong answer is never preferred to a
+      visible failure. The fix is an adapter that follows the engine's own paging, or holding the data
+      in memory. <code>allowPartialResults: true</code> is the deliberate opt-out — a caller who
+      accepts the permissive behaviour keeps the warn-once path — and it is never the default. It does
+      not touch the <code>fullDataset</code> memory guard or the no-residual short-return warning, both
+      of which stand regardless.</p>
+    </div>
+
+    <h3 id="aggregates">Pushing statistics down to the engine</h3>
+    <p class="lead-in">
+      A DuckDB-class engine computes a median or a standard deviation over the whole matching set far
+      faster than pulling every row here to do it. The <code>aggregates</code> config decides, at
+      grid setup, which statistics the engine computes and which the grid does — a design-time
+      developer choice, fixed for the life of the grid, never a runtime toggle and never shown to an
+      end user.
+    </p>
+    <div class="example">
+      <p class="example__label">Push the verified-identical stats, keep the rest exact</p>
+      <pre><code><span class="kw">const</span> source = createPushdownSource({
+  adapter: duckdbAdapter({ connection: conn, from: <span class="str">'trades'</span> }),
+  aggregates: {
+    <span class="cmt">// 'engine' pushes everything expressible; 'engine-if-identical' pushes only</span>
+    <span class="cmt">// the stats whose engine result is verified identical to the grid kernel;</span>
+    <span class="cmt">// 'client' (the default when absent) computes everything here.</span>
+    <span class="kw">default</span>: <span class="str">'engine-if-identical'</span>,
+    overrides: { mode: <span class="str">'client'</span> },   <span class="cmt">// I want the grid's null-when-distinct mode</span>
+  },
+});</code></pre>
+    </div>
+    <div class="why">
+      <p>Every statistic is classified <strong>IDENTICAL</strong> (the engine's result equals the
+      grid's own kernel, verified against it on the same data) or <strong>MAY-DIFFER</strong> (the
+      engine computes it by a method that can differ from the grid's definition — <code>mode</code>
+      returns a value where the grid returns null). The classification drives the docs and
+      build-time provenance, <em>not</em> whether a stat is pushed: that is your choice.
+      <code>weightedQuantile</code> is the one genuine fallback, always client-side, because the
+      engine cannot express the grid's midpoint convention. The published table of every stat, its
+      class and its SQL is generated from one map (<code>STAT_PUSHDOWN</code>) so it cannot drift.</p>
+      <p><strong>No mixed provenance.</strong> An engine figure and a client figure never appear in
+      one result set. Aggregates are pushed only when the filter is <em>fully</em> pushed; a residual
+      filter the engine could not apply forces every aggregate client-side, because an engine number
+      computed over a superset beside a client number over the real set would be wrong-but-plausible.
+      <code>source.lastPlan().aggregates</code> reports, per stat, whether the engine or the grid
+      computed it and the class it was assigned — inspection during the build, not a per-figure
+      runtime marker.</p>
+    </div>
+
     <h2 id="derived">Grids built from other grids</h2>
     <p class="lead-in">
       A derived grid takes its rows from another grid rather than from a load: grouped and
@@ -2982,6 +3144,27 @@ columns: [
       scrolling past a rounded corner is cut by it rather than squaring it off.
     </p>
 
+    <div class="example">
+      <p class="example__label">Zebra striping (opt-in)</p>
+      <pre><code>createGrid(element, {
+  columns, rows,
+  stripedRows: <span class="kw">true</span>,       <span class="cmt">// shade alternate data rows; off by default</span>
+});</code></pre>
+    </div>
+
+    <p class="lead-in">
+      <code>stripedRows</code> shades every other data row. It is strictly opt-in and off by default,
+      so a grid that never mentions it looks exactly as it did on upgrade. Parity is decided by each
+      row's <em>logical</em> index rather than its position in the DOM: rows are virtualised and
+      recycled, so a <code>:nth-child</code> rule would repaint the stripe onto whichever row landed
+      in an odd slot after a scroll, and a logical-index stripe keeps a row shaded consistently across
+      a scroll and across the left-pinned, centre and right-pinned segments of the same row. Group
+      headings, group footers and the grand total are structure rather than data, so they are never
+      striped. The stripe uses the theme's <code>--lattice-surface-alt</code> token, which every
+      palette defines, so dark, high-contrast and terminal are correct without any extra rule, and
+      both selection and hover still win over it.
+    </p>
+
     <h2 id="cards">Cards, lists and feeds</h2>
     <p class="lead-in">
       <code>rowTemplate</code> draws each row with a layout of your own instead of dividing it into