@lotics/app-sdk 0.72.0 → 0.74.0

package/AGENTS.md

@@ -15,7 +15,7 @@ signature; open the file.**
 | Doc | Read it for |
 |---|---|
 | [docs/queries.md](./docs/queries.md) | **The query engine authoring reference** — AST node kinds, per-field-type operator support, filters/params/pruning, free-text search, combining tables (join/union/link/`unnest`/`record_id`), shaping (aggregates, date buckets, windows), runtime refinement bounds, limits & the efficiency playbook. |
-| [docs/data_fetching.md](./docs/data_fetching.md) | The three read hooks (`useQuery`/`useInfiniteQuery`/`usePaginatedQuery`), the `QueryRow` shape (projected columns `unknown`; `__source_record_id`/`__source_table_id` typed but optional), cell readers (`row.*`, `readSelect`, `readMembers`, `readLinks`, `readFiles`, `readLocked`), `useFieldOptions`, caching — **arrival revalidates** (a re-mount renders cache *and* refreshes it in the background, `loading` never flips) — data discipline, the search-as-you-type + record-picker patterns. |
+| [docs/data_fetching.md](./docs/data_fetching.md) | The three read hooks (`useQuery`/`useInfiniteQuery`/`usePaginatedQuery`), the `QueryRow` shape (projected columns `unknown`; `__source_record_id`/`__source_table_id` typed but optional), cell readers (`row.*`, `readSelect`, `readMembers`, `readLinks`, `readFiles`, `readLocked`), `useFieldOptions`, caching — **arrival revalidates** (a re-mount renders cache *and* refreshes it in the background, `loading` never flips) — data discipline, the pagination count as a second full execution (and `total` to suppress it), the search-as-you-type + record-picker patterns. |
 | [docs/mutations.md](./docs/mutations.md) | `useWorkflow` (the ONLY write path), the `WorkflowResult` resolve-never-throw contract, typed inputs, diff-before-update, locked records, `useOptimistic`, `useNewRecord` (client-minted `rec_*` id so a new-record surface never remounts on its first save), read-after-write ordering (a re-read must not overtake an in-flight write). |
 | [docs/workflows.md](./docs/workflows.md) | **The workflow-BODY authoring reference** — the JS subset a `src/workflows/<alias>.ts` body may use: the parse-at-save/never-execute model, opaque `fld_*`/`opt_*` keys, expression sources + link descent, every step form (tool call, `agent`, waits, `validate`, `return`), the accepted sugar and its canonical lowering, helpers + callback rules, record-write surfaces, the traps, the bright line, and the verify loop — `check` (the only local gate: the app's own `npm run typecheck` never sees a body) → `dry_run_workflow` (static green is not a run) → `set`. |
 | [docs/files.md](./docs/files.md) | Files end to end — `useFileUpload`, `useAttachments`, `readFiles`/presigned URLs (**a bearer credential for the bytes** — never logged, reported, or persisted), workflow-generated files, **naming a zip's entries** (`{ id, name }` per file — a file name, never a path), preview pairing, filter operators, the server-side delivery bounds. **Uploads declare a `fidelity`** (`standard` / `high` / `original`) — the app picks how much of the image survives storage; use `high` whenever text must stay legible. |

package/dist/src/hooks.d.ts

@@ -181,6 +181,41 @@ export interface InfiniteQueryOptions extends BaseQueryOptions {
 export interface PaginatedQueryOptions extends BaseQueryOptions {
     /** Rows per page. Default 25. */
     pageSize?: number;
+    /**
+     * The total, when the SCREEN already knows it — which suppresses the hook's
+     * own count request entirely.
+     *
+     * Three states, all of them in the type: **omitted** → the hook counts;
+     * **`null`** → yours, not resolved yet; **a number** → yours, use it. `null`
+     * is what makes the option usable at all, because the natural source is an
+     * aggregate that is still loading on the first render — with only
+     * "number-or-nothing" the hook would fire the count it exists to avoid and
+     * throw the result away the moment the real number landed. So read it as
+     * `?? null`:
+     *
+     * ```tsx
+     * const summary = useQuery("orderStats", params);   // one ungrouped aggregate
+     * const rows = usePaginatedQuery("orders", params, {
+     *   pageSize: 100,
+     *   total: (summary.rows[0]?.row_count as number | undefined) ?? null,
+     * });
+     * ```
+     *
+     * Until the number arrives the hook reports `total: undefined` and `hasMore`
+     * falls back to "the page came back full" — exactly its behaviour while a
+     * count is in flight.
+     *
+     * Worth reaching for when a screen already renders the same figure: a `count`
+     * request re-executes the whole named query server-side, so a summary reading
+     * "N items" beside a table of those N rows is otherwise paying for that number
+     * twice, and both executions grow with the filtered set.
+     *
+     * The number must count the SAME set the query returns — the hook derives
+     * `totalPages` and `hasMore` from it and cannot tell that it doesn't.
+     *
+     * `refetch()` does not refresh it; it is yours, so refresh its source.
+     */
+    total?: number | null;
 }
 /**
  * Trigger a workflow by alias from the app's manifest.

package/dist/src/hooks.js

@@ -207,6 +207,10 @@ export function usePaginatedQuery(alias, params, opts) {
     const sort = opts?.sort && opts.sort.length > 0 ? opts.sort : undefined;
     const filter = opts?.filter;
     const mockRows = getMockRows(alias);
+    // `null` = the caller owns the total and it hasn't resolved; `undefined`
+    // (or omitted) = the hook counts. See `PaginatedQueryOptions.total`.
+    const suppliedTotal = opts?.total;
+    const callerOwnsTotal = suppliedTotal !== undefined;
     // The result-set identity. When it changes, `page` derives back to 0 (not via
     // an effect, so the stale page never fires a wasted fetch) and the count key
     // changes (recount). `setPage` re-stamps the current identity.
@@ -228,13 +232,20 @@ export function usePaginatedQuery(alias, params, opts) {
     // Keep the previous page's rows on screen while the next page loads.
     { keepPreviousData: true, ...swrConfig(revalidateOnFocus) });
     // Count key omits page AND sort — one count per result-set identity, reused
-    // across page clicks and re-sorts.
-    const countKey = mockRows || !enabled
+    // across page clicks and re-sorts. Null when the caller supplies the total:
+    // SWR issues nothing for a null key, so the second scan never happens.
+    const countKey = mockRows || !enabled || callerOwnsTotal
         ? null
         : ["app-query-count", alias, params ?? {}, filter ?? null];
     const countSwr = useSWR(countKey, () => rpc("query", { alias, params: params ?? {}, filter, count: true }), swrConfig(revalidateOnFocus));
     const rows = mockRows ?? rowsSwr.data?.rows ?? [];
-    const total = mockRows ? mockRows.length : countSwr.data?.total;
+    // A supplied `null` reads out as `undefined` — "not known yet" is one state
+    // to the consumer whether the hook is counting or the caller is.
+    const total = mockRows
+        ? mockRows.length
+        : callerOwnsTotal
+            ? (suppliedTotal ?? undefined)
+            : countSwr.data?.total;
     const totalPages = total != null ? Math.max(1, Math.ceil(total / pageSize)) : undefined;
     const hasMore = total != null ? (page + 1) * pageSize < total : rows.length === pageSize;
     const refetch = useCallback(() => {

package/docs/data_fetching.md

@@ -91,7 +91,8 @@ server validates system conditions by `type` and never reads `field_key` on them
   rendered. The next focus revalidation or an explicit `refetch()` re-runs it.
 - **`refetch()`** re-runs the query. Call it after a known mutation point — a successful
   `useWorkflow` call — to pull the latest state (see [./mutations.md](./mutations.md)).
-  `usePaginatedQuery.refetch()` refreshes both the page and the count.
+  `usePaginatedQuery.refetch()` refreshes the page, and the count when the hook owns it
+  (a caller-supplied `total` is the caller's to refresh).
 - **Ambient-chat mutations refetch automatically.** When the member's ambient chat agent (see
   [./ai.md](./ai.md#useaicontextslot-context--tell-the-ambient-chat-what-the-member-is-looking-at))
   finishes a turn that mutated records, the host pushes **every mounted query hook** to re-read —
@@ -147,6 +148,10 @@ Offset-only (the `usePaginatedQuery` numbered pages and the `useQuery` / manual
 - **`count: true` counts the filtered set**, ignoring sort/limit/offset — `usePaginatedQuery`
   issues it automatically. The count and the page are separate requests, so under concurrent
   writes `total` can briefly disagree with what paging finds.
+- **A count is a full re-execution of the named query, not a cheap lookup.** It costs what the
+  query costs and grows with the filtered set — on a union/unpivot pipeline that is a whole extra
+  scan. When the screen already renders that same number from an aggregate, hand it to
+  `usePaginatedQuery` as `total` rather than paying for it twice (below).
 
 ### `useQuery`
 
@@ -189,7 +194,8 @@ const { rows, total, totalPages, page, setPage, hasMore } =
 ```
 
 The page-model hook behind a numbered table (pairs with `@lotics/ui` `Pagination`). It owns the
-page cursor and fetches two things: the current page, and a `count` over the filtered set.
+page cursor and fetches two things: the current page, and a `count` over the filtered set — unless
+you supply the total yourself.
 
 - **Result-set identity is `(params, filter)`.** Changing either resets to page 0 and recounts.
   Changing only `sort` does **neither** — the count is sort-independent and you stay on the same
@@ -203,6 +209,29 @@ page cursor and fetches two things: the current page, and a `count` over the fil
 - A page change **keeps the previous rows on screen** while the next page loads (built-in
   `keepPreviousData`) — gate any skeleton on `loading && rows.length === 0`, never on `loading`
   alone, or every page click collapses the table.
+- **`total` in the options suppresses the count request entirely.** Reach for it when the screen
+  already has that figure — a summary reading "N items" beside a table of those N rows is
+  otherwise executing the same query twice, and both executions grow with the set.
+
+  ```tsx
+  const summary = useQuery("orderStats", params);        // one ungrouped aggregate
+  const rows = usePaginatedQuery("orders", params, {
+    pageSize: 100,
+    total: (summary.rows[0]?.row_count as number | undefined) ?? null,
+  });
+  ```
+
+  **Three states, all of them in the type**: omitted → the hook counts; `null` → yours, not
+  resolved yet; a number → yours, use it. The `?? null` is what makes it work — the natural source
+  is an aggregate still loading on the first render, and without a way to say "mine, pending" the
+  hook would fire the very count it exists to avoid and discard it a moment later. Until the number
+  arrives the hook reports `total: undefined` and `hasMore` falls back to "the page came back
+  full", exactly as while a count is in flight.
+
+  `refetch()` does not refresh a supplied total — it is yours, so refresh its source.
+
+  The number must count the **same set** the query returns — `totalPages` and `hasMore` derive
+  from it and the hook cannot tell that it doesn't.
 
 ## Standalone (public) transport
 
@@ -214,9 +243,11 @@ keyset `cursor`. Design a public app around this:
 - **Runtime `sort` / `filter` are ignored** — every hook's `opts.sort` / `opts.filter` is dropped,
   so a standalone app can't order or narrow a query at the call site. Bake ordering and scoping
   into the query **template** (or drive them through declared `params`), not the refinement options.
-- **`usePaginatedQuery.total` stays `undefined`** — the `count` request never resolves, so
-  `totalPages` never lands and `hasMore` falls back to "the current page came back full". A
-  numbered "Page 1 of N" control has no N; render it defensively, or prefer `useInfiniteQuery`.
+- **`usePaginatedQuery` gets no count** — the `count` request is dropped in transit, so a numbered
+  "Page 1 of N" control has no N unless you supply one. Pass `total` from a figure the app already
+  reads (an aggregate query survives the thin transport fine); with neither, `totalPages` never
+  lands, `hasMore` falls back to "the current page came back full", and `useInfiniteQuery` is the
+  better shape.
 - **`useInfiniteQuery.loadMore()` never advances past the first page** — with the cursor dropped,
   no `next_cursor` comes back, so `hasMore` is `false` after page one. For a standalone browse,
   size the template `limit` (or a `params`-driven page) to return the whole set in one fetch.

package/docs/mutations.md

@@ -328,7 +328,9 @@ const onClose = async (recordId: string) => {
 
 `refetch` re-runs the query in the background while the current rows stay on screen (no
 flash to a spinner — `loading` stays false during revalidation). `usePaginatedQuery`'s
-`refetch` re-runs both the current page and the total count. Focus revalidation
+`refetch` re-runs the current page, and the count when the hook owns it — a total you supplied
+yourself is yours to refresh, so a write that changes the row COUNT must also refresh whatever
+you read it from, or the page moves while "of N" does not. Focus revalidation
 (`revalidateOnFocus`, default on) eventually self-corrects stale data, but never rely on it
 in place of an explicit refetch after a write the user is watching for.
 

package/docs/queries.md

@@ -121,7 +121,8 @@ Delivery-layer enrichment (applied to the response, per request):
 
 - **`files` cells** — each entry gains a presigned `url` + `thumbnail_url` (24 h TTL) and
   `size` (bytes) + `created_at`, resolved from the file object at read. Presigning is
-  server-bounded (§10). Public apps hand out direct presigned URLs — anonymous-fetchable,
+  server-bounded (§10) — and it is per ENTRY, which is why a list that renders one thumbnail
+  should project the column with `limit` (§3) rather than the whole array. Public apps hand out direct presigned URLs — anonymous-fetchable,
   time-boxed. (App workflow-execute responses presign returned files the same way, so their
   URLs also work for anonymous public-app viewers — see [files.md](./files.md).)
 - **`select_member` cells** — bare member-id arrays become
@@ -199,6 +200,20 @@ indexes live (§10). `sort` entries are `{ field_key, order: "asc"|"desc", blank
   (§4), never a relabel. An uncastable combination is rejected at deploy.
 - **Project only what you render.** A bare `from_table` ships every column — including `files`
   cells with storage keys — to the client (over-exposure + the presign ceiling at scale).
+- **A files column takes `limit` — bound it when the surface renders a THUMBNAIL, not the
+  collection.** `{ "type": "files", "output": "photo", "source": "fld_…", "limit": 1 }` returns
+  the first entry per cell, in stored order, cut in SQL so the rest is never read or signed.
+  Without it the cell yields the whole array and the response pays to resolve every entry it
+  will never show: a register over containers averaging 33 gate photos each resolved ~832 files
+  to display 25 thumbnails, within 2.4× of the response-wide presign ceiling (§10) that would
+  reject the query outright. The bound is per CELL and caps at `FILES_PROJECTION_MAX_LIMIT`
+  (10) — a row wanting more than a handful is asking for the collection, which belongs to the
+  record surface that opens it. It is a different axis from the query's own `limit`, which
+  counts ROWS; `limit` on any non-`files` column — or on a computed source, which has no cell
+  to bound — is rejected at deploy rather than ignored. **On a UNION, set it on every arm.**
+  Arms align by column name and type, and `limit` is neither, so an arm that omits it still
+  yields whole cells — the query stays correct and silently costs what the bound was there to
+  avoid.
 
 ### `filter` — predicate over derived columns
 
@@ -783,7 +798,9 @@ template as derived nodes, in this order: `filter` (narrow) → `sort` (order)
   `next_cursor` (null on the last page). Seek stays O(page) and never skips/duplicates a row as the
   set shifts, falling back to offset for a multi-key sort it can't seek.
 - **`count: true`** — returns `{ total }` only: a single-row COUNT over the *filtered* set,
-  ignoring sort/limit/offset. Drives "Page 1 of N".
+  ignoring sort/limit/offset. Drives "Page 1 of N". It is a **second full execution** of the
+  template, not a cheap lookup off the page request — it costs what the query costs and grows with
+  the filtered set, which on a union/unpivot pipeline is a whole extra scan.
 
 **A terminal aggregate cannot be refined — plan the total accordingly.** Because refinement wraps
 *around* the template over its **output** columns, a query that has already collapsed (a `group`
@@ -803,7 +820,10 @@ sends `limit: pageSize, offset: 0` (a cap, not pagination) plus `opts.sort`/`opt
 `useInfiniteQuery` pages by **keyset** (`keyset: true` + the prior `next_cursor`), so its scroll
 never skips or duplicates a row; `usePaginatedQuery` owns the page cursor, pages by `offset`, and
 issues a `count` keyed on `(alias, params, filter)` — independent of page and sort, so paging and
-re-sorting never recount, while changing `(params, filter)` resets to page 0 and recounts.
+re-sorting never recount, while changing `(params, filter)` resets to page 0 and recounts. Given
+its `total` option it issues none at all: a screen that already renders that figure from an
+aggregate hands it over rather than buying it twice
+([data_fetching](./data_fetching.md) → `usePaginatedQuery`).
 
 Build `filter` from UI column-filters with `columnFilterToConditions` (`@lotics/ui`); prefer
 `useFieldOptions` for a select filter's option set.
@@ -811,9 +831,10 @@ Build `filter` from UI column-filters with `columnFilterToConditions` (`@lotics/
 **Warning (transport gaps):** the embedded product host and the `lotics app dev` forwarder pass
 `sort`/`filter`/`count` through. The **standalone public transport** (`<slug>.lotics.app`)
 currently forwards only `alias`/`params`/`limit`/`offset` — runtime `sort`/`filter` are
-silently ignored there and `count` never resolves (`usePaginatedQuery.total` stays
-`undefined`). A standalone app must bake ordering/scoping into the template (or params) rather
-than rely on runtime refinement.
+silently ignored there and a `count` never resolves, so a standalone pager has no "of N" unless the
+app supplies one through `usePaginatedQuery`'s `total` (a plain aggregate query survives the thin
+transport). A standalone app must bake ordering/scoping into the template (or params) rather than
+rely on runtime refinement.
 
 **Pagination semantics.** `useQuery` / `usePaginatedQuery` and manual `rpc("query", { limit,
 offset })` are **offset** — a deep page costs the server the full skipped prefix (page 400 of a
@@ -919,8 +940,21 @@ and is not — and a playbook rule applied to the wrong query costs effort while
     the server's concurrent-query gate (503 busy) on every load. Group by ALL the dimensions
     at once (`by: [a, b, c, …]`) and fold each facet client-side by summing over the others:
     `count` and `sum` fold exactly. `unique` does NOT fold across groups (the same value can
-    appear in many groups) — keep a separate query for each distinct-count you render.
-11. **Re-derive a query when its tables change shape.** A `join` or `union` that exists to bridge
+    appear in many groups), so a rendered distinct-count needs its own query — **unless there
+    are no group keys**, where `by: []` carries sums, counts and distinct counts together in one
+    scan. (It counts distinct *present* values, §8 — so a pre-filter that excludes the rows whose
+    value is blank buys nothing but its own scan.) Which makes the rule's own corollary the thing
+    to check first: **group keys are not free scaffolding.** They are the expensive half of a
+    group — a `select`/jsonb key especially — so keys kept for a facet the screen no longer
+    renders cost a real multiple of the ungrouped aggregate, and a query that outlived its cards
+    is invisible because it still returns the right answer.
+11. **A paginated table's count is a second full execution.** `usePaginatedQuery` issues
+    `count: true` alongside the page, and a count re-runs the whole query — it costs what the
+    query costs and grows with the filtered set, so on a union/unpivot pipeline it is a whole
+    extra scan, not a cheap lookup. When the screen already renders that figure from an
+    aggregate, hand it over as the hook's `total` instead of buying it twice
+    ([data_fetching](./data_fetching.md) → `usePaginatedQuery`).
+12. **Re-derive a query when its tables change shape.** A `join` or `union` that exists to bridge
     two tables becomes pure cost the moment those tables become one — and nothing fails, because
     it keeps returning the right answer at the old price. Migrations that merge, move or back-fill
     a table are exactly when this happens, and exactly when nobody re-reads the queries. After

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@lotics/app-sdk",
-  "version": "0.72.0",
+  "version": "0.74.0",
   "description": "Runtime SDK for Lotics custom-code apps — typed hooks, postMessage bridge, mount entry point",
   "type": "module",
   "exports": {