@lotics/app-sdk 0.100.1 → 0.101.0

package/docs/queries.md

@@ -1,12 +1,9 @@
 # Queries — the query engine authoring reference
 
-Every read a custom-code app performs is a **named query**: a fixed AST template declared in
-`package.json#lotics.queries`, validated and compiled at deploy, and invoked by alias via
-`useQuery` / `useInfiniteQuery` / `usePaginatedQuery` (see [data_fetching.md](./data_fetching.md)
-for the hook-side contract). This document is the authoring reference for the query engine
-itself: the node kinds, the source expressions, the per-field-type operator support, filters and
-params, search, cross-table composition, server-side shaping, runtime refinement, and the
-engine's hard limits.
+Every read an app performs is a **named query**: a fixed AST template bound to the app by alias
+(`set_app_query`), validated and compiled when it is written, and invoked by alias
+through the read hooks ([data_fetching.md](./data_fetching.md)). This is the authoring reference
+for the query engine itself.
 
 ---
 
@@ -14,32 +11,29 @@ engine's hard limits.
 
 ### Declaration
 
-Queries live in the app's `package.json` under `lotics.queries` — an alias → declaration map:
+Queries live on the app, as an alias → declaration map. `set_app_query` binds one alias at a time
+(`set_app_queries` several), each as a new version of the app:
 
 ```jsonc
+// set_app_query { "app_id": "app_…", "alias": "openOrders", "declaration": … }
 {
-  "lotics": {
-    "queries": {
-      "openOrders": {
-        "ast": {
-          "kind": "project",
-          "from": {
-            "kind": "from_table",
-            "table_id": "tbl_orders",
-            "filter": { "node_type": "condition", "field_key": "status",
-                        "operator": "has_any_of", "value": ["opt_open"] }
-          },
-          "columns": ["order_code", "customer", "total", "created"]
-        }
-      },
-      "orderByCode": {
-        "ast": { /* … a filter with "{{params.code}}" … */ },
-        "params": { "code": { "type": "text" } },
-        "description": "One order by its code, with its customer and line total."
-      }
-    }
+  "ast": {
+    "kind": "project",
+    "from": {
+      "kind": "from_table",
+      "table_id": "tbl_orders",
+      "filter": { "node_type": "condition", "field_key": "status",
+                  "operator": "has_any_of", "value": ["opt_open"] }
+    },
+    "columns": ["order_code", "customer", "total", "created"]
   }
 }
+// set_app_query { "app_id": "app_…", "alias": "orderByCode", "declaration": … }
+{
+  "ast": { /* … a filter with "{{params.code}}" … */ },
+  "params": { "code": { "type": "text" } },
+  "description": "One order by its code, with its customer and line total."
+}
 ```
 
 - **`ast`** — a `QueryNode` tree (the 11 node kinds below). It may embed `{{params.<name>}}`
@@ -49,17 +43,14 @@ Queries live in the app's `package.json` under `lotics.queries` — an alias →
   `date_range`, `file`, `json`, `object`, `array`); each param may set `required: false`
   (default is required) and `description`. Nesting is capped at depth 8.
 - **`description`** — one line saying what the query returns, capped at 300 characters. No app
-  code reads it. It is for the **agents** that reach this app's data — the app's own declared
-  agents, and a member's chat while they have the app open — which otherwise see only an alias,
-  a JS identifier that names a query without saying what it covers. Write it for whoever has to
-  choose between your aliases; longer guidance belongs in an agent's own `instructions`.
-- Aliases must be valid JS identifiers (`useQuery("openOrders")` and codegen depend on it).
-
-`lotics app query set <alias>` (or `--all`) pushes this map to the server — a **deploy does
-not**, it echoes the live row back unchanged. The **server holds the canonical template**;
-the app never sends a raw AST over the wire. This is the exposure model: a public app can read
-exactly what its author's queries project — params fill declared value holes and can never
-widen the query's reach (a token in a `table_id` or field-key position fails deploy validation).
+  code reads it: it is what the **agents** that reach this app's data (its declared agents, a
+  member's chat while the app is open) read to choose between your aliases.
+- Aliases must be valid JS identifiers (`useQuery("openOrders")` and the generated types depend on it).
+
+A deploy never touches the bindings; `get_app_query` reads one back. The **server holds the
+canonical template**; the app never sends a raw AST over the wire, so a public app reads exactly
+what its author's queries project — params fill declared value holes and can never widen the
+query's reach (a token in a `table_id` or field-key position fails validation).
 
 ### What deploy validates
 
@@ -105,8 +96,6 @@ executes it inside a bounded transaction.
 - Archived and draft records are always excluded. Rows hidden by a table's row scope
   (`private_filters`) are excluded on every reach — base scans, link extraction, and
   non-self-scoped traversals alike. (Row scopes don't apply when the app owner is an admin.)
-- **Dev loop**: `lotics app dev` forwards query RPCs to the **deployed** manifest. Editing
-  `lotics.queries` locally does nothing until you `lotics app query set`.
 
 ### Results
 
@@ -115,16 +104,12 @@ A result is `{ rows, columns, source_fields_by_table_id }` (`+ total` for a coun
 source_table_id?, source_field_key? }` per column. Rows are plain objects keyed by output
 column name; decode cells with the SDK readers (`row.*`, `readSelect`, `readMembers`,
 `readLinks`, `readFiles` — see [data_fetching.md](./data_fetching.md)). Those output names are also
-the row's TYPE: codegen writes them into `AppQueryColumns`, so reading a column this query does not
-project is a `tsc` error.
+the row's TYPE: the generated `.lotics/app_queries.d.ts` carries them in `AppQueryColumns`, so
+reading a column this query does not project is a `tsc` error.
 
 **`source_fields_by_table_id` is always `{}`** — the key is part of the shape, the map is not
-filled. The server reads the source tables' field definitions to resolve option and member
-cells and then keeps them: sending them beside every page was 94% of a real register
-response's compressed bytes, for a map no reader wanted there. **The complete option sets —
-every option including those no loaded row holds, with colors — are `useFieldOptions`**
-(`/field-options`, which derives them from the same definitions); a cell's own `{ key, label }`
-comes down on the cell. See [members_and_options.md](./members_and_options.md).
+filled. **The complete option sets, with colors, are `useFieldOptions`**; a cell's own
+`{ key, label }` comes down on the cell ([members_and_options.md](./members_and_options.md)).
 
 Delivery-layer enrichment (applied to the response, per request):
 
@@ -145,10 +130,9 @@ Delivery-layer enrichment (applied to the response, per request):
   extraction** carries as of the target field it reads. The one shape that has none is a select
   reached through a **lookup FIELD**: the column addresses the lookup, whose own type is `lookup`,
   not `select` — so the cell stays bare `opt_*` keys AND `useFieldOptions` omits the column
-  entirely, and reading `.label` off it renders the option KEY. Carry the keys out
-  (`readSelect(cell).map((o) => o.key)`) and resolve them against an alias that projects the
-  underlying field DIRECTLY — `useFieldOptions` takes no params, so any alias whose schema
-  contains it works.
+  entirely, and reading `.label` off it renders the option KEY. Resolve the keys
+  (`readSelect(cell).map((o) => o.key)`) against any alias that projects the underlying field
+  DIRECTLY.
 - **`number` columns** arrive as JS numbers (Postgres `numeric` strings are coerced; a
   high-precision decimal that would lose digits stays a string). **`date`/`datetime`** columns
   arrive as canonical wall-clock strings (`YYYY-MM-DD` / `YYYY-MM-DDTHH:mm`).
@@ -163,11 +147,15 @@ declare these; output names starting with `__source_` / `__src_field_`, or equal
 drops addressing (grouped results can't be written through or matched by `record_id`); a `join`
 keeps the **left** side's addressing.
 
-`__created_at` / `__updated_at` are DISPLAY values, not just reserved names: a table with no date
-field of its own still has a chronological anchor. They arrive as ISO-8601 **instants** with an
-offset (`"2026-07-05T09:00:00.000Z"`) — not the timezone-less wall-clock a `date`/`datetime` cell
-carries — so read them with `readCreatedAt(row)` / `readUpdatedAt(row)`, which parse them as
-instants ([./data_fetching.md](./data_fetching.md)).
+A caller outside the app's organization — a public-app visitor, or a member of another
+organization — receives your output columns and the five addressing columns above, and nothing
+else: no `__src_field_*`, and `columns` carries each column's `name`, `type` and `nullable`
+only. Read what a row holds through your projection and the `read*` helpers, never through
+`__src_field_*`.
+
+`__created_at` / `__updated_at` arrive as ISO-8601 **instants** with an offset
+(`"2026-07-05T09:00:00.000Z"`), not a wall-clock — read them with `readCreatedAt(row)` /
+`readUpdatedAt(row)` ([./data_fetching.md](./data_fetching.md)).
 
 ---
 
@@ -217,17 +205,12 @@ indexes live (§10). `sort` entries are `{ field_key, order: "asc"|"desc", blank
   cells with storage keys — to the client (over-exposure + the presign ceiling at scale).
 - **A files column takes `limit` — bound it when the surface renders a THUMBNAIL, not the
   collection.** `{ "type": "files", "output": "photo", "source": "fld_…", "limit": 1 }` returns
-  the first entry per cell, in stored order, cut in SQL so the rest is never read or signed.
-  Without it the cell yields the whole array and the response pays to resolve every entry it
-  will never show — at scale, into the response-wide presign ceiling (§10). The bound is per
-  CELL and caps at `FILES_PROJECTION_MAX_LIMIT` (10) — a row wanting more than a handful is
-  asking for the collection, which belongs to the record surface that opens it. It is a
-  different axis from the query's own `limit`, which counts ROWS; `limit` on any non-`files`
-  column — or on a computed source, which has no cell to bound — is rejected at deploy rather
-  than ignored. **On a UNION, set it on every arm.**
-  Arms align by column name and type, and `limit` is neither, so an arm that omits it still
-  yields whole cells — the query stays correct and silently costs what the bound was there to
-  avoid.
+  the first entry per cell, in stored order, cut in SQL so the rest is never read or signed —
+  without it the response pays to resolve every entry toward the presign ceiling (§10). The bound
+  is per CELL, caps at `FILES_PROJECTION_MAX_LIMIT` (10), and is a different axis from the
+  query's own `limit`, which counts ROWS; `limit` on any non-`files` column or computed source is
+  rejected at deploy. **On a UNION, set it on every arm** — arms align by name and type, not by
+  `limit`, so an arm that omits it still yields whole cells.
 
 ### `filter` — predicate over derived columns
 
@@ -377,11 +360,12 @@ A `QuerySource` appears in `project.columns[].source`, `unpivot.passthrough[].so
 | `"field_key"` (bare string) | Passthrough of an input column | the column's type |
 | `{ "literal": v }` | Constant (`string \| number \| boolean \| null`) | inferred (`null` takes the column's declared type) |
 | `{ "eq": [a, b] }` / `{ "neq": [a, b] }` | Null-safe equality (`IS [NOT] DISTINCT FROM` — `null eq null` is `true`) | boolean |
-| `{ "isEmpty": s }` / `{ "isNotEmpty": s }` | Emptiness test — over an array-valued column: empty at NULL / JSON null / `[]`; otherwise NULL-or-blank-text | boolean |
+| `{ "isEmpty": s }` / `{ "isNotEmpty": s }` | Emptiness test — over an array-valued source (a column or a `link` read of type `select`, `select_member`, `select_record_link` or `files`): empty at NULL / JSON null / `[]`; over anything else: empty at NULL or blank text | boolean |
 | `{ "coalesce": [s, …] }` | First non-null | declared |
 | `{ "concat": [s, …] }` | Text concatenation (NULLs render empty) | text |
 | `{ "record_id": true }` | The row's **own** record id — requires a direct `from_table` parent; always text, non-null, never writable | text |
-| `{ "link": { "source": "customer", "field": "Company name" } }` | A field on the **first** linked record of a `select_record_link` field (correlated per-row subquery) | the target field's type |
+| `{ "link": { "source": "customer", "field": "Company name" } }` | A field on the **first** linked record of a `select_record_link` field | the target field's type |
+| `{ "link_agg": { "source": "orders", "field": "Amount", "operation": "sum" } }` | `sum` / `avg` / `min` / `max` / `count` / `string_agg` over **every** linked record, one value per row. `count` counts the linked records, whether or not `field` holds a value; with no linked record it is `0` and the rest are NULL | declared |
 | `{ "expression": "…" }` | jexpr escape hatch (below) | declared |
 
 **Every variant in that table except the bare field key and `record_id` needs an explicit
@@ -401,14 +385,23 @@ a scoped-out or stale link extracts as NULL.
 **Warning:** only the **first** link in the cell is followed. Multi-link analytics go through
 `unnest` + join (§8). And because `field` resolves by name first, renaming a field on the
 target table can silently re-point (or break) the extraction — prefer field **keys** when the
-target's names are volatile. Each extraction is a correlated subquery evaluated per output
-row — cheap on a filtered detail read, expensive over thousands of rows.
+target's names are volatile.
+
+**Cost.** Every `link` column over one `source` shares one read of the linked record per row, and
+every `link_agg` column over one `source` shares one aggregate, so ten columns through `customer` cost
+about what one does. A source whose columns nothing above the `project` reads costs nothing: a `count`
+over the projection skips it. When a `limit` sits over the `project` — through any `sort`s and a
+`filter` between — a source no filter condition, sort key or seek key reads is read only for the rows
+the page keeps. A source they do read is read for every row before the sort, so sorting a large table
+on a link column costs a linked-record read per row. An `unpivot` reads its sources for every row.
 
 An extracted column **is** addressed to the field it reads on the target table, so the delivery
 layer treats it like any other field-backed column: an extracted `select` enriches to
 `{key,label}` and appears in `useFieldOptions`, and an extracted date resolves relative-date
-filters in the source field's timezone. It is still never **writable** — a correlated subquery is
-not a write path, and `writable_target` rejects it.
+filters in the source field's timezone. It is still never **writable** — a read through a link is
+not a write path, and `writable_target` rejects it. `link_agg` has the same parent and `source` rules,
+carries no field addressing (an aggregate of many cells is no one field's value), and its `operation`
+must be valid over `field`'s type — a refusal lists the valid ones.
 
 ### The expression escape hatch
 
@@ -468,12 +461,12 @@ derived surfaces share one implementation.
 
 | Field type | Column type | Source-layer operators | Derived-layer differences | Sort / group / aggregate notes |
 | --- | --- | --- | --- | --- |
-| text | `text` | `equals`, `not_equals`, `contains`, `does_not_contain`, `starts_with`, `ends_with`, `is_empty`, `is_not_empty`, `is_any_of`, `is_none_of` | same set | Sort is lexicographic in the database's default collation — no locale control. Text comparisons normalize both sides (trim + case-insensitive). The LIKE family (`contains`, `does_not_contain`, `starts_with`, `ends_with`) additionally folds **diacritics**, so `contains "ha noi"` matches `"Hà Nội"`; the identity operators (`equals`, `not_equals`, `is_any_of`, `is_none_of`) do **not** — they compare a stored value, and `mã` is not `ma`. |
-| number | `number` | `equals`, `not_equals`, `greater_than`, `less_than`, `greater_than_or_equal_to`, `less_than_or_equal_to`, `is_empty`, `is_not_empty` | same set | Full numeric aggregate set (§8). |
-| date / datetime | `date` / `datetime` (datetime when the field's format includes time) | `on`, `before`, `after`, `on_or_before`, `on_or_after`, `between`, `time_of_day`, `is_empty`, `is_not_empty` — values are `DateTimePoint`s (below) | same set | Day-level filters on datetime values expand to the full-day window. Sortable; bucketable in `group.by` (§8); `earliest`/`latest`/`min`/`max`/`date_range` aggregate. |
+| text | `text` | `equals`, `not_equals`, `contains`, `does_not_contain`, `starts_with`, `ends_with`, `is_empty`, `is_not_empty`, `is_any_of`, `is_none_of`, `contains_any_of` | same set | Sort is lexicographic in the database's default collation — no locale control. Text comparisons normalize both sides (trim + case-insensitive). The LIKE family (`contains`, `does_not_contain`, `starts_with`, `ends_with`, and `contains_any_of`, whose value is a list and which holds when ANY listed word is in the cell) additionally folds **diacritics**, so `contains "ha noi"` matches `"Hà Nội"`; the identity operators (`equals`, `not_equals`, `is_any_of`, `is_none_of`) do **not** — they compare a stored value, and `mã` is not `ma`. |
+| number | `number` | `equals`, `not_equals`, `greater_than`, `less_than`, `greater_than_or_equal_to`, `less_than_or_equal_to`, `is_empty`, `is_not_empty` | same set — except a figure read in each row's own unit or currency (`unit_field` / `currency_field`, or a rollup totalling in its row's), which a comparison matches only at the source layer: a condition on a column passing it straight through moves there, anywhere else it is refused | Full numeric aggregate set (§8). A comparison on a figure read per row states `unit_option`, the key of the option of that select its value is in — `{ "field_key": "amount", "operator": "greater_than", "value": 100, "unit_option": "opt_usd" }` — and matches only rows holding that option; rows in a measured unit of the same dimension compare converted (`≥ 1` in `t` matches `1500` in `kg`), a counted unit or a currency never. One stating none is refused. |
+| date / datetime | `date` / `datetime` (datetime when the field's format includes time) | `on`, `before`, `after`, `on_or_before`, `on_or_after`, `between`, `time_of_day`, `is_empty`, `is_not_empty` — values are `DateTimePoint`s (below) | same set | Day-level filters on datetime values expand to the full-day window; a `date` column compares on the day alone, so a time in the filter value is ignored. A reduced-precision value (`"2026-05"`) is its period start, and a value with no time is at `00:00` for `time_of_day`. Sortable; bucketable in `group.by` (§8); `earliest`/`latest`/`min`/`max`/`date_range` aggregate. |
 | boolean | `boolean` | `equals` (value `true`/`false`; `false` matches NULL/missing) | same | `checked`/`unchecked`/`percent_*` aggregate (`unchecked` counts false **or** empty). |
-| select (single & multi) | `select` | `has_any_of`, `has_none_of`, `has_all_of`, `is_empty`, `is_not_empty` — values are **option keys** (`opt_*`), never labels | the **runtime** `filter` (§9) also takes option **names**, resolving each to its key — an unknown one is refused rather than matching nothing, and the value stays an array either way; a select column the query DERIVES instead of projecting from a field carries no option set, so addressing one there is refused | Cells are arrays even for single-selects. Sorting a select column in a query orders by raw JSON, **not** configured option order. `unnest` fans option keys. |
-| select_member | `select_member` | select ops + `is_current_member`, `is_not_current_member` (field-scoped, no value) | same — `is_current_member` **works at the runtime layer** too | `unnest` fans member ids. Anonymous public request: current-member binds the app owner (§1). |
+| select (single & multi) | `select` | `has_any_of`, `has_none_of`, `has_all_of`, `is_empty`, `is_not_empty` — values are **option keys** (`opt_*`), never labels | the **runtime** `filter` (§9) also takes option **names**, resolving each to its key — an unknown one is refused rather than matching nothing, and the value stays an array either way; a select column the query DERIVES instead of projecting from a field carries no option set, so addressing one there is refused | Cells are arrays even for single-selects. Sorting a select column orders by its options' order (a multi-select by its first option), an empty cell last; a select the query derives carries no options and orders by raw JSON. `unnest` fans option keys. |
+| select_member | `select_member` | select ops + `is_current_member`, `is_not_current_member` (field-scoped, no value; `is_not_current_member` matches every cell not holding the requester, an empty one included) | same — `is_current_member` **works at the runtime layer** too | `unnest` fans member ids. Anonymous public request: current-member binds the app owner (§1). |
 | select_record_link | `select_record_link` | membership by linked-record **id**: `has_any_of`, `has_none_of`, `has_all_of`; text over the cached **display**: `contains`, `not_contains`, `starts_with`, `ends_with`; `is_empty`, `is_not_empty` | same — id-membership works at the runtime layer (converged `[{id}]` containment) | Sorting orders by raw JSON, not display text — project the display (link extraction / `display_output`) and sort that. `unnest` fans link ids (+ display). |
 | files | `files` | `has_filename`, `has_mime_type` (substring), `has_file_count` (exact count), `is_empty`, `is_not_empty` | same | Presign-enriched at delivery (§1); `unnest` fans file ids. Only presence-counting aggregates. |
 | formula | its declared output type (`number`/`text`/`boolean`/`date`/`datetime`; `json` until inferred) | filtered by the output type's operators; `#ERROR:` cells are guarded — they count as empty and never match value operators | same | Extracted as a real scalar, so numeric formulas feed `sum`/`avg`/sorts. A **datetime-output** formula matches day-level date filters (full-day expansion). |
@@ -490,14 +483,11 @@ derived surfaces share one implementation.
 | `current_member` | `in_any_group` (value: group ids — gates by the *viewer's* group membership) | source layer only |
 | `record_id` | `is_any_of`, `is_none_of` (value: record ids) | source, derived, **and** runtime layers; rejected over a `group` |
 
-**Warning (silent no-op):** a condition whose operator has no translation for the field's type
-at the source layer — e.g. the date-*range* operators (`overlaps`, `within`, `starts_before`,
-`duration_*`) — contributes **no constraint** rather than erroring. The same applies to a
-condition whose value is `null`/absent. At the *derived* layer, unsupported operators fail
-loudly instead. Positive list operators with an **empty list** (`is_any_of: []`,
-`has_any_of: []`) match **nothing** at the derived layer, but are a silent no-op (no
-constraint — like a null value) at the source layer; negative ones (`is_none_of: []`,
-`has_none_of: []`) constrain nothing at either layer.
+**A value that states nothing constrains nothing.** A list operator with an **empty list**
+(`is_any_of: []`, `contains_any_of: []`, `has_any_of: []`, `is_none_of: []`, `has_none_of: []`) contributes no
+constraint at either layer, and so does a source-layer condition whose value is `null`/absent.
+The date-*range* operators (`overlaps`, `within`, `starts_before`, `duration_*`) are refused on
+the query path at both layers.
 
 ### DateTimePoint
 
@@ -638,8 +628,8 @@ It AND-s with `filter` (search *within* a scope) and is templatable
 (`"search": "{{params.q}}"`). An empty or whitespace-only term matches everything; an
 unresolved optional token prunes to match-all (§6).
 
-**Search-as-you-type over a LARGE table uses `search`, never a `contains` OR-group.** Per-field
-`contains` is unindexed — a zero-match keystroke forces a full-partition scan that hangs the
+**Search-as-you-type over a LARGE table uses `search`, never a `contains` OR-group alone.** Per-field
+`contains` is unindexed — a zero-match keystroke forces a full-table scan that hangs the
 picker. (It is not accent-sensitive; the LIKE family folds diacritics, §5.) Gate the fetch on a
 non-empty term client-side (`enabled`), or first paint dumps the table.
 
@@ -650,7 +640,11 @@ into. A box captioned "find a person" then matches rows that merely share a stag
 owner, and folding makes near-homographs collide (`tiến` ≡ `tiền`). When the box means *find
 this person*, bound it: an OR-group of `contains` over the two or three identity fields (name,
 national id, phone), with the term param `required: false` so an empty box prunes the group to
-match-all. Use `search` when the box genuinely means *find this anywhere in the record*.
+match-all, and put the same term in `search` beside it: the search document holds every field the
+OR-group reads, folded alike for diacritics and case, so the pair returns the OR-group's rows, found
+through the trigram index rather than a scan. (A term whose letters `contains` folds further — `ß`,
+`ł`, curly quotes — can miss a row.) Use `search` alone when the box genuinely means *find this
+anywhere in the record*.
 
 ---
 
@@ -690,9 +684,8 @@ sizes on a shipment, the tags on a ticket) without a second query.
 - **`distinct`** defaults to **true**: three containers sized 40HC/40HC/20DC give `20DC, 40HC`.
   Pass `false` to keep every occurrence. Values are always sorted, so the column doesn't
   reshuffle between reads.
-- **`max_values`** (default 20, max 100) caps the emitted values so an unbounded child set can't
-  produce a giant cell. It is a silent cap — pair it with a `unique` aggregate over the same
-  column to render an honest `+N more`.
+- **`max_values`** (default 20, max 100) caps the emitted values silently — pair it with a
+  `unique` aggregate over the same column to render an honest `+N more`.
 - **`separator`** defaults to `", "` (max 8 chars).
 - Empty values are dropped by the same emptiness contract below, so a partly-blank column
   doesn't emit empty slots. A group with nothing present yields NULL.
@@ -700,15 +693,17 @@ sizes on a shipment, the tags on a ticket) without a second query.
   are mutually exclusive in SQL. Setting `distinct` / `separator` / `max_values` on any other
   operation is rejected rather than ignored.
 
-It is deliberately **not** a rollup field type: a rollup persists its value into record data and
-rewrites it on every child change, and an unbounded concatenation does not belong in a stored
-cell. Query-time only.
+It is query-time only, never a rollup field type: an unbounded concatenation does not belong in a
+stored cell.
 
 **The one emptiness contract.** `filled`/`empty`/`unique`/`percent_*` use the same definition
 of "present" as the filter layer's `is_empty` and the `isEmpty` source: array-valued cells are
 empty at NULL / JSON `null` / `[]`; text at NULL or blank (whitespace-only); opaque json at
 NULL / JSON `null`; other scalars at SQL NULL. A cleared multi-select or a whitespace-only text
-cell counts **empty** everywhere — filters and aggregates never disagree.
+cell counts **empty** everywhere — filters and aggregates never disagree. A cleared cell stored
+as JSON `null` reads as NULL wherever a query reads it: a `group` key puts it in one empty group
+with the cells that lack the key, `from_table.sort` places it by `blank_position`, and `isEmpty`
+over a `link` read of it is `true`.
 
 ### Date-bucket group keys
 
@@ -738,17 +733,14 @@ Sort by the bucket column for a time series; filter it with date operators for a
 window. **Always bucket server-side** — shipping raw rows to bucket in JS burns the row cap and
 the timeout for nothing.
 
-A `window` node carries two independent column lists — **`aggregates`** (frame aggregates) and
-**`functions`** (ranking / navigation). At least one must be non-empty; a node may carry both.
-
-**`aggregates` — the OVER-legal aggregate subset.** Only operations that compile to a single
+**Window `aggregates` — the OVER-legal aggregate subset.** Only operations that compile to a single
 legal SQL window call are accepted: **`count`, `sum`, `avg`, `min`, `max`, `earliest`, `latest`,
 `filled`, `checked`, `unchecked`.** The rest cannot take an OVER clause (`median` is an
 ordered-set aggregate; `unique`/`percent_unique`/`string_agg` need DISTINCT;
 `range`/`empty`/`date_range`/`percent_*` compose multiple calls) — rejected at deploy. The optional `frame` applies **only**
 to these; a `frame` on a window with no `aggregates` is rejected as dead config.
 
-**`functions` — ranking / navigation.** Each is `{ "output", "fn", … }` (its own arg shape, no
+**Window `functions` — ranking / navigation.** Each is `{ "output", "fn", … }` (its own arg shape, no
 `operation`/`type` — the output type is fixed or inherited). They never take a `frame` and
 **require a non-empty `order_by`** (ranking without an order is nondeterministic):
 
@@ -807,24 +799,34 @@ template as derived nodes, in this order: `filter` (narrow) → `sort` (order)
   layer; `record_id` allowed). A `field_key` that isn't a projected output column → 400. This
   is the exposure invariant: a caller can refine what the query already exposes, never widen
   it. Record-link membership (`has_any_of` by id) **works** here; so does `is_current_member`
-  on a member column. Traversals / `locked` / `current_member` do not — bake those into the
+  on a member column. Words (`contains`, `starts_with`, …) on an option or a member column — its
+  own, or a lookup of one — match the option's or the member's NAME, and on a text lookup match
+  the words its column carries, all folded for case and marks like text — so one search over
+  text, options, people and what a row looks up finds what the reader reads. A number, a date or
+  a rollup has no words; narrow it by range. Traversals / `locked` / `current_member` do not — bake those into the
   template.
 - **`sort`** — `[{ field_key, order, blank_position? }]` over output columns — the typed
   `QuerySortKey` the hooks accept. `blank_position` is `"top" | "bottom"`, defaulting to
-  `"bottom"`. It matters on a **multi-key** sort over columns whose blankness IS the ranking —
-  a pipeline's per-step date stamps, where the first non-blank column gives the row's rung.
-  Listing the ladder top-down with blanks at the bottom orders it furthest-along-first; the
-  ascending reading needs `blank_position: "top"` on every key, and without it that direction
-  cannot be expressed at all.
-- **`limit` / `offset` / keyset `cursor`** — pagination. `useQuery` / `usePaginatedQuery` page by
-  `offset` (`limit` clamped to the 10,000-row cap, §10); `useInfiniteQuery` opts into **keyset
+  `"bottom"` — the only way to express the ascending reading of a **multi-key** sort over
+  columns whose blankness IS the ranking (a pipeline's per-step date stamps).
+- **`limit` / `offset` / keyset `cursor`** — pagination. `useQuery` with `limit` or `page` reads by
+  `offset` (`limit` clamped to the 10,000-row cap, §10); `useQuery` with `more` opts into **keyset
   (seek)** by sending `keyset: true` + the prior page's `cursor`, and the server returns the next
   `next_cursor` (null on the last page). Seek stays O(page) and never skips/duplicates a row as the
   set shifts, falling back to offset for a multi-key sort it can't seek.
-- **`count: true`** — returns `{ total }` only: a single-row COUNT over the *filtered* set,
-  ignoring sort/limit/offset. Drives "Page 1 of N". It is a **second full execution** of the
-  template, not a cheap lookup off the page request — it costs what the query costs and grows with
-  the filtered set, which on a union/unpivot pipeline is a whole extra scan.
+- **`count: true`** — returns `{ total }` only: a COUNT over the *filtered* set, ignoring
+  sort/limit/offset. Drives "Page 1 of N". With **`count_by: "<column>"`** the same scan also
+  returns `counts` — rows per value of that projected column (a multi-value row under each value,
+  a row holding none in `total` only); a column whose values are not plain (a link) → 400. It is a
+  **second full execution** of the template — sent only where a read asks for its `total`, and
+  one count serves every read of the same set ([data_fetching](./data_fetching.md)).
+- **`aggregate: { by?, day?, sum? }`** — the groups of the *filtered* set in place of its rows: one
+  row per value of up to two `by` output columns (a cell holding several values groups by all of
+  them) and per day of a `day` output date, each holding its `by` columns, `__day`, `__count`, and
+  `__sum` of a `sum` output number (`null` where no row holds one). Every column it names is an
+  output column, like `filter`'s, so what leaves is counts and sums of what a page of the same
+  query returns. Sent with `params` and `filter` alone; `truncated` past the row cap. One read
+  over a list's rows serves the figures above it — its filter is the list's own.
 
 **A terminal aggregate cannot be refined — plan the total accordingly.** Because refinement wraps
 *around* the template over its **output** columns, a query that has already collapsed (a `group`
@@ -836,21 +838,12 @@ runtime facet (a filter TREE is not a scalar param), and leaving the total unfil
 that follows the facet next to a sum that does not. **Project the summed column plus the columns
 the facet can name, fetch under the same runtime `filter`, and total in the browser** — correct and
 facet-aware, but it ships every matching row, so it holds only where the scope is already small
-(one member's own records, a single project). A workspace-wide total that must follow a runtime
-facet has no server-side shape today.
+(one member's own records, a single project). A workspace-wide total that follows a runtime facet
+has no server-side shape.
 
-The SDK hooks map onto this directly — the hook-side contract is
-[data_fetching](./data_fetching.md).
-
-Build `filter` from UI column-filters with `columnFilterToConditions` (`@lotics/ui`); prefer
-`useFieldOptions` for a select filter's option set.
-
-**Warning (transport gaps):** the embedded product host and the `lotics app dev` forwarder pass
-`sort`/`filter`/`count` through. The **standalone public transport** (the app's own origin)
-forwards only `alias`/`params`/`limit`/`offset` — runtime `sort`/`filter` are silently ignored
-there and a `count` never resolves ([data_fetching](./data_fetching.md) → Standalone transport).
-A standalone app must bake ordering/scoping into the template (or params) rather than rely on
-runtime refinement.
+**The standalone public transport drops runtime `sort`/`filter`/`count`**
+([data_fetching](./data_fetching.md#standalone-public-transport)), so a standalone app bakes
+ordering and scoping into the template or its params.
 
 ---
 
@@ -872,8 +865,8 @@ may embed SQL — is server-logged only). Deploy-time and validation errors are
 
 ### Index reality, in plain terms
 
-Records live in one big partitioned store; a `from_table` scan narrows to the table's partition
-slice, and within it:
+Records are stored partitioned by workspace, so a query reads only its own workspace's slice; a
+`from_table` narrows that slice to the table's rows, and within them:
 
 - **GIN-served (fast at any size):** *positive* exact-membership filters at the **source
   layer** — `select`/`select_member`/`select_record_link` `has_any_of` / `has_all_of`, and
@@ -882,12 +875,11 @@ slice, and within it:
 - **B-tree-served (automatic for deployed queries):** text `equals`, number and date
   comparisons (exact and range), and `sort` fields — for fields referenced in a **deployed
   named query's template**. The platform provisions a partial expression index per referenced
-  field automatically: built online whenever a query is written (`app query set`, or a chat
-  draft's publish — a deploy does not write queries), re-synced daily,
+  field automatically: built online whenever a query is bound (`set_app_query` or `apply_model`), re-synced daily,
   capped at 8 per table (fields past the cap fall back to the scan tier, with a server WARN).
   A `{{params.…}}` value hole doesn't change this — the field key is static in the template,
   so it still gets its index. Index-seek speed at any table size once provisioned.
-- **Partition scan (linear in table size):** everything else — text `contains`, negations
+- **Table scan (linear in table size):** everything else — text `contains`, negations
   (`has_none_of`, `is_none_of`, `not_*`), emptiness, files predicates, and predicates/sorts on
   fields that appear **only** in the runtime `filter`/`sort` options rather than the deployed
   template. Fine on thousands of rows; on very large tables these dominate latency and are the
@@ -910,35 +902,29 @@ per-table queries you merge client-side; the merge already happens in the index.
 sort key to be a bare field projection — a computed/literal/type-overridden sort column falls back
 to the whole-union sort.)
 
-**An aggregate arm you cannot filter costs its whole table, every execution.** The shape to watch
-is the mirror of the one above: a `join` whose right side is a `group` over one or more entire
-tables — a lookup built by scanning everything in order to decorate a small left side. Rule 1 is
-no help here, and that is what makes it easy to ship: the arm keys on a value the LEFT side
-supplies at run time (a code, a serial number), so there is no static predicate to push down. The
-left side's size is irrelevant; you pay for the arms. As a unit to budget with: a four-table arm
-totalling ~14k rows is **seconds, not milliseconds**, on every single execution. Prefer a
-materialized **lookup/rollup field** on the row (the platform keeps it current and it filters at
-the source), or drop the arm entirely if the value it fetches now lives on the row already.
-
-**Measure before you change a shape, and after.** Latency is observable per alias: each `useQuery`
-goes out as its own RPC, so a screen's cold load attributes a duration to every query by name in
-the browser's network panel. Time it, fix the one that dominates, time it again. Reasoning
-alone mis-ranks these — the union above looks expensive and is fast, the join here looks ordinary
-and is not — and a playbook rule applied to the wrong query costs effort while proving nothing.
+**An aggregate arm you cannot filter costs its whole table, every execution.** A `join` whose right
+side is a `group` over entire tables, keyed on a value the LEFT side supplies at run time, has no
+static predicate to push down — a four-table arm totalling ~14k rows is **seconds, not
+milliseconds**, on every execution. Prefer a materialized **lookup/rollup field** on the row.
+
+**Measure before you change a shape, and after.** Each `useQuery` goes out as its own RPC, so the
+browser's network panel times every query by name. Reasoning alone mis-ranks these — the union
+above looks expensive and is fast, the join here looks ordinary and is not.
 
 ### The authoring rules
 
 1. **Filter at the source.** Push every static predicate into `from_table.filter`.
 2. **Match the operator to what the box MEANS.** *Find this anywhere in the record* → `search`.
-   *Find this person/order* → a `contains` OR-group over the identity fields (§7) — the
-   whole-record index will surface rows that only share an owner or a status.
+   *Find this person/order* → a `contains` OR-group over the identity fields, with the same term in
+   `search` beside it (§7) — `search` alone surfaces rows that only share an owner or a status,
+   and the OR-group alone scans.
 3. **Aggregate and bucket server-side.** A dashboard reads grouped rows, never raw rows it
    reduces in JS — raw-row shipping burns the 10k cap, the timeout, and bandwidth at once.
 4. **Project narrow.** Every un-rendered column is wasted bytes; every un-rendered `files`
    column risks the presign ceiling and over-exposes storage metadata.
 5. **Files columns only where rendered.** A list view projects no files; the detail query does.
 6. **Avoid deep offsets.** Filter first so the browsable set is small; sort by a stable key.
-7. **Don't bake a `limit` into a browsable query** — it caps the total; let `pageSize` drive.
+7. **Don't bake a `limit` into a browsable query** — it caps the total; let `page` drive.
 8. **Parameterize lookups.** A code/id lookup is a `{{params.x}}` filter (or a `record_id`
    runtime filter) returning one row — never load-all-then-find in JS.
 9. **Expect and handle the three runtime outcomes**: a 400 with a message (surface it), the
@@ -951,24 +937,12 @@ and is not — and a playbook rule applied to the wrong query costs effort while
     `count` and `sum` fold exactly. `unique` does NOT fold across groups (the same value can
     appear in many groups), so a rendered distinct-count needs its own query — **unless there
     are no group keys**, where `by: []` carries sums, counts and distinct counts together in one
-    scan. (It counts distinct *present* values, §8 — so a pre-filter that excludes the rows whose
-    value is blank buys nothing but its own scan.) Which makes the rule's own corollary the thing
-    to check first: **group keys are not free scaffolding.** They are the expensive half of a
-    group — a `select`/jsonb key especially — so keys kept for a facet the screen no longer
-    renders cost a real multiple of the ungrouped aggregate, and a query that outlived its cards
-    is invisible because it still returns the right answer.
-11. **A paginated table's count is a second full execution.** `usePaginatedQuery` issues
-    `count: true` alongside the page, and a count re-runs the whole query — it costs what the
-    query costs and grows with the filtered set, so on a union/unpivot pipeline it is a whole
-    extra scan, not a cheap lookup. When the screen already renders that figure from an
-    aggregate, hand it over as the hook's `total` instead of buying it twice
-    ([data_fetching](./data_fetching.md) → `usePaginatedQuery`).
-12. **Re-derive a query when its tables change shape.** A `join` or `union` that exists to bridge
-    two tables becomes pure cost the moment those tables become one — and nothing fails, because
-    it keeps returning the right answer at the old price. Migrations that merge, move or back-fill
-    a table are exactly when this happens, and exactly when nobody re-reads the queries. After
-    one, open every query over the affected tables and ask what it would look like written today,
-    not what it needs to keep working.
+    scan. **Group keys are not free scaffolding** — a `select`/jsonb key especially — so keys
+    kept for a facet the screen no longer renders cost a real multiple of the ungrouped aggregate.
+11. **Re-derive a query when its tables change shape.** A `join` or `union` that bridged two
+    tables becomes pure cost once they are one, and keeps returning the right answer at the old
+    price — so after a migration that merges, moves or back-fills a table, re-read every query
+    over it.
 
 ---
 
@@ -1066,40 +1040,3 @@ A lookup projection (and unnesting the lookup column) yields only the **first**
 value (§5). To read the looked-up field across *all* links: unnest the **link column**, join to
 the target on `record_id`, and project the field from the target side — the link-identity join
 above, with `region` replaced by whatever the lookup pointed at.
-
----
-
-## 12. Current limitations
-
-- **No pivot/crosstab node.** Row-values-to-columns happens client-side over grouped results
-  (§8).
-- **No `first_value`/`last_value`/`nth_value` navigation** — the `functions` set is `row_number`,
-  `rank`, `dense_rank`, `percent_rank`, `cume_dist`, `ntile`, `lag`, `lead` (§8). Frame aggregates
-  remain the 10-item OVER-legal subset.
-- **`useQuery` / numbered pagination is offset.** Deep pages cost the full skipped prefix and can
-  shift under concurrent writes; `useInfiniteQuery` uses keyset (seek), which stays O(page) and
-  never skips or duplicates rows (§9).
-- **No collation control.** Text ORDER BY uses the database default collation —
-  locale-specific alphabetical order (e.g. accented-letter ordering) is not configurable.
-- **Select / member / link columns sort by raw JSON** in queries — not by configured option
-  order or display text. Sort a projected text form instead.
-- **Lookup projection returns the first element**; unnesting a lookup column fans only the
-  first linked record's value. All-values access goes through unnest-the-link + join (§11).
-- **Link extraction follows the first link only**, resolves the target field by display name
-  before key, and costs a correlated subquery per row (§3).
-- **Join is single-column equality** with SQL-category-matched types; addressing follows the
-  left side (§2).
-- **Text / number / date range predicates are unindexed within the partition** — the usual
-  cause of slow queries and timeouts on large tables; positive membership filters and `search`
-  are the indexed paths (§10).
-- **Traversals, `locked`, and `current_member` conditions are source-layer only** — not
-  available in derived or runtime filters (§6).
-- **The standalone public transport drops runtime `sort`/`filter`/`count`** — refinement is
-  silently ignored and paginated totals never resolve there; bake ordering/scoping into the
-  template for standalone apps (§9).
-- **Unsupported operators at the source layer are silent no-ops** (notably the date-range
-  operator family) — the condition contributes no constraint rather than erroring (§5).
-- **The expression allowlist is small and flat** — no indexing, nested access, or functions
-  beyond the §3 list; compute anything richer client-side from projected columns.
-- **`from_table` is the only leaf** — a query reads tables; there is no way to query a view,
-  another query, or an external source.