@lotics/app-sdk 0.63.0 → 0.64.0

package/AGENTS.md

@@ -16,7 +16,7 @@ signature; open the file.**
 |---|---|
 | [docs/queries.md](./docs/queries.md) | **The query engine authoring reference** — AST node kinds, per-field-type operator support, filters/params/pruning, free-text search, combining tables (join/union/link/`unnest`/`record_id`), shaping (aggregates, date buckets, windows), runtime refinement bounds, limits & the efficiency playbook. |
 | [docs/data_fetching.md](./docs/data_fetching.md) | The three read hooks (`useQuery`/`useInfiniteQuery`/`usePaginatedQuery`), the `QueryRow` shape (projected columns `unknown`; `__source_record_id`/`__source_table_id` typed but optional), cell readers (`row.*`, `readSelect`, `readMembers`, `readLinks`, `readFiles`, `readLocked`), `useFieldOptions`, data discipline, the search-as-you-type + record-picker patterns. |
-| [docs/mutations.md](./docs/mutations.md) | `useWorkflow` (the ONLY write path), the `WorkflowResult` resolve-never-throw contract, typed inputs, diff-before-update, locked records, `useOptimistic`, read-after-write ordering (a re-read must not overtake an in-flight write). |
+| [docs/mutations.md](./docs/mutations.md) | `useWorkflow` (the ONLY write path), the `WorkflowResult` resolve-never-throw contract, typed inputs, diff-before-update, locked records, `useOptimistic`, `useNewRecord` (client-minted `rec_*` id so a new-record surface never remounts on its first save), read-after-write ordering (a re-read must not overtake an in-flight write). |
 | [docs/workflows.md](./docs/workflows.md) | **The workflow-BODY authoring reference** — the JS subset a `src/workflows/<alias>.ts` body may use: the parse-at-save/never-execute model, opaque `fld_*`/`opt_*` keys, expression sources + link descent, every step form (tool call, `agent`, waits, `validate`, `return`), the accepted sugar and its canonical lowering, helpers + callback rules, record-write surfaces, the traps, the bright line, and the verify loop — `check` (the only local gate: the app's own `npm run typecheck` never sees a body) → `dry_run_workflow` (static green is not a run) → `set`. |
 | [docs/files.md](./docs/files.md) | Files end to end — `useFileUpload`, `useAttachments`, `readFiles`/presigned URLs (**a bearer credential for the bytes** — never logged, reported, or persisted), workflow-generated files, **naming a zip's entries** (`{ id, name }` per file — a file name, never a path), preview pairing, filter operators, the server-side delivery bounds. **Uploads declare a `fidelity`** (`standard` / `high` / `original`) — the app picks how much of the image survives storage; use `high` whenever text must stay legible. |
 | [docs/members_and_options.md](./docs/members_and_options.md) | People + select options + comments — `useMembers`, `useFieldOptions`, `useViewer`, `useComments`, and the `@lotics/ui` components they feed. |

package/dist/src/index.d.ts

@@ -40,6 +40,8 @@ export { row, readLinks, readFiles, readLocked } from "./row.js";
 export type { ResolvedLink, AppFile } from "./row.js";
 export { useOptimistic } from "./use_optimistic.js";
 export type { OptimisticApi } from "./use_optimistic.js";
+export { useNewRecord, newRecordId } from "./new_record.js";
+export type { NewRecordApi } from "./new_record.js";
 export { useRecents } from "./use_recents.js";
 export type { RecentsApi, RecentsOptions } from "./use_recents.js";
 export { useUrlState } from "./use_url_state.js";

package/dist/src/index.js

@@ -28,6 +28,7 @@ export { readMembers } from "./members.js";
 export { readSelect } from "./select.js";
 export { row, readLinks, readFiles, readLocked } from "./row.js";
 export { useOptimistic } from "./use_optimistic.js";
+export { useNewRecord, newRecordId } from "./new_record.js";
 export { useRecents } from "./use_recents.js";
 export { useUrlState } from "./use_url_state.js";
 export { urlParam } from "./url_params.js";

package/dist/src/new_record.d.ts

@@ -0,0 +1,69 @@
+/**
+ * Mint a record id locally, in the shape the platform mints.
+ *
+ * The generator is restated here rather than imported because the canonical one
+ * lives in a package that is not published, and this one is. The duplication is
+ * safe by construction rather than by discipline: the server validates the shape on
+ * every write, so a client that drifted would be rejected loudly at the first call
+ * instead of quietly persisting a malformed primary key.
+ *
+ * Uses `crypto.getRandomValues` — available in every browser this SDK runs in — and
+ * rejection-samples so each character is uniformly drawn from the alphabet. A plain
+ * `% 62` over bytes would bias the first 8 characters, which is a poor property for
+ * something used as a primary key.
+ */
+export declare function newRecordId(): string;
+export interface NewRecordApi<P> {
+    /**
+     * The id this surface writes under, stable from the first render — before the
+     * record exists, and unchanged by its creation.
+     */
+    id: string;
+    /**
+     * Persist a patch. The first call creates the record, every later one updates it.
+     * Calls are serialised, so firing several before the first resolves is safe.
+     *
+     * Rejects with whatever `create`/`update` rejected with; a failed create leaves the
+     * record uncreated and the next call will try again.
+     */
+    save: (patch: P) => Promise<void>;
+}
+/**
+ * A record that does not exist yet, named before it does.
+ *
+ * The problem this removes: when the server mints the id, a surface editing a new
+ * record has nothing to identify it by until the first write returns. Everything
+ * keyed on that id — the route, the drawer, a list selection — therefore changes
+ * identity mid-edit, which React resolves by remounting the surface the user is
+ * typing into. Minting the id locally makes it stable from the first render, so the
+ * create stops being an event the UI has to survive.
+ *
+ * Creation still happens on the first write, not on mount: a surface the user opens
+ * and abandons should leave nothing behind.
+ *
+ * The transport is the caller's. Like `useOptimistic`, this hook has no idea how the
+ * app persists anything — it takes `create` and `update` thunks and owns only the id
+ * and the ordering, which is the part that is easy to get wrong:
+ *
+ * - Two blur-saves fired before the first resolves must not both create. That is a
+ *   duplicate record, and it is the failure this exists to prevent.
+ * - A save arriving mid-create must wait for it, or it updates a row that is not
+ *   there yet.
+ * - A create that FAILS must not latch. Otherwise every later save updates a record
+ *   that was never written, and the user's work goes nowhere while looking saved.
+ *
+ * ```tsx
+ * const { id, save } = useNewRecord({
+ *   create: (id, patch) => createCustomer({ record_id: id, ...patch }),
+ *   update: (id, patch) => updateCustomer({ record_id: id, ...patch }),
+ *   onCreated: refetch,
+ * });
+ * <InlineText onBlur={(name) => save({ name })} />
+ * ```
+ */
+export declare function useNewRecord<P>(opts: {
+    create: (id: string, patch: P) => Promise<unknown>;
+    update: (id: string, patch: P) => Promise<unknown>;
+    /** Runs once, after the record first exists. Pass a list `refetch` to reveal it. */
+    onCreated?: (id: string) => void;
+}): NewRecordApi<P>;

package/dist/src/new_record.js

@@ -0,0 +1,117 @@
+import { useCallback, useEffect, useRef } from "react";
+const ID_ALPHABET = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
+const ID_LENGTH = 12;
+/**
+ * Mint a record id locally, in the shape the platform mints.
+ *
+ * The generator is restated here rather than imported because the canonical one
+ * lives in a package that is not published, and this one is. The duplication is
+ * safe by construction rather than by discipline: the server validates the shape on
+ * every write, so a client that drifted would be rejected loudly at the first call
+ * instead of quietly persisting a malformed primary key.
+ *
+ * Uses `crypto.getRandomValues` — available in every browser this SDK runs in — and
+ * rejection-samples so each character is uniformly drawn from the alphabet. A plain
+ * `% 62` over bytes would bias the first 8 characters, which is a poor property for
+ * something used as a primary key.
+ */
+export function newRecordId() {
+    const max = 256 - (256 % ID_ALPHABET.length);
+    let out = "";
+    const buf = new Uint8Array(ID_LENGTH * 2);
+    while (out.length < ID_LENGTH) {
+        crypto.getRandomValues(buf);
+        for (let i = 0; i < buf.length && out.length < ID_LENGTH; i++) {
+            if (buf[i] < max)
+                out += ID_ALPHABET[buf[i] % ID_ALPHABET.length];
+        }
+    }
+    return `rec_${out}`;
+}
+/**
+ * A record that does not exist yet, named before it does.
+ *
+ * The problem this removes: when the server mints the id, a surface editing a new
+ * record has nothing to identify it by until the first write returns. Everything
+ * keyed on that id — the route, the drawer, a list selection — therefore changes
+ * identity mid-edit, which React resolves by remounting the surface the user is
+ * typing into. Minting the id locally makes it stable from the first render, so the
+ * create stops being an event the UI has to survive.
+ *
+ * Creation still happens on the first write, not on mount: a surface the user opens
+ * and abandons should leave nothing behind.
+ *
+ * The transport is the caller's. Like `useOptimistic`, this hook has no idea how the
+ * app persists anything — it takes `create` and `update` thunks and owns only the id
+ * and the ordering, which is the part that is easy to get wrong:
+ *
+ * - Two blur-saves fired before the first resolves must not both create. That is a
+ *   duplicate record, and it is the failure this exists to prevent.
+ * - A save arriving mid-create must wait for it, or it updates a row that is not
+ *   there yet.
+ * - A create that FAILS must not latch. Otherwise every later save updates a record
+ *   that was never written, and the user's work goes nowhere while looking saved.
+ *
+ * ```tsx
+ * const { id, save } = useNewRecord({
+ *   create: (id, patch) => createCustomer({ record_id: id, ...patch }),
+ *   update: (id, patch) => updateCustomer({ record_id: id, ...patch }),
+ *   onCreated: refetch,
+ * });
+ * <InlineText onBlur={(name) => save({ name })} />
+ * ```
+ */
+export function useNewRecord(opts) {
+    const idRef = useRef("");
+    if (idRef.current === "")
+        idRef.current = newRecordId();
+    // Callbacks are read through a ref so `save` keeps a stable identity across
+    // renders — it is typically handed to an `onBlur`, and a new function every render
+    // would re-bind every field on every keystroke.
+    //
+    // Updated in an effect rather than during render: React only sanctions writing a
+    // ref while rendering for one-time initialisation (as `idRef` above does), because
+    // a render that is discarded would otherwise leave the ref holding props that never
+    // committed. `save` runs from event handlers, which are always after commit, so the
+    // effect is early enough.
+    const optsRef = useRef(opts);
+    useEffect(() => {
+        optsRef.current = opts;
+    });
+    /** The in-flight or settled create. `null` means the record does not exist yet. */
+    const createdRef = useRef(null);
+    /** Tail of the write chain, so saves apply in the order the user made them. */
+    const queueRef = useRef(Promise.resolve());
+    const save = useCallback((patch) => {
+        const run = async () => {
+            const id = idRef.current;
+            const { create, update, onCreated } = optsRef.current;
+            if (createdRef.current === null) {
+                const attempt = (async () => {
+                    await create(id, patch);
+                })();
+                createdRef.current = attempt;
+                try {
+                    await attempt;
+                }
+                catch (error) {
+                    // Unlatch. The record was not written, so the next save has to be free to
+                    // create it — leaving the promise in place would send every later write to
+                    // a row that does not exist.
+                    createdRef.current = null;
+                    throw error;
+                }
+                onCreated?.(id);
+                return;
+            }
+            await createdRef.current;
+            await update(id, patch);
+        };
+        // Chained on both settle paths: one failed save must not strand the ones behind
+        // it, but they still have to run in order.
+        const result = queueRef.current.then(run, run);
+        queueRef.current = result.catch(() => undefined);
+        return result;
+    }, []);
+    return { id: idRef.current, save };
+}

package/docs/ai.md

@@ -209,6 +209,8 @@ Sessions are scoped to the authenticated member who ran them: two members using
 |---|---|---|
 | **Perceived natively** | `image/*`, `application/pdf` | A vision / document part — the agent literally sees it |
 | **Materialized** | Word (`.docx`/`.doc`), Excel (`.xlsx`/`.xls`), CSV, and text (`.txt`, `.md`, `.json`, `.eml`, `.html`, `.xml`, `.yaml`) | Read server-side by the same engines `view_files` uses and inlined into the run's message, truncated at 40,000 characters with the agent told when that happened |
+
+**Pictures inside a Word document are delivered as images**, so a scanned page, an ID card photographed into a `.docx`, or a screenshot pasted into one is READ, not merely described. A document whose content is entirely pictures carries no text at all — it used to arrive as empty paragraphs plus a size in centimetres, and the agent correctly reported it could not read it. Any picture that is skipped or unresolvable is counted and stated in the text, so a partially-read document never reads as a complete one.
 | **Unreadable** | Archives, audio, video | The run fails immediately, naming the file — a fact about the upload, not a gap in your configuration |
 
 Every file in a `multi` input is materialized — nothing is collapsed to the first. Get the ids from [`useFileUpload` / `useAttachments`](./files.md).
@@ -223,6 +225,7 @@ Every file in a `multi` input is materialized — nothing is collapsed to the fi
 - **Owner-billed.** Token usage is recorded against the app's organization's AI credits. The quota is enforced before the first model call (a `run()` rejection) and re-checked between steps (a mid-run exhaustion settles the run as an error).
 - **Hard duration cap: 20 minutes per run.** A capped run settles as an error — but with whatever partial transcript and captured output it produced, recoverable from the persisted run.
 - **Concurrency cap: 5 in-flight runs per member.** Exceeding it rejects with "Too many agent runs in progress".
+- **One tool result may not exceed 512 KiB.** A tool's result is persisted into the run's transcript and replayed on every later step, so an oversized one is paid again per step until the run stops fitting in a request. A tool that overruns fails by name ("Tool \"x\" returned N bytes…") rather than being delivered; narrow the call, or use a tool that returns a reference instead of the bytes. Reaching this from a normal call means too much was asked for at once — read a range, not a whole sheet.
 
 **Limitation (dev harness):** `lotics app dev` streams runs but does not forward the run id, so in the dev loop `cancel()` degrades to a local `abort()` (the run keeps executing server-side) and connection-drop recovery is unavailable. Both work fully in the embedded product.
 

package/docs/mutations.md

@@ -539,3 +539,40 @@ This is the full read → mutate → reconcile loop: `useQuery` reads, `row.*` c
 one workflow per (table, field) you mutate — typed and narrow, never a generic
 `setField(any_field)` that hands the client write access to every field
 ([security](./security.md)).
+
+## Editing a record that does not exist yet: `useNewRecord`
+
+A "new record" surface — a drawer, a detail page reached from a **New** button — has a problem
+before its first write: it has no id. If the server mints one, then everything keyed on that id
+(the route, the drawer, a list selection) *changes identity* the moment the first field saves,
+and React answers an identity change by remounting the surface the user is typing into.
+
+Mint the id on the client instead. `create_records` accepts caller-supplied `rec_*` ids, so the
+id is known from the first render and the create stops being an event the UI has to survive:
+
+```tsx
+const { id, save } = useNewRecord({
+  create: (id, patch) => createCustomer({ record_id: id, ...patch }),
+  update: (id, patch) => updateCustomer({ record_id: id, ...patch }),
+  onCreated: list.refetch,
+});
+
+<InlineText onBlur={(name) => save({ name })} />
+```
+
+- **The record is still created on the first write, not on mount** — a surface the user opens
+  and abandons leaves nothing behind.
+- **`save` serialises.** The first call creates, every later one updates, and calls made before
+  the create resolves queue behind it. Two fields blurred in quick succession therefore produce
+  one record, not two. This is the whole reason to use the hook rather than a `useRef` latch:
+  hand-rolled, the racing-blur case creates a duplicate.
+- **A failed create does not latch.** The next `save` retries the create, rather than updating a
+  row that was never written while the UI looks like it saved.
+- `id` and `save` are stable across renders, so `save` can be bound directly to an `onBlur`.
+
+Your `create` workflow must pass the id through to `create_records` as `ids: [record_id]`.
+Creation is creation: an id that already exists is a conflict, never an overwrite — so the
+workflow cannot be used to clobber another record by guessing its id.
+
+Use `newRecordId()` directly if you need the id outside a hook (routing to the surface before
+mounting it, say). It mints the same `rec_*` shape the server validates.

package/docs/queries.md

@@ -853,6 +853,22 @@ per-table queries you merge client-side; the merge already happens in the index.
 sort key to be a bare field projection — a computed/literal/type-overridden sort column falls back
 to the whole-union sort.)
 
+**An aggregate arm you cannot filter costs its whole table, every execution.** The shape to watch
+is the mirror of the one above: a `join` whose right side is a `group` over one or more entire
+tables — a lookup built by scanning everything in order to decorate a small left side. Rule 1 is
+no help here, and that is what makes it easy to ship: the arm keys on a value the LEFT side
+supplies at run time (a code, a serial number), so there is no static predicate to push down. The
+left side's size is irrelevant; you pay for the arms. As a unit to budget with: a four-table arm
+totalling ~14k rows is **seconds, not milliseconds**, on every single execution. Prefer a
+materialized **lookup/rollup field** on the row (the platform keeps it current and it filters at
+the source), or drop the arm entirely if the value it fetches now lives on the row already.
+
+**Measure before you change a shape, and after.** Latency is observable per alias: each `useQuery`
+goes out as its own RPC, so a screen's cold load attributes a duration to every query by name in
+the browser's network panel. Time it, fix the one that dominates, time it again. Reasoning
+alone mis-ranks these — the union above looks expensive and is fast, the join here looks ordinary
+and is not — and a playbook rule applied to the wrong query costs effort while proving nothing.
+
 ### The authoring rules
 
 1. **Filter at the source.** Push every static predicate into `from_table.filter`.
@@ -875,6 +891,12 @@ to the whole-union sort.)
     at once (`by: [a, b, c, …]`) and fold each facet client-side by summing over the others:
     `count` and `sum` fold exactly. `unique` does NOT fold across groups (the same value can
     appear in many groups) — keep a separate query for each distinct-count you render.
+11. **Re-derive a query when its tables change shape.** A `join` or `union` that exists to bridge
+    two tables becomes pure cost the moment those tables become one — and nothing fails, because
+    it keeps returning the right answer at the old price. Migrations that merge, move or back-fill
+    a table are exactly when this happens, and exactly when nobody re-reads the queries. After
+    one, open every query over the affected tables and ask what it would look like written today,
+    not what it needs to keep working.
 
 ---
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@lotics/app-sdk",
-  "version": "0.63.0",
+  "version": "0.64.0",
   "description": "Runtime SDK for Lotics custom-code apps — typed hooks, postMessage bridge, mount entry point",
   "type": "module",
   "exports": {