@ultimat3/query 31.4.1 → 31.5.0

package/CLAUDE.md

@@ -14,6 +14,7 @@ Owns the `query` primitive: reads, live reads, cursors, the incremental matcher.
 |---|---|
 | `query.ts` | the primitive: `query()`, `describeQuery`, `queryHash`; the front door for the read path |
 | `read.ts` | **the one read path** (`runQuery`, `sourceFor`) + the private declaration store `sql` lives in |
+| `parsed-input.ts` | `parsedQueryInput`: the parse alone, the input every `(query, input)` key hashes |
 | `rows-override.ts`, `adapter.ts` | `@ultimat3/query/adapter`'s `overrideRows(query, rows \| fn \| undefined)`: canned rows where `sql()` is, answered in `buildSource` AFTER the parse and the policy, never cached (`rows-override.test.ts`). Never on the barrel — `@ultimat3/testing`'s `stub()` is its one caller |
 | `contract-test.ts` | `.contract()`, the generated suite. The three `X_CONTRACT_*` codes and `X_CONTRACT_DRIFT` are borrowed (`QUERY_BORROWED_ERROR_CODES`) |
 | `exercise.ts` | the exercise twin. "Reached" means `sql()` built a source, read off the refusal (`beforeSql`), because `read.ts` keeps no counter. No transaction: a read writes nothing |
@@ -217,7 +218,9 @@ Owns the `query` primitive: reads, live reads, cursors, the incremental matcher.
   `min(rowsLeft, first + 1)`, and a limited read's cursor carries the rows served so far at the
   tail of its key (after the typed id). A cursor without it is `X_CURSOR_INVALID` on a limited
   read. `Builder.seek()` takes the smaller limit too. `pagination-limit.test.ts`.
-- **A `RowProvider` may be a list, a sync function or an async one** (`source.test.ts`).
+- **A `RowProvider` may be a list, a sync function or an async one** (`source.test.ts`). A
+  function is HANDED the `RowWindow` (`after`, `limit`, `order`, `filters`) on every `execute()`,
+  and the builder re-applies all four over its answer (`pagination-window.test.ts`).
 - **A cursor is a position, not a row**: `isAfterKey` (`source.ts`) is the one definition of "after";
   never reintroduce a row lookup. The seek predicate is spelled out per key
   (`(a < $1) or (a = $2 and id > $3)`), never a row-value comparison.
@@ -252,14 +255,17 @@ Owns the `query` primitive: reads, live reads, cursors, the incremental matcher.
   `Map`, `Set`. `query-hash.test.ts` pins it at `queryHash` and `cacheKeyFor`. `stable.ts` keeps
   `columnOf`; core's `isJsonObject` is imported from core (`X_HELPER_COPY` refuses a re-export).
 - The cursor codec is `@ultimat3/core`'s; this package supplies only the scope (`queryHash(name,
-  input)`). An unverified cursor is `X_CURSOR_INVALID` (`CursorInvalidError`, core's — imported from `@ultimat3/core`, never re-exported).
+  input)`) — of the RAW input (known limit: a stripped key re-scopes a cursor; kept so outstanding
+  cursors stay valid). A live hash is the parsed input's (`builtFor`). An unverified cursor is `X_CURSOR_INVALID` (`CursorInvalidError`, core's — imported from `@ultimat3/core`, never re-exported).
 
 ## Invariants — caching
 
 - **The request memo holds the read, not the rows**: `readOnce` publishes the in-flight promise
   before its first await; a rejection is evicted. `requestMemo(ctx)` is `Map<string, Promise<unknown>>`.
 - **Every read is memoized; only a `cache:` read goes through the tier** (`readThrough`). One key
-  function, `cacheKeyFor`.
+  function, `cacheKeyFor`, over the PARSED input `admitAndBuild` returns — never `raw`, or a
+  stripped `?junk=N` is an entry and a read per N (`read-cache-key.test.ts`). Outside, a key
+  hashes `parsedQueryInput(query, raw)`.
 - **A cache key carries the read's AUTHORITY**: `readAuthority(actor, scope)` is its one producer and
   `cacheKeyFor`'s fourth argument is required. `scope` defaults to `'actor'`; `'tenant'` with no
   `orgId` (`undefined`, `''` or `null` — `orgless()`) narrows to the actor. The authority is JSON.

package/README.md

@@ -218,6 +218,7 @@ forgetting the policy, and the reason is what tells the next reader which of the
 |---|---|
 | `query.ts` | the primitive, `describeQuery`, `queryHash` |
 | `read.ts` | the one read path — `runQuery`, `sourceFor` — and the declaration store |
+| `parsed-input.ts` | `parsedQueryInput(query, raw)` — the input every key over `(query, input)` hashes |
 | `facade.ts` | binds each projection to the query; re-implements none of them |
 | `mcp-tool.ts` | the MCP read descriptor |
 | `client.ts` | the typed read client (browser-safe) |
@@ -399,6 +400,14 @@ HMAC-signed, and bound to one query + arguments — a cursor from another query
 `query.page(input, { first, after })` is the only way to ask for one — `paginate` backs it and is
 not exported, because a page is the read's own answer rather than an imported helper.
 
+**A function `RowProvider` is handed the window** (`RowWindow`: `after`, `limit`, `order`,
+`filters`; `As of 31.4.3`). `from('posts', (window) => repo.page(window))` pushes the cursor down,
+so a provider that reads a bounded window (a repository's top N) reaches every row: called with
+nothing, it served page one's rows to every page and page two came back short. `from()` re-applies
+the filter, order, seek and cut over what comes back, so ignoring the argument is still correct —
+but a provider that honours `limit` must honour `after`, `order` and `filters` too
+(`pagination-window.test.ts`; the reference app's `org-posts.ts` is the worked push-down).
+
 ### The same page over HTTP — `?_first=` and `?_after=`
 
 The route reads two controls off the search string BEFORE the schema sees it, and with either
@@ -538,6 +547,12 @@ up. Keys are `query:<name>:<authority>:<input fingerprint>:<tags>`. An action's
 and a query's `cache.tags` meet in the one graph owned by `@ultimat3/cache`, because there is one
 registry and this package holds no store of its own.
 
+**The input fingerprint is the PARSED input, never the raw payload** (`As of 31.4.3`). A key the
+schema strips — `?junk=1`, `?junk=2`, … — is one entry and one read, not a new entry per value
+that evicts a hot one (`read-cache-key.test.ts`). Anything else keyed on `(query, input)` asks for
+the same value: `parsedQueryInput(query, raw)` is the parse alone (`X_INPUT_INVALID` on a refused
+input; no actor, policy, limit or `sql()`), then `queryHash(name, parsed)`.
+
 **The authority is who the read was answered for, and it is not optional.** `sql(input, ctx)` is
 handed the context and `@ultimat3/entity` derives every tenant predicate from `ctx.actor.orgId`,
 never from the input — so a key made of the name, the input and the tags did not identify a read's

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ultimat3/query",
-  "version": "31.4.1",
+  "version": "31.5.0",
   "description": "The query primitive: a policy-checked read, optionally live, with cursor pagination and an incremental matcher",
   "keywords": [
     "ultimate",
@@ -49,12 +49,12 @@
     "test": "bun test"
   },
   "dependencies": {
-    "@ultimat3/cache": "31.4.1",
-    "@ultimat3/core": "31.4.1",
-    "@ultimat3/db": "31.4.1",
-    "@ultimat3/entity": "31.4.1",
-    "@ultimat3/http": "31.4.1",
-    "@ultimat3/policy": "31.4.1",
-    "@ultimat3/schema": "31.4.1"
+    "@ultimat3/cache": "31.5.0",
+    "@ultimat3/core": "31.5.0",
+    "@ultimat3/db": "31.5.0",
+    "@ultimat3/entity": "31.5.0",
+    "@ultimat3/http": "31.5.0",
+    "@ultimat3/policy": "31.5.0",
+    "@ultimat3/schema": "31.5.0"
   }
 }

package/src/index.ts

@@ -128,6 +128,8 @@ export { MAX_WIRE_PAGE_SIZE } from './page-controls';
  * `@ultimat3/core`'s — one place to encode, decode or re-key a cursor.
  */
 export type { Page, PaginateArgs } from './pagination';
+/** The input as the schema keeps it — what a key over `(query, input)` hashes (`queryHash`). */
+export { parsedQueryInput } from './parsed-input';
 export type { QueryPolicy, QuerySubject, QuerySurface } from './policy-gate';
 /**
  * The one authz gate for reads, named after what it guards. The display label
@@ -171,7 +173,7 @@ export {
 /** What a `single: true` read answers — one rule, also `@ultimat3/mcp`'s served tool's. */
 export { readAnswer } from './single-answer';
 /** `Builder` is a type only: `from()` is the one way to build one, so there is no `new Builder()`. */
-export type { Builder, RowProvider, SqlSource, SqlText } from './source';
+export type { Builder, RowProvider, RowWindow, SqlSource, SqlText } from './source';
 /** `isAfterKey` is the one definition of "after this position" — both seek paths use it. */
 export { from, isAfterKey } from './source';
 export type { ExplainResult, QuerySqlInfo } from './sql';

package/src/live.ts

@@ -18,7 +18,7 @@ import type { AnyQuery, Query } from './query';
 import { queryHash } from './query';
 import type { QueryLimitCaller } from './rate-limit-gate';
 import { spendReadLimit } from './rate-limit-gate';
-import { queryName, sourceFor } from './read';
+import { builtFor, queryName } from './read';
 import { assertNoSealedKey } from './sealed-shape';
 import type { QueryShape, SeekKey } from './shape';
 import { seekKeyOf } from './shape';
@@ -131,7 +131,9 @@ export async function toLiveQuery<TInput extends StandardSchemaV1, TRow extends
   options: ToLiveOptions = {},
 ): Promise<LiveQuery> {
   const name = queryName(target);
-  const source = await sourceFor(target, input, {
+  // `builtFor`, not `sourceFor`: the hash below is the PARSED input's, so a key the schema strips
+  // (`{ q, junk }`) is the same live identity and cursor scope as `{ q }` (`read-cache-key.test.ts`).
+  const { source, input: parsed } = await builtFor(target, input, {
     ...(options.ctx === undefined ? {} : { ctx: options.ctx }),
     // The boolean stays this layer's spelling because it has exactly ONE reason, stated on
     // `ToLiveOptions.enforce` above; `sourceFor` takes that reason as a string so every skipped
@@ -150,7 +152,7 @@ export async function toLiveQuery<TInput extends StandardSchemaV1, TRow extends
   assertNoSealedKey(name, shape);
 
   const epoch = options.epoch ?? liveEpoch();
-  const hash = queryHash(name, input);
+  const hash = queryHash(name, parsed);
   const reads = [...new Set([shape.entity, ...tagKeys(target.cache?.tags ?? [])])].sort();
   // The query's own policy object, not a copy: the same authz a direct read runs.
   const policy = target.policy;

package/src/parsed-input.ts

@@ -0,0 +1,16 @@
+// A read's input as its declared schema keeps it — what every key over `(query, input)` hashes.
+// Keyed raw, an unknown key the schema strips (`?junk=N`) minted a new entry per value; the read
+// cache keys on the parse inside `read.ts`, and a live window (`@ultimat3/realtime`) asks here.
+
+import type { AnyQuery } from './query';
+import { defOf, queryName } from './read';
+import { parseQueryInput } from './validate';
+
+/**
+ * The parse alone: no actor, no policy, no rate limit, no `sql()`. `X_INPUT_INVALID` for an input
+ * the schema refuses, exactly as the read path answers it. The caller still reads through
+ * `sourceFor` / `.live()` — this decides a KEY, never an admission.
+ */
+export function parsedQueryInput(target: AnyQuery, raw: unknown): Promise<unknown> {
+  return parseQueryInput(defOf(target).input, raw, queryName(target));
+}

package/src/read.ts

@@ -127,18 +127,35 @@ export function runQuery(
  * `explain` and the MCP tool all build on this, so none of them re-implement the
  * front half and none of them can skip the policy while doing it.
  */
-export function sourceFor(
+export async function sourceFor(
   target: AnyQuery,
   raw: unknown,
   options: SourceOptions = {},
 ): Promise<SqlSource<object>> {
-  return asActor(options, (ctx) => {
+  return (await builtFor(target, raw, options)).source;
+}
+
+/**
+ * `sourceFor` keeping the input `admit` parsed — `toLiveQuery` hashes THAT, so a key the schema
+ * strips cannot mint a second live identity. Internal: never re-exported from `src/index.ts`.
+ */
+export function builtFor(
+  target: AnyQuery,
+  raw: unknown,
+  options: SourceOptions = {},
+): Promise<{ readonly source: SqlSource<object>; readonly input: unknown }> {
+  return asActor(options, async (ctx) => {
     const audit = auditFor(target, ctx, options);
     // A source handed out is executed by its caller — `@ultimat3/mcp`'s served tool among them —
     // so an audited read records per `execute()`, and a build that refuses records here.
-    return audit === null
-      ? buildSource(target, raw, ctx, options, null)
-      : auditedBuild(audit, () => buildSource(target, raw, ctx, options, audit.trace));
+    if (audit === null) return admitAndBuild(target, raw, ctx, options, null);
+    let input: unknown;
+    const source = await auditedBuild(audit, async () => {
+      const built = await admitAndBuild(target, raw, ctx, options, audit.trace);
+      input = built.input;
+      return built.source;
+    });
+    return { source, input };
   });
 }
 
@@ -252,7 +269,7 @@ async function readRowsIn(
 ): Promise<readonly object[]> {
   const def = defOf(target);
   const name = queryName(target);
-  const source = await buildSource(target, raw, ctx, options, trace);
+  const { source, input } = await admitAndBuild(target, raw, ctx, options, trace);
   // Whether THIS call executed, for the audit record: a memo or tier hit never calls `read`, and
   // its record says so (`replayed: true`) rather than being skipped.
   if (trace !== null) trace.replayed = true;
@@ -275,7 +292,9 @@ async function readRowsIn(
   // nothing — a memo is already per-ctx — and it is the whole of what the tier was missing: keyed
   // on the name, the input and the tags alone, the process-wide tier handed one org's rows to the
   // next org that asked for them. `actor` when the declaration named no scope, always.
-  const key = cacheKeyFor(name, raw, tags, readAuthority(ctx.actor, def.cache?.scope ?? 'actor'));
+  // The PARSED input, never `raw`: a key the schema strips (`?junk=N`) would otherwise be a new
+  // entry and a database read per value, each evicting a hot one (`read-cache-key.test.ts`).
+  const key = cacheKeyFor(name, input, tags, readAuthority(ctx.actor, def.cache?.scope ?? 'actor'));
   // `fresh` is the caller saying no cache may answer this one — the memo included, a memo being
   // a cache whose lifetime is the request. It still *publishes* into the memo: this read is the
   // newest answer the request has, so the next plain read of the key joins it rather than the
@@ -335,7 +354,28 @@ async function buildSource(
   options: SourceOptions,
   trace: ReadTrace | null,
 ): Promise<SqlSource<object>> {
+  return (await admitAndBuild(target, raw, ctx, options, trace)).source;
+}
+
+/** `buildSource`, keeping the admitted input: the cache key is what the policy decided about. */
+async function admitAndBuild(
+  target: AnyQuery,
+  raw: unknown,
+  ctx: Ctx,
+  options: SourceOptions,
+  trace: ReadTrace | null,
+): Promise<{ readonly source: SqlSource<object>; readonly input: unknown }> {
   const input = await admit(target, raw, ctx, options, trace);
+  return { source: sourceOver(target, input, ctx, options), input };
+}
+
+/** `sql()` — or the canned rows — over an input `admit` already let through. */
+function sourceOver(
+  target: AnyQuery,
+  input: unknown,
+  ctx: Ctx,
+  options: SourceOptions,
+): SqlSource<object> {
   const canned = rowsOverrideOf(defOf(target));
   if (canned !== undefined) {
     const rows = canned;

package/src/source.ts

@@ -40,15 +40,34 @@ export interface SqlSource<TRow> {
   seek?(after: SeekKey | null, limit: number): SqlSource<TRow>;
 }
 
+/**
+ * The window a source is asked for, handed to a function `RowProvider` on every `execute()`.
+ * `after` is the keyset position (rows strictly past it in `order`), `limit` the most rows wanted,
+ * `order` the order actually served (the declared keys, then `id` once paged), `filters` every
+ * `.where()`/`.compare()` on the chain. The builder re-applies all four over what comes back, so
+ * a provider may honour any of them or none — but one that honours `limit` must also honour
+ * `after`, `order` and `filters`, or the cut lands before the rows it was meant to keep.
+ */
+export interface RowWindow {
+  readonly after: SeekKey | null;
+  readonly limit: number | null;
+  readonly order: readonly OrderKey[];
+  readonly filters: readonly Filter[];
+}
+
 /**
  * Rows, or a function that answers them. The function half is `readonly TRow[] | Promise<…>`
  * because `execute()` **awaits** whatever it returns — declaring only the promise refused a
  * synchronous provider the implementation has always accepted, which is a repo method that
  * already has its page in hand, and every in-memory fixture.
+ *
+ * The function is handed the `RowWindow`. Called with nothing, a repository's top N was the same N
+ * rows on every page: rows past it were unreachable and page two came back short. A provider that
+ * reads a window pushes `after` and `limit` down; one that reads everything ignores the argument.
  */
 export type RowProvider<TRow> =
   | readonly TRow[]
-  | (() => readonly TRow[] | Promise<readonly TRow[]>);
+  | ((window: RowWindow) => readonly TRow[] | Promise<readonly TRow[]>);
 
 /**
  * In-memory reference source. `from<Post>('posts', rows).where({ orgId }).orderBy('createdAt')`
@@ -141,11 +160,20 @@ export class Builder<TRow extends object> implements SqlSource<TRow> {
   }
 
   async execute(): Promise<readonly TRow[]> {
-    const source = typeof this.rows === 'function' ? await this.rows() : this.rows;
+    const keys = this.servedOrder();
+    const source =
+      typeof this.rows === 'function'
+        ? await this.rows({
+            after: this.after,
+            limit: this.rowLimit,
+            order: keys,
+            filters: this.filters,
+          })
+        : this.rows;
     this.assertCarried(source);
     const kindOf = kindsOf(this.entity);
+    // Idempotent over a window the provider already cut: filter, sort, seek and slice again.
     let result = source.filter((row) => matchesFilters(row, this.filters, kindOf));
-    const keys = this.servedOrder();
     if (keys.length > 0) {
       result = [...result].sort((a, b) => compareRows(a, b, keys, kindOf));
     }