okengine 0.11.2 → 0.13.0

package/site/content/docs/elements/store.mdx

@@ -25,6 +25,81 @@ name).
 
 The relational facet: tables, schema, and typed single-table sessions through `fx.store`.
 
+The Console SQL band opens a query console (`POST /console/store/sql`) for
+DML and DDL. ⌘Enter runs the current statement or selection; ⌘⇧Enter runs
+the whole script sequentially and stops on the first error.
+
+EXPLAIN / EXPLAIN ANALYZE wrap the current statement. A script with more
+than one result opens a tab per statement.
+
+The table grid can insert a row (`POST /console/store/edit` without `id`).
+Double-click a tab to rename; Save keeps named queries. Completions cover
+Manifest tables, columns, and keywords.
+
+Classified PII stays masked unless the toolbar **PII** control is on
+(`revealPii: true`, audited).
+
+**Gate** views the result as Operator (default, bypasses RLS), public, a
+selected user (via their matching policy), or a Manifest policy gate.
+`asGate` sets `oke.gate` on postgres / pglite for `current_setting('oke.gate', true)`.
+
+The SQL band **Schema** control opens a visualizer of Manifest tables and
+relations — declared `.references()` plus inferred `*_id` columns — not the
+query-console Schema rail.
+
+<Callout title="Query performance">
+  SQL **Performance** reads `pg_stat_statements` and live locks
+  (`QUERY /console/store/sql/stats` and `/locks`).
+
+Alpine needs preload plus `CREATE EXTENSION`. Index Advisor is opt-in on
+[Postgres](/docs/recipes/postgres#query-performance).
+
+KV **Performance** reads Redis-wire `INFO` / `COMMANDSTATS` / `SLOWLOG`
+(`QUERY /console/store/kv/stats`). In-process `memory` is unsupported.
+
+INFO is instance-wide (`StoreKvStatsServerWideGap`). SLOWLOG args collapse
+until reveal (`StoreKvStatsSlowlogArgsGap`). No hot-key table.
+
+</Callout>
+
+**Colorize** groups tables by prefix and FK family. Click a related table to
+dim the rest; click the canvas to show all again.
+
+Edges mark cardinality with icons (many-to-one, one-to-one, many-to-many,
+self). Cards and the query Schema rail mark primary (key), foreign (link),
+and unique (diamond) columns.
+
+Each SQL store also lists **Indexes**, **Functions**, **Triggers**,
+**Extensions**, and **RLS Policies**.
+
+Indexes, Functions, and Triggers use the same flat form as RLS Policies —
+paired fields, compact templates, and a pinned SQL editor — then run
+`CREATE INDEX` / `CREATE FUNCTION` / `CREATE TRIGGER`.
+
+**Advanced** unlocks the leftover Postgres knobs (`CONCURRENTLY` / `INCLUDE`,
+`SECURITY DEFINER` / `PARALLEL`, `CONSTRAINT` / `UPDATE OF`).
+
+Extensions list built-in contrib plus a **Library** (featured cards, search,
+category filter) for TimescaleDB, PostGIS, and `pg_cron`.
+
+Install reviews the `CREATE EXTENSION` SQL; **Advanced** can set `SCHEMA`,
+`VERSION`, and `CASCADE` (`pg_catalog` is refused).
+
+When a newer packaged version exists, the name chip shows **Upgrade**
+(`ALTER EXTENSION … UPDATE`). Names link to the project or Postgres docs.
+
+Each SQL table row shows whether Postgres RLS is on (emerald shield)
+or off (muted shield). KV, files, and index have no badge. Enabling
+RLS on create flips that shield.
+
+**RLS Policies** lists `pg_policies`. Create policy pins the SQL preview;
+**Templates** slides a flush library beside it (every `FOR` command, owner,
+`oke.gate`). Or edit `roles` / `USING` / `WITH CHECK` in the grid (`ALTER POLICY`).
+
+Create policy picks a Gate posture (Read / Write / Both), or **Policy &
+scope** for `gate.public`, `gate.policy`, and `gate.scope`. `TO public`
+stays on the SQL — those picks are not Postgres roles.
+
 ### Quick start
 
 <Steps>
@@ -115,24 +190,25 @@ const notesR = store.resource(db, notes, {
     filter: "all", // ?col=op.value — "all" | Column[] | "none"
     order: "all", // ?order=col.desc
   },
-  unit: "notes",
 });
 
-const mounted = on(http.resource("/notes", notesR.all()));
+const mounted = on(http.resource("/notes", notesR.all()).gate.public.live());
 ```
 
+`.gate(...)` / `.live()` chain like `http.get` — gates on every verb, live on list and get.
+
 The list endpoint's URL is the whole query language:
 
-| Param                               | Meaning                                             | Example                 |
-| ----------------------------------- | --------------------------------------------------- | ----------------------- |
-| `?cursor=` / `?offset=` / `?limit=` | paginate (keyset when `cursor` columns are set)     | `?limit=20&cursor=eyJ…` |
-| `?search=` (`?q=`)                  | substring match over `search` columns               | `?q=invoice`            |
-| `?col=op.value`                     | filter — ops `eq ne gt gte lt lte like ilike in is` | `?title=like.%draft%`   |
-| `?or=(…)` / `?and=(…)`              | grouped boolean filters                             | `?or=(a.eq.1,b.eq.2)`   |
-| `?order=`                           | sort                                                | `?order=createdAt.desc` |
-| `?select=`                          | project columns                                     | `?select=id,title`      |
+| Param                               | Meaning                                                   | Example                               |
+| ----------------------------------- | --------------------------------------------------------- | ------------------------------------- |
+| `?cursor=` / `?offset=` / `?limit=` | paginate (keyset when `cursor` columns are set)           | `?limit=20&cursor=eyJ…`               |
+| `?search=` (`?q=`)                  | substring match over `search` columns                     | `?q=invoice`                          |
+| `?col=op.value`                     | filter — `eq neq gt gte lt lte like ilike in is` + `not.` | `?title=like.*draft*`                 |
+| `?or=(…)` / `?and=(…)`              | grouped filters, nested `not.and` / `not.or`              | `?or=(a.eq.1,not.and(b.gt.2,c.lt.9))` |
+| `?order=`                           | sort (`col.desc`, comma-separated)                        | `?order=createdAt.desc`               |
+| `?select=`                          | project columns                                           | `?select=id,title`                    |
 
-Responses follow the Stripe-style envelope: `{ data, meta: { nextCursor, hasNextPage }, error }`. `create` answers **201**, `remove` answers **204**, and a missing row is a typed `NotFound` — never a crash.
+Responses follow the Stripe-style envelope: `{ data, meta, error }`. Clients page with `page.next()` or `for await (const page of api.notes.list({ limit: 20 }))` — `meta.next` / `meta.prev` are the next request (`{ cursor }`), not flags. `mode` is diagnostic. `create` answers **201**, `remove` answers **204**, and a missing row is a typed `NotFound` — never a crash.
 
 #### An admin table
 
@@ -158,9 +234,37 @@ list: { mode: "offset", filter: "none", limit: 20 },
 
 **Consequence:** a request that filters on a forbidden column — `?secret=eq.x` — fails with **422** and the exact message `unknown list param "secret"`. Filterable columns are a whitelist, never an accident.
 
+### Handwritten HTTP lists
+
+`store.resource` is the usual CRUD path. When you write the list Flow yourself, prefer the same Stripe envelope — `out` is the item array, pagination sits on `meta`:
+
+```typescript
+export const list = on(
+  http.get("/notes").gate.public,
+  flow("notes.list", {
+    out: z.array(Note),
+    do: async (input, fx) => {
+      const rows = await fx.store(db).select().from(notes);
+      return fx.json.withQuery(rows, input);
+    },
+  }),
+);
+```
+
+Zero-config: `?q=` searches every string field, extra keys auto-eq except path `id`, and PostgREST `?col=op.value` / `?order=` / `?select=` are open. Nested lists filter the parent in the load — `input.id` is not a column eq.
+
+<Callout title="Preferred, not required">
+  Any declared `out` is valid — `{ notes: […] }` still works. The array + `fx.json.withQuery`
+  shape matches `store.resource` and the typed client (`data` is the page, `meta` is the pager).
+</Callout>
+
+**Consequence:** returning `{ items, count }` from `do` nests as `{ data: { items, count }, error: null }` — the pager is no longer top-level `meta`.
+
+`fx.json.with({ data, meta })` stays for a page you already built. A third argument locks the surface or switches to keyset (`mode: "cursor"`, `cursor: ["createdAt", "id"]`).
+
 ### Querying by hand
 
-When `store.resource` is too conventional, `fx.store(db)` is the full single-table session:
+When the list is not conventional CRUD, `fx.store(db)` is the full single-table session:
 
 ```typescript
 // select — chain where / orderBy / limit / offset in any order
@@ -232,11 +336,9 @@ export const relations = store.schema.relations({ links, daily }, (r) => ({
 | `oke db generate` | Write versioned SQL under `drizzle/` for review                |
 | `oke db migrate`  | Apply those files — human or CI, **never** at boot             |
 
-`oke dev` auto-pushes when you save a schema _input_ (`schema.decl.ts`,
-hand-written `schema.ts`, `app.ts` for plugin tables, or `drizzle.config.ts`) —
-not when emit rewrites `schema.drizzle.ts`. A definition error in
-`schema.decl.ts` prints `schema.decl.ts has an error — …` (red ●) and keeps the
-session running — it is not framed as a quiet skip.
+`oke dev` auto-pushes on schema _inputs_ (`schema.decl.ts`, hand-written `schema.ts`,
+`app.ts` plugin tables, `drizzle.config.ts`) — not when emit rewrites `schema.drizzle.ts`.
+A `schema.decl.ts` error prints `schema.decl.ts has an error — …` (red ●) and keeps running.
 
 Opt out with `--no-db-push` or `db: { autoPush: false }`. `prod` **never**
 auto-applies DDL; a missing table fails as **OKE1101** (`oke db migrate`).
@@ -269,8 +371,8 @@ Not wired this round — readiness only.
 
 #### Seeding
 
-`oke db seed` loads `defineSeed` from `src/seed/index.ts` — **never at boot**.
-Seed proves existence (bootstrap rows); it does not correct schema-adjacent data over time.
+`oke db seed` loads `defineSeed` from `src/db/seed/index.ts` — **never at boot**.
+`name` is this app's seed (template or example). The prompt and `.oke/state.json` key off it.
 
 <StoreSeeding />
 
@@ -290,6 +392,8 @@ import { db } from "../core";
 import { notes } from "../schema.decl";
 
 export default defineSeed({
+  name: "notes",
+  description: "Welcome + sample notes",
   essential: async (fx: Fx) => {
     await fx
       .store(db)
@@ -321,6 +425,8 @@ import { seedSamples } from "./dev/samples";
 import { seedWebhook } from "./prod/webhook";
 
 export default defineSeed({
+  name: "notes",
+  description: "Welcome + sample notes",
   essential: [seedWelcome],
   dev: [seedSamples],
   prod: [seedWebhook],
@@ -421,6 +527,12 @@ SQL-backed index driver `pgvector` shares this facet's connection — see [Advan
 The key-value facet: short-lived cache and session data behind `fx.store`. One namespace per
 declaration; drivers swap `memory` ↔ `redis` per environment.
 
+The Console KV band opens a query console for `list`, `get`, `set`, `delete`, and `ttl`.
+Prettify rewrites commands to call form and indents `set` JSON.
+
+The same band’s **Performance** control opens engine telemetry on the `redis`
+driver. In-process `memory` is unsupported (`KvStatsUnsupported`).
+
 ### Quick start
 
 <Steps>
@@ -454,17 +566,19 @@ do: async ({ userId, data }, fx) => {
 
 ### Handle API
 
-| Call                    | Effect  | Returns    | Notes                                     |
-| ----------------------- | ------- | ---------- | ----------------------------------------- |
-| `get(key)`              | `read`  | `unknown`  | Missing keys resolve per driver           |
-| `set(key, value, ttl?)` | `write` | `void`     | Values should be JSON-serializable        |
-| `delete(key)`           | `write` | `boolean`  | Whether a key was removed                 |
-| `list(prefix?)`         | `read`  | `string[]` | Prefix filter; powers the Console browser |
+| Call                    | Effect  | Returns          | Notes                                                                    |
+| ----------------------- | ------- | ---------------- | ------------------------------------------------------------------------ |
+| `get(key)`              | `read`  | `unknown`        | Missing keys resolve per driver                                          |
+| `set(key, value, ttl?)` | `write` | `void`           | Values should be JSON-serializable                                       |
+| `delete(key)`           | `write` | `boolean`        | Whether a key was removed                                                |
+| `list(prefix?)`         | `read`  | `string[]`       | Prefix filter; Console browse shows TTL and size                         |
+| `ttlMs(key)`            | `read`  | `number \| null` | Remaining expiry; `null` when the key has none. Console command is `ttl` |
 
 ### TTL
 
 Optional third argument on `set` is a duration string: `(\d+)(ms|s|m|h|d)` — e.g. `"30m"`, `"1h"`,
-`"5000ms"`. Same call — opposite physics under each driver:
+`"5000ms"`. Same call — opposite physics under each driver. Console KV browse edits that TTL
+(empty clears expiry) and can add a key through the same `set` path.
 
 <StoreKvTtl />
 
@@ -518,6 +632,26 @@ images: {
 
 Missing Redis URL fails boot loudly: `oke boot: redis driver needs REDIS_URL`.
 
+### Durable KV
+
+Default `store.kv("sessions")` is cache-shaped — a Redis recreate drops keys.
+
+`{ durable: true }` persists that namespace in your SQL database (`oke_kv` JSONB on
+`DATABASE_URL`). Not a per-key Redis flag, not Flow `durable`, not tier-1 auto-cache.
+
+```typescript
+export const sessions = store.kv("sessions", { description: "Session cache" });
+export const ledger = store.kv("ledger", { durable: true, description: "Idempotency keys" });
+```
+
+Gate rates and Signal stay on `REDIS_URL`. Missing `DATABASE_URL` with the postgres driver
+fails boot: `oke boot: durable store.kv needs DATABASE_URL`.
+
+<Callout title="Not per-key, not Flow durable, not auto-cache">
+  Durable KV is a JSONB table on the same Postgres as `store.sql`. Flow `durable` journals steps.
+  Tier-1 auto-cache is an in-process Map and never reads `store.kv`.
+</Callout>
+
 Driver id stays `redis` for every image below — same `REDIS_URL`, zero Flow changes.
 Redis is the default because it is the most mature and battle-tested; Valkey and
 Dragonfly are equally legitimate opt-in pins.
@@ -544,7 +678,8 @@ Dragonfly are equally legitimate opt-in pins.
 <StoreFacetMark facet="files" />
 
 The blob facet: opaque objects (`Uint8Array` or string) on the same handle as the optional image
-pipeline. Drivers swap `fs` ↔ `s3` (plus `memory` for tests).
+pipeline. Drivers swap `fs` ↔ `s3` (plus `memory` for tests). The Console Files band is a folder
+browser — upload, download, delete — not a signed-URL mint.
 
 ### Quick start
 
@@ -581,12 +716,12 @@ do: async ({ key, body }, fx) => {
 
 ### Blob CRUD — `put` / `get` / `delete` / `list`
 
-| Call             | Effect  | Returns              | Notes                                |
-| ---------------- | ------- | -------------------- | ------------------------------------ |
-| `put(key, data)` | `write` | `void`               | `data` is `Uint8Array` or `string`   |
-| `get(key)`       | `read`  | `Uint8Array \| null` | Missing key → `null`                 |
-| `delete(key)`    | `write` | `boolean`            | Whether an object was removed        |
-| `list(prefix?)`  | `read`  | `string[]`           | Prefix filter; Console browse source |
+| Call             | Effect  | Returns              | Notes                                        |
+| ---------------- | ------- | -------------------- | -------------------------------------------- |
+| `put(key, data)` | `write` | `void`               | `data` is `Uint8Array` or `string`           |
+| `get(key)`       | `read`  | `Uint8Array \| null` | Missing key → `null`                         |
+| `delete(key)`    | `write` | `boolean`            | Whether an object was removed                |
+| `list(prefix?)`  | `read`  | `string[]`           | Prefix filter; Console folder browser source |
 
 ```typescript
 const files = fx.store(uploads);
@@ -600,11 +735,12 @@ await files.delete("docs/readme.txt");
 
 Keys are opaque strings the driver stores as-is. Prefer path-like prefixes (`photos/`, `exports/`).
 
-| Helper / rule               | Meaning                                                                  |
-| --------------------------- | ------------------------------------------------------------------------ |
-| `contentAddressedKey(data)` | From `okengine/store` — stable sha256 hex of the bytes (dedupe-friendly) |
-| Leading `/` or `..`         | Rejected by the `fs` driver (`Invalid object key`)                       |
-| Non-ASCII keys              | Console warns `non_ascii_key` — may break S3 signed URLs                 |
+| Helper / rule                      | Meaning                                                                                                                                                  |
+| ---------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `contentAddressedKey(data)`        | From `okengine/store` — stable sha256 hex of the bytes (dedupe-friendly)                                                                                 |
+| `safeFileObjectKey(name, prefix?)` | URL-safe ASCII key (`file-a1b2c3d4.pdf`). Keep the original name in the catalog.                                                                         |
+| Leading `/` or `..`                | Rejected by the `fs` driver (`Invalid object key`)                                                                                                       |
+| Non-ASCII keys                     | Console warns `non_ascii_key`. Console puts rewrite the key and store `originalName` in `.oke/catalog.json` (and `file_objects` when that table exists). |
 
 ### Per-environment Files drivers
 
@@ -615,7 +751,7 @@ drivers: {
   },
 },
 images: {
-  store: { files: "rustfs/rustfs:1.0.0-beta.11" },
+  store: { files: "rustfs/rustfs:1.0.0-rc.2" },
 },
 ```
 
@@ -634,9 +770,9 @@ images: {
 
 **Known limit:** `fs` is single-host. Under horizontal scale each replica sees its own filesystem (silently inconsistent object views). Boot prints a one-shot warning when `drivers.store.files` is `fs`. Use `s3` for `dev`/`prod` (create-oke templates already do).
 
-<Callout title="Operator tools can browse, not edit bytes">
-  Operator tools can list keys and delete them. Direct edit is **KV + SQL only** — blob bodies are
-  not patched.
+<Callout title="Console Files is a folder browser">
+  Inspect the **bucket**. Click a file to preview in a sheet. Uploads use a URL-safe key; the
+  original name lives in the object catalog. Bytes live on `memory`, `fs`, or `s3`.
 </Callout>
 
 ### Images — `image` / `putImage`
@@ -734,11 +870,17 @@ One declaration shape — two search physics. The handle is a **discriminated un
 
 <StoreIndexModes />
 
-| Call (both)  | Effect  | Returns       |
-| ------------ | ------- | ------------- |
-| `upsert(…)`  | `write` | `void`        |
-| `search(…)`  | `read`  | hits / result |
-| `delete(id)` | `write` | `boolean`     |
+| Call (both)    | Effect  | Returns               |
+| -------------- | ------- | --------------------- |
+| `upsert(…)`    | `write` | `void`                |
+| `search(…)`    | `read`  | hits / result         |
+| `list(limit?)` | `read`  | documents (id + meta) |
+| `delete(id)`   | `write` | `boolean`             |
+
+<Callout title="Console Index is a search box">
+  Type a phrase to find documents by id and stored meta. A comma-separated vector still runs an ANN
+  probe. Meilisearch uses full-text `q`.
+</Callout>
 
 TypeScript rejects a vector query against a text index (and vice versa) at compile time — never at
 runtime.
@@ -789,7 +931,7 @@ drivers: {
   },
 },
 images: {
-  store: { index: "getmeili/meilisearch:v1.37" },
+  store: { index: "getmeili/meilisearch:v1.53" },
 },
 ```
 
@@ -862,6 +1004,12 @@ same PII masking — see the Callout under [Querying by hand](#querying-by-hand)
 ## Troubleshooting
 
 <Accordions>
+<Accordion title="Unable to resolve table name from value">
+
+A `store.schema.table` column named `name` used to shadow the SQL table name.
+Inserts and selects now resolve it from the declaration. Keep the `name` column.
+
+</Accordion>
 <Accordion title="OKE1101 — missing table in prod">
 
 Schema DDL never runs automatically in `prod`. Under `oke dev`, `db.autoPush` (default `true`) runs `oke db push`. For production use `oke db generate` + `oke db migrate`.
@@ -881,6 +1029,11 @@ You are on offset paging. Switch `list` to keyset by setting `cursor` columns wi
 
 `count: "exact"` runs `COUNT(*)` per page. Set `count: "none"` in the `list` options to skip it, or use keyset mode where totals are rarely needed.
 
+</Accordion>
+<Accordion title="List data is { items, count } instead of an array">
+
+Success already wraps `{ data, error: null }`. Returning a page object from `do` nests the pager inside `data`. Prefer `out: z.array(Item)` and `return fx.json.withQuery(rows, input)`. Any other declared `out` is still valid.
+
 </Accordion>
 <Accordion title="KV TTL never expires in tests">
 
@@ -901,6 +1054,16 @@ The vector length must match `store.index(…, { dims })` (runtime default `3` i
 
 Unlike KV/Files, unset `drivers.store.index` does **not** promote to `pgvector` in `dev` — it stays `memory`. Set the map explicitly (and pair SQL for `pgvector`).
 
+</Accordion>
+<Accordion title="Console warns Non-ASCII object key">
+
+`fx.store(files).put` stores the key as-is. Console uploads mint an ASCII key and keep the original name in `.oke/catalog.json`. Re-upload to rewrite an existing object.
+
+</Accordion>
+<Accordion title="oke dev skipped the seed prompt">
+
+`.oke/state.json` stores `seededAt` plus this app's `defineSeed({ name })` — notes vs keel are different ids. A new name re-asks. Clear those keys to prompt again.
+
 </Accordion>
 </Accordions>