okengine 0.6.0 → 0.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (162) hide show
  1. package/AGENTS.md +2 -2
  2. package/README.md +98 -71
  3. package/package.json +4 -4
  4. package/site/content/docs/console/clock.mdx +25 -5
  5. package/site/content/docs/console/signals.mdx +13 -4
  6. package/site/content/docs/elements/ai.mdx +29 -2
  7. package/site/content/docs/elements/channel.mdx +90 -20
  8. package/site/content/docs/elements/clock.mdx +56 -24
  9. package/site/content/docs/elements/flow.mdx +2 -0
  10. package/site/content/docs/elements/gate.mdx +8 -1
  11. package/site/content/docs/elements/signal.mdx +173 -29
  12. package/site/content/docs/elements/store.mdx +464 -73
  13. package/site/content/docs/elements/vault.mdx +15 -5
  14. package/site/content/docs/get-started/basic-usage.mdx +13 -6
  15. package/site/content/docs/get-started/installation.mdx +49 -13
  16. package/site/content/docs/get-started/introduction.mdx +1 -1
  17. package/site/content/docs/reference/cli.md +30 -2
  18. package/site/content/docs/reference/configuration.mdx +18 -17
  19. package/site/content/docs/reference/environment-variables.mdx +44 -18
  20. package/site/content/docs/reference/errors.mdx +12 -11
  21. package/site/content/docs/reference/fx.mdx +15 -13
  22. package/site/content/docs/reference/security.md +3 -1
  23. package/src/cli/ai-setup/ai-setup.test.ts +144 -0
  24. package/src/cli/ai-setup/apply.ts +214 -0
  25. package/src/cli/ai-setup/catalog.ts +263 -0
  26. package/src/cli/ai-setup/detect-ollama.ts +166 -0
  27. package/src/cli/ai-setup/index.ts +228 -0
  28. package/src/cli/ai-setup/prompts.ts +649 -0
  29. package/src/cli/ai-setup/recommend.test.ts +100 -0
  30. package/src/cli/ai-setup/recommend.ts +203 -0
  31. package/src/cli/ai.ts +38 -0
  32. package/src/cli/db-auto-push.test.ts +6 -1
  33. package/src/cli/db-auto-push.ts +8 -3
  34. package/src/cli/dev-db-push.test.ts +59 -0
  35. package/src/cli/dev.test.ts +40 -0
  36. package/src/cli/dev.ts +55 -1
  37. package/src/cli/docker-clean.test.ts +179 -0
  38. package/src/cli/docker-clean.ts +406 -0
  39. package/src/cli/docker.ts +16 -4
  40. package/src/cli/ensure-drizzle-config.ts +1 -1
  41. package/src/cli/hero-meta.test.ts +6 -5
  42. package/src/cli/hero-meta.ts +7 -1
  43. package/src/cli/index.ts +5 -0
  44. package/src/cli/load-config.ts +4 -4
  45. package/src/cli/openbao-bootstrap.test.ts +1 -0
  46. package/src/cli/openbao-bootstrap.ts +9 -2
  47. package/src/cli/openbao-restart.integration.test.ts +106 -97
  48. package/src/cli/registry.ts +79 -1
  49. package/src/client/live-gap.test.ts +35 -0
  50. package/src/client/transport.test.ts +21 -0
  51. package/src/client/transport.ts +27 -5
  52. package/src/compiler/fixtures/skyport/oke.config.ts +1 -1
  53. package/src/config/define-config.test.ts +6 -6
  54. package/src/config/index.ts +1 -1
  55. package/src/console/server/console.test.ts +45 -0
  56. package/src/console/server/flows.ts +39 -9
  57. package/src/console/server/i18n.ts +41 -0
  58. package/src/console/server/vault.ts +12 -38
  59. package/src/console/ui/dist/assets/index-ClX0ZCe5.js +10 -0
  60. package/src/console/ui/dist/assets/{panel-access-BGv45snf.js → panel-access-ri1X9Otj.js} +1 -1
  61. package/src/console/ui/dist/assets/{panel-ai-B2S7LEii.js → panel-ai-DLHjRoJk.js} +1 -1
  62. package/src/console/ui/dist/assets/{panel-architecture-D7UJh91v.js → panel-architecture-DI3bQF3J.js} +1 -1
  63. package/src/console/ui/dist/assets/{panel-channels-9T3ybqRu.js → panel-channels-BDYWX29p.js} +1 -1
  64. package/src/console/ui/dist/assets/{panel-clock-Cb1UXGRQ.js → panel-clock-i-JpTjVB.js} +1 -1
  65. package/src/console/ui/dist/assets/{panel-diff-DmYbKWmN.js → panel-diff-CFuWxiXy.js} +1 -1
  66. package/src/console/ui/dist/assets/{panel-flows-PiHwT55z.js → panel-flows-B8TRhcoe.js} +1 -1
  67. package/src/console/ui/dist/assets/{panel-gates-BQGYXvjT.js → panel-gates-CfkBgco7.js} +1 -1
  68. package/src/console/ui/dist/assets/{panel-overview-BBnRO18l.js → panel-overview-DONDzBd2.js} +1 -1
  69. package/src/console/ui/dist/assets/{panel-plugins-D0PsmVw2.js → panel-plugins-DDJwTehL.js} +1 -1
  70. package/src/console/ui/dist/assets/{panel-runs-CWuRDe0r.js → panel-runs-D5zf-D9c.js} +1 -1
  71. package/src/console/ui/dist/assets/{panel-signals-Bbg4ewpP.js → panel-signals-Dsxdu_AR.js} +1 -1
  72. package/src/console/ui/dist/assets/{panel-store-CPCbsDRa.js → panel-store-hy7O8HOs.js} +1 -1
  73. package/src/console/ui/dist/assets/{panel-traces-DVAzuA_S.js → panel-traces-CBiAe5go.js} +1 -1
  74. package/src/console/ui/dist/assets/{panel-vault-D1_MvOmo.js → panel-vault-Za1GyJfM.js} +1 -1
  75. package/src/console/ui/dist/index.html +1 -1
  76. package/src/console/ui/shell/components/ui.tsx +5 -1
  77. package/src/console/ui/shell/setup/Wizard.tsx +26 -5
  78. package/src/docker/cleanup.test.ts +193 -0
  79. package/src/docker/cleanup.ts +355 -0
  80. package/src/docker/docker.test.ts +4 -0
  81. package/src/docker/dockerfile.integration.test.ts +126 -119
  82. package/src/docker/index.ts +19 -0
  83. package/src/docker/recipes/ollama.ts +10 -5
  84. package/src/docker/stack.integration.test.ts +118 -102
  85. package/src/drivers/ai-ollama-tools.integration.test.ts +8 -6
  86. package/src/drivers/ai-ollama.integration.test.ts +3 -19
  87. package/src/drivers/ai-ollama.ts +45 -2
  88. package/src/drivers/ai-providers.test.ts +31 -0
  89. package/src/drivers/channel-fcm.ts +49 -53
  90. package/src/drivers/channel-msegat.ts +61 -0
  91. package/src/drivers/channel-sently-map.ts +57 -0
  92. package/src/drivers/channel-sently.test.ts +99 -0
  93. package/src/drivers/channel-sndr.ts +28 -0
  94. package/src/drivers/channel-taqnyat.ts +57 -0
  95. package/src/drivers/channel-types.ts +79 -2
  96. package/src/drivers/channel-unifonic.ts +26 -43
  97. package/src/drivers/channel-wa-cloud.ts +33 -47
  98. package/src/drivers/channel-webpush.ts +39 -239
  99. package/src/drivers/index.ts +7 -0
  100. package/src/drivers/signal-engine.ts +98 -25
  101. package/src/drivers/signal-nats.ts +4 -4
  102. package/src/drivers/signal-postgres.ts +158 -64
  103. package/src/drivers/signal-redis.ts +4 -4
  104. package/src/drivers/signal-types.ts +61 -2
  105. package/src/drivers/vault-driver-removal.test.ts +5 -1
  106. package/src/elements/channel/costs.test.ts +2 -2
  107. package/src/elements/channel/costs.ts +14 -2
  108. package/src/elements/channel/mime.ts +11 -0
  109. package/src/elements/channel/runtime.ts +94 -0
  110. package/src/elements/channel/sndr-webhooks.test.ts +26 -0
  111. package/src/elements/channel.ts +10 -1
  112. package/src/elements/clock/chaos-child.ts +150 -0
  113. package/src/elements/clock/chaos.test.ts +204 -0
  114. package/src/elements/clock/reconcile.ts +110 -0
  115. package/src/elements/clock/runtime.ts +11 -0
  116. package/src/elements/clock.test.ts +164 -2
  117. package/src/elements/clock.ts +1 -0
  118. package/src/elements/gate/runtime.ts +6 -0
  119. package/src/elements/index.ts +9 -0
  120. package/src/elements/signal/chaos-child.ts +41 -4
  121. package/src/elements/signal/declare.ts +4 -1
  122. package/src/elements/signal/delivery-modes.test.ts +179 -0
  123. package/src/elements/signal/key-ordering.test.ts +306 -0
  124. package/src/elements/signal/lease-reclaim.test.ts +123 -0
  125. package/src/elements/signal/optional-emit.test.ts +76 -0
  126. package/src/elements/signal/order-lifecycle.test.ts +130 -0
  127. package/src/elements/signal/orphan-messages.test.ts +152 -0
  128. package/src/elements/signal/runtime.ts +11 -4
  129. package/src/elements/signal/schema-emit.test.ts +107 -0
  130. package/src/elements/signal.test.ts +89 -5
  131. package/src/elements/store/files-fx.ts +104 -0
  132. package/src/elements/store/files-image.test.ts +251 -0
  133. package/src/elements/store/files-image.ts +494 -0
  134. package/src/elements/store/runtime.ts +60 -0
  135. package/src/elements/store.ts +24 -0
  136. package/src/elements/vault/boot-chain.ts +150 -0
  137. package/src/elements/vault/runtime.ts +8 -0
  138. package/src/kernel/app.ts +3 -1
  139. package/src/kernel/boot-bind/channel.test.ts +68 -3
  140. package/src/kernel/boot-bind/channel.ts +93 -2
  141. package/src/kernel/boot-bind/clock.ts +52 -7
  142. package/src/kernel/boot-bind/gate.ts +71 -3
  143. package/src/kernel/boot-bind/honor-config.test.ts +247 -0
  144. package/src/kernel/boot-bind/signal.ts +64 -7
  145. package/src/kernel/boot-bind/vault.ts +27 -8
  146. package/src/kernel/boot.ts +10 -2
  147. package/src/kernel/errors.ts +8 -2
  148. package/src/kernel/fx.test.ts +13 -0
  149. package/src/kernel/fx.ts +27 -4
  150. package/src/plugins/auth-delivery.mailpit.integration.test.ts +10 -4
  151. package/src/release/exports.test.ts +26 -0
  152. package/src/release/exports.ts +64 -5
  153. package/src/release/index.ts +5 -0
  154. package/src/release/measure.exports.test.ts +13 -1
  155. package/src/release/measure.ts +76 -13
  156. package/src/release/official-plugins.ts +46 -0
  157. package/src/release/readme.test.ts +30 -2
  158. package/src/runtime/dev-request-log.test.ts +20 -1
  159. package/src/runtime/dev-request-log.ts +38 -0
  160. package/src/term.test.ts +15 -0
  161. package/src/term.ts +8 -3
  162. package/src/console/ui/dist/assets/index-CjxwRGVv.js +0 -10
@@ -1,18 +1,31 @@
1
1
  ---
2
2
  title: "Store"
3
- description: "Data at rest SQL tables, KV cache, file blobs, and vector search, declared once and swapped per environment by driver."
3
+ description: "Data at rest behind one handle and four independent facets sql · kv · files · index — swapped per environment by driver."
4
4
  icon: "Database"
5
5
  source: "docs/spec/unified-theory.md"
6
6
  ---
7
7
 
8
- Store is where your app's **data at rest** lives. It covers four kinds of storage relational tables, key-value cache, file blobs, and vector search behind one declaration style. Your flow code never changes between SQLite on your laptop and Postgres in production; only the driver does.
8
+ Store is one handle, four independent facets**sql · kv · files · index**. Each has its own
9
+ shape; nothing forces you to use more than one. Flow code stays the same across environments;
10
+ only the driver changes.
9
11
 
10
12
  <Callout title="The one rule">
11
13
  Drivers are named after **protocols**, not vendors (`postgres`, `redis`, `s3` — never `neon` or
12
14
  `minio`). Vendor choice lives in the `images` map of `oke.config.ts`.
13
15
  </Callout>
14
16
 
15
- ## Quick start
17
+ <StoreFacets />
18
+
19
+ Every facet accepts optional `description` — a human title in the Console (falls back to the store
20
+ name).
21
+
22
+ ## SQL
23
+
24
+ <StoreFacetMark facet="sql" />
25
+
26
+ The relational facet: tables, schema, and typed single-table sessions through `fx.store`.
27
+
28
+ ### Quick start
16
29
 
17
30
  <Steps>
18
31
 
@@ -86,25 +99,7 @@ export const createNote = on(
86
99
 
87
100
  </Steps>
88
101
 
89
- ## The four facets
90
-
91
- Pick the facet that matches the physics of your data. Each declaration is one line; the runtime handle shows what flows can do with it.
92
-
93
- <StoreFacets />
94
-
95
- Every facet accepts optional `description` — a human title in the Console
96
- (falls back to the store name). `store.index` also takes `dims`.
97
-
98
- ```typescript
99
- export const cache = store.kv("sessions", { description: "Session cache" });
100
- export const uploads = store.files("attachments");
101
- export const embeddings = store.index("docs", {
102
- dims: 1536,
103
- description: "Document embeddings",
104
- });
105
- ```
106
-
107
- ## CRUD without boilerplate — `store.resource`
102
+ ### CRUD without boilerplate — `store.resource`
108
103
 
109
104
  Five conventional endpoints (list, create, get, update, remove) expand from one declaration. Each is an ordinary Flow underneath — same contracts, same `fx`:
110
105
 
@@ -139,7 +134,31 @@ The list endpoint's URL is the whole query language:
139
134
 
140
135
  Responses follow the Stripe-style envelope: `{ data, meta: { nextCursor, hasNextPage }, error }`. `create` answers **201**, `remove` answers **204**, and a missing row is a typed `NotFound` — never a crash.
141
136
 
142
- ## Querying by hand
137
+ #### An admin table
138
+
139
+ ```typescript
140
+ list: { mode: "offset", count: "exact", limit: 20 },
141
+ ```
142
+
143
+ **Consequence:** `count: "exact"` (the offset default) runs `COUNT(*)` to fill `meta.total`. On a huge table that count is the real cost — set `count: "none"` to return only `meta.offset`.
144
+
145
+ #### An infinite feed
146
+
147
+ ```typescript
148
+ list: { cursor: [notes.createdAt, notes.id], direction: "desc", limit: 20, maxLimit: 100 },
149
+ ```
150
+
151
+ **Consequence:** keyset (cursor) paging is the default when `cursor` columns are set — pages stay stable while new rows are inserted, where offset pages would shift and show duplicates.
152
+
153
+ #### A public, restricted endpoint
154
+
155
+ ```typescript
156
+ list: { mode: "offset", filter: "none", limit: 20 },
157
+ ```
158
+
159
+ **Consequence:** a request that filters on a forbidden column — `?secret=eq.x` — fails with **422** and the exact message `unknown list param "secret"`. Filterable columns are a whitelist, never an accident.
160
+
161
+ ### Querying by hand
143
162
 
144
163
  When `store.resource` is too conventional, `fx.store(db)` is the full single-table session:
145
164
 
@@ -165,7 +184,7 @@ await fx.store(db).delete(notes).where(lt(notes.createdAt, cutoff));
165
184
  table shows up explicitly in the Manifest's `reads` / `writes` — powering caching and PII masking.
166
185
  </Callout>
167
186
 
168
- ## Schema — declare once, generate per dialect
187
+ ### Schema — declare once, generate per dialect
169
188
 
170
189
  The recommended path: declare tables ORM-agnostically, then let `oke db` emit real Drizzle for the active dialect (`sqliteTable` locally, `pgTable` in docker/prod) into `src/schema.generated.ts`.
171
190
 
@@ -179,7 +198,7 @@ The recommended path: declare tables ORM-agnostically, then let `oke db` emit re
179
198
  | `.describe("…")` | human title in the Console (falls back to key) |
180
199
  | `.references(() => col, { onDelete })` | foreign key |
181
200
 
182
- ### Foreign keys and relations
201
+ #### Foreign keys and relations
183
202
 
184
203
  Declare FKs on fields, and relation metadata once per schema:
185
204
 
@@ -205,7 +224,7 @@ export const relations = store.schema.relations({ links, daily }, (r) => ({
205
224
  There is no separate API and no `.through()` — the junction is an ordinary table.
206
225
  </Callout>
207
226
 
208
- ### Syncing the schema
227
+ #### Syncing the schema
209
228
 
210
229
  | Command | When |
211
230
  | ----------------- | -------------------------------------------------------------- |
@@ -213,7 +232,12 @@ export const relations = store.schema.relations({ links, daily }, (r) => ({
213
232
  | `oke db generate` | Write versioned SQL under `drizzle/` for review |
214
233
  | `oke db migrate` | Apply those files — human or CI, **never** at boot |
215
234
 
216
- `oke dev` (local) auto-pushes when the schema file changes — opt out with `--no-db-push` or `db: { autoPush: false }`. Docker/prod **never** auto-apply DDL; a missing table fails loudly as **OKE1101**, telling you to run `oke db migrate`.
235
+ `oke dev` (local) auto-pushes when you save a schema _input_ (`schema.decl.ts`,
236
+ hand-written `schema.ts`, `app.ts` for plugin tables, or `drizzle.config.ts`) —
237
+ not when emit rewrites `schema.generated.ts`.
238
+
239
+ Opt out with `--no-db-push` or `db: { autoPush: false }`. Docker/prod **never**
240
+ auto-apply DDL; a missing table fails as **OKE1101** (`oke db migrate`).
217
241
 
218
242
  <Callout title="Escape hatch">
219
243
  Hand-written Drizzle in `src/schema.ts` stays supported — if there is nothing to emit, the emit
@@ -221,30 +245,17 @@ export const relations = store.schema.relations({ links, daily }, (r) => ({
221
245
  extending an app-owned table with plugin columns is not supported in v1.
222
246
  </Callout>
223
247
 
224
- ## Per-environment drivers
248
+ ### Per-environment SQL drivers
225
249
 
226
250
  Same flow code, different backends — configured once in `oke.config.ts`:
227
251
 
228
- ```typescript title="oke.config.ts"
229
- drivers: {
230
- store: {
231
- sql: { local: "sqlite", docker: "postgres", test: "memory", prod: "postgres" },
232
- kv: { local: "memory", docker: "redis", test: "memory", prod: "redis" },
233
- files: { local: "fs", docker: "s3", test: "memory", prod: "s3" },
234
- },
235
- },
236
- ```
237
-
238
- | Facet | Local | Docker / prod | Runs as |
239
- | ------- | -------- | ------------- | --------------------------------------- |
240
- | `sql` | `sqlite` | `postgres` | file on disk → container + named volume |
241
- | `kv` | `memory` | `redis` | in-process → container |
242
- | `files` | `fs` | `s3` | project folder → RustFS container |
243
- | `index` | `memory` | `pgvector` | in-process → pgvector image |
252
+ | Facet | Local | Docker / prod | Runs as |
253
+ | ----- | -------- | ------------- | --------------------------------------- |
254
+ | `sql` | `sqlite` | `postgres` | file on disk → container + named volume |
244
255
 
245
- Container images come from the `images` map — change the vendor by changing the pin, never the driver id.
256
+ Defaults: `sqlite` locally, `postgres` in docker/prod, `memory` in test. Container images come from the `images` map — change the vendor by changing the pin, never the driver id.
246
257
 
247
- ### Opt-in SQL drivers
258
+ #### Opt-in SQL drivers
248
259
 
249
260
  Two more `store.sql` drivers cover specific parity needs. Both are optional peers — `bun add` them yourself; neither changes the `sqlite` local default.
250
261
 
@@ -258,33 +269,357 @@ Two more `store.sql` drivers cover specific parity needs. Both are optional peer
258
269
  Choose it for dialect/pgvector parity with prod — never as a faster local default.
259
270
  </Callout>
260
271
 
261
- A SQL-backed index shares the sql facet's already-open connection, so configure the pair together:
272
+ SQL-backed index drivers (`pgvector`, `libsql`) share this facet's connection see [Advanced composing facets together](#advanced--composing-facets-together).
273
+
274
+ ## KV
275
+
276
+ <StoreFacetMark facet="kv" />
277
+
278
+ The key-value facet: short-lived cache and session data behind `fx.store`. One namespace per
279
+ declaration; drivers swap `memory` ↔ `redis` per environment.
280
+
281
+ ### Quick start
282
+
283
+ <Steps>
284
+
285
+ <Step>
286
+ ### Declare a namespace
287
+
288
+ ```typescript
289
+ import { store } from "okengine";
290
+
291
+ export const sessions = store.kv("sessions", { description: "Session cache" });
292
+ // ref → kv:sessions
293
+ ```
294
+
295
+ </Step>
296
+
297
+ <Step>
298
+ ### Read and write in a Flow
299
+
300
+ ```typescript
301
+ do: async ({ userId, data }, fx) => {
302
+ const kv = fx.store(sessions);
303
+ await kv.set(`user:${userId}`, data, "30m");
304
+ return await kv.get(`user:${userId}`);
305
+ },
306
+ ```
307
+
308
+ </Step>
309
+
310
+ </Steps>
311
+
312
+ ### Handle API
313
+
314
+ | Call | Effect | Returns | Notes |
315
+ | ----------------------- | ------- | ---------- | ----------------------------------------- |
316
+ | `get(key)` | `read` | `unknown` | Missing keys resolve per driver |
317
+ | `set(key, value, ttl?)` | `write` | `void` | Values should be JSON-serializable |
318
+ | `delete(key)` | `write` | `boolean` | Whether a key was removed |
319
+ | `list(prefix?)` | `read` | `string[]` | Prefix filter; powers the Console browser |
320
+
321
+ ### TTL
322
+
323
+ Optional third argument on `set` is a duration string: `(\d+)(ms|s|m|h|d)` — e.g. `"30m"`, `"1h"`,
324
+ `"5000ms"`. Same call — opposite physics under each driver:
325
+
326
+ <StoreKvTtl />
327
+
328
+ **Consequence:** a TTL you rely on in prod does nothing under the local `memory` driver. Test expiry
329
+ against `redis` (docker) when lifetime matters.
330
+
331
+ ### Examples — what follows from each choice
332
+
333
+ #### A session with expiry
334
+
335
+ ```typescript
336
+ await fx.store(sessions).set(`sess:${token}`, { userId }, "7d");
337
+ ```
338
+
339
+ **Consequence:** under `redis`, the key vanishes after seven days without a sweeper Flow. Under
340
+ `memory`, it lives until process exit or an explicit `delete`.
341
+
342
+ #### Browse by prefix
343
+
344
+ ```typescript
345
+ const keys = await fx.store(sessions).list("sess:");
346
+ ```
347
+
348
+ **Consequence:** `list` is how the [Console · Store](/docs/console/store) browser enumerates a
349
+ namespace. Prefer stable key prefixes (`user:`, `sess:`) so both your Flows and the Console stay
350
+ readable.
351
+
352
+ ### Per-environment KV drivers
262
353
 
263
354
  ```typescript title="oke.config.ts"
264
355
  drivers: {
265
356
  store: {
266
- sql: { local: "libsql" },
267
- index: { local: "libsql" },
357
+ kv: { local: "memory", docker: "redis", test: "memory", prod: "redis" },
358
+ },
359
+ },
360
+ images: {
361
+ "store.kv": "redis:8-alpine",
362
+ },
363
+ ```
364
+
365
+ | Facet | Local | Docker / prod | Runs as |
366
+ | ----- | -------- | ------------- | ---------------------- |
367
+ | `kv` | `memory` | `redis` | in-process → container |
368
+
369
+ | Variable | When |
370
+ | ------------------ | -------------------------------- |
371
+ | `REDIS_URL` | Required for the `redis` driver |
372
+ | `OKE_STORE_KV_URL` | Override if `REDIS_URL` is unset |
373
+ | `OKE_KV_DRIVER` | Force a driver id (docker) |
374
+
375
+ Missing Redis URL fails boot loudly: `oke boot: redis driver needs REDIS_URL`.
376
+
377
+ <Callout title="Dry-run refuses KV writes">
378
+ `set` and `delete` throw `DryRunWriteIsolationError` during dry-run — the runtime will not risk a
379
+ double-write against a shared Redis. Reads (`get` / `list`) still run.
380
+ </Callout>
381
+
382
+ ## Files
383
+
384
+ <StoreFacetMark facet="files" />
385
+
386
+ The blob facet: opaque objects (`Uint8Array` or string) on the same handle as the optional image
387
+ pipeline. Drivers swap `fs` ↔ `s3` (plus `memory` for tests).
388
+
389
+ ### Quick start
390
+
391
+ <Steps>
392
+
393
+ <Step>
394
+ ### Declare a bucket
395
+
396
+ ```typescript
397
+ import { store } from "okengine";
398
+
399
+ export const uploads = store.files("attachments", {
400
+ description: "User uploads",
401
+ });
402
+ // ref → files:attachments
403
+ ```
404
+
405
+ </Step>
406
+
407
+ <Step>
408
+ ### Put and get in a Flow
409
+
410
+ ```typescript
411
+ do: async ({ key, body }, fx) => {
412
+ const files = fx.store(uploads);
413
+ await files.put(key, body);
414
+ return await files.get(key); // Uint8Array | null
415
+ },
416
+ ```
417
+
418
+ </Step>
419
+
420
+ </Steps>
421
+
422
+ ### Blob CRUD — `put` / `get` / `delete` / `list`
423
+
424
+ | Call | Effect | Returns | Notes |
425
+ | ---------------- | ------- | -------------------- | ------------------------------------ |
426
+ | `put(key, data)` | `write` | `void` | `data` is `Uint8Array` or `string` |
427
+ | `get(key)` | `read` | `Uint8Array \| null` | Missing key → `null` |
428
+ | `delete(key)` | `write` | `boolean` | Whether an object was removed |
429
+ | `list(prefix?)` | `read` | `string[]` | Prefix filter; Console browse source |
430
+
431
+ ```typescript
432
+ const files = fx.store(uploads);
433
+ await files.put("docs/readme.txt", "hello");
434
+ const bytes = await files.get("docs/readme.txt");
435
+ const keys = await files.list("docs/");
436
+ await files.delete("docs/readme.txt");
437
+ ```
438
+
439
+ ### Keys
440
+
441
+ Keys are opaque strings the driver stores as-is. Prefer path-like prefixes (`photos/`, `exports/`).
442
+
443
+ | Helper / rule | Meaning |
444
+ | --------------------------- | ------------------------------------------------------------------------ |
445
+ | `contentAddressedKey(data)` | From `okengine/store` — stable sha256 hex of the bytes (dedupe-friendly) |
446
+ | Leading `/` or `..` | Rejected by the `fs` driver (`Invalid object key`) |
447
+ | Non-ASCII keys | Console warns `non_ascii_key` — may break S3 signed URLs |
448
+
449
+ ### Per-environment Files drivers
450
+
451
+ ```typescript title="oke.config.ts"
452
+ drivers: {
453
+ store: {
454
+ files: { local: "fs", docker: "s3", test: "memory", prod: "s3" },
268
455
  },
269
456
  },
457
+ images: {
458
+ "store.files": "rustfs/rustfs:1.0.0-beta.11",
459
+ },
460
+ ```
461
+
462
+ | Facet | Local | Docker / prod | Runs as |
463
+ | ------- | ----- | ------------- | --------------------------------- |
464
+ | `files` | `fs` | `s3` | project folder → RustFS container |
465
+
466
+ | Variable | When |
467
+ | ----------------------------- | -------------------------------------- |
468
+ | `S3_BUCKET` | Required bucket for the `s3` driver |
469
+ | `OKE_STORE_FILES_DB` | Override bucket name |
470
+ | `S3_ENDPOINT` · keys · region | S3-compatible endpoint and credentials |
471
+ | `OKE_FILES_DRIVER` | Force a driver id (docker) |
472
+
473
+ `memory` is the test default. Local `fs` writes under a temp root when no binding `root` is set.
474
+
475
+ <Callout title="Console can browse, not edit bytes">
476
+ The [Console · Store](/docs/console/store) lists keys and can delete them. Direct edit is **KV +
477
+ SQL only** — blob bodies are not patched from the Console.
478
+ </Callout>
479
+
480
+ ### Images — `image` / `putImage`
481
+
482
+ For photos, use Bun's built-in `Bun.Image` pipeline on the same handle (requires Bun `>=1.3.14`).
483
+ `putImage` is one write that fans into several keys:
484
+
485
+ <StoreFilesVariants />
486
+
487
+ ```typescript
488
+ const meta = await fx.store(uploads).image(key).metadata();
489
+ // → { width, height, format }
490
+
491
+ await fx
492
+ .store(uploads)
493
+ .image(key)
494
+ .resize(400, 400, { fit: "inside" })
495
+ .webp({ quality: 80 })
496
+ .put(`thumbs/${key}.webp`);
497
+
498
+ const result = await fx.store(uploads).putImage("photos/x.jpg", bytes, {
499
+ variants: {
500
+ thumb: { resize: [128, 128, { fit: "inside" }], webp: { quality: 80 } },
501
+ medium: { resize: [800], webp: { quality: 85 } },
502
+ },
503
+ placeholder: true, // ThumbHash LQIP data URL
504
+ });
505
+ // variants: photos/x.thumb.webp, photos/x.medium.webp
270
506
  ```
271
507
 
508
+ | Call | Effect | Notes |
509
+ | ----------------------------------------------------------------------- | ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
510
+ | `image(key\|bytes).metadata()` / `bytes()` / `blob()` / `placeholder()` | `read` when source is a key | Chain `resize` · `rotate` · `flip` · `flop` · `modulate`, then `jpeg` / `png` / `webp` (portable); `heic` / `avif` fall back to WebP when the OS codec is missing |
511
+ | `image(…).put(outKey)` | `read` + `write` (key source) | Persist the encoded result |
512
+ | `putImage(key, data, opts?)` | `write` | Original + named variants; keys are `{stem}.{variant}.{ext}` |
513
+
514
+ #### Image options
515
+
516
+ | Option | Default | Meaning |
517
+ | ------------- | ----------------------- | ---------------------------------------------------- |
518
+ | `maxPixels` | **16 MP** (`4096×4096`) | Decode ceiling — raise it or pass `false` to opt out |
519
+ | `autoOrient` | `true` | Apply JPEG EXIF orientation before transforms |
520
+ | `variants` | — | Named `{ resize?, jpeg\|png\|webp\|… }` specs |
521
+ | `placeholder` | — | When `true`, return a ThumbHash LQIP data URL |
522
+
523
+ `putImage` returns `{ key, meta, variants, placeholder? }` — `variants` maps name → object key.
524
+
525
+ ## Index
526
+
527
+ <StoreFacetMark facet="index" />
528
+
529
+ The search / similarity facet: vector ANN and opt-in full-text, both behind `store.index`. The
530
+ handle is a **discriminated union** on `driverId` — vector methods and text methods never mix.
531
+
532
+ ### Quick start
533
+
534
+ <Steps>
535
+
536
+ <Step>
537
+ ### Declare a vector index
538
+
539
+ ```typescript
540
+ import { store } from "okengine";
541
+
542
+ export const embeddings = store.index("docs", {
543
+ dims: 1536,
544
+ description: "Document embeddings",
545
+ });
546
+ // ref → index:docs
547
+ ```
548
+
549
+ `dims` defaults to `3` at open if omitted — set it to match your embedder.
550
+
551
+ </Step>
552
+
553
+ <Step>
554
+ ### Upsert and search in a Flow
555
+
556
+ ```typescript
557
+ do: async ({ id, vector, query }, fx) => {
558
+ const idx = fx.store(embeddings);
559
+ await idx.upsert(id, vector, { title: "Intro" });
560
+ return await idx.search(query, 5);
561
+ // → [{ id, score, meta? }, …] score = cosine similarity
562
+ },
563
+ ```
564
+
565
+ </Step>
566
+
567
+ </Steps>
568
+
569
+ ### Handle API — vector vs text
570
+
571
+ One declaration shape — two search physics. The handle is a **discriminated union** on `driverId`:
572
+
573
+ <StoreIndexModes />
574
+
575
+ | Call (both) | Effect | Returns |
576
+ | ------------ | ------- | ------------- |
577
+ | `upsert(…)` | `write` | `void` |
578
+ | `search(…)` | `read` | hits / result |
579
+ | `delete(id)` | `write` | `boolean` |
580
+
581
+ TypeScript rejects a vector query against a text index (and vice versa) at compile time — never at
582
+ runtime.
583
+
584
+ ### Vector ANN
585
+
272
586
  | `store.index` id | Shares connection from | Real ANN via |
273
587
  | ---------------- | -------------------------- | ------------------------------------ |
274
- | `memory` | — (in-process, default) | |
588
+ | `memory` | — (in-process, default) | Full scan cosine |
275
589
  | `pgvector` | `postgres` or `pglite` sql | HNSW + `cosineDistance` |
276
590
  | `libsql` | `libsql` sql | `libsql_vector_idx` + `vector_top_k` |
277
591
 
278
- **Consequence:** a configured SQL-backed index that cannot reach its engine — missing peer, missing `vector` extension, wrong sql driver — fails loudly at first use. It never silently falls back to `memory`.
592
+ **Consequence:** a configured SQL-backed index that cannot reach its engine — missing peer, missing
593
+ `vector` extension, wrong sql driver — fails loudly at first use. It never silently falls back to
594
+ `memory`. A vector whose length ≠ `dims` throws `vector dims X !== index dims Y`.
595
+
596
+ Pair SQL + index under [Advanced — composing facets together](#advanced--composing-facets-together).
279
597
 
280
598
  ### Full-text search — `meilisearch` (opt-in)
281
599
 
282
- `meilisearch` is a fourth `store.index` backend, opt-in only — `memory` stays the default. It is a genuinely different capability from the vector ANN drivers: typo-tolerant, faceted full-text relevance, not cosine similarity. The two models are kept apart by a **discriminated union** on `driverId`, so TypeScript rejects a vector query against a text index (and vice versa) at compile time — never at runtime:
600
+ `meilisearch` is a fourth `store.index` backend, opt-in only — `memory` stays the default when
601
+ `drivers.store.index` is unset. Typo-tolerant, faceted full-text relevance — not cosine similarity.
283
602
 
284
- | `driverId` | `upsert` | `search` | `score` means |
285
- | -------------------------------- | --------------------- | ---------------------------------------- | --------------------------- |
286
- | `memory` / `pgvector` / `libsql` | `(id, vector, meta?)` | `search(vector, topK?)` → `IndexHit[]` | cosine similarity |
287
- | `meilisearch` | `(id, document)` | `search(q, opts?)` `{ hits, facets? }` | relevance (`_rankingScore`) |
603
+ ```typescript
604
+ export const kb = store.index("kb", { description: "Help articles" });
605
+
606
+ do: async ({ id, doc, q }, fx) => {
607
+ const idx = fx.store(kb); // driverId: "meilisearch"
608
+ await idx.upsert(id, { id, ...doc });
609
+ return await idx.search(q, {
610
+ topK: 10,
611
+ filter: "category = guides",
612
+ facets: ["category"],
613
+ });
614
+ // → { hits, facetDistribution? }
615
+ },
616
+ ```
617
+
618
+ | Search option | Meaning |
619
+ | ------------- | ------------------------------------------------ |
620
+ | `topK` | Max hits (Meilisearch `limit`, default 10) |
621
+ | `filter` | Filter expression — attribute must be filterable |
622
+ | `facets` | Attributes to aggregate into `facetDistribution` |
288
623
 
289
624
  ```typescript title="oke.config.ts"
290
625
  drivers: {
@@ -303,35 +638,67 @@ images: {
303
638
  - **Not for vectors:** `ai.embed` / `fx.search` stay vector-only; pointing an embed `into` a meilisearch index fails loud (embeddings don't apply).
304
639
  - **Alpine caveat:** the raw glibc binary fails on Alpine/musl _hosts_; the official image is musl-clean. Install the binary for your platform (e.g. Homebrew / install script), or use `--docker`.
305
640
 
306
- ## Privacy built in
641
+ ### Per-environment Index drivers
307
642
 
308
- Columns tagged `.pii()` or `.sensitive()` are masked at the store boundaryflows, logs, and the Console see a mask, not the value. Revealing cleartext PII requires an explicit `pii:reveal` gate on the flow, so access is a permission, not a convention.
643
+ Unlike KV/Files, an unset `drivers.store.index` map stays **`memory` in every environment**there is no docker fallback to `pgvector`. Opt in explicitly:
644
+
645
+ ```typescript title="oke.config.ts"
646
+ drivers: {
647
+ store: {
648
+ sql: { local: "libsql", docker: "postgres", prod: "postgres" },
649
+ index: { local: "libsql", docker: "pgvector", prod: "pgvector" },
650
+ },
651
+ },
652
+ ```
309
653
 
310
- ## Examples — what follows from each choice
654
+ | Variable | When |
655
+ | --------------------- | ------------------------------------------------- |
656
+ | `OKE_INDEX_DRIVER` | Force a driver id |
657
+ | `OKE_STORE_INDEX_URL` | Required for `meilisearch` |
658
+ | `OKE_STORE_INDEX_KEY` | Meilisearch API key (`MEILI_MASTER_KEY` fallback) |
311
659
 
312
- ### An admin table
660
+ ### With AI embeddings
313
661
 
314
- ```typescript
315
- list: { mode: "offset", count: "exact", limit: 20 },
316
- ```
662
+ `ai.embed` writes into a vector `store.index`; Flows search it with `fx.search` (or
663
+ `fx.store(index).search`). See [AI](/docs/elements/ai) this path is vector-only and will not
664
+ target meilisearch.
317
665
 
318
- **Consequence:** `count: "exact"` (the offset default) runs `COUNT(*)` to fill `meta.total`. On a huge table that count is the real cost set `count: "none"` to return only `meta.offset`.
666
+ ## Advancedcomposing facets together
319
667
 
320
- ### An infinite feed
668
+ Facets are independent declarations. A Flow may touch several in one `do` — for example store a files key on a SQL row, or `upsert` into an index after a SQL write. There is no special sync API: `store.resource` `search:` is SQL substring match, not meilisearch.
321
669
 
322
- ```typescript
323
- list: { cursor: [notes.createdAt, notes.id], direction: "desc", limit: 20, maxLimit: 100 },
324
- ```
670
+ One real coupling: a SQL-backed index shares the sql facet's already-open connection, so configure the pair together:
325
671
 
326
- **Consequence:** keyset (cursor) paging is the default when `cursor` columns are set — pages stay stable while new rows are inserted, where offset pages would shift and show duplicates.
672
+ ```typescript title="oke.config.ts"
673
+ drivers: {
674
+ store: {
675
+ sql: { local: "libsql" },
676
+ index: { local: "libsql" },
677
+ },
678
+ },
679
+ ```
327
680
 
328
- ### A public, restricted endpoint
681
+ Same idea for `pgvector` with `postgres` or `pglite`. The full per-environment map looks like this:
329
682
 
330
- ```typescript
331
- list: { mode: "offset", filter: "none", limit: 20 },
683
+ ```typescript title="oke.config.ts"
684
+ drivers: {
685
+ store: {
686
+ sql: { local: "sqlite", docker: "postgres", test: "memory", prod: "postgres" },
687
+ kv: { local: "memory", docker: "redis", test: "memory", prod: "redis" },
688
+ files: { local: "fs", docker: "s3", test: "memory", prod: "s3" },
689
+ },
690
+ },
332
691
  ```
333
692
 
334
- **Consequence:** a request that filters on a forbidden column `?secret=eq.x` fails with **422** and the exact message `unknown list param "secret"`. Filterable columns are a whitelist, never an accident.
693
+ Container images come from the `images` mapchange the vendor by changing the pin, never the driver id.
694
+
695
+ ## Advanced — with other elements
696
+
697
+ Columns tagged `.pii()` or `.sensitive()` are masked at the store boundary — flows, logs, and the Console see a mask, not the value. Revealing cleartext PII requires an explicit `pii:reveal` gate on the flow, so access is a permission, not a convention.
698
+
699
+ Optional `description` on a store (and `.describe("…")` on fields) becomes the human title in the
700
+ [Console · Store](/docs/console/store) browser. Single-table `reads` / `writes` in the Manifest
701
+ power cache invalidation and that same PII masking — see the Callout under [Querying by hand](#querying-by-hand).
335
702
 
336
703
  ## Troubleshooting
337
704
 
@@ -355,14 +722,38 @@ You are on offset paging. Switch `list` to keyset by setting `cursor` columns wi
355
722
 
356
723
  `count: "exact"` runs `COUNT(*)` per page. Set `count: "none"` in the `list` options to skip it, or use keyset mode where totals are rarely needed.
357
724
 
725
+ </Accordion>
726
+ <Accordion title="KV TTL never expires locally">
727
+
728
+ The local default is `memory`, which ignores TTL on `set`. Run against `redis` (`oke dev --docker` or `drivers.store.kv.local: "redis"` with `REDIS_URL`) when you need real expiry.
729
+
730
+ </Accordion>
731
+ <Accordion title="oke boot: redis / meilisearch / S3 URL missing">
732
+
733
+ Driver-backed facets fail boot when their URL is absent — Redis needs `REDIS_URL`, meilisearch needs `OKE_STORE_INDEX_URL`, S3 needs `S3_BUCKET`. Docker mode expects `oke dev -d` to write these into the compose env.
734
+
735
+ </Accordion>
736
+ <Accordion title="vector dims X !== index dims Y">
737
+
738
+ The vector length must match `store.index(…, { dims })` (runtime default `3` if unset). Align `dims` with your embedder, or fix the vector you pass to `upsert` / `search`.
739
+
740
+ </Accordion>
741
+ <Accordion title="Index stays on memory in docker">
742
+
743
+ Unlike KV/Files, unset `drivers.store.index` does **not** promote to `pgvector` in docker — it stays `memory`. Set the map explicitly (and pair SQL for `pgvector` / `libsql`).
744
+
358
745
  </Accordion>
359
746
  </Accordions>
360
747
 
361
748
  ## Learn more
362
749
 
363
750
  - [Flow](/docs/elements/flow) — the `fx.store` session inside `do`
751
+ - [AI](/docs/elements/ai) — `ai.embed` into a vector `store.index`, searched via `fx.search`
752
+ - [Gate](/docs/elements/gate) — `pii:reveal` and other permissions on flows
364
753
  - [Console · Store](/docs/console/store) — browse data, cache keys, PII masking
365
754
  - [CLI Reference](/docs/reference/cli) — `oke db push` · `generate` · `migrate`
755
+ - [Configuration](/docs/reference/configuration) — `drivers.store` maps and `images` pins
756
+ - [Environment variables](/docs/reference/environment-variables) — Redis · S3 · meilisearch URLs
366
757
 
367
758
  ## Next
368
759