okengine 0.6.1 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (167) hide show
  1. package/AGENTS.md +2 -2
  2. package/README.md +73 -181
  3. package/package.json +3 -3
  4. package/site/content/docs/console/clock.mdx +25 -5
  5. package/site/content/docs/console/signals.mdx +13 -4
  6. package/site/content/docs/elements/ai.mdx +29 -2
  7. package/site/content/docs/elements/channel.mdx +42 -25
  8. package/site/content/docs/elements/clock.mdx +59 -25
  9. package/site/content/docs/elements/flow.mdx +8 -2
  10. package/site/content/docs/elements/gate.mdx +8 -1
  11. package/site/content/docs/elements/signal.mdx +173 -29
  12. package/site/content/docs/elements/store.mdx +595 -73
  13. package/site/content/docs/elements/vault.mdx +15 -5
  14. package/site/content/docs/get-started/basic-usage.mdx +13 -6
  15. package/site/content/docs/get-started/installation.mdx +52 -14
  16. package/site/content/docs/get-started/introduction.mdx +1 -1
  17. package/site/content/docs/plugins/magic-link.mdx +42 -0
  18. package/site/content/docs/plugins/phone-number.mdx +78 -17
  19. package/site/content/docs/plugins/two-factor.mdx +1 -0
  20. package/site/content/docs/reference/cli.md +32 -2
  21. package/site/content/docs/reference/configuration.mdx +15 -13
  22. package/site/content/docs/reference/environment-variables.mdx +32 -18
  23. package/site/content/docs/reference/errors.mdx +12 -11
  24. package/site/content/docs/reference/fx.mdx +15 -13
  25. package/site/content/docs/reference/security.md +3 -1
  26. package/src/cli/ai-setup/ai-setup.test.ts +144 -0
  27. package/src/cli/ai-setup/apply.ts +214 -0
  28. package/src/cli/ai-setup/catalog.ts +263 -0
  29. package/src/cli/ai-setup/detect-ollama.ts +166 -0
  30. package/src/cli/ai-setup/index.ts +228 -0
  31. package/src/cli/ai-setup/prompts.ts +649 -0
  32. package/src/cli/ai-setup/recommend.test.ts +100 -0
  33. package/src/cli/ai-setup/recommend.ts +203 -0
  34. package/src/cli/ai.ts +38 -0
  35. package/src/cli/db-auto-push.test.ts +6 -1
  36. package/src/cli/db-auto-push.ts +8 -3
  37. package/src/cli/db-seed.ts +359 -0
  38. package/src/cli/db.test.ts +341 -3
  39. package/src/cli/db.ts +75 -8
  40. package/src/cli/dev-db-push.test.ts +59 -0
  41. package/src/cli/dev.test.ts +40 -0
  42. package/src/cli/dev.ts +55 -1
  43. package/src/cli/docker-clean.test.ts +179 -0
  44. package/src/cli/docker-clean.ts +406 -0
  45. package/src/cli/docker.ts +16 -4
  46. package/src/cli/ensure-drizzle-config.ts +1 -1
  47. package/src/cli/hero-meta.test.ts +6 -5
  48. package/src/cli/hero-meta.ts +7 -1
  49. package/src/cli/index.ts +5 -0
  50. package/src/cli/load-config.images.test.ts +22 -0
  51. package/src/cli/load-config.ts +7 -2
  52. package/src/cli/openbao-bootstrap.test.ts +1 -0
  53. package/src/cli/openbao-bootstrap.ts +9 -2
  54. package/src/cli/registry.ts +116 -2
  55. package/src/client/live-gap.test.ts +35 -0
  56. package/src/client/transport.test.ts +21 -0
  57. package/src/client/transport.ts +27 -5
  58. package/src/compiler/effects-infer.ts +1 -0
  59. package/src/compiler/fixtures/skyport/oke.config.ts +1 -1
  60. package/src/config/define-config.test.ts +6 -6
  61. package/src/config/index.ts +5 -1
  62. package/src/console/server/console.test.ts +45 -0
  63. package/src/console/server/flows.ts +39 -9
  64. package/src/console/server/i18n.ts +41 -0
  65. package/src/console/server/vault.ts +12 -38
  66. package/src/console/ui/dist/assets/index-ClX0ZCe5.js +10 -0
  67. package/src/console/ui/dist/assets/{panel-access-BGv45snf.js → panel-access-ri1X9Otj.js} +1 -1
  68. package/src/console/ui/dist/assets/{panel-ai-B2S7LEii.js → panel-ai-DLHjRoJk.js} +1 -1
  69. package/src/console/ui/dist/assets/{panel-architecture-D7UJh91v.js → panel-architecture-DI3bQF3J.js} +1 -1
  70. package/src/console/ui/dist/assets/{panel-channels-9T3ybqRu.js → panel-channels-BDYWX29p.js} +1 -1
  71. package/src/console/ui/dist/assets/{panel-clock-Cb1UXGRQ.js → panel-clock-i-JpTjVB.js} +1 -1
  72. package/src/console/ui/dist/assets/{panel-diff-DmYbKWmN.js → panel-diff-CFuWxiXy.js} +1 -1
  73. package/src/console/ui/dist/assets/{panel-flows-PiHwT55z.js → panel-flows-B8TRhcoe.js} +1 -1
  74. package/src/console/ui/dist/assets/{panel-gates-BQGYXvjT.js → panel-gates-CfkBgco7.js} +1 -1
  75. package/src/console/ui/dist/assets/{panel-overview-BBnRO18l.js → panel-overview-DONDzBd2.js} +1 -1
  76. package/src/console/ui/dist/assets/{panel-plugins-D0PsmVw2.js → panel-plugins-DDJwTehL.js} +1 -1
  77. package/src/console/ui/dist/assets/{panel-runs-CWuRDe0r.js → panel-runs-D5zf-D9c.js} +1 -1
  78. package/src/console/ui/dist/assets/{panel-signals-Bbg4ewpP.js → panel-signals-Dsxdu_AR.js} +1 -1
  79. package/src/console/ui/dist/assets/{panel-store-CPCbsDRa.js → panel-store-hy7O8HOs.js} +1 -1
  80. package/src/console/ui/dist/assets/{panel-traces-DVAzuA_S.js → panel-traces-CBiAe5go.js} +1 -1
  81. package/src/console/ui/dist/assets/{panel-vault-D1_MvOmo.js → panel-vault-Za1GyJfM.js} +1 -1
  82. package/src/console/ui/dist/index.html +1 -1
  83. package/src/console/ui/shell/components/ui.tsx +5 -1
  84. package/src/console/ui/shell/setup/Wizard.tsx +26 -5
  85. package/src/docker/cleanup.test.ts +193 -0
  86. package/src/docker/cleanup.ts +355 -0
  87. package/src/docker/docker.test.ts +4 -0
  88. package/src/docker/index.ts +19 -0
  89. package/src/docker/recipes/ollama.ts +10 -5
  90. package/src/drivers/ai-ollama.ts +45 -2
  91. package/src/drivers/ai-providers.test.ts +31 -0
  92. package/src/drivers/channel-sently.test.ts +8 -0
  93. package/src/drivers/channel-taqnyat-mail.ts +34 -0
  94. package/src/drivers/channel-types.ts +71 -0
  95. package/src/drivers/clock-postgres.test.ts +258 -0
  96. package/src/drivers/clock-postgres.ts +410 -0
  97. package/src/drivers/index.ts +21 -0
  98. package/src/drivers/journal-postgres.test.ts +175 -0
  99. package/src/drivers/journal-postgres.ts +492 -0
  100. package/src/drivers/signal-engine.ts +98 -25
  101. package/src/drivers/signal-nats.ts +4 -4
  102. package/src/drivers/signal-postgres.ts +158 -64
  103. package/src/drivers/signal-redis.ts +4 -4
  104. package/src/drivers/signal-types.ts +61 -2
  105. package/src/drivers/vault-driver-removal.test.ts +5 -1
  106. package/src/elements/channel/runtime.ts +51 -0
  107. package/src/elements/channel.test.ts +71 -0
  108. package/src/elements/clock/chaos-child.ts +389 -0
  109. package/src/elements/clock/chaos.test.ts +204 -0
  110. package/src/elements/clock/durable.ts +7 -0
  111. package/src/elements/clock/reconcile.ts +110 -0
  112. package/src/elements/clock/runtime.ts +13 -0
  113. package/src/elements/clock.test.ts +164 -2
  114. package/src/elements/clock.ts +2 -1
  115. package/src/elements/gate/runtime.ts +6 -0
  116. package/src/elements/signal/chaos-child.ts +41 -4
  117. package/src/elements/signal/declare.ts +4 -1
  118. package/src/elements/signal/delivery-modes.test.ts +179 -0
  119. package/src/elements/signal/key-ordering.test.ts +306 -0
  120. package/src/elements/signal/lease-reclaim.test.ts +123 -0
  121. package/src/elements/signal/optional-emit.test.ts +76 -0
  122. package/src/elements/signal/order-lifecycle.test.ts +130 -0
  123. package/src/elements/signal/orphan-messages.test.ts +152 -0
  124. package/src/elements/signal/runtime.ts +11 -4
  125. package/src/elements/signal/schema-emit.test.ts +107 -0
  126. package/src/elements/signal.test.ts +89 -5
  127. package/src/elements/store/files-fx.ts +104 -0
  128. package/src/elements/store/files-image.test.ts +251 -0
  129. package/src/elements/store/files-image.ts +494 -0
  130. package/src/elements/store/runtime.ts +60 -0
  131. package/src/elements/store/seed.test.ts +27 -0
  132. package/src/elements/store/seed.ts +68 -0
  133. package/src/elements/store/sql-session.test.ts +39 -0
  134. package/src/elements/store/sql-session.ts +55 -0
  135. package/src/elements/store/upsert-app.test.ts +103 -0
  136. package/src/elements/store.ts +29 -0
  137. package/src/elements/vault/boot-chain.ts +150 -0
  138. package/src/elements/vault/runtime.ts +8 -0
  139. package/src/index.ts +15 -0
  140. package/src/kernel/app.ts +168 -15
  141. package/src/kernel/boot-bind/channel.test.ts +16 -0
  142. package/src/kernel/boot-bind/channel.ts +13 -0
  143. package/src/kernel/boot-bind/clock.ts +63 -7
  144. package/src/kernel/boot-bind/gate.ts +71 -3
  145. package/src/kernel/boot-bind/honor-config.test.ts +348 -0
  146. package/src/kernel/boot-bind/journal.ts +89 -0
  147. package/src/kernel/boot-bind/signal.ts +64 -7
  148. package/src/kernel/boot-bind/vault.ts +27 -8
  149. package/src/kernel/boot.test.ts +6 -4
  150. package/src/kernel/boot.ts +63 -15
  151. package/src/kernel/concurrency.ts +1 -1
  152. package/src/kernel/errors.ts +8 -2
  153. package/src/kernel/fx.test.ts +19 -0
  154. package/src/kernel/fx.ts +153 -9
  155. package/src/kernel/index.ts +6 -0
  156. package/src/kernel/journal-boot.test.ts +397 -0
  157. package/src/kernel/journal-suspend.ts +35 -0
  158. package/src/kernel/journal.test.ts +142 -0
  159. package/src/kernel/journal.ts +202 -27
  160. package/src/plugins/auth-methods.security.test.ts +10 -7
  161. package/src/plugins/phone-number.ts +67 -10
  162. package/src/plugins/taqnyat.live.test.ts +174 -0
  163. package/src/runtime/dev-request-log.test.ts +20 -1
  164. package/src/runtime/dev-request-log.ts +38 -0
  165. package/src/term.test.ts +15 -0
  166. package/src/term.ts +8 -3
  167. package/src/console/ui/dist/assets/index-CjxwRGVv.js +0 -10
@@ -1,18 +1,31 @@
1
1
  ---
2
2
  title: "Store"
3
- description: "Data at rest SQL tables, KV cache, file blobs, and vector search, declared once and swapped per environment by driver."
3
+ description: "Data at rest behind one handle and four independent facets sql · kv · files · index — swapped per environment by driver."
4
4
  icon: "Database"
5
5
  source: "docs/spec/unified-theory.md"
6
6
  ---
7
7
 
8
- Store is where your app's **data at rest** lives. It covers four kinds of storage relational tables, key-value cache, file blobs, and vector search behind one declaration style. Your flow code never changes between SQLite on your laptop and Postgres in production; only the driver does.
8
+ Store is one handle, four independent facets**sql · kv · files · index**. Each has its own
9
+ shape; nothing forces you to use more than one. Flow code stays the same across environments;
10
+ only the driver changes.
9
11
 
10
12
  <Callout title="The one rule">
11
13
  Drivers are named after **protocols**, not vendors (`postgres`, `redis`, `s3` — never `neon` or
12
14
  `minio`). Vendor choice lives in the `images` map of `oke.config.ts`.
13
15
  </Callout>
14
16
 
15
- ## Quick start
17
+ <StoreFacets />
18
+
19
+ Every facet accepts optional `description` — a human title in the Console (falls back to the store
20
+ name).
21
+
22
+ ## SQL
23
+
24
+ <StoreFacetMark facet="sql" />
25
+
26
+ The relational facet: tables, schema, and typed single-table sessions through `fx.store`.
27
+
28
+ ### Quick start
16
29
 
17
30
  <Steps>
18
31
 
@@ -86,25 +99,7 @@ export const createNote = on(
86
99
 
87
100
  </Steps>
88
101
 
89
- ## The four facets
90
-
91
- Pick the facet that matches the physics of your data. Each declaration is one line; the runtime handle shows what flows can do with it.
92
-
93
- <StoreFacets />
94
-
95
- Every facet accepts optional `description` — a human title in the Console
96
- (falls back to the store name). `store.index` also takes `dims`.
97
-
98
- ```typescript
99
- export const cache = store.kv("sessions", { description: "Session cache" });
100
- export const uploads = store.files("attachments");
101
- export const embeddings = store.index("docs", {
102
- dims: 1536,
103
- description: "Document embeddings",
104
- });
105
- ```
106
-
107
- ## CRUD without boilerplate — `store.resource`
102
+ ### CRUD without boilerplate — `store.resource`
108
103
 
109
104
  Five conventional endpoints (list, create, get, update, remove) expand from one declaration. Each is an ordinary Flow underneath — same contracts, same `fx`:
110
105
 
@@ -139,7 +134,31 @@ The list endpoint's URL is the whole query language:
139
134
 
140
135
  Responses follow the Stripe-style envelope: `{ data, meta: { nextCursor, hasNextPage }, error }`. `create` answers **201**, `remove` answers **204**, and a missing row is a typed `NotFound` — never a crash.
141
136
 
142
- ## Querying by hand
137
+ #### An admin table
138
+
139
+ ```typescript
140
+ list: { mode: "offset", count: "exact", limit: 20 },
141
+ ```
142
+
143
+ **Consequence:** `count: "exact"` (the offset default) runs `COUNT(*)` to fill `meta.total`. On a huge table that count is the real cost — set `count: "none"` to return only `meta.offset`.
144
+
145
+ #### An infinite feed
146
+
147
+ ```typescript
148
+ list: { cursor: [notes.createdAt, notes.id], direction: "desc", limit: 20, maxLimit: 100 },
149
+ ```
150
+
151
+ **Consequence:** keyset (cursor) paging is the default when `cursor` columns are set — pages stay stable while new rows are inserted, where offset pages would shift and show duplicates.
152
+
153
+ #### A public, restricted endpoint
154
+
155
+ ```typescript
156
+ list: { mode: "offset", filter: "none", limit: 20 },
157
+ ```
158
+
159
+ **Consequence:** a request that filters on a forbidden column — `?secret=eq.x` — fails with **422** and the exact message `unknown list param "secret"`. Filterable columns are a whitelist, never an accident.
160
+
161
+ ### Querying by hand
143
162
 
144
163
  When `store.resource` is too conventional, `fx.store(db)` is the full single-table session:
145
164
 
@@ -165,7 +184,7 @@ await fx.store(db).delete(notes).where(lt(notes.createdAt, cutoff));
165
184
  table shows up explicitly in the Manifest's `reads` / `writes` — powering caching and PII masking.
166
185
  </Callout>
167
186
 
168
- ## Schema — declare once, generate per dialect
187
+ ### Schema — declare once, generate per dialect
169
188
 
170
189
  The recommended path: declare tables ORM-agnostically, then let `oke db` emit real Drizzle for the active dialect (`sqliteTable` locally, `pgTable` in docker/prod) into `src/schema.generated.ts`.
171
190
 
@@ -179,7 +198,7 @@ The recommended path: declare tables ORM-agnostically, then let `oke db` emit re
179
198
  | `.describe("…")` | human title in the Console (falls back to key) |
180
199
  | `.references(() => col, { onDelete })` | foreign key |
181
200
 
182
- ### Foreign keys and relations
201
+ #### Foreign keys and relations
183
202
 
184
203
  Declare FKs on fields, and relation metadata once per schema:
185
204
 
@@ -205,7 +224,7 @@ export const relations = store.schema.relations({ links, daily }, (r) => ({
205
224
  There is no separate API and no `.through()` — the junction is an ordinary table.
206
225
  </Callout>
207
226
 
208
- ### Syncing the schema
227
+ #### Syncing the schema
209
228
 
210
229
  | Command | When |
211
230
  | ----------------- | -------------------------------------------------------------- |
@@ -213,7 +232,143 @@ export const relations = store.schema.relations({ links, daily }, (r) => ({
213
232
  | `oke db generate` | Write versioned SQL under `drizzle/` for review |
214
233
  | `oke db migrate` | Apply those files — human or CI, **never** at boot |
215
234
 
216
- `oke dev` (local) auto-pushes when the schema file changes — opt out with `--no-db-push` or `db: { autoPush: false }`. Docker/prod **never** auto-apply DDL; a missing table fails loudly as **OKE1101**, telling you to run `oke db migrate`.
235
+ `oke dev` (local) auto-pushes when you save a schema _input_ (`schema.decl.ts`,
236
+ hand-written `schema.ts`, `app.ts` for plugin tables, or `drizzle.config.ts`) —
237
+ not when emit rewrites `schema.generated.ts`.
238
+
239
+ Opt out with `--no-db-push` or `db: { autoPush: false }`. Docker/prod **never**
240
+ auto-apply DDL; a missing table fails as **OKE1101** (`oke db migrate`).
241
+
242
+ #### Multiple environments
243
+
244
+ Local can `oke db push` freely — safe to wipe and regenerate.
245
+
246
+ Staging/prod accumulate versioned SQL under `drizzle/` (`oke db generate`).
247
+ `oke db migrate` applies only unrecorded files in order — behind DBs catch up.
248
+
249
+ | Environment | Sync |
250
+ | -------------- | -------------------------------------------------------------- |
251
+ | Local | `oke db push` (or auto-push from `oke dev`) |
252
+ | Staging / prod | `oke db generate` → review files → `oke db migrate` on that DB |
253
+
254
+ #### Seeding
255
+
256
+ `oke db seed` loads `defineSeed` from `src/seed/index.ts` — **never at boot**.
257
+ Seed proves existence (bootstrap rows); it does not correct schema-adjacent data over time.
258
+
259
+ <StoreSeeding />
260
+
261
+ | env | `essential` | `dev` | `prod` |
262
+ | -------- | ----------- | ----- | ------ |
263
+ | `local` | yes | yes | no |
264
+ | `docker` | yes | yes | no |
265
+ | `test` | yes | no | no |
266
+ | `prod` | yes | no | yes |
267
+
268
+ `docker` is a laptop profile with prod-shaped drivers — still development, so `dev` runs.
269
+ Outside `test`, exactly one of `dev` / `prod` runs with `essential`.
270
+
271
+ **Simple form** — everything inline:
272
+
273
+ ```ts
274
+ import { defineSeed, type Fx } from "okengine";
275
+ import { db } from "../core";
276
+ import { notes } from "../schema.decl";
277
+
278
+ export default defineSeed({
279
+ essential: async (fx: Fx) => {
280
+ await fx
281
+ .store(db)
282
+ .upsert(
283
+ notes,
284
+ { id: "welcome" },
285
+ { id: "welcome", title: "Welcome", body: "…", createdAt: 1 },
286
+ );
287
+ },
288
+ dev: async (fx: Fx) => {
289
+ await fx
290
+ .store(db)
291
+ .upsert(
292
+ notes,
293
+ { id: "sample-ideas" },
294
+ { id: "sample-ideas", title: "Ideas", body: "…", createdAt: 2 },
295
+ );
296
+ },
297
+ // prod: async (fx) => { /* e.g. register a real external webhook URL */ },
298
+ });
299
+ ```
300
+
301
+ **Complex form** — one function per file, compose with arrays (order = run order):
302
+
303
+ ```ts
304
+ import { defineSeed } from "okengine";
305
+ import { seedWelcome } from "./essential/welcome";
306
+ import { seedSamples } from "./dev/samples";
307
+ import { seedWebhook } from "./prod/webhook";
308
+
309
+ export default defineSeed({
310
+ essential: [seedWelcome],
311
+ dev: [seedSamples],
312
+ prod: [seedWebhook],
313
+ });
314
+ ```
315
+
316
+ | `upsert` outcome | When |
317
+ | ----------------- | ------------------------------------------ |
318
+ | `upserted` | no row matched `matchOn` — insert |
319
+ | `already-existed` | match found; default leaves it alone |
320
+ | `changed` | match found and `{ onExisting: "update" }` |
321
+
322
+ Default upsert is **insert-if-missing**. Pass `{ onExisting: "update" }` only when this
323
+ call should also rewrite other columns on a match — opt-in per call, never global.
324
+
325
+ ```ts
326
+ // Safe bootstrap — second seed run leaves the row alone
327
+ await fx.store(db).upsert(
328
+ notes,
329
+ { id: "welcome" },
330
+ {
331
+ id: "welcome",
332
+ title: "Welcome",
333
+ body: "Your Notes API is ready.",
334
+ createdAt: 1,
335
+ },
336
+ );
337
+
338
+ // Refresh copy on every seed — match stays; title/body are rewritten
339
+ await fx.store(db).upsert(
340
+ notes,
341
+ { id: "welcome" },
342
+ {
343
+ id: "welcome",
344
+ title: "Welcome (updated)",
345
+ body: "New bootstrap copy.",
346
+ createdAt: 1,
347
+ },
348
+ { onExisting: "update" },
349
+ );
350
+ ```
351
+
352
+ **Consequence:** default keeps operator edits safe across re-seeds; `onExisting: "update"`
353
+ is for seed-owned rows you intentionally overwrite. Schema-wide data fixes still belong in
354
+ migrations, not seed.
355
+
356
+ | Env | Confirm |
357
+ | ----------------- | ----------------------------------------------------- |
358
+ | `local` / `test` | none |
359
+ | `docker` / `prod` | print DB target; type the env name (`--force` for CI) |
360
+
361
+ ```bash
362
+ oke db seed # resolve env like other oke db commands (.oke/mode)
363
+ oke db seed --env docker # explicit override
364
+ oke db seed --env prod --force
365
+ ```
366
+
367
+ <Callout title="Seed vs migrate">
368
+ Permanent reference-data corrections go through `oke db generate` → `migrate`, not repeated seed.
369
+ Optional `drizzle-seed` may be used inside a `dev` function body only — never as the seed system
370
+ backbone.
371
+ </Callout>
217
372
 
218
373
  <Callout title="Escape hatch">
219
374
  Hand-written Drizzle in `src/schema.ts` stays supported — if there is nothing to emit, the emit
@@ -221,30 +376,17 @@ export const relations = store.schema.relations({ links, daily }, (r) => ({
221
376
  extending an app-owned table with plugin columns is not supported in v1.
222
377
  </Callout>
223
378
 
224
- ## Per-environment drivers
379
+ ### Per-environment SQL drivers
225
380
 
226
381
  Same flow code, different backends — configured once in `oke.config.ts`:
227
382
 
228
- ```typescript title="oke.config.ts"
229
- drivers: {
230
- store: {
231
- sql: { local: "sqlite", docker: "postgres", test: "memory", prod: "postgres" },
232
- kv: { local: "memory", docker: "redis", test: "memory", prod: "redis" },
233
- files: { local: "fs", docker: "s3", test: "memory", prod: "s3" },
234
- },
235
- },
236
- ```
383
+ | Facet | Local | Docker / prod | Runs as |
384
+ | ----- | -------- | ------------- | --------------------------------------- |
385
+ | `sql` | `sqlite` | `postgres` | file on disk → container + named volume |
237
386
 
238
- | Facet | Local | Docker / prod | Runs as |
239
- | ------- | -------- | ------------- | --------------------------------------- |
240
- | `sql` | `sqlite` | `postgres` | file on disk → container + named volume |
241
- | `kv` | `memory` | `redis` | in-process → container |
242
- | `files` | `fs` | `s3` | project folder → RustFS container |
243
- | `index` | `memory` | `pgvector` | in-process → pgvector image |
387
+ Defaults: `sqlite` locally, `postgres` in docker/prod, `memory` in test. Container images come from the `images` map — change the vendor by changing the pin, never the driver id.
244
388
 
245
- Container images come from the `images` map — change the vendor by changing the pin, never the driver id.
246
-
247
- ### Opt-in SQL drivers
389
+ #### Opt-in SQL drivers
248
390
 
249
391
  Two more `store.sql` drivers cover specific parity needs. Both are optional peers — `bun add` them yourself; neither changes the `sqlite` local default.
250
392
 
@@ -258,33 +400,357 @@ Two more `store.sql` drivers cover specific parity needs. Both are optional peer
258
400
  Choose it for dialect/pgvector parity with prod — never as a faster local default.
259
401
  </Callout>
260
402
 
261
- A SQL-backed index shares the sql facet's already-open connection, so configure the pair together:
403
+ SQL-backed index drivers (`pgvector`, `libsql`) share this facet's connection see [Advanced composing facets together](#advanced--composing-facets-together).
404
+
405
+ ## KV
406
+
407
+ <StoreFacetMark facet="kv" />
408
+
409
+ The key-value facet: short-lived cache and session data behind `fx.store`. One namespace per
410
+ declaration; drivers swap `memory` ↔ `redis` per environment.
411
+
412
+ ### Quick start
413
+
414
+ <Steps>
415
+
416
+ <Step>
417
+ ### Declare a namespace
418
+
419
+ ```typescript
420
+ import { store } from "okengine";
421
+
422
+ export const sessions = store.kv("sessions", { description: "Session cache" });
423
+ // ref → kv:sessions
424
+ ```
425
+
426
+ </Step>
427
+
428
+ <Step>
429
+ ### Read and write in a Flow
430
+
431
+ ```typescript
432
+ do: async ({ userId, data }, fx) => {
433
+ const kv = fx.store(sessions);
434
+ await kv.set(`user:${userId}`, data, "30m");
435
+ return await kv.get(`user:${userId}`);
436
+ },
437
+ ```
438
+
439
+ </Step>
440
+
441
+ </Steps>
442
+
443
+ ### Handle API
444
+
445
+ | Call | Effect | Returns | Notes |
446
+ | ----------------------- | ------- | ---------- | ----------------------------------------- |
447
+ | `get(key)` | `read` | `unknown` | Missing keys resolve per driver |
448
+ | `set(key, value, ttl?)` | `write` | `void` | Values should be JSON-serializable |
449
+ | `delete(key)` | `write` | `boolean` | Whether a key was removed |
450
+ | `list(prefix?)` | `read` | `string[]` | Prefix filter; powers the Console browser |
451
+
452
+ ### TTL
453
+
454
+ Optional third argument on `set` is a duration string: `(\d+)(ms|s|m|h|d)` — e.g. `"30m"`, `"1h"`,
455
+ `"5000ms"`. Same call — opposite physics under each driver:
456
+
457
+ <StoreKvTtl />
458
+
459
+ **Consequence:** a TTL you rely on in prod does nothing under the local `memory` driver. Test expiry
460
+ against `redis` (docker) when lifetime matters.
461
+
462
+ ### Examples — what follows from each choice
463
+
464
+ #### A session with expiry
465
+
466
+ ```typescript
467
+ await fx.store(sessions).set(`sess:${token}`, { userId }, "7d");
468
+ ```
469
+
470
+ **Consequence:** under `redis`, the key vanishes after seven days without a sweeper Flow. Under
471
+ `memory`, it lives until process exit or an explicit `delete`.
472
+
473
+ #### Browse by prefix
474
+
475
+ ```typescript
476
+ const keys = await fx.store(sessions).list("sess:");
477
+ ```
478
+
479
+ **Consequence:** `list` is how the [Console · Store](/docs/console/store) browser enumerates a
480
+ namespace. Prefer stable key prefixes (`user:`, `sess:`) so both your Flows and the Console stay
481
+ readable.
482
+
483
+ ### Per-environment KV drivers
262
484
 
263
485
  ```typescript title="oke.config.ts"
264
486
  drivers: {
265
487
  store: {
266
- sql: { local: "libsql" },
267
- index: { local: "libsql" },
488
+ kv: { local: "memory", docker: "redis", test: "memory", prod: "redis" },
268
489
  },
269
490
  },
491
+ images: {
492
+ "store.kv": "redis:8-alpine",
493
+ },
270
494
  ```
271
495
 
496
+ | Facet | Local | Docker / prod | Runs as |
497
+ | ----- | -------- | ------------- | ---------------------- |
498
+ | `kv` | `memory` | `redis` | in-process → container |
499
+
500
+ | Variable | When |
501
+ | ------------------ | -------------------------------- |
502
+ | `REDIS_URL` | Required for the `redis` driver |
503
+ | `OKE_STORE_KV_URL` | Override if `REDIS_URL` is unset |
504
+ | `OKE_KV_DRIVER` | Force a driver id (docker) |
505
+
506
+ Missing Redis URL fails boot loudly: `oke boot: redis driver needs REDIS_URL`.
507
+
508
+ <Callout title="Dry-run refuses KV writes">
509
+ `set` and `delete` throw `DryRunWriteIsolationError` during dry-run — the runtime will not risk a
510
+ double-write against a shared Redis. Reads (`get` / `list`) still run.
511
+ </Callout>
512
+
513
+ ## Files
514
+
515
+ <StoreFacetMark facet="files" />
516
+
517
+ The blob facet: opaque objects (`Uint8Array` or string) on the same handle as the optional image
518
+ pipeline. Drivers swap `fs` ↔ `s3` (plus `memory` for tests).
519
+
520
+ ### Quick start
521
+
522
+ <Steps>
523
+
524
+ <Step>
525
+ ### Declare a bucket
526
+
527
+ ```typescript
528
+ import { store } from "okengine";
529
+
530
+ export const uploads = store.files("attachments", {
531
+ description: "User uploads",
532
+ });
533
+ // ref → files:attachments
534
+ ```
535
+
536
+ </Step>
537
+
538
+ <Step>
539
+ ### Put and get in a Flow
540
+
541
+ ```typescript
542
+ do: async ({ key, body }, fx) => {
543
+ const files = fx.store(uploads);
544
+ await files.put(key, body);
545
+ return await files.get(key); // Uint8Array | null
546
+ },
547
+ ```
548
+
549
+ </Step>
550
+
551
+ </Steps>
552
+
553
+ ### Blob CRUD — `put` / `get` / `delete` / `list`
554
+
555
+ | Call | Effect | Returns | Notes |
556
+ | ---------------- | ------- | -------------------- | ------------------------------------ |
557
+ | `put(key, data)` | `write` | `void` | `data` is `Uint8Array` or `string` |
558
+ | `get(key)` | `read` | `Uint8Array \| null` | Missing key → `null` |
559
+ | `delete(key)` | `write` | `boolean` | Whether an object was removed |
560
+ | `list(prefix?)` | `read` | `string[]` | Prefix filter; Console browse source |
561
+
562
+ ```typescript
563
+ const files = fx.store(uploads);
564
+ await files.put("docs/readme.txt", "hello");
565
+ const bytes = await files.get("docs/readme.txt");
566
+ const keys = await files.list("docs/");
567
+ await files.delete("docs/readme.txt");
568
+ ```
569
+
570
+ ### Keys
571
+
572
+ Keys are opaque strings the driver stores as-is. Prefer path-like prefixes (`photos/`, `exports/`).
573
+
574
+ | Helper / rule | Meaning |
575
+ | --------------------------- | ------------------------------------------------------------------------ |
576
+ | `contentAddressedKey(data)` | From `okengine/store` — stable sha256 hex of the bytes (dedupe-friendly) |
577
+ | Leading `/` or `..` | Rejected by the `fs` driver (`Invalid object key`) |
578
+ | Non-ASCII keys | Console warns `non_ascii_key` — may break S3 signed URLs |
579
+
580
+ ### Per-environment Files drivers
581
+
582
+ ```typescript title="oke.config.ts"
583
+ drivers: {
584
+ store: {
585
+ files: { local: "fs", docker: "s3", test: "memory", prod: "s3" },
586
+ },
587
+ },
588
+ images: {
589
+ "store.files": "rustfs/rustfs:1.0.0-beta.11",
590
+ },
591
+ ```
592
+
593
+ | Facet | Local | Docker / prod | Runs as |
594
+ | ------- | ----- | ------------- | --------------------------------- |
595
+ | `files` | `fs` | `s3` | project folder → RustFS container |
596
+
597
+ | Variable | When |
598
+ | ----------------------------- | -------------------------------------- |
599
+ | `S3_BUCKET` | Required bucket for the `s3` driver |
600
+ | `OKE_STORE_FILES_DB` | Override bucket name |
601
+ | `S3_ENDPOINT` · keys · region | S3-compatible endpoint and credentials |
602
+ | `OKE_FILES_DRIVER` | Force a driver id (docker) |
603
+
604
+ `memory` is the test default. Local `fs` writes under a temp root when no binding `root` is set.
605
+
606
+ <Callout title="Console can browse, not edit bytes">
607
+ The [Console · Store](/docs/console/store) lists keys and can delete them. Direct edit is **KV +
608
+ SQL only** — blob bodies are not patched from the Console.
609
+ </Callout>
610
+
611
+ ### Images — `image` / `putImage`
612
+
613
+ For photos, use Bun's built-in `Bun.Image` pipeline on the same handle (requires Bun `>=1.3.14`).
614
+ `putImage` is one write that fans into several keys:
615
+
616
+ <StoreFilesVariants />
617
+
618
+ ```typescript
619
+ const meta = await fx.store(uploads).image(key).metadata();
620
+ // → { width, height, format }
621
+
622
+ await fx
623
+ .store(uploads)
624
+ .image(key)
625
+ .resize(400, 400, { fit: "inside" })
626
+ .webp({ quality: 80 })
627
+ .put(`thumbs/${key}.webp`);
628
+
629
+ const result = await fx.store(uploads).putImage("photos/x.jpg", bytes, {
630
+ variants: {
631
+ thumb: { resize: [128, 128, { fit: "inside" }], webp: { quality: 80 } },
632
+ medium: { resize: [800], webp: { quality: 85 } },
633
+ },
634
+ placeholder: true, // ThumbHash LQIP data URL
635
+ });
636
+ // variants: photos/x.thumb.webp, photos/x.medium.webp
637
+ ```
638
+
639
+ | Call | Effect | Notes |
640
+ | ----------------------------------------------------------------------- | ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
641
+ | `image(key\|bytes).metadata()` / `bytes()` / `blob()` / `placeholder()` | `read` when source is a key | Chain `resize` · `rotate` · `flip` · `flop` · `modulate`, then `jpeg` / `png` / `webp` (portable); `heic` / `avif` fall back to WebP when the OS codec is missing |
642
+ | `image(…).put(outKey)` | `read` + `write` (key source) | Persist the encoded result |
643
+ | `putImage(key, data, opts?)` | `write` | Original + named variants; keys are `{stem}.{variant}.{ext}` |
644
+
645
+ #### Image options
646
+
647
+ | Option | Default | Meaning |
648
+ | ------------- | ----------------------- | ---------------------------------------------------- |
649
+ | `maxPixels` | **16 MP** (`4096×4096`) | Decode ceiling — raise it or pass `false` to opt out |
650
+ | `autoOrient` | `true` | Apply JPEG EXIF orientation before transforms |
651
+ | `variants` | — | Named `{ resize?, jpeg\|png\|webp\|… }` specs |
652
+ | `placeholder` | — | When `true`, return a ThumbHash LQIP data URL |
653
+
654
+ `putImage` returns `{ key, meta, variants, placeholder? }` — `variants` maps name → object key.
655
+
656
+ ## Index
657
+
658
+ <StoreFacetMark facet="index" />
659
+
660
+ The search / similarity facet: vector ANN and opt-in full-text, both behind `store.index`. The
661
+ handle is a **discriminated union** on `driverId` — vector methods and text methods never mix.
662
+
663
+ ### Quick start
664
+
665
+ <Steps>
666
+
667
+ <Step>
668
+ ### Declare a vector index
669
+
670
+ ```typescript
671
+ import { store } from "okengine";
672
+
673
+ export const embeddings = store.index("docs", {
674
+ dims: 1536,
675
+ description: "Document embeddings",
676
+ });
677
+ // ref → index:docs
678
+ ```
679
+
680
+ `dims` defaults to `3` at open if omitted — set it to match your embedder.
681
+
682
+ </Step>
683
+
684
+ <Step>
685
+ ### Upsert and search in a Flow
686
+
687
+ ```typescript
688
+ do: async ({ id, vector, query }, fx) => {
689
+ const idx = fx.store(embeddings);
690
+ await idx.upsert(id, vector, { title: "Intro" });
691
+ return await idx.search(query, 5);
692
+ // → [{ id, score, meta? }, …] score = cosine similarity
693
+ },
694
+ ```
695
+
696
+ </Step>
697
+
698
+ </Steps>
699
+
700
+ ### Handle API — vector vs text
701
+
702
+ One declaration shape — two search physics. The handle is a **discriminated union** on `driverId`:
703
+
704
+ <StoreIndexModes />
705
+
706
+ | Call (both) | Effect | Returns |
707
+ | ------------ | ------- | ------------- |
708
+ | `upsert(…)` | `write` | `void` |
709
+ | `search(…)` | `read` | hits / result |
710
+ | `delete(id)` | `write` | `boolean` |
711
+
712
+ TypeScript rejects a vector query against a text index (and vice versa) at compile time — never at
713
+ runtime.
714
+
715
+ ### Vector ANN
716
+
272
717
  | `store.index` id | Shares connection from | Real ANN via |
273
718
  | ---------------- | -------------------------- | ------------------------------------ |
274
- | `memory` | — (in-process, default) | |
719
+ | `memory` | — (in-process, default) | Full scan cosine |
275
720
  | `pgvector` | `postgres` or `pglite` sql | HNSW + `cosineDistance` |
276
721
  | `libsql` | `libsql` sql | `libsql_vector_idx` + `vector_top_k` |
277
722
 
278
- **Consequence:** a configured SQL-backed index that cannot reach its engine — missing peer, missing `vector` extension, wrong sql driver — fails loudly at first use. It never silently falls back to `memory`.
723
+ **Consequence:** a configured SQL-backed index that cannot reach its engine — missing peer, missing
724
+ `vector` extension, wrong sql driver — fails loudly at first use. It never silently falls back to
725
+ `memory`. A vector whose length ≠ `dims` throws `vector dims X !== index dims Y`.
726
+
727
+ Pair SQL + index under [Advanced — composing facets together](#advanced--composing-facets-together).
279
728
 
280
729
  ### Full-text search — `meilisearch` (opt-in)
281
730
 
282
- `meilisearch` is a fourth `store.index` backend, opt-in only — `memory` stays the default. It is a genuinely different capability from the vector ANN drivers: typo-tolerant, faceted full-text relevance, not cosine similarity. The two models are kept apart by a **discriminated union** on `driverId`, so TypeScript rejects a vector query against a text index (and vice versa) at compile time — never at runtime:
731
+ `meilisearch` is a fourth `store.index` backend, opt-in only — `memory` stays the default when
732
+ `drivers.store.index` is unset. Typo-tolerant, faceted full-text relevance — not cosine similarity.
283
733
 
284
- | `driverId` | `upsert` | `search` | `score` means |
285
- | -------------------------------- | --------------------- | ---------------------------------------- | --------------------------- |
286
- | `memory` / `pgvector` / `libsql` | `(id, vector, meta?)` | `search(vector, topK?)` → `IndexHit[]` | cosine similarity |
287
- | `meilisearch` | `(id, document)` | `search(q, opts?)` `{ hits, facets? }` | relevance (`_rankingScore`) |
734
+ ```typescript
735
+ export const kb = store.index("kb", { description: "Help articles" });
736
+
737
+ do: async ({ id, doc, q }, fx) => {
738
+ const idx = fx.store(kb); // driverId: "meilisearch"
739
+ await idx.upsert(id, { id, ...doc });
740
+ return await idx.search(q, {
741
+ topK: 10,
742
+ filter: "category = guides",
743
+ facets: ["category"],
744
+ });
745
+ // → { hits, facetDistribution? }
746
+ },
747
+ ```
748
+
749
+ | Search option | Meaning |
750
+ | ------------- | ------------------------------------------------ |
751
+ | `topK` | Max hits (Meilisearch `limit`, default 10) |
752
+ | `filter` | Filter expression — attribute must be filterable |
753
+ | `facets` | Attributes to aggregate into `facetDistribution` |
288
754
 
289
755
  ```typescript title="oke.config.ts"
290
756
  drivers: {
@@ -303,35 +769,67 @@ images: {
303
769
  - **Not for vectors:** `ai.embed` / `fx.search` stay vector-only; pointing an embed `into` a meilisearch index fails loud (embeddings don't apply).
304
770
  - **Alpine caveat:** the raw glibc binary fails on Alpine/musl _hosts_; the official image is musl-clean. Install the binary for your platform (e.g. Homebrew / install script), or use `--docker`.
305
771
 
306
- ## Privacy built in
772
+ ### Per-environment Index drivers
307
773
 
308
- Columns tagged `.pii()` or `.sensitive()` are masked at the store boundaryflows, logs, and the Console see a mask, not the value. Revealing cleartext PII requires an explicit `pii:reveal` gate on the flow, so access is a permission, not a convention.
774
+ Unlike KV/Files, an unset `drivers.store.index` map stays **`memory` in every environment**there is no docker fallback to `pgvector`. Opt in explicitly:
309
775
 
310
- ## Examples — what follows from each choice
776
+ ```typescript title="oke.config.ts"
777
+ drivers: {
778
+ store: {
779
+ sql: { local: "libsql", docker: "postgres", prod: "postgres" },
780
+ index: { local: "libsql", docker: "pgvector", prod: "pgvector" },
781
+ },
782
+ },
783
+ ```
311
784
 
312
- ### An admin table
785
+ | Variable | When |
786
+ | --------------------- | ------------------------------------------------- |
787
+ | `OKE_INDEX_DRIVER` | Force a driver id |
788
+ | `OKE_STORE_INDEX_URL` | Required for `meilisearch` |
789
+ | `OKE_STORE_INDEX_KEY` | Meilisearch API key (`MEILI_MASTER_KEY` fallback) |
313
790
 
314
- ```typescript
315
- list: { mode: "offset", count: "exact", limit: 20 },
316
- ```
791
+ ### With AI embeddings
317
792
 
318
- **Consequence:** `count: "exact"` (the offset default) runs `COUNT(*)` to fill `meta.total`. On a huge table that count is the real cost — set `count: "none"` to return only `meta.offset`.
793
+ `ai.embed` writes into a vector `store.index`; Flows search it with `fx.search` (or
794
+ `fx.store(index).search`). See [AI](/docs/elements/ai) — this path is vector-only and will not
795
+ target meilisearch.
319
796
 
320
- ### An infinite feed
797
+ ## Advanced composing facets together
321
798
 
322
- ```typescript
323
- list: { cursor: [notes.createdAt, notes.id], direction: "desc", limit: 20, maxLimit: 100 },
324
- ```
799
+ Facets are independent declarations. A Flow may touch several in one `do` — for example store a files key on a SQL row, or `upsert` into an index after a SQL write. There is no special sync API: `store.resource` `search:` is SQL substring match, not meilisearch.
325
800
 
326
- **Consequence:** keyset (cursor) paging is the default when `cursor` columns are set — pages stay stable while new rows are inserted, where offset pages would shift and show duplicates.
801
+ One real coupling: a SQL-backed index shares the sql facet's already-open connection, so configure the pair together:
327
802
 
328
- ### A public, restricted endpoint
803
+ ```typescript title="oke.config.ts"
804
+ drivers: {
805
+ store: {
806
+ sql: { local: "libsql" },
807
+ index: { local: "libsql" },
808
+ },
809
+ },
810
+ ```
329
811
 
330
- ```typescript
331
- list: { mode: "offset", filter: "none", limit: 20 },
812
+ Same idea for `pgvector` with `postgres` or `pglite`. The full per-environment map looks like this:
813
+
814
+ ```typescript title="oke.config.ts"
815
+ drivers: {
816
+ store: {
817
+ sql: { local: "sqlite", docker: "postgres", test: "memory", prod: "postgres" },
818
+ kv: { local: "memory", docker: "redis", test: "memory", prod: "redis" },
819
+ files: { local: "fs", docker: "s3", test: "memory", prod: "s3" },
820
+ },
821
+ },
332
822
  ```
333
823
 
334
- **Consequence:** a request that filters on a forbidden column `?secret=eq.x` fails with **422** and the exact message `unknown list param "secret"`. Filterable columns are a whitelist, never an accident.
824
+ Container images come from the `images` mapchange the vendor by changing the pin, never the driver id.
825
+
826
+ ## Advanced — with other elements
827
+
828
+ Columns tagged `.pii()` or `.sensitive()` are masked at the store boundary — flows, logs, and the Console see a mask, not the value. Revealing cleartext PII requires an explicit `pii:reveal` gate on the flow, so access is a permission, not a convention.
829
+
830
+ Optional `description` on a store (and `.describe("…")` on fields) becomes the human title in the
831
+ [Console · Store](/docs/console/store) browser. Single-table `reads` / `writes` in the Manifest
832
+ power cache invalidation and that same PII masking — see the Callout under [Querying by hand](#querying-by-hand).
335
833
 
336
834
  ## Troubleshooting
337
835
 
@@ -355,14 +853,38 @@ You are on offset paging. Switch `list` to keyset by setting `cursor` columns wi
355
853
 
356
854
  `count: "exact"` runs `COUNT(*)` per page. Set `count: "none"` in the `list` options to skip it, or use keyset mode where totals are rarely needed.
357
855
 
856
+ </Accordion>
857
+ <Accordion title="KV TTL never expires locally">
858
+
859
+ The local default is `memory`, which ignores TTL on `set`. Run against `redis` (`oke dev --docker` or `drivers.store.kv.local: "redis"` with `REDIS_URL`) when you need real expiry.
860
+
861
+ </Accordion>
862
+ <Accordion title="oke boot: redis / meilisearch / S3 URL missing">
863
+
864
+ Driver-backed facets fail boot when their URL is absent — Redis needs `REDIS_URL`, meilisearch needs `OKE_STORE_INDEX_URL`, S3 needs `S3_BUCKET`. Docker mode expects `oke dev -d` to write these into the compose env.
865
+
866
+ </Accordion>
867
+ <Accordion title="vector dims X !== index dims Y">
868
+
869
+ The vector length must match `store.index(…, { dims })` (runtime default `3` if unset). Align `dims` with your embedder, or fix the vector you pass to `upsert` / `search`.
870
+
871
+ </Accordion>
872
+ <Accordion title="Index stays on memory in docker">
873
+
874
+ Unlike KV/Files, unset `drivers.store.index` does **not** promote to `pgvector` in docker — it stays `memory`. Set the map explicitly (and pair SQL for `pgvector` / `libsql`).
875
+
358
876
  </Accordion>
359
877
  </Accordions>
360
878
 
361
879
  ## Learn more
362
880
 
363
881
  - [Flow](/docs/elements/flow) — the `fx.store` session inside `do`
882
+ - [AI](/docs/elements/ai) — `ai.embed` into a vector `store.index`, searched via `fx.search`
883
+ - [Gate](/docs/elements/gate) — `pii:reveal` and other permissions on flows
364
884
  - [Console · Store](/docs/console/store) — browse data, cache keys, PII masking
365
885
  - [CLI Reference](/docs/reference/cli) — `oke db push` · `generate` · `migrate`
886
+ - [Configuration](/docs/reference/configuration) — `drivers.store` maps and `images` pins
887
+ - [Environment variables](/docs/reference/environment-variables) — Redis · S3 · meilisearch URLs
366
888
 
367
889
  ## Next
368
890