okengine 0.6.0 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +2 -2
- package/README.md +98 -71
- package/package.json +4 -4
- package/site/content/docs/console/clock.mdx +25 -5
- package/site/content/docs/console/signals.mdx +13 -4
- package/site/content/docs/elements/ai.mdx +29 -2
- package/site/content/docs/elements/channel.mdx +90 -20
- package/site/content/docs/elements/clock.mdx +56 -24
- package/site/content/docs/elements/flow.mdx +2 -0
- package/site/content/docs/elements/gate.mdx +8 -1
- package/site/content/docs/elements/signal.mdx +173 -29
- package/site/content/docs/elements/store.mdx +464 -73
- package/site/content/docs/elements/vault.mdx +15 -5
- package/site/content/docs/get-started/basic-usage.mdx +13 -6
- package/site/content/docs/get-started/installation.mdx +49 -13
- package/site/content/docs/get-started/introduction.mdx +1 -1
- package/site/content/docs/reference/cli.md +30 -2
- package/site/content/docs/reference/configuration.mdx +18 -17
- package/site/content/docs/reference/environment-variables.mdx +44 -18
- package/site/content/docs/reference/errors.mdx +12 -11
- package/site/content/docs/reference/fx.mdx +15 -13
- package/site/content/docs/reference/security.md +3 -1
- package/src/cli/ai-setup/ai-setup.test.ts +144 -0
- package/src/cli/ai-setup/apply.ts +214 -0
- package/src/cli/ai-setup/catalog.ts +263 -0
- package/src/cli/ai-setup/detect-ollama.ts +166 -0
- package/src/cli/ai-setup/index.ts +228 -0
- package/src/cli/ai-setup/prompts.ts +649 -0
- package/src/cli/ai-setup/recommend.test.ts +100 -0
- package/src/cli/ai-setup/recommend.ts +203 -0
- package/src/cli/ai.ts +38 -0
- package/src/cli/db-auto-push.test.ts +6 -1
- package/src/cli/db-auto-push.ts +8 -3
- package/src/cli/dev-db-push.test.ts +59 -0
- package/src/cli/dev.test.ts +40 -0
- package/src/cli/dev.ts +55 -1
- package/src/cli/docker-clean.test.ts +179 -0
- package/src/cli/docker-clean.ts +406 -0
- package/src/cli/docker.ts +16 -4
- package/src/cli/ensure-drizzle-config.ts +1 -1
- package/src/cli/hero-meta.test.ts +6 -5
- package/src/cli/hero-meta.ts +7 -1
- package/src/cli/index.ts +5 -0
- package/src/cli/load-config.ts +4 -4
- package/src/cli/openbao-bootstrap.test.ts +1 -0
- package/src/cli/openbao-bootstrap.ts +9 -2
- package/src/cli/openbao-restart.integration.test.ts +106 -97
- package/src/cli/registry.ts +79 -1
- package/src/client/live-gap.test.ts +35 -0
- package/src/client/transport.test.ts +21 -0
- package/src/client/transport.ts +27 -5
- package/src/compiler/fixtures/skyport/oke.config.ts +1 -1
- package/src/config/define-config.test.ts +6 -6
- package/src/config/index.ts +1 -1
- package/src/console/server/console.test.ts +45 -0
- package/src/console/server/flows.ts +39 -9
- package/src/console/server/i18n.ts +41 -0
- package/src/console/server/vault.ts +12 -38
- package/src/console/ui/dist/assets/index-ClX0ZCe5.js +10 -0
- package/src/console/ui/dist/assets/{panel-access-BGv45snf.js → panel-access-ri1X9Otj.js} +1 -1
- package/src/console/ui/dist/assets/{panel-ai-B2S7LEii.js → panel-ai-DLHjRoJk.js} +1 -1
- package/src/console/ui/dist/assets/{panel-architecture-D7UJh91v.js → panel-architecture-DI3bQF3J.js} +1 -1
- package/src/console/ui/dist/assets/{panel-channels-9T3ybqRu.js → panel-channels-BDYWX29p.js} +1 -1
- package/src/console/ui/dist/assets/{panel-clock-Cb1UXGRQ.js → panel-clock-i-JpTjVB.js} +1 -1
- package/src/console/ui/dist/assets/{panel-diff-DmYbKWmN.js → panel-diff-CFuWxiXy.js} +1 -1
- package/src/console/ui/dist/assets/{panel-flows-PiHwT55z.js → panel-flows-B8TRhcoe.js} +1 -1
- package/src/console/ui/dist/assets/{panel-gates-BQGYXvjT.js → panel-gates-CfkBgco7.js} +1 -1
- package/src/console/ui/dist/assets/{panel-overview-BBnRO18l.js → panel-overview-DONDzBd2.js} +1 -1
- package/src/console/ui/dist/assets/{panel-plugins-D0PsmVw2.js → panel-plugins-DDJwTehL.js} +1 -1
- package/src/console/ui/dist/assets/{panel-runs-CWuRDe0r.js → panel-runs-D5zf-D9c.js} +1 -1
- package/src/console/ui/dist/assets/{panel-signals-Bbg4ewpP.js → panel-signals-Dsxdu_AR.js} +1 -1
- package/src/console/ui/dist/assets/{panel-store-CPCbsDRa.js → panel-store-hy7O8HOs.js} +1 -1
- package/src/console/ui/dist/assets/{panel-traces-DVAzuA_S.js → panel-traces-CBiAe5go.js} +1 -1
- package/src/console/ui/dist/assets/{panel-vault-D1_MvOmo.js → panel-vault-Za1GyJfM.js} +1 -1
- package/src/console/ui/dist/index.html +1 -1
- package/src/console/ui/shell/components/ui.tsx +5 -1
- package/src/console/ui/shell/setup/Wizard.tsx +26 -5
- package/src/docker/cleanup.test.ts +193 -0
- package/src/docker/cleanup.ts +355 -0
- package/src/docker/docker.test.ts +4 -0
- package/src/docker/dockerfile.integration.test.ts +126 -119
- package/src/docker/index.ts +19 -0
- package/src/docker/recipes/ollama.ts +10 -5
- package/src/docker/stack.integration.test.ts +118 -102
- package/src/drivers/ai-ollama-tools.integration.test.ts +8 -6
- package/src/drivers/ai-ollama.integration.test.ts +3 -19
- package/src/drivers/ai-ollama.ts +45 -2
- package/src/drivers/ai-providers.test.ts +31 -0
- package/src/drivers/channel-fcm.ts +49 -53
- package/src/drivers/channel-msegat.ts +61 -0
- package/src/drivers/channel-sently-map.ts +57 -0
- package/src/drivers/channel-sently.test.ts +99 -0
- package/src/drivers/channel-sndr.ts +28 -0
- package/src/drivers/channel-taqnyat.ts +57 -0
- package/src/drivers/channel-types.ts +79 -2
- package/src/drivers/channel-unifonic.ts +26 -43
- package/src/drivers/channel-wa-cloud.ts +33 -47
- package/src/drivers/channel-webpush.ts +39 -239
- package/src/drivers/index.ts +7 -0
- package/src/drivers/signal-engine.ts +98 -25
- package/src/drivers/signal-nats.ts +4 -4
- package/src/drivers/signal-postgres.ts +158 -64
- package/src/drivers/signal-redis.ts +4 -4
- package/src/drivers/signal-types.ts +61 -2
- package/src/drivers/vault-driver-removal.test.ts +5 -1
- package/src/elements/channel/costs.test.ts +2 -2
- package/src/elements/channel/costs.ts +14 -2
- package/src/elements/channel/mime.ts +11 -0
- package/src/elements/channel/runtime.ts +94 -0
- package/src/elements/channel/sndr-webhooks.test.ts +26 -0
- package/src/elements/channel.ts +10 -1
- package/src/elements/clock/chaos-child.ts +150 -0
- package/src/elements/clock/chaos.test.ts +204 -0
- package/src/elements/clock/reconcile.ts +110 -0
- package/src/elements/clock/runtime.ts +11 -0
- package/src/elements/clock.test.ts +164 -2
- package/src/elements/clock.ts +1 -0
- package/src/elements/gate/runtime.ts +6 -0
- package/src/elements/index.ts +9 -0
- package/src/elements/signal/chaos-child.ts +41 -4
- package/src/elements/signal/declare.ts +4 -1
- package/src/elements/signal/delivery-modes.test.ts +179 -0
- package/src/elements/signal/key-ordering.test.ts +306 -0
- package/src/elements/signal/lease-reclaim.test.ts +123 -0
- package/src/elements/signal/optional-emit.test.ts +76 -0
- package/src/elements/signal/order-lifecycle.test.ts +130 -0
- package/src/elements/signal/orphan-messages.test.ts +152 -0
- package/src/elements/signal/runtime.ts +11 -4
- package/src/elements/signal/schema-emit.test.ts +107 -0
- package/src/elements/signal.test.ts +89 -5
- package/src/elements/store/files-fx.ts +104 -0
- package/src/elements/store/files-image.test.ts +251 -0
- package/src/elements/store/files-image.ts +494 -0
- package/src/elements/store/runtime.ts +60 -0
- package/src/elements/store.ts +24 -0
- package/src/elements/vault/boot-chain.ts +150 -0
- package/src/elements/vault/runtime.ts +8 -0
- package/src/kernel/app.ts +3 -1
- package/src/kernel/boot-bind/channel.test.ts +68 -3
- package/src/kernel/boot-bind/channel.ts +93 -2
- package/src/kernel/boot-bind/clock.ts +52 -7
- package/src/kernel/boot-bind/gate.ts +71 -3
- package/src/kernel/boot-bind/honor-config.test.ts +247 -0
- package/src/kernel/boot-bind/signal.ts +64 -7
- package/src/kernel/boot-bind/vault.ts +27 -8
- package/src/kernel/boot.ts +10 -2
- package/src/kernel/errors.ts +8 -2
- package/src/kernel/fx.test.ts +13 -0
- package/src/kernel/fx.ts +27 -4
- package/src/plugins/auth-delivery.mailpit.integration.test.ts +10 -4
- package/src/release/exports.test.ts +26 -0
- package/src/release/exports.ts +64 -5
- package/src/release/index.ts +5 -0
- package/src/release/measure.exports.test.ts +13 -1
- package/src/release/measure.ts +76 -13
- package/src/release/official-plugins.ts +46 -0
- package/src/release/readme.test.ts +30 -2
- package/src/runtime/dev-request-log.test.ts +20 -1
- package/src/runtime/dev-request-log.ts +38 -0
- package/src/term.test.ts +15 -0
- package/src/term.ts +8 -3
- package/src/console/ui/dist/assets/index-CjxwRGVv.js +0 -10
|
@@ -1,18 +1,31 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: "Store"
|
|
3
|
-
description: "Data at rest
|
|
3
|
+
description: "Data at rest behind one handle and four independent facets — sql · kv · files · index — swapped per environment by driver."
|
|
4
4
|
icon: "Database"
|
|
5
5
|
source: "docs/spec/unified-theory.md"
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
-
Store is
|
|
8
|
+
Store is one handle, four independent facets — **sql · kv · files · index**. Each has its own
|
|
9
|
+
shape; nothing forces you to use more than one. Flow code stays the same across environments;
|
|
10
|
+
only the driver changes.
|
|
9
11
|
|
|
10
12
|
<Callout title="The one rule">
|
|
11
13
|
Drivers are named after **protocols**, not vendors (`postgres`, `redis`, `s3` — never `neon` or
|
|
12
14
|
`minio`). Vendor choice lives in the `images` map of `oke.config.ts`.
|
|
13
15
|
</Callout>
|
|
14
16
|
|
|
15
|
-
|
|
17
|
+
<StoreFacets />
|
|
18
|
+
|
|
19
|
+
Every facet accepts optional `description` — a human title in the Console (falls back to the store
|
|
20
|
+
name).
|
|
21
|
+
|
|
22
|
+
## SQL
|
|
23
|
+
|
|
24
|
+
<StoreFacetMark facet="sql" />
|
|
25
|
+
|
|
26
|
+
The relational facet: tables, schema, and typed single-table sessions through `fx.store`.
|
|
27
|
+
|
|
28
|
+
### Quick start
|
|
16
29
|
|
|
17
30
|
<Steps>
|
|
18
31
|
|
|
@@ -86,25 +99,7 @@ export const createNote = on(
|
|
|
86
99
|
|
|
87
100
|
</Steps>
|
|
88
101
|
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
Pick the facet that matches the physics of your data. Each declaration is one line; the runtime handle shows what flows can do with it.
|
|
92
|
-
|
|
93
|
-
<StoreFacets />
|
|
94
|
-
|
|
95
|
-
Every facet accepts optional `description` — a human title in the Console
|
|
96
|
-
(falls back to the store name). `store.index` also takes `dims`.
|
|
97
|
-
|
|
98
|
-
```typescript
|
|
99
|
-
export const cache = store.kv("sessions", { description: "Session cache" });
|
|
100
|
-
export const uploads = store.files("attachments");
|
|
101
|
-
export const embeddings = store.index("docs", {
|
|
102
|
-
dims: 1536,
|
|
103
|
-
description: "Document embeddings",
|
|
104
|
-
});
|
|
105
|
-
```
|
|
106
|
-
|
|
107
|
-
## CRUD without boilerplate — `store.resource`
|
|
102
|
+
### CRUD without boilerplate — `store.resource`
|
|
108
103
|
|
|
109
104
|
Five conventional endpoints (list, create, get, update, remove) expand from one declaration. Each is an ordinary Flow underneath — same contracts, same `fx`:
|
|
110
105
|
|
|
@@ -139,7 +134,31 @@ The list endpoint's URL is the whole query language:
|
|
|
139
134
|
|
|
140
135
|
Responses follow the Stripe-style envelope: `{ data, meta: { nextCursor, hasNextPage }, error }`. `create` answers **201**, `remove` answers **204**, and a missing row is a typed `NotFound` — never a crash.
|
|
141
136
|
|
|
142
|
-
|
|
137
|
+
#### An admin table
|
|
138
|
+
|
|
139
|
+
```typescript
|
|
140
|
+
list: { mode: "offset", count: "exact", limit: 20 },
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
**Consequence:** `count: "exact"` (the offset default) runs `COUNT(*)` to fill `meta.total`. On a huge table that count is the real cost — set `count: "none"` to return only `meta.offset`.
|
|
144
|
+
|
|
145
|
+
#### An infinite feed
|
|
146
|
+
|
|
147
|
+
```typescript
|
|
148
|
+
list: { cursor: [notes.createdAt, notes.id], direction: "desc", limit: 20, maxLimit: 100 },
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
**Consequence:** keyset (cursor) paging is the default when `cursor` columns are set — pages stay stable while new rows are inserted, where offset pages would shift and show duplicates.
|
|
152
|
+
|
|
153
|
+
#### A public, restricted endpoint
|
|
154
|
+
|
|
155
|
+
```typescript
|
|
156
|
+
list: { mode: "offset", filter: "none", limit: 20 },
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
**Consequence:** a request that filters on a forbidden column — `?secret=eq.x` — fails with **422** and the exact message `unknown list param "secret"`. Filterable columns are a whitelist, never an accident.
|
|
160
|
+
|
|
161
|
+
### Querying by hand
|
|
143
162
|
|
|
144
163
|
When `store.resource` is too conventional, `fx.store(db)` is the full single-table session:
|
|
145
164
|
|
|
@@ -165,7 +184,7 @@ await fx.store(db).delete(notes).where(lt(notes.createdAt, cutoff));
|
|
|
165
184
|
table shows up explicitly in the Manifest's `reads` / `writes` — powering caching and PII masking.
|
|
166
185
|
</Callout>
|
|
167
186
|
|
|
168
|
-
|
|
187
|
+
### Schema — declare once, generate per dialect
|
|
169
188
|
|
|
170
189
|
The recommended path: declare tables ORM-agnostically, then let `oke db` emit real Drizzle for the active dialect (`sqliteTable` locally, `pgTable` in docker/prod) into `src/schema.generated.ts`.
|
|
171
190
|
|
|
@@ -179,7 +198,7 @@ The recommended path: declare tables ORM-agnostically, then let `oke db` emit re
|
|
|
179
198
|
| `.describe("…")` | human title in the Console (falls back to key) |
|
|
180
199
|
| `.references(() => col, { onDelete })` | foreign key |
|
|
181
200
|
|
|
182
|
-
|
|
201
|
+
#### Foreign keys and relations
|
|
183
202
|
|
|
184
203
|
Declare FKs on fields, and relation metadata once per schema:
|
|
185
204
|
|
|
@@ -205,7 +224,7 @@ export const relations = store.schema.relations({ links, daily }, (r) => ({
|
|
|
205
224
|
There is no separate API and no `.through()` — the junction is an ordinary table.
|
|
206
225
|
</Callout>
|
|
207
226
|
|
|
208
|
-
|
|
227
|
+
#### Syncing the schema
|
|
209
228
|
|
|
210
229
|
| Command | When |
|
|
211
230
|
| ----------------- | -------------------------------------------------------------- |
|
|
@@ -213,7 +232,12 @@ export const relations = store.schema.relations({ links, daily }, (r) => ({
|
|
|
213
232
|
| `oke db generate` | Write versioned SQL under `drizzle/` for review |
|
|
214
233
|
| `oke db migrate` | Apply those files — human or CI, **never** at boot |
|
|
215
234
|
|
|
216
|
-
`oke dev` (local) auto-pushes when
|
|
235
|
+
`oke dev` (local) auto-pushes when you save a schema _input_ (`schema.decl.ts`,
|
|
236
|
+
hand-written `schema.ts`, `app.ts` for plugin tables, or `drizzle.config.ts`) —
|
|
237
|
+
not when emit rewrites `schema.generated.ts`.
|
|
238
|
+
|
|
239
|
+
Opt out with `--no-db-push` or `db: { autoPush: false }`. Docker/prod **never**
|
|
240
|
+
auto-apply DDL; a missing table fails as **OKE1101** (`oke db migrate`).
|
|
217
241
|
|
|
218
242
|
<Callout title="Escape hatch">
|
|
219
243
|
Hand-written Drizzle in `src/schema.ts` stays supported — if there is nothing to emit, the emit
|
|
@@ -221,30 +245,17 @@ export const relations = store.schema.relations({ links, daily }, (r) => ({
|
|
|
221
245
|
extending an app-owned table with plugin columns is not supported in v1.
|
|
222
246
|
</Callout>
|
|
223
247
|
|
|
224
|
-
|
|
248
|
+
### Per-environment SQL drivers
|
|
225
249
|
|
|
226
250
|
Same flow code, different backends — configured once in `oke.config.ts`:
|
|
227
251
|
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
sql: { local: "sqlite", docker: "postgres", test: "memory", prod: "postgres" },
|
|
232
|
-
kv: { local: "memory", docker: "redis", test: "memory", prod: "redis" },
|
|
233
|
-
files: { local: "fs", docker: "s3", test: "memory", prod: "s3" },
|
|
234
|
-
},
|
|
235
|
-
},
|
|
236
|
-
```
|
|
237
|
-
|
|
238
|
-
| Facet | Local | Docker / prod | Runs as |
|
|
239
|
-
| ------- | -------- | ------------- | --------------------------------------- |
|
|
240
|
-
| `sql` | `sqlite` | `postgres` | file on disk → container + named volume |
|
|
241
|
-
| `kv` | `memory` | `redis` | in-process → container |
|
|
242
|
-
| `files` | `fs` | `s3` | project folder → RustFS container |
|
|
243
|
-
| `index` | `memory` | `pgvector` | in-process → pgvector image |
|
|
252
|
+
| Facet | Local | Docker / prod | Runs as |
|
|
253
|
+
| ----- | -------- | ------------- | --------------------------------------- |
|
|
254
|
+
| `sql` | `sqlite` | `postgres` | file on disk → container + named volume |
|
|
244
255
|
|
|
245
|
-
Container images come from the `images` map — change the vendor by changing the pin, never the driver id.
|
|
256
|
+
Defaults: `sqlite` locally, `postgres` in docker/prod, `memory` in test. Container images come from the `images` map — change the vendor by changing the pin, never the driver id.
|
|
246
257
|
|
|
247
|
-
|
|
258
|
+
#### Opt-in SQL drivers
|
|
248
259
|
|
|
249
260
|
Two more `store.sql` drivers cover specific parity needs. Both are optional peers — `bun add` them yourself; neither changes the `sqlite` local default.
|
|
250
261
|
|
|
@@ -258,33 +269,357 @@ Two more `store.sql` drivers cover specific parity needs. Both are optional peer
|
|
|
258
269
|
Choose it for dialect/pgvector parity with prod — never as a faster local default.
|
|
259
270
|
</Callout>
|
|
260
271
|
|
|
261
|
-
|
|
272
|
+
SQL-backed index drivers (`pgvector`, `libsql`) share this facet's connection — see [Advanced — composing facets together](#advanced--composing-facets-together).
|
|
273
|
+
|
|
274
|
+
## KV
|
|
275
|
+
|
|
276
|
+
<StoreFacetMark facet="kv" />
|
|
277
|
+
|
|
278
|
+
The key-value facet: short-lived cache and session data behind `fx.store`. One namespace per
|
|
279
|
+
declaration; drivers swap `memory` ↔ `redis` per environment.
|
|
280
|
+
|
|
281
|
+
### Quick start
|
|
282
|
+
|
|
283
|
+
<Steps>
|
|
284
|
+
|
|
285
|
+
<Step>
|
|
286
|
+
### Declare a namespace
|
|
287
|
+
|
|
288
|
+
```typescript
|
|
289
|
+
import { store } from "okengine";
|
|
290
|
+
|
|
291
|
+
export const sessions = store.kv("sessions", { description: "Session cache" });
|
|
292
|
+
// ref → kv:sessions
|
|
293
|
+
```
|
|
294
|
+
|
|
295
|
+
</Step>
|
|
296
|
+
|
|
297
|
+
<Step>
|
|
298
|
+
### Read and write in a Flow
|
|
299
|
+
|
|
300
|
+
```typescript
|
|
301
|
+
do: async ({ userId, data }, fx) => {
|
|
302
|
+
const kv = fx.store(sessions);
|
|
303
|
+
await kv.set(`user:${userId}`, data, "30m");
|
|
304
|
+
return await kv.get(`user:${userId}`);
|
|
305
|
+
},
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
</Step>
|
|
309
|
+
|
|
310
|
+
</Steps>
|
|
311
|
+
|
|
312
|
+
### Handle API
|
|
313
|
+
|
|
314
|
+
| Call | Effect | Returns | Notes |
|
|
315
|
+
| ----------------------- | ------- | ---------- | ----------------------------------------- |
|
|
316
|
+
| `get(key)` | `read` | `unknown` | Missing keys resolve per driver |
|
|
317
|
+
| `set(key, value, ttl?)` | `write` | `void` | Values should be JSON-serializable |
|
|
318
|
+
| `delete(key)` | `write` | `boolean` | Whether a key was removed |
|
|
319
|
+
| `list(prefix?)` | `read` | `string[]` | Prefix filter; powers the Console browser |
|
|
320
|
+
|
|
321
|
+
### TTL
|
|
322
|
+
|
|
323
|
+
Optional third argument on `set` is a duration string: `(\d+)(ms|s|m|h|d)` — e.g. `"30m"`, `"1h"`,
|
|
324
|
+
`"5000ms"`. Same call — opposite physics under each driver:
|
|
325
|
+
|
|
326
|
+
<StoreKvTtl />
|
|
327
|
+
|
|
328
|
+
**Consequence:** a TTL you rely on in prod does nothing under the local `memory` driver. Test expiry
|
|
329
|
+
against `redis` (docker) when lifetime matters.
|
|
330
|
+
|
|
331
|
+
### Examples — what follows from each choice
|
|
332
|
+
|
|
333
|
+
#### A session with expiry
|
|
334
|
+
|
|
335
|
+
```typescript
|
|
336
|
+
await fx.store(sessions).set(`sess:${token}`, { userId }, "7d");
|
|
337
|
+
```
|
|
338
|
+
|
|
339
|
+
**Consequence:** under `redis`, the key vanishes after seven days without a sweeper Flow. Under
|
|
340
|
+
`memory`, it lives until process exit or an explicit `delete`.
|
|
341
|
+
|
|
342
|
+
#### Browse by prefix
|
|
343
|
+
|
|
344
|
+
```typescript
|
|
345
|
+
const keys = await fx.store(sessions).list("sess:");
|
|
346
|
+
```
|
|
347
|
+
|
|
348
|
+
**Consequence:** `list` is how the [Console · Store](/docs/console/store) browser enumerates a
|
|
349
|
+
namespace. Prefer stable key prefixes (`user:`, `sess:`) so both your Flows and the Console stay
|
|
350
|
+
readable.
|
|
351
|
+
|
|
352
|
+
### Per-environment KV drivers
|
|
262
353
|
|
|
263
354
|
```typescript title="oke.config.ts"
|
|
264
355
|
drivers: {
|
|
265
356
|
store: {
|
|
266
|
-
|
|
267
|
-
|
|
357
|
+
kv: { local: "memory", docker: "redis", test: "memory", prod: "redis" },
|
|
358
|
+
},
|
|
359
|
+
},
|
|
360
|
+
images: {
|
|
361
|
+
"store.kv": "redis:8-alpine",
|
|
362
|
+
},
|
|
363
|
+
```
|
|
364
|
+
|
|
365
|
+
| Facet | Local | Docker / prod | Runs as |
|
|
366
|
+
| ----- | -------- | ------------- | ---------------------- |
|
|
367
|
+
| `kv` | `memory` | `redis` | in-process → container |
|
|
368
|
+
|
|
369
|
+
| Variable | When |
|
|
370
|
+
| ------------------ | -------------------------------- |
|
|
371
|
+
| `REDIS_URL` | Required for the `redis` driver |
|
|
372
|
+
| `OKE_STORE_KV_URL` | Override if `REDIS_URL` is unset |
|
|
373
|
+
| `OKE_KV_DRIVER` | Force a driver id (docker) |
|
|
374
|
+
|
|
375
|
+
Missing Redis URL fails boot loudly: `oke boot: redis driver needs REDIS_URL`.
|
|
376
|
+
|
|
377
|
+
<Callout title="Dry-run refuses KV writes">
|
|
378
|
+
`set` and `delete` throw `DryRunWriteIsolationError` during dry-run — the runtime will not risk a
|
|
379
|
+
double-write against a shared Redis. Reads (`get` / `list`) still run.
|
|
380
|
+
</Callout>
|
|
381
|
+
|
|
382
|
+
## Files
|
|
383
|
+
|
|
384
|
+
<StoreFacetMark facet="files" />
|
|
385
|
+
|
|
386
|
+
The blob facet: opaque objects (`Uint8Array` or string) on the same handle as the optional image
|
|
387
|
+
pipeline. Drivers swap `fs` ↔ `s3` (plus `memory` for tests).
|
|
388
|
+
|
|
389
|
+
### Quick start
|
|
390
|
+
|
|
391
|
+
<Steps>
|
|
392
|
+
|
|
393
|
+
<Step>
|
|
394
|
+
### Declare a bucket
|
|
395
|
+
|
|
396
|
+
```typescript
|
|
397
|
+
import { store } from "okengine";
|
|
398
|
+
|
|
399
|
+
export const uploads = store.files("attachments", {
|
|
400
|
+
description: "User uploads",
|
|
401
|
+
});
|
|
402
|
+
// ref → files:attachments
|
|
403
|
+
```
|
|
404
|
+
|
|
405
|
+
</Step>
|
|
406
|
+
|
|
407
|
+
<Step>
|
|
408
|
+
### Put and get in a Flow
|
|
409
|
+
|
|
410
|
+
```typescript
|
|
411
|
+
do: async ({ key, body }, fx) => {
|
|
412
|
+
const files = fx.store(uploads);
|
|
413
|
+
await files.put(key, body);
|
|
414
|
+
return await files.get(key); // Uint8Array | null
|
|
415
|
+
},
|
|
416
|
+
```
|
|
417
|
+
|
|
418
|
+
</Step>
|
|
419
|
+
|
|
420
|
+
</Steps>
|
|
421
|
+
|
|
422
|
+
### Blob CRUD — `put` / `get` / `delete` / `list`
|
|
423
|
+
|
|
424
|
+
| Call | Effect | Returns | Notes |
|
|
425
|
+
| ---------------- | ------- | -------------------- | ------------------------------------ |
|
|
426
|
+
| `put(key, data)` | `write` | `void` | `data` is `Uint8Array` or `string` |
|
|
427
|
+
| `get(key)` | `read` | `Uint8Array \| null` | Missing key → `null` |
|
|
428
|
+
| `delete(key)` | `write` | `boolean` | Whether an object was removed |
|
|
429
|
+
| `list(prefix?)` | `read` | `string[]` | Prefix filter; Console browse source |
|
|
430
|
+
|
|
431
|
+
```typescript
|
|
432
|
+
const files = fx.store(uploads);
|
|
433
|
+
await files.put("docs/readme.txt", "hello");
|
|
434
|
+
const bytes = await files.get("docs/readme.txt");
|
|
435
|
+
const keys = await files.list("docs/");
|
|
436
|
+
await files.delete("docs/readme.txt");
|
|
437
|
+
```
|
|
438
|
+
|
|
439
|
+
### Keys
|
|
440
|
+
|
|
441
|
+
Keys are opaque strings the driver stores as-is. Prefer path-like prefixes (`photos/`, `exports/`).
|
|
442
|
+
|
|
443
|
+
| Helper / rule | Meaning |
|
|
444
|
+
| --------------------------- | ------------------------------------------------------------------------ |
|
|
445
|
+
| `contentAddressedKey(data)` | From `okengine/store` — stable sha256 hex of the bytes (dedupe-friendly) |
|
|
446
|
+
| Leading `/` or `..` | Rejected by the `fs` driver (`Invalid object key`) |
|
|
447
|
+
| Non-ASCII keys | Console warns `non_ascii_key` — may break S3 signed URLs |
|
|
448
|
+
|
|
449
|
+
### Per-environment Files drivers
|
|
450
|
+
|
|
451
|
+
```typescript title="oke.config.ts"
|
|
452
|
+
drivers: {
|
|
453
|
+
store: {
|
|
454
|
+
files: { local: "fs", docker: "s3", test: "memory", prod: "s3" },
|
|
268
455
|
},
|
|
269
456
|
},
|
|
457
|
+
images: {
|
|
458
|
+
"store.files": "rustfs/rustfs:1.0.0-beta.11",
|
|
459
|
+
},
|
|
460
|
+
```
|
|
461
|
+
|
|
462
|
+
| Facet | Local | Docker / prod | Runs as |
|
|
463
|
+
| ------- | ----- | ------------- | --------------------------------- |
|
|
464
|
+
| `files` | `fs` | `s3` | project folder → RustFS container |
|
|
465
|
+
|
|
466
|
+
| Variable | When |
|
|
467
|
+
| ----------------------------- | -------------------------------------- |
|
|
468
|
+
| `S3_BUCKET` | Required bucket for the `s3` driver |
|
|
469
|
+
| `OKE_STORE_FILES_DB` | Override bucket name |
|
|
470
|
+
| `S3_ENDPOINT` · keys · region | S3-compatible endpoint and credentials |
|
|
471
|
+
| `OKE_FILES_DRIVER` | Force a driver id (docker) |
|
|
472
|
+
|
|
473
|
+
`memory` is the test default. Local `fs` writes under a temp root when no binding `root` is set.
|
|
474
|
+
|
|
475
|
+
<Callout title="Console can browse, not edit bytes">
|
|
476
|
+
The [Console · Store](/docs/console/store) lists keys and can delete them. Direct edit is **KV +
|
|
477
|
+
SQL only** — blob bodies are not patched from the Console.
|
|
478
|
+
</Callout>
|
|
479
|
+
|
|
480
|
+
### Images — `image` / `putImage`
|
|
481
|
+
|
|
482
|
+
For photos, use Bun's built-in `Bun.Image` pipeline on the same handle (requires Bun `>=1.3.14`).
|
|
483
|
+
`putImage` is one write that fans into several keys:
|
|
484
|
+
|
|
485
|
+
<StoreFilesVariants />
|
|
486
|
+
|
|
487
|
+
```typescript
|
|
488
|
+
const meta = await fx.store(uploads).image(key).metadata();
|
|
489
|
+
// → { width, height, format }
|
|
490
|
+
|
|
491
|
+
await fx
|
|
492
|
+
.store(uploads)
|
|
493
|
+
.image(key)
|
|
494
|
+
.resize(400, 400, { fit: "inside" })
|
|
495
|
+
.webp({ quality: 80 })
|
|
496
|
+
.put(`thumbs/${key}.webp`);
|
|
497
|
+
|
|
498
|
+
const result = await fx.store(uploads).putImage("photos/x.jpg", bytes, {
|
|
499
|
+
variants: {
|
|
500
|
+
thumb: { resize: [128, 128, { fit: "inside" }], webp: { quality: 80 } },
|
|
501
|
+
medium: { resize: [800], webp: { quality: 85 } },
|
|
502
|
+
},
|
|
503
|
+
placeholder: true, // ThumbHash LQIP data URL
|
|
504
|
+
});
|
|
505
|
+
// variants: photos/x.thumb.webp, photos/x.medium.webp
|
|
270
506
|
```
|
|
271
507
|
|
|
508
|
+
| Call | Effect | Notes |
|
|
509
|
+
| ----------------------------------------------------------------------- | ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
510
|
+
| `image(key\|bytes).metadata()` / `bytes()` / `blob()` / `placeholder()` | `read` when source is a key | Chain `resize` · `rotate` · `flip` · `flop` · `modulate`, then `jpeg` / `png` / `webp` (portable); `heic` / `avif` fall back to WebP when the OS codec is missing |
|
|
511
|
+
| `image(…).put(outKey)` | `read` + `write` (key source) | Persist the encoded result |
|
|
512
|
+
| `putImage(key, data, opts?)` | `write` | Original + named variants; keys are `{stem}.{variant}.{ext}` |
|
|
513
|
+
|
|
514
|
+
#### Image options
|
|
515
|
+
|
|
516
|
+
| Option | Default | Meaning |
|
|
517
|
+
| ------------- | ----------------------- | ---------------------------------------------------- |
|
|
518
|
+
| `maxPixels` | **16 MP** (`4096×4096`) | Decode ceiling — raise it or pass `false` to opt out |
|
|
519
|
+
| `autoOrient` | `true` | Apply JPEG EXIF orientation before transforms |
|
|
520
|
+
| `variants` | — | Named `{ resize?, jpeg\|png\|webp\|… }` specs |
|
|
521
|
+
| `placeholder` | — | When `true`, return a ThumbHash LQIP data URL |
|
|
522
|
+
|
|
523
|
+
`putImage` returns `{ key, meta, variants, placeholder? }` — `variants` maps name → object key.
|
|
524
|
+
|
|
525
|
+
## Index
|
|
526
|
+
|
|
527
|
+
<StoreFacetMark facet="index" />
|
|
528
|
+
|
|
529
|
+
The search / similarity facet: vector ANN and opt-in full-text, both behind `store.index`. The
|
|
530
|
+
handle is a **discriminated union** on `driverId` — vector methods and text methods never mix.
|
|
531
|
+
|
|
532
|
+
### Quick start
|
|
533
|
+
|
|
534
|
+
<Steps>
|
|
535
|
+
|
|
536
|
+
<Step>
|
|
537
|
+
### Declare a vector index
|
|
538
|
+
|
|
539
|
+
```typescript
|
|
540
|
+
import { store } from "okengine";
|
|
541
|
+
|
|
542
|
+
export const embeddings = store.index("docs", {
|
|
543
|
+
dims: 1536,
|
|
544
|
+
description: "Document embeddings",
|
|
545
|
+
});
|
|
546
|
+
// ref → index:docs
|
|
547
|
+
```
|
|
548
|
+
|
|
549
|
+
`dims` defaults to `3` at open if omitted — set it to match your embedder.
|
|
550
|
+
|
|
551
|
+
</Step>
|
|
552
|
+
|
|
553
|
+
<Step>
|
|
554
|
+
### Upsert and search in a Flow
|
|
555
|
+
|
|
556
|
+
```typescript
|
|
557
|
+
do: async ({ id, vector, query }, fx) => {
|
|
558
|
+
const idx = fx.store(embeddings);
|
|
559
|
+
await idx.upsert(id, vector, { title: "Intro" });
|
|
560
|
+
return await idx.search(query, 5);
|
|
561
|
+
// → [{ id, score, meta? }, …] score = cosine similarity
|
|
562
|
+
},
|
|
563
|
+
```
|
|
564
|
+
|
|
565
|
+
</Step>
|
|
566
|
+
|
|
567
|
+
</Steps>
|
|
568
|
+
|
|
569
|
+
### Handle API — vector vs text
|
|
570
|
+
|
|
571
|
+
One declaration shape — two search physics. The handle is a **discriminated union** on `driverId`:
|
|
572
|
+
|
|
573
|
+
<StoreIndexModes />
|
|
574
|
+
|
|
575
|
+
| Call (both) | Effect | Returns |
|
|
576
|
+
| ------------ | ------- | ------------- |
|
|
577
|
+
| `upsert(…)` | `write` | `void` |
|
|
578
|
+
| `search(…)` | `read` | hits / result |
|
|
579
|
+
| `delete(id)` | `write` | `boolean` |
|
|
580
|
+
|
|
581
|
+
TypeScript rejects a vector query against a text index (and vice versa) at compile time — never at
|
|
582
|
+
runtime.
|
|
583
|
+
|
|
584
|
+
### Vector ANN
|
|
585
|
+
|
|
272
586
|
| `store.index` id | Shares connection from | Real ANN via |
|
|
273
587
|
| ---------------- | -------------------------- | ------------------------------------ |
|
|
274
|
-
| `memory` | — (in-process, default) |
|
|
588
|
+
| `memory` | — (in-process, default) | Full scan cosine |
|
|
275
589
|
| `pgvector` | `postgres` or `pglite` sql | HNSW + `cosineDistance` |
|
|
276
590
|
| `libsql` | `libsql` sql | `libsql_vector_idx` + `vector_top_k` |
|
|
277
591
|
|
|
278
|
-
**Consequence:** a configured SQL-backed index that cannot reach its engine — missing peer, missing
|
|
592
|
+
**Consequence:** a configured SQL-backed index that cannot reach its engine — missing peer, missing
|
|
593
|
+
`vector` extension, wrong sql driver — fails loudly at first use. It never silently falls back to
|
|
594
|
+
`memory`. A vector whose length ≠ `dims` throws `vector dims X !== index dims Y`.
|
|
595
|
+
|
|
596
|
+
Pair SQL + index under [Advanced — composing facets together](#advanced--composing-facets-together).
|
|
279
597
|
|
|
280
598
|
### Full-text search — `meilisearch` (opt-in)
|
|
281
599
|
|
|
282
|
-
`meilisearch` is a fourth `store.index` backend, opt-in only — `memory` stays the default
|
|
600
|
+
`meilisearch` is a fourth `store.index` backend, opt-in only — `memory` stays the default when
|
|
601
|
+
`drivers.store.index` is unset. Typo-tolerant, faceted full-text relevance — not cosine similarity.
|
|
283
602
|
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
603
|
+
```typescript
|
|
604
|
+
export const kb = store.index("kb", { description: "Help articles" });
|
|
605
|
+
|
|
606
|
+
do: async ({ id, doc, q }, fx) => {
|
|
607
|
+
const idx = fx.store(kb); // driverId: "meilisearch"
|
|
608
|
+
await idx.upsert(id, { id, ...doc });
|
|
609
|
+
return await idx.search(q, {
|
|
610
|
+
topK: 10,
|
|
611
|
+
filter: "category = guides",
|
|
612
|
+
facets: ["category"],
|
|
613
|
+
});
|
|
614
|
+
// → { hits, facetDistribution? }
|
|
615
|
+
},
|
|
616
|
+
```
|
|
617
|
+
|
|
618
|
+
| Search option | Meaning |
|
|
619
|
+
| ------------- | ------------------------------------------------ |
|
|
620
|
+
| `topK` | Max hits (Meilisearch `limit`, default 10) |
|
|
621
|
+
| `filter` | Filter expression — attribute must be filterable |
|
|
622
|
+
| `facets` | Attributes to aggregate into `facetDistribution` |
|
|
288
623
|
|
|
289
624
|
```typescript title="oke.config.ts"
|
|
290
625
|
drivers: {
|
|
@@ -303,35 +638,67 @@ images: {
|
|
|
303
638
|
- **Not for vectors:** `ai.embed` / `fx.search` stay vector-only; pointing an embed `into` a meilisearch index fails loud (embeddings don't apply).
|
|
304
639
|
- **Alpine caveat:** the raw glibc binary fails on Alpine/musl _hosts_; the official image is musl-clean. Install the binary for your platform (e.g. Homebrew / install script), or use `--docker`.
|
|
305
640
|
|
|
306
|
-
|
|
641
|
+
### Per-environment Index drivers
|
|
307
642
|
|
|
308
|
-
|
|
643
|
+
Unlike KV/Files, an unset `drivers.store.index` map stays **`memory` in every environment** — there is no docker fallback to `pgvector`. Opt in explicitly:
|
|
644
|
+
|
|
645
|
+
```typescript title="oke.config.ts"
|
|
646
|
+
drivers: {
|
|
647
|
+
store: {
|
|
648
|
+
sql: { local: "libsql", docker: "postgres", prod: "postgres" },
|
|
649
|
+
index: { local: "libsql", docker: "pgvector", prod: "pgvector" },
|
|
650
|
+
},
|
|
651
|
+
},
|
|
652
|
+
```
|
|
309
653
|
|
|
310
|
-
|
|
654
|
+
| Variable | When |
|
|
655
|
+
| --------------------- | ------------------------------------------------- |
|
|
656
|
+
| `OKE_INDEX_DRIVER` | Force a driver id |
|
|
657
|
+
| `OKE_STORE_INDEX_URL` | Required for `meilisearch` |
|
|
658
|
+
| `OKE_STORE_INDEX_KEY` | Meilisearch API key (`MEILI_MASTER_KEY` fallback) |
|
|
311
659
|
|
|
312
|
-
###
|
|
660
|
+
### With AI embeddings
|
|
313
661
|
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
662
|
+
`ai.embed` writes into a vector `store.index`; Flows search it with `fx.search` (or
|
|
663
|
+
`fx.store(index).search`). See [AI](/docs/elements/ai) — this path is vector-only and will not
|
|
664
|
+
target meilisearch.
|
|
317
665
|
|
|
318
|
-
|
|
666
|
+
## Advanced — composing facets together
|
|
319
667
|
|
|
320
|
-
|
|
668
|
+
Facets are independent declarations. A Flow may touch several in one `do` — for example store a files key on a SQL row, or `upsert` into an index after a SQL write. There is no special sync API: `store.resource` `search:` is SQL substring match, not meilisearch.
|
|
321
669
|
|
|
322
|
-
|
|
323
|
-
list: { cursor: [notes.createdAt, notes.id], direction: "desc", limit: 20, maxLimit: 100 },
|
|
324
|
-
```
|
|
670
|
+
One real coupling: a SQL-backed index shares the sql facet's already-open connection, so configure the pair together:
|
|
325
671
|
|
|
326
|
-
|
|
672
|
+
```typescript title="oke.config.ts"
|
|
673
|
+
drivers: {
|
|
674
|
+
store: {
|
|
675
|
+
sql: { local: "libsql" },
|
|
676
|
+
index: { local: "libsql" },
|
|
677
|
+
},
|
|
678
|
+
},
|
|
679
|
+
```
|
|
327
680
|
|
|
328
|
-
|
|
681
|
+
Same idea for `pgvector` with `postgres` or `pglite`. The full per-environment map looks like this:
|
|
329
682
|
|
|
330
|
-
```typescript
|
|
331
|
-
|
|
683
|
+
```typescript title="oke.config.ts"
|
|
684
|
+
drivers: {
|
|
685
|
+
store: {
|
|
686
|
+
sql: { local: "sqlite", docker: "postgres", test: "memory", prod: "postgres" },
|
|
687
|
+
kv: { local: "memory", docker: "redis", test: "memory", prod: "redis" },
|
|
688
|
+
files: { local: "fs", docker: "s3", test: "memory", prod: "s3" },
|
|
689
|
+
},
|
|
690
|
+
},
|
|
332
691
|
```
|
|
333
692
|
|
|
334
|
-
|
|
693
|
+
Container images come from the `images` map — change the vendor by changing the pin, never the driver id.
|
|
694
|
+
|
|
695
|
+
## Advanced — with other elements
|
|
696
|
+
|
|
697
|
+
Columns tagged `.pii()` or `.sensitive()` are masked at the store boundary — flows, logs, and the Console see a mask, not the value. Revealing cleartext PII requires an explicit `pii:reveal` gate on the flow, so access is a permission, not a convention.
|
|
698
|
+
|
|
699
|
+
Optional `description` on a store (and `.describe("…")` on fields) becomes the human title in the
|
|
700
|
+
[Console · Store](/docs/console/store) browser. Single-table `reads` / `writes` in the Manifest
|
|
701
|
+
power cache invalidation and that same PII masking — see the Callout under [Querying by hand](#querying-by-hand).
|
|
335
702
|
|
|
336
703
|
## Troubleshooting
|
|
337
704
|
|
|
@@ -355,14 +722,38 @@ You are on offset paging. Switch `list` to keyset by setting `cursor` columns wi
|
|
|
355
722
|
|
|
356
723
|
`count: "exact"` runs `COUNT(*)` per page. Set `count: "none"` in the `list` options to skip it, or use keyset mode where totals are rarely needed.
|
|
357
724
|
|
|
725
|
+
</Accordion>
|
|
726
|
+
<Accordion title="KV TTL never expires locally">
|
|
727
|
+
|
|
728
|
+
The local default is `memory`, which ignores TTL on `set`. Run against `redis` (`oke dev --docker` or `drivers.store.kv.local: "redis"` with `REDIS_URL`) when you need real expiry.
|
|
729
|
+
|
|
730
|
+
</Accordion>
|
|
731
|
+
<Accordion title="oke boot: redis / meilisearch / S3 URL missing">
|
|
732
|
+
|
|
733
|
+
Driver-backed facets fail boot when their URL is absent — Redis needs `REDIS_URL`, meilisearch needs `OKE_STORE_INDEX_URL`, S3 needs `S3_BUCKET`. Docker mode expects `oke dev -d` to write these into the compose env.
|
|
734
|
+
|
|
735
|
+
</Accordion>
|
|
736
|
+
<Accordion title="vector dims X !== index dims Y">
|
|
737
|
+
|
|
738
|
+
The vector length must match `store.index(…, { dims })` (runtime default `3` if unset). Align `dims` with your embedder, or fix the vector you pass to `upsert` / `search`.
|
|
739
|
+
|
|
740
|
+
</Accordion>
|
|
741
|
+
<Accordion title="Index stays on memory in docker">
|
|
742
|
+
|
|
743
|
+
Unlike KV/Files, unset `drivers.store.index` does **not** promote to `pgvector` in docker — it stays `memory`. Set the map explicitly (and pair SQL for `pgvector` / `libsql`).
|
|
744
|
+
|
|
358
745
|
</Accordion>
|
|
359
746
|
</Accordions>
|
|
360
747
|
|
|
361
748
|
## Learn more
|
|
362
749
|
|
|
363
750
|
- [Flow](/docs/elements/flow) — the `fx.store` session inside `do`
|
|
751
|
+
- [AI](/docs/elements/ai) — `ai.embed` into a vector `store.index`, searched via `fx.search`
|
|
752
|
+
- [Gate](/docs/elements/gate) — `pii:reveal` and other permissions on flows
|
|
364
753
|
- [Console · Store](/docs/console/store) — browse data, cache keys, PII masking
|
|
365
754
|
- [CLI Reference](/docs/reference/cli) — `oke db push` · `generate` · `migrate`
|
|
755
|
+
- [Configuration](/docs/reference/configuration) — `drivers.store` maps and `images` pins
|
|
756
|
+
- [Environment variables](/docs/reference/environment-variables) — Redis · S3 · meilisearch URLs
|
|
366
757
|
|
|
367
758
|
## Next
|
|
368
759
|
|