@lunora/cli 1.0.0-alpha.45 → 1.0.0-alpha.451

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (141) hide show
  1. package/LICENSE.md +33 -0
  2. package/README.md +3 -1
  3. package/dist/bin.mjs +3 -10
  4. package/dist/cli.d.mts +31 -0
  5. package/dist/cli.d.ts +31 -0
  6. package/dist/cli.mjs +1 -0
  7. package/dist/index.d.mts +1096 -386
  8. package/dist/index.d.ts +1096 -386
  9. package/dist/index.mjs +1 -19
  10. package/dist/packem_chunks/cli.mjs +2 -0
  11. package/dist/packem_chunks/dispatch.mjs +1 -0
  12. package/dist/packem_chunks/handler.mjs +17 -158
  13. package/dist/packem_chunks/handler10.mjs +1 -16
  14. package/dist/packem_chunks/handler11.mjs +1 -22
  15. package/dist/packem_chunks/handler12.mjs +1 -192
  16. package/dist/packem_chunks/handler13.mjs +1 -131
  17. package/dist/packem_chunks/handler14.mjs +3 -65
  18. package/dist/packem_chunks/handler15.mjs +1 -58
  19. package/dist/packem_chunks/handler16.mjs +1 -80
  20. package/dist/packem_chunks/handler17.mjs +1 -43
  21. package/dist/packem_chunks/handler18.mjs +1 -105
  22. package/dist/packem_chunks/handler19.mjs +7 -170
  23. package/dist/packem_chunks/handler2.mjs +1 -114
  24. package/dist/packem_chunks/handler20.mjs +2 -94
  25. package/dist/packem_chunks/handler21.mjs +1 -94
  26. package/dist/packem_chunks/handler22.mjs +3 -0
  27. package/dist/packem_chunks/handler23.mjs +3 -0
  28. package/dist/packem_chunks/handler24.mjs +1 -0
  29. package/dist/packem_chunks/handler25.mjs +2 -0
  30. package/dist/packem_chunks/handler26.mjs +1 -0
  31. package/dist/packem_chunks/handler27.mjs +99 -0
  32. package/dist/packem_chunks/handler28.mjs +15 -0
  33. package/dist/packem_chunks/handler29.mjs +1 -0
  34. package/dist/packem_chunks/handler3.mjs +1 -204
  35. package/dist/packem_chunks/handler4.mjs +1 -33
  36. package/dist/packem_chunks/handler5.mjs +3 -49
  37. package/dist/packem_chunks/handler6.mjs +1 -91
  38. package/dist/packem_chunks/handler7.mjs +1 -42
  39. package/dist/packem_chunks/handler8.mjs +1 -174
  40. package/dist/packem_chunks/handler9.mjs +2 -315
  41. package/dist/packem_chunks/runCodegenCommand.mjs +4 -52
  42. package/dist/packem_chunks/runDevCommand.mjs +6 -0
  43. package/dist/packem_chunks/runInitCommand.mjs +212 -1432
  44. package/dist/packem_chunks/runResetCommand.mjs +1 -41
  45. package/dist/packem_chunks/runRpcCommand.mjs +1 -68
  46. package/dist/packem_shared/DEFAULT_IMPORT_BATCH_SIZE-5xxKz-zB.mjs +1 -0
  47. package/dist/packem_shared/EXIT_CODE-DVjUgIaG.mjs +1 -0
  48. package/dist/packem_shared/admin-token-VdUnvnKW.mjs +1 -0
  49. package/dist/packem_shared/admin-url-DJB8P6Jh.mjs +1 -0
  50. package/dist/packem_shared/api-spec-lYyq4vL6.mjs +1 -0
  51. package/dist/packem_shared/binding-manifest-file-CQ1eQYy1.mjs +2 -0
  52. package/dist/packem_shared/buildRegistryIndex-Dc8D8AR6.mjs +1 -0
  53. package/dist/packem_shared/cap-error-body-YBKO32BF.mjs +1 -0
  54. package/dist/packem_shared/catalog-CahzmDLV.mjs +1 -0
  55. package/dist/packem_shared/cli-manifest-DoQ_3bzs.mjs +1 -0
  56. package/dist/packem_shared/cloudflare-credentials-BVmZnDZk.mjs +1 -0
  57. package/dist/packem_shared/codegen-error-AmH54ofi.mjs +3 -0
  58. package/dist/packem_shared/command-B6vu47iR.mjs +1 -0
  59. package/dist/packem_shared/commands-DC3ZX1yk.mjs +13 -0
  60. package/dist/packem_shared/createLogger-4TujevyB.mjs +2 -0
  61. package/dist/packem_shared/createRecordingSpawner-01ZngIwK.mjs +1 -0
  62. package/dist/packem_shared/deploy-target-resolve-BTCW1Iv4.mjs +1 -0
  63. package/dist/packem_shared/diffSnapshots-htqNZT8Q.mjs +7 -0
  64. package/dist/packem_shared/docker-DpVxvYpL.mjs +1 -0
  65. package/dist/packem_shared/free-port-DjbFhaaT.mjs +1 -0
  66. package/dist/packem_shared/health-probe-Bw9JFfIy.mjs +1 -0
  67. package/dist/packem_shared/import-DoAUdyfU.mjs +13 -0
  68. package/dist/packem_shared/index-oHZBGBwB.mjs +1 -0
  69. package/dist/packem_shared/insertSchemaExtension-CbTH3lCQ.mjs +10 -0
  70. package/dist/packem_shared/lint-ignore-report-DKZagpqk.mjs +2 -0
  71. package/dist/packem_shared/mcp-config-file-GR1Y_CVw.mjs +2 -0
  72. package/dist/packem_shared/open-url-EnKy--w-.mjs +1 -0
  73. package/dist/packem_shared/output-format-DWe_tkTR.mjs +2 -0
  74. package/dist/packem_shared/parseManifest-CwPTKdtS.mjs +1 -0
  75. package/dist/packem_shared/path-containment-CgxYZggb.mjs +1 -0
  76. package/dist/packem_shared/plan-BH4ZfiFs.mjs +5 -0
  77. package/dist/packem_shared/planDevCommand-I8-NB_CN.mjs +1 -0
  78. package/dist/packem_shared/platform-diagnostics-Cn2g6Jh-.mjs +4 -0
  79. package/dist/packem_shared/prompt-cancelled-BvsNxg_Q.mjs +1 -0
  80. package/dist/packem_shared/render-lunora-error-DUo0pA61.mjs +2 -0
  81. package/dist/packem_shared/resolve-Bdc9KEUv.mjs +5 -0
  82. package/dist/packem_shared/resolve-target-C_ZloTvd.mjs +1 -0
  83. package/dist/packem_shared/runAddCommand-CTblwN9O.mjs +1 -0
  84. package/dist/packem_shared/runDeployCommand-C5L6ZhGV.mjs +1 -0
  85. package/dist/packem_shared/runExportCommand-DiMCQ-Il.mjs +5 -0
  86. package/dist/packem_shared/runMigrateGenerateCommand-Bd2W9t1k.mjs +11 -0
  87. package/dist/packem_shared/schemaIrToSnapshot-BHuNOfgs.mjs +1 -0
  88. package/dist/packem_shared/shared-D-zCOmgY.mjs +1 -0
  89. package/dist/packem_shared/storage-BLLoZRJa.mjs +1 -0
  90. package/dist/packem_shared/tui-prompts-B3YwUhGw.mjs +4 -0
  91. package/dist/packem_shared/variables-CPYyJtsL.mjs +1 -0
  92. package/dist/packem_shared/vectorize-metadata-DOB1akh6.mjs +2 -0
  93. package/dist/packem_shared/wrangler-name-Dsk5K1f-.mjs +1 -0
  94. package/dist/packem_shared/wrangler-secrets-DlSlWvRH.mjs +1 -0
  95. package/package.json +43 -19
  96. package/skills/README.md +36 -17
  97. package/skills/lunora/SKILL.md +158 -65
  98. package/skills/lunora-cloudflare-audit/SKILL.md +163 -0
  99. package/skills/lunora-create-package/SKILL.md +97 -84
  100. package/skills/lunora-deploy/SKILL.md +175 -105
  101. package/skills/lunora-functions/SKILL.md +367 -60
  102. package/skills/lunora-migration-helper/SKILL.md +168 -137
  103. package/skills/lunora-performance-audit/SKILL.md +137 -95
  104. package/skills/lunora-quickstart/SKILL.md +136 -136
  105. package/skills/lunora-realtime/SKILL.md +217 -97
  106. package/skills/lunora-setup-auth/SKILL.md +203 -101
  107. package/skills/lunora-setup-hyperdrive/SKILL.md +138 -109
  108. package/skills/lunora-setup-hyperdrive-global/SKILL.md +125 -92
  109. package/skills/lunora-setup-mail/SKILL.md +85 -113
  110. package/skills/lunora-setup-scheduler/SKILL.md +132 -96
  111. package/skills/lunora-setup-storage/SKILL.md +189 -95
  112. package/dist/packem_chunks/planDevCommand.mjs +0 -541
  113. package/dist/packem_chunks/runDeployCommand.mjs +0 -594
  114. package/dist/packem_chunks/runMigrateGenerateCommand.mjs +0 -397
  115. package/dist/packem_shared/COMMANDS-B0ftFD_3.mjs +0 -948
  116. package/dist/packem_shared/DEFAULT_IMPORT_BATCH_SIZE-Ck-2bU08.mjs +0 -244
  117. package/dist/packem_shared/admin-url-4UzT-CI4.mjs +0 -19
  118. package/dist/packem_shared/api-spec-CtA6ilu4.mjs +0 -13
  119. package/dist/packem_shared/buildRegistryIndex-BcYe607_.mjs +0 -38
  120. package/dist/packem_shared/codegen-error-DJG-ghs_.mjs +0 -31
  121. package/dist/packem_shared/command-D3lB_4Az.mjs +0 -19
  122. package/dist/packem_shared/commands-B-gR09Z_.mjs +0 -845
  123. package/dist/packem_shared/createLogger-B40gPzQo.mjs +0 -78
  124. package/dist/packem_shared/createRecordingSpawner-DxI3mebw.mjs +0 -43
  125. package/dist/packem_shared/detect-package-manager-DYp7n3mJ.mjs +0 -61
  126. package/dist/packem_shared/diffSnapshots-BeDvvNiF.mjs +0 -161
  127. package/dist/packem_shared/docker-hMQ97KSQ.mjs +0 -21
  128. package/dist/packem_shared/insertSchemaExtension-DAqbfr9Z.mjs +0 -64
  129. package/dist/packem_shared/open-url-Dfq6fAyT.mjs +0 -41
  130. package/dist/packem_shared/output-format-wUvAN6AL.mjs +0 -17
  131. package/dist/packem_shared/parseArgs-YXFuKdEk.mjs +0 -56
  132. package/dist/packem_shared/parseManifest--vZf2FY1.mjs +0 -94
  133. package/dist/packem_shared/prompt-cancelled-APzX1Im-.mjs +0 -9
  134. package/dist/packem_shared/resolve-target-qbsJ_5sF.mjs +0 -16
  135. package/dist/packem_shared/runAddCommand-bnY6-HKb.mjs +0 -4
  136. package/dist/packem_shared/schema-drift-gate-BtBt0as0.mjs +0 -79
  137. package/dist/packem_shared/schemaIrToSnapshot-DdsljJT-.mjs +0 -43
  138. package/dist/packem_shared/storage-BIsph-Vk.mjs +0 -84
  139. package/dist/packem_shared/tui-prompts-BjEN8XgP.mjs +0 -658
  140. package/dist/packem_shared/wrangler-name-cy4yhm9j.mjs +0 -12
  141. package/dist/packem_shared/wrangler-secrets-P2_ZUR-k.mjs +0 -47
@@ -1,90 +1,65 @@
1
1
  ---
2
2
  name: lunora-setup-hyperdrive
3
- description: Connects an existing Postgres/MySQL database to a Lunora app from an action via Cloudflare Hyperdrive. Use for `@lunora/hyperdrive`, `ctx.sql`, `createHyperdrive` + `fromPostgresJs`/`fromNodePg`/`fromMysql2`, the `HYPERDRIVE` binding, `wrangler hyperdrive create`, and the action-only / non-reactive guardrails (live queries don't track external writes).
3
+ description: Connects an existing, externally owned Postgres/MySQL database to a Lunora app through Cloudflare Hyperdrive. Lunora does not own its schema. Covers `@lunora/hyperdrive`, the action-only, non-reactive `ctx.sql` client (`createHyperdrive` + `fromPostgresJs`/`fromNodePg`/`fromMysql2`, wired with the app builder's `.hyperdrive()`), the `HYPERDRIVE` binding and `wrangler hyperdrive create`, and making external rows live through a projection or a `.source()` ingest table. Use when the user wants to "query our existing Postgres", "read the legacy DB from an action", "connect Neon/RDS/Supabase", or sync external rows into Lunora, or sees `ctx.sql` / `hyperdrive_outside_action` errors. For storing Lunora's own `.global()` tables in Postgres/MySQL with reactive live queries, use `lunora-setup-hyperdrive-global` instead.
4
4
  ---
5
5
 
6
6
  # Lunora Setup Hyperdrive
7
7
 
8
- Wire an **existing** Postgres/MySQL database into a Lunora app using
9
- `@lunora/hyperdrive`, which surfaces a [Cloudflare
10
- Hyperdrive](https://developers.cloudflare.com/hyperdrive/) binding's connection
11
- string and a driver-agnostic `ctx.sql` client — usable **only inside an
12
- `action`**.
13
-
14
- > **Integrate, don't replace.** Hyperdrive talks to a database Lunora has no
15
- > visibility into. Queries through `ctx.sql` are **non-deterministic** (forbidden
16
- > in `query`/`mutation`, enforced by the `hyperdrive_outside_action` advisor
17
- > lint) and external writes are **invisible to Lunora live queries** —
18
- > subscriptions will NOT re-run when external rows change. Use it to read/write a
19
- > legacy DB from an action; if you want that data reactive, write a projection
20
- > into a `defineSchema` DO/D1 table.
21
-
22
- ## When to Use
23
-
24
- - Reading/writing an existing Postgres or MySQL database from a Lunora action.
25
- - Incrementally migrating off a legacy DB while standing up Lunora alongside it.
26
- - Backfilling/syncing external rows into reactive Lunora tables (projection).
27
-
28
- ## When Not to Use
29
-
30
- - You want the external data to be **reactive** out of the box — it can't be;
31
- Hyperdrive writes don't hit the change-feed. Project into a `defineSchema`
32
- table instead.
33
- - You're reaching for `ctx.sql` inside a `query` or `mutation` — forbidden by
34
- design (the advisor lint flags it). Move the access into an `action`.
35
- - The app has no Lunora backend yet — use `lunora-quickstart` first.
36
- - You'd be replacing `defineSchema` entirely — then you lose realtime, OCC,
37
- optimistic updates, the offline queue, and the advisors. Use Lunora's own data
38
- layer for app state.
39
-
40
- ## Workflow
41
-
42
- 1. Install `@lunora/hyperdrive` and one SQL driver.
43
- 2. Create the Hyperdrive config (`wrangler hyperdrive create`) and add the
44
- `HYPERDRIVE` binding to `wrangler.jsonc`.
45
- 3. Regenerate types with `lunora codegen` so `ctx.sql` appears on `ActionCtx`.
46
- 4. Read/write the external DB from an action via `ctx.sql`.
47
- 5. (Optional) Project external rows into a `defineSchema` table to make them
48
- reactive.
49
-
50
- ## Step 1: Install the package + a driver
8
+ `@lunora/hyperdrive` exposes a Cloudflare Hyperdrive binding as a
9
+ driver-agnostic `ctx.sql` client that is available only inside an `action`. It integrates
10
+ a database Lunora doesn't own. Don't use it to replace `defineSchema`, because you'd lose
11
+ realtime, OCC, optimistic updates, the offline queue and the advisors.
12
+
13
+ > **`ctx.sql` is not reactive.** Lunora cannot see this database, so subscriptions
14
+ > do not re-run when its rows change, including rows you write through `ctx.sql`.
15
+ > Queries are non-deterministic, so `ctx.sql` exists only on `ActionCtx`.
16
+ > The `hyperdrive_outside_action` advisor lint flags any use in a
17
+ > `query`/`mutation`. To make the data live, see Step 5. If you instead want Lunora-owned tables
18
+ > stored in Postgres/MySQL and fully reactive, use `lunora-setup-hyperdrive-global`.
19
+
20
+ ## Step 1: Install the package + one driver
51
21
 
52
22
  ```bash
53
23
  pnpm add @lunora/hyperdrive
54
- # choose ONE driver (optional peer deps — none is bundled):
55
- pnpm add postgres # postgres.js → fromPostgresJs ($1, $2 placeholders)
56
- # or
57
- pnpm add pg # node-postgres → fromNodePg ($1, $2 placeholders)
58
- # or
59
- pnpm add mysql2 # mysql2 → fromMysql2 (? placeholders)
24
+ pnpm add postgres # → fromPostgresJs ($1, $2 placeholders)
25
+ # or: pnpm add pg → fromNodePg ($1, $2)
26
+ # or: pnpm add mysql2 → fromMysql2 (?)
60
27
  ```
61
28
 
29
+ The drivers are optional peer dependencies, so none ships with the package.
30
+ The package never rewrites SQL, so use your driver's native placeholders.
31
+
62
32
  ## Step 2: Create the binding
63
33
 
34
+ Nothing to write by hand: `lunora dev` and `lunora deploy` add a `HYPERDRIVE` binding
35
+ to `wrangler.jsonc` with a placeholder `id`.
36
+
37
+ - **Dev.** `ctx.sql` runs on an embedded Postgres (PGlite), data in
38
+ `.wrangler/state/lunora-pglite`. Install its two optional packages:
39
+
64
40
  ```bash
65
- wrangler hyperdrive create my-db --connection-string="postgres://user:pass@host:5432/db" # gitleaks:allow -- placeholder, not a real secret
41
+ pnpm add -D @electric-sql/pglite @electric-sql/pglite-socket
66
42
  ```
67
43
 
68
- This prints an `id`. Add the binding to `wrangler.jsonc`, using
69
- `localConnectionString` so `lunora dev` connects to your DB directly (no edge
70
- proxy locally):
71
-
72
- ```jsonc
73
- {
74
- "hyperdrive": [
75
- {
76
- "binding": "HYPERDRIVE",
77
- "id": "<the id from the command above>",
78
- "localConnectionString": "postgres://user:pass@localhost:5432/db", // gitleaks:allow -- placeholder, not a real secret
79
- },
80
- ],
81
- }
44
+ To use your own database, set `localConnectionString` on the binding or
45
+ `CLOUDFLARE_HYPERDRIVE_LOCAL_CONNECTION_STRING_<BINDING>`; neither is overridden.
46
+
47
+ - **Deploy.** Give `lunora deploy` the origin and a token that may edit Hyperdrive:
48
+
49
+ ```bash
50
+ LUNORA_HYPERDRIVE_URL="postgres://user:pass@host:5432/db" lunora deploy # gitleaks:allow -- placeholder, not a real secret
82
51
  ```
83
52
 
84
- Lunora validates this binding (errors if `binding` is missing; warns if `id` is
85
- empty/placeholder — it can't connect), but it does **not** auto-write the `id`:
86
- that's a remote resource only `wrangler hyperdrive create` can mint. Importing
87
- `@lunora/hyperdrive` surfaces a hint reminding you to add it.
53
+ (`CLOUDFLARE_API_TOKEN` with Hyperdrive: Edit, and `CLOUDFLARE_ACCOUNT_ID`; the variable is
54
+ `LUNORA_<BINDING>_URL`.) Lunora creates the config, or reuses one of the same name, and writes
55
+ its `id` into `wrangler.jsonc`. The connection string is never written or logged.
56
+
57
+ To manage the config yourself, run `wrangler hyperdrive create my-db --connection-string=...`
58
+ and put the `id` in the binding. Validation fails without a `binding`.
59
+
60
+ Hyperdrive caches read results (up to 60 s plus 15 s stale by default) and a write does not
61
+ invalidate them. For reads that must see your own writes, add a second config created with
62
+ `--caching-disabled` and bind it under another name.
88
63
 
89
64
  ## Step 3: Regenerate types
90
65
 
@@ -92,63 +67,117 @@ that's a remote resource only `wrangler hyperdrive create` can mint. Importing
92
67
  lunora codegen
93
68
  ```
94
69
 
95
- When codegen sees `ctx.sql` used, it adds `sql: SqlClient` to **`ActionCtx`
96
- only** — never `QueryCtx`/`MutationCtx` — with a JSDoc restating the
97
- determinism/realtime caveat.
70
+ When `ctx.sql` is used, codegen adds `readonly sql: SqlClient` to `ActionCtx` only. It also
71
+ emits a `.hyperdrive()` method on the generated app builder.
98
72
 
99
- ## Step 4: Use `ctx.sql` from an action
73
+ ## Step 4: Wire the client on the app builder
100
74
 
101
- ```ts
75
+ `ctx.sql` is `readonly`, so assigning it in a handler is a `TS2540` error.
76
+ Codegen can't build the client because it depends on your driver, so supply a
77
+ thunk instead. It runs once per shard construction, not per request:
78
+
79
+ ```ts title="src/server.ts"
80
+ import type { HyperdriveLike } from "@lunora/hyperdrive";
102
81
  import { createHyperdrive, fromPostgresJs } from "@lunora/hyperdrive";
103
82
  import postgres from "postgres";
104
83
 
105
- import { action, v } from "@lunora/server";
84
+ import { defineApp } from "../lunora/_generated/app.js";
106
85
 
107
- export const listLegacyOrders = action.input({ orgId: v.string() }).action(async ({ ctx, args: { orgId } }) => {
108
- const { connectionString } = createHyperdrive(ctx.env.HYPERDRIVE);
109
- ctx.sql = fromPostgresJs(postgres(connectionString));
86
+ const app = defineApp<Env>()
87
+ .shard((env) => env.SHARD)
88
+ .hyperdrive((env) => fromPostgresJs(postgres(createHyperdrive(env.HYPERDRIVE as HyperdriveLike).connectionString, { fetch_types: false, max: 5 })))
89
+ .build();
110
90
 
111
- return ctx.sql.query<{ id: string; total: number }>("select id, total from orders where org = $1", [orgId]);
91
+ export const ShardDO = app.ShardDO;
92
+ ```
93
+
94
+ Then query from any action:
95
+
96
+ ```ts
97
+ import { LunoraError } from "lunorash/server";
98
+
99
+ import { action, v } from "#lunora/_generated/server.js";
100
+
101
+ export const listLegacyOrders = action.input({ orgId: v.string() }).action(async ({ ctx, args: { orgId } }) => {
102
+ if (!ctx.auth.userId) {
103
+ throw new LunoraError("UNAUTHORIZED", "not signed in");
104
+ }
105
+
106
+ // `orgId` comes from the client: the join scopes it to orgs the caller belongs to.
107
+ return ctx.sql.query<{ id: string; total: number }>(
108
+ "select o.id, o.total from orders o join org_members m on m.org_id = o.org where o.org = $1 and m.user_id = $2",
109
+ [orgId, ctx.auth.userId],
110
+ );
112
111
  });
113
112
  ```
114
113
 
115
- The package never rewrites SQL — use your driver's native placeholders (`$1` for
116
- Postgres, `?` for MySQL).
114
+ **Verify:** call the action from the studio's function runner and check that it returns rows.
117
115
 
118
- ## Step 5 (optional): Make it reactive via a projection
116
+ ## Step 5 (optional): Make external data reactive
119
117
 
120
- External writes don't hit the change-feed, so a subscription won't re-fire on a
121
- Postgres change. To make external data reactive, write a projection into a
122
- `defineSchema` table from the same action — that write _is_ tracked:
118
+ Pick one approach:
119
+
120
+ **A. Projection from an action.** This works when your app is the writer. Write the row into a
121
+ `defineSchema` table through a mutation. That write is tracked:
123
122
 
124
123
  ```ts
124
+ import { LunoraError } from "lunorash/server";
125
+
126
+ import { internal } from "#lunora/_generated/internal.js";
127
+
125
128
  const [row] = await ctx.sql.query<{ id: string; total: number }>("select id, total from orders where id = $1", [id]);
126
129
 
127
- // This write re-runs live queries reading `orders`:
128
- await ctx.runMutation("orders:upsert", { id: row.id, total: row.total });
130
+ if (!row) {
131
+ throw new LunoraError("NOT_FOUND", `order ${id} not found`);
132
+ }
133
+
134
+ // Re-runs live queries over `orders`. ctx.run* takes a generated reference, not a "file:fn" string.
135
+ await ctx.runMutation(internal.orders.upsert, { id: row.id, total: row.total });
129
136
  ```
130
137
 
138
+ **B. A `.source()` table.** This works when other systems also write the database. Lunora polls
139
+ the external query on the shard's alarm, diffs the result, materializes it into the
140
+ DO's SQLite and notifies subscribers, so you don't write an action or a cron:
141
+
142
+ ```ts
143
+ // lunora/schema.ts
144
+ messages: defineTable({ body: v.string(), channelId: v.string() })
145
+ .shardBy("channelId")
146
+ .source({
147
+ binding: "HYPERDRIVE",
148
+ query: 'select id, body, channel_id as "channelId" from messages where channel_id = $1',
149
+ tenantBy: (shardKey) => [shardKey], // required under .shardBy(): the tenant boundary
150
+ }),
151
+ ```
152
+
153
+ ```ts
154
+ // worker entry: one client per binding name
155
+ .sourceClient((env, binding) => fromPostgresJs(postgres((env[binding] as { connectionString: string }).connectionString, { fetch_types: false, max: 5 })))
156
+ ```
157
+
158
+ How `.source()` behaves:
159
+
160
+ - Freshness depends on the poll interval: every alarm tick by default,
161
+ `refresh: { everyMs }` to throttle it, or `"manual"`.
162
+ - The table is read-only from Lunora: rows come only from the ingest loop.
163
+ - The default `mode: "full-pull"` re-reads the whole slice on every tick, which
164
+ works up to about 10k rows. For larger slices, use `mode: "incremental"` with a
165
+ `cursor`, plus either `reconcileEveryMs` or `softDeleteColumn` so deletes are
166
+ still detected.
167
+ - A source table can't also be `.global()`.
168
+
131
169
  ## Common Pitfalls
132
170
 
133
- 1. **`ctx.sql` in a query/mutation.** It's action-only by design; the
134
- `hyperdrive_outside_action` advisor lint flags it. Move the SQL into an
135
- action.
136
- 2. **Expecting external writes to be reactive.** They aren't — Lunora can't see
137
- them. Project into a `defineSchema` table if you need reactivity.
138
- 3. **Placeholder / empty `id`.** A Hyperdrive binding with no real `id` can't
139
- connect (Lunora warns). Run `wrangler hyperdrive create` and paste the id.
140
- 4. **Bundling a driver as a hard dependency.** Drivers are optional peer deps —
141
- install the one you use; don't assume one ships with the package.
142
- 5. **Wrong placeholder syntax.** `$1, $2` for Postgres drivers, `?` for mysql2.
143
- The package does not rewrite SQL.
144
-
145
- ## Checklist
146
-
147
- - [ ] `@lunora/hyperdrive` + one driver (`postgres`/`pg`/`mysql2`) installed.
148
- - [ ] `wrangler hyperdrive create` run; `HYPERDRIVE` binding added to
149
- `wrangler.jsonc` with a real `id` and a `localConnectionString` for dev.
150
- - [ ] `lunora codegen` run so `ctx.sql` is typed on `ActionCtx`.
151
- - [ ] All `ctx.sql` access lives in actions (advisor clean — no
152
- `hyperdrive_outside_action`).
153
- - [ ] If reactivity is needed, external rows are projected into a `defineSchema`
154
- table.
171
+ 1. **`ctx.sql` in a query/mutation.** It's action-only. Move the SQL into an action.
172
+ 2. **Expecting `ctx.sql` writes to update subscriptions.** They don't. Use Step 5.
173
+ 3. **Placeholder `id` at deploy.** The deploy stops until `LUNORA_<BINDING>_URL` is set
174
+ (Lunora then creates the config) or the real `id` is pasted in. An empty `id` is not
175
+ provisioned: wrangler rejects it, so remove the entry and let Lunora add it again.
176
+ 4. **Wrong placeholders.** Use `$1` for `postgres`/`pg` and `?` for `mysql2`.
177
+ 5. **Array or JSON params on postgres.js with `fetch_types: false`.** A JS array bound to
178
+ `= ANY($1)` fails as a malformed array literal: bind `JSON.stringify(list)` to
179
+ `IN (SELECT jsonb_array_elements_text($1::text::jsonb))`. Cast JSON params `$1::text::jsonb`,
180
+ since a bare `$1::jsonb` gets your string JSON-encoded twice.
181
+ 6. **Unscoped `.source()` under `.shardBy()`.** Without `tenantBy`, every tenant's
182
+ rows are copied into each shard. `defineSchema` throws, and the
183
+ `external_source_unscoped` lint fails the build.
@@ -1,66 +1,63 @@
1
1
  ---
2
2
  name: lunora-setup-hyperdrive-global
3
- description: Use PlanetScale (Postgres/MySQL via Cloudflare Hyperdrive) as a first-class, REACTIVE `.global()` storage backend for a Lunora app — and migrate an existing D1 `.global()` dataset to it. Use for `@lunora/hyperdrive/global`, `.global({ backend: "hyperdrive" })`, `createHyperdriveGlobalCtxDb`, the app-builder `.hyperdriveGlobal()` declaration, the `HYPERDRIVE` binding pointing at PlanetScale, and `lunora migrate d1-to-hyperdrive`. NOT the same as `@lunora/hyperdrive` (which is an action-only, non-reactive `ctx.sql`).
3
+ description: Stores a Lunora app's own `.global()` tables in Postgres or MySQL (PlanetScale, Neon, RDS, or any Hyperdrive-reachable database) through Cloudflare Hyperdrive instead of D1. Lunora owns the schema, and live queries stay fully reactive. Also migrates an existing D1 `.global()` dataset with `lunora migrate d1-to-hyperdrive`. Covers `.global({ backend: "hyperdrive" })`, `@lunora/hyperdrive/global` (`buildPgExec` / `buildMysqlExec`, `createHyperdriveGlobalCtxDb`), the app builder's `.hyperdriveGlobal({ engine, exec })`, and the `HYPERDRIVE` binding. Use when the user wants to "move global tables to PlanetScale/Postgres", "use Postgres instead of D1", or "migrate off D1", or edits `.global({ backend: "hyperdrive" })`. For querying an existing, externally owned database from an action (non-reactive `ctx.sql`), use `lunora-setup-hyperdrive` instead.
4
4
  ---
5
5
 
6
6
  # Lunora Setup: Hyperdrive Global Backend (Postgres/MySQL)
7
7
 
8
- Store Lunora `.global()` tables in a **Postgres or MySQL database reached
9
- through [Cloudflare Hyperdrive](https://developers.cloudflare.com/hyperdrive/)**
10
- — **PlanetScale**, Neon, RDS, or any Hyperdrive-reachable database — instead of
11
- D1, as a **drop-in, fully reactive** backend. (PlanetScale on Cloudflare is the
12
- headline use case; nothing here is PlanetScale-specific.)
8
+ Store `.global()` (cross-tenant) tables in a Postgres or MySQL database reached
9
+ through Cloudflare Hyperdrive instead of D1. PlanetScale on Cloudflare is the
10
+ typical case, but nothing here is PlanetScale-specific.
13
11
 
14
- > **`@lunora/hyperdrive/global` vs. `@lunora/hyperdrive` (`ctx.sql`).** The
15
- > `ctx.sql` surface is an action-only, **non-reactive** escape hatch for
16
- > integrating a legacy database. `@lunora/hyperdrive/global` is different: Lunora
17
- > **owns** the schema (column-per-field, like D1) and routes every write through
18
- > its own store core, so live queries stay reactive — the writer is injected as
19
- > `globalDb` and the shard DO's broadcast hook re-runs subscriptions exactly as
20
- > with D1.
21
-
22
- ## When to Use
23
-
24
- - You want `.global()` (cross-tenant) tables to live in PlanetScale rather than
25
- D1 — e.g. for larger datasets, existing SQL tooling, or unified Cloudflare
26
- billing.
27
- - You're migrating an existing D1 `.global()` dataset to PlanetScale.
12
+ > **This is not `ctx.sql`.** `@lunora/hyperdrive`'s `ctx.sql`
13
+ > (`lunora-setup-hyperdrive`) is an action-only, non-reactive client for a
14
+ > database Lunora doesn't own. `.global({ backend: "hyperdrive" })` is the opposite:
15
+ > Lunora owns the tables (one column per field, like D1), and every write goes
16
+ > through Lunora's store core. Subscriptions re-run exactly as they do with D1.
17
+ > The live-query guarantee covers only writes made through Lunora (`ctx.db`).
18
+ > Rows written to the database directly are invisible to live queries.
28
19
 
29
20
  ## When Not to Use
30
21
 
31
- - You only need to read/write a **legacy** external DB from an action — use
32
- `@lunora/hyperdrive` (`ctx.sql`) instead; it's lighter and action-scoped.
33
- - Your global data is small and D1 is fine — bare `.global()` (D1) needs no
34
- extra binding or driver.
22
+ - You only need to read or write a legacy database from an action: use
23
+ `lunora-setup-hyperdrive`.
24
+ - Your global data is small and D1 is fine: bare `.global()` needs no extra
25
+ binding or driver.
35
26
 
36
27
  ## How it works
37
28
 
38
- PlanetScale reuses Lunora's dialect-parameterized store core (the same one D1
39
- uses): a `SqlDialect` (Postgres or MySQL) shapes the SQL, and a Hyperdrive-backed
40
- `SqlExec` runs it from inside the Durable Object that hosts the global writer.
41
- Values are stored SQLite-shaped (boolean → 1/0, JSON → text/json, bigint →
42
- decimal), so the value codec is shared with D1.
29
+ The same dialect-parameterized store core that backs D1 runs here with a
30
+ Postgres or MySQL `SqlDialect`. A Hyperdrive-backed `SqlExec` executes it from
31
+ inside the Durable Object that hosts the global writer. Values are stored in the
32
+ same shapes D1 uses (boolean → 1/0, JSON → text/json, bigint → decimal).
43
33
 
44
- ## Step 1: Install the package + a driver
34
+ ## Step 1: Install
45
35
 
46
36
  ```bash
47
- pnpm add @lunora/hyperdrive/global
48
- # choose the driver matching your engine (optional peer deps — none is bundled):
49
- pnpm add postgres # PlanetScale Postgres → fromPostgresJs
50
- # or
51
- pnpm add mysql2 # PlanetScale MySQL → mysql2/promise
37
+ pnpm add @lunora/hyperdrive @lunora/sql-store
38
+ pnpm add postgres # Postgres engine → buildPgExec(fromPostgresJs(...))
39
+ # or: pnpm add mysql2 → buildMysqlExec(mysql2/promise pool), MySQL 8.0+
52
40
  ```
53
41
 
54
- ## Step 2: Create the Hyperdrive binding pointing at PlanetScale
42
+ `@lunora/hyperdrive/global` is a subpath export of `@lunora/hyperdrive`, not a
43
+ separate package. Codegen requires `@lunora/sql-store` too, because the generated
44
+ `_generated/app.ts` imports its `SqlExec` types. Install the driver that matches
45
+ your engine. Drivers are optional peer dependencies.
55
46
 
56
- Create a PlanetScale database from the Cloudflare dashboard (unified billing),
57
- then a Hyperdrive config over its connection string:
47
+ ## Step 2: Create the Hyperdrive binding
58
48
 
59
49
  ```bash
60
- wrangler hyperdrive create my-db --connection-string="postgres://user:pass@host/db" # gitleaks:allow -- placeholder
50
+ wrangler hyperdrive create my-db --connection-string="postgres://user:pass@host/db" --caching-disabled # gitleaks:allow -- placeholder
61
51
  ```
62
52
 
63
- Add the binding to `wrangler.jsonc` (use `localConnectionString` for `lunora dev`):
53
+ **Query caching must be off** for the config behind `.global()`. Hyperdrive
54
+ caches reads by default (60 s plus 15 s stale-while-revalidate) and a write does
55
+ not invalidate them, so the store would read its own rows stale: OCC retries
56
+ reread the cached version and fail with spurious conflicts, and live queries
57
+ re-run onto old data. For a config that already exists, run
58
+ `wrangler hyperdrive update <id> --caching-disabled`. When `lunora deploy`
59
+ creates the config from `LUNORA_<BINDING>_URL`, it turns caching off itself. For a
60
+ binding that already has an id, it only warns (given Cloudflare API credentials).
64
61
 
65
62
  ```jsonc
66
63
  {
@@ -70,10 +67,14 @@ Add the binding to `wrangler.jsonc` (use `localConnectionString` for `lunora dev
70
67
  }
71
68
  ```
72
69
 
73
- > **Read-your-writes:** point the Hyperdrive config at the **primary** (or pin
74
- > writes to it) so a write then immediate read isn't served by a stale replica.
70
+ A binding is required. Without any `hyperdrive` entry, config validation errors
71
+ and `lunora dev` / `lunora deploy` won't start. You choose the name: the
72
+ validator only checks that at least one binding exists, and your `exec` selector picks it.
73
+
74
+ Point Hyperdrive at the primary (or pin writes to it) so you can read your own
75
+ writes. A replica can serve a stale read right after a write.
75
76
 
76
- ## Step 3: Mark tables `.global({ backend: "hyperdrive" })`
77
+ ## Step 3: Mark tables
77
78
 
78
79
  ```ts
79
80
  // lunora/schema.ts
@@ -84,88 +85,120 @@ export default defineSchema({
84
85
  });
85
86
  ```
86
87
 
87
- A bare `.global()` stays on D1. **One backend per app** — mixing D1- and
88
- PlanetScale-backed global tables in the same app isn't supported yet (codegen
89
- errors clearly).
88
+ Bare `.global()` stays on D1. An app can use only one global backend: codegen fails
89
+ if the schema mixes D1 and Hyperdrive global tables.
90
90
 
91
- ## Step 4: Wire `.hyperdriveGlobal()` in the app composition
91
+ ## Step 4: Wire `.hyperdriveGlobal()`
92
92
 
93
- Codegen emits a `.hyperdriveGlobal()` builder method. Supply the engine and an `exec`
94
- built from the Hyperdrive binding (cache the driver on the DO instance; rebuild
95
- lazily after hibernation):
93
+ Codegen emits a `.hyperdriveGlobal({ engine, exec, origin? })` builder method.
94
+ `exec` builds a `SqlExec` from `env`:
96
95
 
97
96
  ```ts
98
- import { buildPgExec } from "@lunora/hyperdrive/global";
99
97
  import { fromPostgresJs } from "@lunora/hyperdrive";
98
+ import { buildPgExec } from "@lunora/hyperdrive/global";
100
99
  import postgres from "postgres";
101
100
 
102
- export default defineApp<Env>()
101
+ import { defineApp } from "../lunora/_generated/app.js";
102
+
103
+ const app = defineApp<Env>()
103
104
  .shard((env) => env.SHARD)
104
105
  .hyperdriveGlobal({
105
106
  engine: "postgres",
106
- exec: (env) => buildPgExec(fromPostgresJs(postgres(env.HYPERDRIVE.connectionString))),
107
- });
107
+ exec: (env) => buildPgExec(fromPostgresJs(postgres(env.HYPERDRIVE.connectionString, { fetch_types: false, max: 5 }))),
108
+ })
109
+ .build();
110
+
111
+ export const ShardDO = app.ShardDO;
108
112
  ```
109
113
 
110
- For MySQL, create the pool with the **`FOUND_ROWS`** flag (the optimistic-concurrency
111
- guard needs matched-row counts, or idempotent writes raise spurious conflicts):
114
+ For MySQL, create the pool with the `FOUND_ROWS` flag. The optimistic-concurrency
115
+ guard needs matched-row counts. Without the flag, an idempotent write reports
116
+ zero affected rows and raises a spurious conflict. `buildMysqlExec` throws at
117
+ construction when the flag is missing. If it can't read the flag, it warns once instead.
112
118
 
113
119
  ```ts
114
120
  import { buildMysqlExec } from "@lunora/hyperdrive/global";
115
121
  import mysql from "mysql2/promise";
116
122
 
123
+ // inside the builder chain:
117
124
  .hyperdriveGlobal({
118
125
  engine: "mysql",
119
- exec: (env) => buildMysqlExec(mysql.createPool({ uri: env.HYPERDRIVE.connectionString, flags: ["FOUND_ROWS"] })),
120
- });
126
+ // disableEval: mysql2 needs it to run on Workers.
127
+ exec: (env) => buildMysqlExec(mysql.createPool({ uri: env.HYPERDRIVE.connectionString, disableEval: true, flags: ["FOUND_ROWS"] })),
128
+ })
121
129
  ```
122
130
 
123
- ## Step 5: Regenerate + run
131
+ Outside the app builder, for example in tests or a custom host,
132
+ `createHyperdriveGlobalCtxDb({ engine, exec, ... })` builds the same reactive writer
133
+ directly.
134
+
135
+ ## Step 5: Regenerate and run
124
136
 
125
137
  ```bash
126
- lunora codegen # wires config.hyperdriveGlobal and the .hyperdriveGlobal() method
138
+ lunora codegen
127
139
  lunora dev
128
140
  ```
129
141
 
130
- Tables auto-provision on first use (the runtime runs the PlanetScale DDL through
131
- the dialect — no manual migration needed), and subscriptions are reactive.
142
+ Tables provision automatically on first use, because the runtime runs the DDL
143
+ through the dialect. You don't write or commit SQL migrations. `CREATE TABLE IF NOT EXISTS` never
144
+ reshapes an existing table, though. When a column's validator starts accepting
145
+ null, the store reports the `ALTER` statement to run. MySQL tables created before
146
+ the `utf8mb4_0900_bin` column collation need a one-off
147
+ `ALTER TABLE … CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_bin`.
148
+ See `lunora-migration-helper` for schema changes.
132
149
 
133
- ## Migrating an existing D1 dataset → PlanetScale
150
+ **Verify:** subscribe to a query over a hyperdrive-backed table, write through a
151
+ mutation, and confirm that the subscription updates.
134
152
 
135
- Blue-green: deploy the new PlanetScale-backed worker alongside the old D1 one,
136
- then copy the `.global()` data:
153
+ Postgres deployments can also serve `ctx.vectors` from pgvector instead of a Vectorize
154
+ binding with `createPgVectorIndex` (from `@lunora/hyperdrive/global`). It supports
155
+ equality-only metadata filters.
156
+
157
+ ## Migrating an existing D1 dataset
158
+
159
+ The flow is blue-green: deploy the Hyperdrive-backed worker next to the D1 one,
160
+ then copy the data:
137
161
 
138
162
  ```bash
139
163
  lunora migrate d1-to-hyperdrive \
140
- --from-url https://old-d1.example.com --from-token "$D1_ADMIN_TOKEN" \
141
- --to-url https://new-ps.example.com --to-token "$PS_ADMIN_TOKEN" \
142
- --tables settings,orders # omit to move every global table
143
- # --out dump.ndjson # keep the intermediate dump to inspect
164
+ --from-url https://old-d1.example.com --from-token "$D1_ADMIN_TOKEN" \
165
+ --to-url https://new-hd.example.com --to-token "$HD_ADMIN_TOKEN" \
166
+ --tables settings,orders # omit to move every global table; --out dump.ndjson keeps the dump
144
167
  ```
145
168
 
146
- It streams the source's global rows to NDJSON, imports them into the target
147
- (whose `.global({ backend: "hyperdrive" })` tables route the writes to
148
- PlanetScale), and verifies the row counts match. Rows whose `_id` already exists
149
- are reported as conflicts, not duplicated.
150
-
151
- > **Direct external writes** to PlanetScale (bypassing Lunora) are invisible to
152
- > live queries — same as D1. Let Lunora own the writes.
169
+ The command exports the source's global rows to NDJSON, imports them into the
170
+ target, and compares row counts. It refuses to run when the source and target
171
+ resolve to the same URL, and non-localhost URLs must use https. Rows whose `_id`
172
+ already exists in the target are reported as conflicts and not duplicated.
173
+ A count mismatch is a warning: inspect it with `--out`, resolve it, and re-run.
174
+ Tokens default to `--token` / `LUNORA_ADMIN_TOKEN`.
175
+
176
+ To roll back, run `lunora migrate hyperdrive-to-d1` with the same flags, the
177
+ Hyperdrive deployment as `--from-url` and the D1 one as `--to-url`. The target's
178
+ `.global()` backend decides where the rows land, so the copy works the same way
179
+ in both directions.
180
+
181
+ Without `--replace` the rollback restores rows missing from D1, not edits made
182
+ on Hyperdrive since cutover: a row D1 already holds is a conflict and keeps its
183
+ D1 values. Add `--replace` (plus `--tables` naming the `.global()` tables, and
184
+ `--yes` for a remote target) to overwrite those
185
+ rows with the Hyperdrive values, or keep the dump with `--out` and reconcile by
186
+ hand. Neither mode carries deletions over: a row deleted on Hyperdrive since
187
+ cutover is still in D1 afterwards, so remove those by hand.
153
188
 
154
189
  ## Common Pitfalls
155
190
 
156
- 1. **Mixing backends.** One global backend per app (codegen errors on a mix).
157
- 2. **Missing driver.** `postgres`/`mysql2` are optional peer deps — install the
158
- one matching your engine.
159
- 3. **Stale-replica reads.** Point Hyperdrive at the primary for read-your-writes.
160
- 4. **Expecting external writes to be reactive.** They aren't — route writes
161
- through Lunora (`ctx.db`).
162
-
163
- ## Checklist
164
-
165
- - [ ] `@lunora/hyperdrive/global` + the engine's driver installed.
166
- - [ ] Hyperdrive binding (`HYPERDRIVE`) created over the PlanetScale connection
167
- string, with a `localConnectionString` for dev.
168
- - [ ] Target tables marked `.global({ backend: "hyperdrive" })` (one backend per app).
169
- - [ ] `.hyperdriveGlobal({ engine, exec })` wired in the app composition.
170
- - [ ] `lunora codegen` run; `lunora dev` serves the tables reactively.
171
- - [ ] (Migrating) `lunora migrate d1-to-hyperdrive` run; row counts verified.
191
+ 1. **Installing `@lunora/hyperdrive/global` as a package name.** It's a subpath.
192
+ Install `@lunora/hyperdrive` and `@lunora/sql-store`.
193
+ 2. **Mixing D1 and Hyperdrive global tables.** Codegen rejects this.
194
+ 3. **Missing `FOUND_ROWS` on MySQL.** It causes spurious OCC conflicts on idempotent writes.
195
+ 4. **Stale-replica reads.** Point Hyperdrive at the primary.
196
+ 5. **Query caching left on.** Spurious OCC conflicts and live queries stuck up to
197
+ 75 s behind writes. Disable caching on the config (`--caching-disabled`). Local dev
198
+ has no cache, so this shows up only once deployed.
199
+ 6. **MySQL with `NO_BACKSLASH_ESCAPES` in `sql_mode`.** Unsupported: statements are
200
+ sent as text and a backslash in a string is escaped as `\\`, which that mode stores as
201
+ two backslashes. Quotes are doubled, so no value can leave its literal under either
202
+ mode. MySQL's default `sql_mode` doesn't include it.
203
+ 7. **Writing to the database directly.** Live queries won't see those writes. Route writes
204
+ through `ctx.db`.