@terreno/mcp 57.0.0-beta.1 → 57.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (222) hide show
  1. package/dist/bootstrap.d.ts +1 -1
  2. package/dist/bootstrap.d.ts.map +1 -1
  3. package/dist/bootstrap.js +381 -191
  4. package/dist/bootstrap.js.map +1 -1
  5. package/dist/docs/guidelines/api/skills/mongoose-schema-safety/SKILL.md +8 -0
  6. package/dist/docs/resources/overview.md +6 -1
  7. package/dist/docs/resources/rtk.md +3 -1
  8. package/dist/docs/resources/syncdb.md +43 -0
  9. package/dist/docs/upgrades/56.0.0-beta.2.md +34 -0
  10. package/dist/docs/upgrades/57.0.0.md +49 -0
  11. package/dist/docs/upgrades/57.1.0.md +126 -0
  12. package/dist/docs/versioned/next/README.md +15 -3
  13. package/dist/docs/versioned/next/explanation/README.md +3 -0
  14. package/dist/docs/versioned/next/explanation/admin-consent-migration.md +15 -0
  15. package/dist/docs/versioned/next/explanation/admin-plugin-frontend.md +12 -0
  16. package/dist/docs/versioned/next/explanation/announcements/rtk-deprecation.md +29 -0
  17. package/dist/docs/versioned/next/explanation/authentication.md +51 -58
  18. package/dist/docs/versioned/next/explanation/local-first-data.md +52 -0
  19. package/dist/docs/versioned/next/explanation/loop-engineering.md +117 -0
  20. package/dist/docs/versioned/next/explanation/repository-settings.md +8 -0
  21. package/dist/docs/versioned/next/explanation/roadmap-process.md +22 -9
  22. package/dist/docs/versioned/next/explanation/roadmap-seed-issues.md +39 -0
  23. package/dist/docs/versioned/next/how-to/README.md +3 -0
  24. package/dist/docs/versioned/next/how-to/admin-add-model.md +41 -0
  25. package/dist/docs/versioned/next/how-to/admin-custom-home.md +31 -0
  26. package/dist/docs/versioned/next/how-to/admin-custom-widget.md +42 -0
  27. package/dist/docs/versioned/next/how-to/admin-import-prebuilt.md +25 -0
  28. package/dist/docs/versioned/next/how-to/circleci.md +173 -0
  29. package/dist/docs/versioned/next/how-to/configure-better-auth.md +13 -10
  30. package/dist/docs/versioned/next/how-to/expose-mcp-tools.md +194 -0
  31. package/dist/docs/versioned/next/how-to/install-agent-skills.md +46 -0
  32. package/dist/docs/versioned/next/how-to/migrate-rtk-to-syncdb.md +26 -1
  33. package/dist/docs/versioned/next/how-to/websocket-integration.md +2 -1
  34. package/dist/docs/versioned/next/implementationPlans/ModularAPI.md +1 -0
  35. package/dist/docs/versioned/next/implementationPlans/README.md +31 -17
  36. package/dist/docs/versioned/next/implementationPlans/admin-improvements-research.md +136 -0
  37. package/dist/docs/versioned/next/implementationPlans/admin-improvements.md +157 -527
  38. package/dist/docs/versioned/next/implementationPlans/admin-only-research.md +4 -0
  39. package/dist/docs/versioned/next/implementationPlans/admin-only.md +3 -1
  40. package/dist/docs/versioned/next/implementationPlans/admin-script-runner.md +1 -0
  41. package/dist/docs/versioned/next/implementationPlans/admin-ui-v2-django-parity.md +1 -0
  42. package/dist/docs/versioned/next/implementationPlans/agentic-sdlc-plugin.md +160 -130
  43. package/dist/docs/versioned/next/implementationPlans/apierror-standard-error-redesign.md +1 -0
  44. package/dist/docs/versioned/next/implementationPlans/b2b-platform-program.md +5 -4
  45. package/dist/docs/versioned/next/implementationPlans/comms-abstraction-research.md +48 -0
  46. package/dist/docs/versioned/next/implementationPlans/comms-abstraction.md +152 -62
  47. package/dist/docs/versioned/next/implementationPlans/comms-adapter-sendgrid.md +9 -9
  48. package/dist/docs/versioned/next/implementationPlans/comms-admin-dashboard.md +1 -1
  49. package/dist/docs/versioned/next/implementationPlans/consent-forms.md +4 -0
  50. package/dist/docs/versioned/next/implementationPlans/design-blend-skill.md +1 -0
  51. package/dist/docs/versioned/next/implementationPlans/docs-site-and-versioning.md +1 -0
  52. package/dist/docs/versioned/next/implementationPlans/feature-flags-openfeature.md +1 -0
  53. package/dist/docs/versioned/next/implementationPlans/feature-flags.md +4 -0
  54. package/dist/docs/versioned/next/implementationPlans/infra-mcp-research.md +1 -0
  55. package/dist/docs/versioned/next/implementationPlans/infra-mcp.md +3 -3
  56. package/dist/docs/versioned/next/implementationPlans/mcp-boost-parity.md +1 -0
  57. package/dist/docs/versioned/next/implementationPlans/migrate-cicd-to-circleci-research.md +1 -0
  58. package/dist/docs/versioned/next/implementationPlans/migrate-cicd-to-circleci.md +2 -2
  59. package/dist/docs/versioned/next/implementationPlans/model-router-actions.md +4 -0
  60. package/dist/docs/versioned/next/implementationPlans/model-router-mcp.md +5 -4
  61. package/dist/docs/versioned/next/implementationPlans/offline-mode.md +3 -1
  62. package/dist/docs/versioned/next/implementationPlans/oss-launch-program.md +11 -2
  63. package/dist/docs/versioned/next/implementationPlans/pluggable-database-sqlite-research.md +150 -0
  64. package/dist/docs/versioned/next/implementationPlans/pluggable-database-sqlite.md +247 -0
  65. package/dist/docs/versioned/next/implementationPlans/positioning-django-rails-universal.md +1 -1
  66. package/dist/docs/versioned/next/implementationPlans/public-roadmap-github.md +2 -2
  67. package/dist/docs/versioned/next/implementationPlans/rbac-permissions.md +52 -15
  68. package/dist/docs/versioned/next/implementationPlans/rtk-to-syncdb-migration-docs.md +13 -12
  69. package/dist/docs/versioned/next/implementationPlans/syncdb-codegen-research.md +137 -0
  70. package/dist/docs/versioned/next/implementationPlans/syncdb-codegen.md +63 -59
  71. package/dist/docs/versioned/next/implementationPlans/syncdb-local-first.md +1 -0
  72. package/dist/docs/versioned/next/implementationPlans/syncdb-phase-c-design.md +1 -0
  73. package/dist/docs/versioned/next/implementationPlans/terreno-langfuse-integration.md +1 -0
  74. package/dist/docs/versioned/next/implementationPlans/terreno-syncdb-2.md +1 -0
  75. package/dist/docs/versioned/next/implementationPlans/upgrade-banner.md +4 -0
  76. package/dist/docs/versioned/next/reference/README.md +6 -1
  77. package/dist/docs/versioned/next/reference/admin-config.md +51 -0
  78. package/dist/docs/versioned/next/reference/ai.md +15 -1
  79. package/dist/docs/versioned/next/reference/api.md +18 -0
  80. package/dist/docs/versioned/next/reference/comms.md +88 -7
  81. package/dist/docs/versioned/next/reference/environment-variables.md +9 -1
  82. package/dist/docs/versioned/next/reference/{rtk.md → legacy/rtk.md} +11 -7
  83. package/dist/docs/versioned/next/reference/lifecycle-plugin.md +54 -0
  84. package/dist/docs/versioned/next/reference/mcp-server.md +7 -0
  85. package/dist/docs/versioned/next/reference/syncdb.md +30 -3
  86. package/dist/docs/versioned/next/tasks/README.md +21 -10
  87. package/dist/docs/versioned/next/tasks/admin-improvements.md +189 -353
  88. package/dist/docs/versioned/next/tasks/admin-only.md +2 -0
  89. package/dist/docs/versioned/next/tasks/agentic-sdlc-plugin.md +54 -160
  90. package/dist/docs/versioned/next/tasks/comms-abstraction.md +30 -24
  91. package/dist/docs/versioned/next/tasks/comms-adapter-sendgrid.md +4 -4
  92. package/dist/docs/versioned/next/tasks/consent-forms.md +2 -0
  93. package/dist/docs/versioned/next/tasks/feature-flags.md +2 -0
  94. package/dist/docs/versioned/next/tasks/migrate-cicd-to-circleci.md +14 -10
  95. package/dist/docs/versioned/next/tasks/model-router-actions.md +2 -0
  96. package/dist/docs/versioned/next/tasks/model-router-mcp.md +27 -27
  97. package/dist/docs/versioned/next/tasks/offline-mode.md +2 -0
  98. package/dist/docs/versioned/next/tasks/pluggable-database-sqlite.md +44 -0
  99. package/dist/docs/versioned/next/tasks/public-roadmap-github.md +1 -1
  100. package/dist/docs/versioned/next/tasks/rtk-to-syncdb-migration-docs.md +15 -15
  101. package/dist/docs/versioned/next/tasks/syncdb-codegen.md +37 -31
  102. package/dist/docs/versioned/next/tasks/upgrade-banner.md +2 -0
  103. package/dist/index.js +24 -55
  104. package/dist/index.js.map +1 -1
  105. package/dist/local/localServer.d.ts.map +1 -1
  106. package/dist/local/localServer.js +9 -8
  107. package/dist/local/localServer.js.map +1 -1
  108. package/dist/local/localTools.d.ts +1 -1
  109. package/dist/local/localTools.d.ts.map +1 -1
  110. package/dist/prompts.d.ts.map +1 -1
  111. package/dist/prompts.js +44 -45
  112. package/dist/prompts.js.map +1 -1
  113. package/dist/resources.d.ts.map +1 -1
  114. package/dist/resources.js +9 -2
  115. package/dist/resources.js.map +1 -1
  116. package/dist/tools.d.ts +1 -1
  117. package/dist/tools.d.ts.map +1 -1
  118. package/dist/tools.js +4 -2
  119. package/dist/tools.js.map +1 -1
  120. package/package.json +7 -3
  121. package/src/docs/guidelines/api/skills/mongoose-schema-safety/SKILL.md +8 -0
  122. package/src/docs/resources/overview.md +6 -1
  123. package/src/docs/resources/rtk.md +3 -1
  124. package/src/docs/resources/syncdb.md +43 -0
  125. package/src/docs/upgrades/56.0.0-beta.2.md +34 -0
  126. package/src/docs/upgrades/57.0.0.md +49 -0
  127. package/src/docs/upgrades/57.1.0.md +126 -0
  128. package/src/docs/versioned/next/README.md +15 -3
  129. package/src/docs/versioned/next/explanation/README.md +3 -0
  130. package/src/docs/versioned/next/explanation/admin-consent-migration.md +15 -0
  131. package/src/docs/versioned/next/explanation/admin-plugin-frontend.md +12 -0
  132. package/src/docs/versioned/next/explanation/announcements/rtk-deprecation.md +29 -0
  133. package/src/docs/versioned/next/explanation/authentication.md +51 -58
  134. package/src/docs/versioned/next/explanation/local-first-data.md +52 -0
  135. package/src/docs/versioned/next/explanation/loop-engineering.md +117 -0
  136. package/src/docs/versioned/next/explanation/repository-settings.md +8 -0
  137. package/src/docs/versioned/next/explanation/roadmap-process.md +22 -9
  138. package/src/docs/versioned/next/explanation/roadmap-seed-issues.md +39 -0
  139. package/src/docs/versioned/next/how-to/README.md +3 -0
  140. package/src/docs/versioned/next/how-to/admin-add-model.md +41 -0
  141. package/src/docs/versioned/next/how-to/admin-custom-home.md +31 -0
  142. package/src/docs/versioned/next/how-to/admin-custom-widget.md +42 -0
  143. package/src/docs/versioned/next/how-to/admin-import-prebuilt.md +25 -0
  144. package/src/docs/versioned/next/how-to/circleci.md +173 -0
  145. package/src/docs/versioned/next/how-to/configure-better-auth.md +13 -10
  146. package/src/docs/versioned/next/how-to/expose-mcp-tools.md +194 -0
  147. package/src/docs/versioned/next/how-to/install-agent-skills.md +46 -0
  148. package/src/docs/versioned/next/how-to/migrate-rtk-to-syncdb.md +26 -1
  149. package/src/docs/versioned/next/how-to/websocket-integration.md +2 -1
  150. package/src/docs/versioned/next/implementationPlans/ModularAPI.md +1 -0
  151. package/src/docs/versioned/next/implementationPlans/README.md +31 -17
  152. package/src/docs/versioned/next/implementationPlans/admin-improvements-research.md +136 -0
  153. package/src/docs/versioned/next/implementationPlans/admin-improvements.md +157 -527
  154. package/src/docs/versioned/next/implementationPlans/admin-only-research.md +4 -0
  155. package/src/docs/versioned/next/implementationPlans/admin-only.md +3 -1
  156. package/src/docs/versioned/next/implementationPlans/admin-script-runner.md +1 -0
  157. package/src/docs/versioned/next/implementationPlans/admin-ui-v2-django-parity.md +1 -0
  158. package/src/docs/versioned/next/implementationPlans/agentic-sdlc-plugin.md +160 -130
  159. package/src/docs/versioned/next/implementationPlans/apierror-standard-error-redesign.md +1 -0
  160. package/src/docs/versioned/next/implementationPlans/b2b-platform-program.md +5 -4
  161. package/src/docs/versioned/next/implementationPlans/comms-abstraction-research.md +48 -0
  162. package/src/docs/versioned/next/implementationPlans/comms-abstraction.md +152 -62
  163. package/src/docs/versioned/next/implementationPlans/comms-adapter-sendgrid.md +9 -9
  164. package/src/docs/versioned/next/implementationPlans/comms-admin-dashboard.md +1 -1
  165. package/src/docs/versioned/next/implementationPlans/consent-forms.md +4 -0
  166. package/src/docs/versioned/next/implementationPlans/design-blend-skill.md +1 -0
  167. package/src/docs/versioned/next/implementationPlans/docs-site-and-versioning.md +1 -0
  168. package/src/docs/versioned/next/implementationPlans/feature-flags-openfeature.md +1 -0
  169. package/src/docs/versioned/next/implementationPlans/feature-flags.md +4 -0
  170. package/src/docs/versioned/next/implementationPlans/infra-mcp-research.md +1 -0
  171. package/src/docs/versioned/next/implementationPlans/infra-mcp.md +3 -3
  172. package/src/docs/versioned/next/implementationPlans/mcp-boost-parity.md +1 -0
  173. package/src/docs/versioned/next/implementationPlans/migrate-cicd-to-circleci-research.md +1 -0
  174. package/src/docs/versioned/next/implementationPlans/migrate-cicd-to-circleci.md +2 -2
  175. package/src/docs/versioned/next/implementationPlans/model-router-actions.md +4 -0
  176. package/src/docs/versioned/next/implementationPlans/model-router-mcp.md +5 -4
  177. package/src/docs/versioned/next/implementationPlans/offline-mode.md +3 -1
  178. package/src/docs/versioned/next/implementationPlans/oss-launch-program.md +11 -2
  179. package/src/docs/versioned/next/implementationPlans/pluggable-database-sqlite-research.md +150 -0
  180. package/src/docs/versioned/next/implementationPlans/pluggable-database-sqlite.md +247 -0
  181. package/src/docs/versioned/next/implementationPlans/positioning-django-rails-universal.md +1 -1
  182. package/src/docs/versioned/next/implementationPlans/public-roadmap-github.md +2 -2
  183. package/src/docs/versioned/next/implementationPlans/rbac-permissions.md +52 -15
  184. package/src/docs/versioned/next/implementationPlans/rtk-to-syncdb-migration-docs.md +13 -12
  185. package/src/docs/versioned/next/implementationPlans/syncdb-codegen-research.md +137 -0
  186. package/src/docs/versioned/next/implementationPlans/syncdb-codegen.md +63 -59
  187. package/src/docs/versioned/next/implementationPlans/syncdb-local-first.md +1 -0
  188. package/src/docs/versioned/next/implementationPlans/syncdb-phase-c-design.md +1 -0
  189. package/src/docs/versioned/next/implementationPlans/terreno-langfuse-integration.md +1 -0
  190. package/src/docs/versioned/next/implementationPlans/terreno-syncdb-2.md +1 -0
  191. package/src/docs/versioned/next/implementationPlans/upgrade-banner.md +4 -0
  192. package/src/docs/versioned/next/reference/README.md +6 -1
  193. package/src/docs/versioned/next/reference/admin-config.md +51 -0
  194. package/src/docs/versioned/next/reference/ai.md +15 -1
  195. package/src/docs/versioned/next/reference/api.md +18 -0
  196. package/src/docs/versioned/next/reference/comms.md +88 -7
  197. package/src/docs/versioned/next/reference/environment-variables.md +9 -1
  198. package/src/docs/versioned/next/reference/{rtk.md → legacy/rtk.md} +11 -7
  199. package/src/docs/versioned/next/reference/lifecycle-plugin.md +54 -0
  200. package/src/docs/versioned/next/reference/mcp-server.md +7 -0
  201. package/src/docs/versioned/next/reference/syncdb.md +30 -3
  202. package/src/docs/versioned/next/tasks/README.md +21 -10
  203. package/src/docs/versioned/next/tasks/admin-improvements.md +189 -353
  204. package/src/docs/versioned/next/tasks/admin-only.md +2 -0
  205. package/src/docs/versioned/next/tasks/agentic-sdlc-plugin.md +54 -160
  206. package/src/docs/versioned/next/tasks/comms-abstraction.md +30 -24
  207. package/src/docs/versioned/next/tasks/comms-adapter-sendgrid.md +4 -4
  208. package/src/docs/versioned/next/tasks/consent-forms.md +2 -0
  209. package/src/docs/versioned/next/tasks/feature-flags.md +2 -0
  210. package/src/docs/versioned/next/tasks/migrate-cicd-to-circleci.md +14 -10
  211. package/src/docs/versioned/next/tasks/model-router-actions.md +2 -0
  212. package/src/docs/versioned/next/tasks/model-router-mcp.md +27 -27
  213. package/src/docs/versioned/next/tasks/offline-mode.md +2 -0
  214. package/src/docs/versioned/next/tasks/pluggable-database-sqlite.md +44 -0
  215. package/src/docs/versioned/next/tasks/public-roadmap-github.md +1 -1
  216. package/src/docs/versioned/next/tasks/rtk-to-syncdb-migration-docs.md +15 -15
  217. package/src/docs/versioned/next/tasks/syncdb-codegen.md +37 -31
  218. package/src/docs/versioned/next/tasks/upgrade-banner.md +2 -0
  219. package/dist/docs/versioned/next/implementationPlans/PLAN_INDEX.md +0 -73
  220. package/dist/docs/versioned/next/implementationPlans/syncdb-api-inventory.md +0 -303
  221. package/src/docs/versioned/next/implementationPlans/PLAN_INDEX.md +0 -73
  222. package/src/docs/versioned/next/implementationPlans/syncdb-api-inventory.md +0 -303
@@ -1,6 +1,7 @@
1
1
  # Program: Open Source Launch
2
2
 
3
3
  **Status:** Draft — decisions recorded (defaults accepted 2026-07-29)
4
+ **Roadmap issue:** https://github.com/FlourishHealth/terreno/issues/1094
4
5
  **Priority:** High
5
6
  **Effort:** Epic (multiple IPs)
6
7
  **Owner:** unassigned
@@ -24,7 +25,9 @@ Three pillars, in priority order:
24
25
  2. **Universal by default** (the differentiator vs Django/Rails) — one codebase, three platforms. Not a web framework with a mobile bolt-on.
25
26
  3. **AI-native** (the differentiator vs everything) — two layers, and the docs must present them as complementary:
26
27
  - the **tool layer**: an MCP server with codegen, docs search, and runtime introspection (logs, client state, navigation), plus per-package agent guidelines;
27
- - the **process layer**: the `/terreno-*` agentic SDLC pipeline shipped as an installable plugin — plan, implement test-first, verify in a fresh context, submit with evidence, then own the review loop to mergeable.
28
+ - the **process layer**: the `/terreno-*` lifecycle shipped as an installable plugin —
29
+ shape, implement test-first, verify in a fresh context, submit with evidence, then
30
+ react once per current PR state while an outer loop owns waiting and reinvocation.
28
31
 
29
32
  Agents are a first-class client of the framework, not an afterthought. Django gives you `manage.py startapp`; Terreno gives you a reviewed path from a request to a mergeable PR.
30
33
 
@@ -78,7 +81,13 @@ Combined with the already-shipped hosted tools (`terreno_search_docs`, `terreno_
78
81
 
79
82
  ## The `/terreno-*` SDLC pipeline
80
83
 
81
- The repo already ships the process half of the AI story and **has never documented it**: `.cursor-plugin/marketplace.json` plus `plugins/terreno-planning/` define an installable five-stage pipeline — **`/terreno-1-grow`** (plan), **`/terreno-2-harvest`** (implement), **`/terreno-3-roast`** (verify), **`/terreno-4-brew`** (submit), **`/terreno-5-taste`** (review loop until mergeable). Legacy names (`blend`, `cupping`, `pour`, `dialin`) remain as aliases during the rename — see [`agentic-sdlc-plugin.md`](agentic-sdlc-plugin.md).
84
+ The repo already ships the process half of the AI story: `.cursor-plugin/marketplace.json`
85
+ plus `plugins/terreno-planning/` define five bounded transitions —
86
+ **`/terreno-1-grow`** (shape), **`/terreno-2-pick`** (build),
87
+ **`/terreno-3-roast`** (prove), **`/terreno-4-brew`** (submit), and
88
+ **`/terreno-5-taste`** (react once). The outer loop owns persistence, waiting, retry, and
89
+ escalation. Former command names are a documented hard-cut migration — see
90
+ [`agentic-sdlc-plugin.md`](agentic-sdlc-plugin.md).
82
91
 
83
92
  It encodes real judgment rather than automation: a question-first planning gate that refuses to commit to decisions before they are answered, independent review and test-quality sub-agents spawned in **fresh contexts** after every commit, drift detection against the plan, anti-mocking rules, a hard frontend-evidence gate, and a refusal to push speculative fixes for flaky CI.
84
93
 
@@ -0,0 +1,150 @@
1
+ # Research: Pluggable database layer and SQLite support
2
+
3
+ **Status:** Complete — decisions made, IP written: [pluggable-database-sqlite.md](pluggable-database-sqlite.md)
4
+ **Created:** 2026-08-20
5
+
6
+ ## Scope statement
7
+
8
+ Evaluate how to make the persistence layer of `@terreno/api` (and the packages that build on it) pluggable so that SQLite can be used as an alternative to MongoDB/Mongoose, with special attention to:
9
+
10
+ - Making the current Mongoose handler one implementation behind a database-agnostic seam.
11
+ - Options for **schema definition** that minimize disruption to existing Terreno codebases (which all author `new mongoose.Schema(...)` today).
12
+ - Which framework capabilities can realistically be ported to SQLite in a first release versus which stay Mongo-only behind capability flags.
13
+
14
+ ## PRD summary
15
+
16
+ From the request:
17
+
18
+ - **Problem:** Terreno is MongoDB-only. Every consumer must run a Mongo replica set (even for small apps and local dev), and the positioning docs explicitly rule out relational databases. This limits adoption (SQLite/libSQL is the default for many small deployments, edge platforms, and quick starts).
19
+ - **Goal:** Support SQLite in `@terreno/api` and any other package that needs it; make the Mongoose handler pluggable rather than hard-wired.
20
+ - **Explicit ask:** Present a few options for how to handle schemas with the minimal amount of disruption to existing codebases.
21
+
22
+ ## Codebase findings
23
+
24
+ ### Coupling depth (where Mongoose lives today)
25
+
26
+ There is **no storage abstraction seam** in the framework today — no repository, `DatabaseAdapter`, or dialect-neutral query builder. Mongoose is the schema source of truth and the runtime data API.
27
+
28
+ | Layer | Depth | Key touchpoints |
29
+ |---|---|---|
30
+ | CRUD (`modelRouter` + executors) | Very high | `api/src/api.ts`, `api/src/sync/executors.ts`, `api/src/docLoader.ts` — `find/findById/create/doc.save()/deleteOne/countDocuments/populate`, hydrated `Document`s, soft-delete via query middleware, native `model.collection.findOne` probe |
31
+ | Schema plugins | Very high | `api/src/plugins.ts` — `createdUpdatedPlugin`, `isDeletedPlugin`, `findExactlyOne`/`findOneOrNone`, `upsertPlugin`, custom `DateOnly` `SchemaType`; plus `configurationPlugin`, `syncPlugin`, `errorsPlugin`, `githubUserPlugin` |
32
+ | OpenAPI generation | High | `api/src/populate.ts` `getOpenApiSpecForModel` via `mongoose-to-swagger`; reads `schema.paths`, `options.ref`, virtuals, child schemas, `instance === "Mixed"`; feeds `openApi.ts`, AJV validation (`openApiValidator.ts`), and admin metadata |
33
+ | Auth | High | `api/src/auth.ts` `UserModel extends Model<User>` requiring passport-local-mongoose statics; `betterAuthSetup.ts` hard-wires `mongodbAdapter` off the global `mongoose.connection` |
34
+ | Realtime / sync | Extreme (Mongo-only) | `api/src/realtime/changeStreamWatcher.ts` (`db.watch()`, resume tokens, replica set required), `api/src/sync/models.ts` (`$inc` seq counters, optional `ClientSession`), `feature-flags` `.watch()` |
35
+ | Query language | Public API | modelRouter passes Mongo filter objects through (`$and`/`$or`, `_gte`/`_lte` → `$gte`/`$lte`, `OwnerQueryFilter` returns Mongo filters); `realtime/queryMatcher.ts` re-implements Mongo matching in memory |
36
+
37
+ `mongoose` is a **peerDependency** of `@terreno/api` (`^8 || ^9`); the catalog pins `9.7.4`. Connection lifecycle is the consumer's job (`example-backend/src/utils/database.ts`); the framework assumes the global `mongoose.connection` for change streams and Better Auth.
38
+
39
+ ### Schema definition surface (what "schemas" means downstream)
40
+
41
+ Mongoose schemas are consumed by four downstream systems, all of which would need a story under any schema-handling option:
42
+
43
+ 1. **OpenAPI**: `mongoose-to-swagger` reads `required`, `enum`, `default`, `readOnly`, `description`; populate/ref metadata comes from raw `schema.path(key).options.ref`.
44
+ 2. **Admin** (`admin-backend/src/adminApp.ts`): builds `/admin/config` field metadata from `getOpenApiSpecForModel` + raw `schema.paths` for refs and allowlists.
45
+ 3. **syncdb codegen** ([syncdb-codegen.md](syncdb-codegen.md), approved): generates TypeScript **from the OpenAPI spec**, not from schemas directly — so anything that produces equivalent OpenAPI keeps codegen working unchanged.
46
+ 4. **MCP** (`mcp-server/src/tools.ts` `terreno_generate_model`): scaffolds Mongoose schema source code.
47
+
48
+ ### Package-by-package Mongoose exposure
49
+
50
+ | Package | Exposure |
51
+ |---|---|
52
+ | `api` | Structural (see above) |
53
+ | `admin-backend` | Deep — `Model<any>` config, `schema.paths`, ObjectId search, modelRouter CRUD |
54
+ | `comms` | Deep — own Mongoose models (`PushToken`, `CommsMessage`) + modelRouter + direct queries |
55
+ | `ai` | Deep — own Mongoose models (`AIRequest`, `GptHistory`) + modelRouter + aggregation |
56
+ | `feature-flags` | Deep + Mongo-only `.watch()` |
57
+ | `example-backend` | Canonical consumer (connect lifecycle, models, change streams) |
58
+ | `syncdb` (client) | **None** — transport/protocol only, but the server side of the protocol is Mongo-only today |
59
+ | `rtk`, `ui`, `admin-frontend`, `admin-spa` | None |
60
+ | `mcp-server` | Codegen templates emit Mongoose; local tools connect via mongoose |
61
+ | `test` | mongodb-memory-server harness (`TERRENO_TEST_MONGODB_URI`, replica set option, transaction patches) |
62
+
63
+ ### Useful precedents already in-repo
64
+
65
+ - **String `_id`**: synced models already use String `_id` (hex string default) — precedent for non-ObjectId primary keys.
66
+ - **In-memory Mongo query matcher** (`realtime/queryMatcher.ts`): the framework already treats "a Mongo filter object" as an evaluable spec independent of the database.
67
+ - **Executor seam** (`executeCreate/Update/Delete`): HTTP and sync share one write path — the natural place to insert an adapter.
68
+ - **`TerrenoPlugin` / comms adapter pattern**: the repo already ships pluggable-provider IPs ([comms-abstraction.md](comms-abstraction.md)) — an established shape for "core interface + adapter packages".
69
+ - **zod** (`^4.4.3`) is already a peer of `@terreno/api` and used for action bodies via `zodOpenApi.ts`.
70
+
71
+ ## External findings
72
+
73
+ - **Better Auth** supports SQLite first-class: built-in Kysely adapter accepts `better-sqlite3`, `node:sqlite` (Node ≥ 22.5), and `bun:sqlite` instances, plus a Drizzle adapter with `provider: "sqlite"`. The Mongo-only hard-wiring in `betterAuthSetup.ts` is a Terreno choice, not a Better Auth limitation.
74
+ - **SQLite as a document store** is a well-trodden pattern: one table per collection with a JSON column (`TEXT` + `json_valid` check), queried via the JSON1 functions (`json_extract`), with **generated columns + indexes** for hot query fields. This preserves Mongo-style nested/Mixed/array data without a relational schema migration story.
75
+ - **Mongo-filter → SQL translation** for the subset modelRouter actually emits (equality, `$in`, `$ne`, `$gte`/`$lte`, `$and`/`$or`, `$exists`) is small and testable; the existing `queryMatcher.ts` documents the exact operator surface needed.
76
+ - **Change streams have no SQLite equivalent.** SQLite alternatives are `data_version` pragma polling, update hooks (better-sqlite3 exposes none; libSQL/Turso has limited CDC), or an app-level outbox table. Realtime/sync on SQLite therefore needs a different backend (likely outbox-table + in-process emit) or stays Mongo-only initially.
77
+ - **Driver options**: `bun:sqlite` (zero-dep under Bun, not Node), `better-sqlite3` (Node, native build), `node:sqlite` (release candidate, Node ≥ 22.5), `libsql` (Turso-compatible, adds remote/replica options). Drizzle supports all of them if an ORM layer is wanted.
78
+
79
+ ## Candidate options
80
+
81
+ ### Decision 1 — Schema definition (the user's headline question)
82
+
83
+ **Option A — Mongoose schema stays the authoring format; adapters interpret it.**
84
+ Consumers keep writing `new mongoose.Schema({...})` exactly as today. A `DatabaseAdapter` receives the compiled model; the SQLite adapter reads `schema.paths` (types, refs, required, enum, defaults, descriptions) to derive its storage and to feed OpenAPI/admin unchanged. Mongoose remains a dependency even for SQLite users, but only as a schema DSL + document hydrator (it can define schemas without a connection).
85
+
86
+ - Disruption: **zero** for existing codebases; plugins, `strict: "throw"`, virtuals, and the model-type pattern all keep working.
87
+ - Risk: Mongoose document semantics (hooks, `doc.save()`, query middleware) must be emulated or bypassed on SQLite; hidden Mongo-isms can leak. Mongoose stays a hard dependency of the "database-agnostic" framework, which is philosophically odd but pragmatic.
88
+
89
+ **Option B — New Terreno schema DSL (framework-owned AST).**
90
+ `defineModel({fields: {...}})` produces a framework-native schema AST that compiles to a Mongoose schema on Mongo and to SQLite DDL/JSON layout on SQLite, and directly emits OpenAPI (dropping `mongoose-to-swagger`). Existing codebases migrate via codemod; a compatibility shim could wrap existing Mongoose schemas during transition.
91
+
92
+ - Disruption: **high** (every model file in every consumer changes), but one-time and mostly mechanical.
93
+ - Payoff: the framework owns its schema story end-to-end (OpenAPI, admin, MCP codegen, future Postgres); no per-adapter schema interpretation.
94
+
95
+ **Option C — zod as the schema source.**
96
+ Models defined as zod objects (already in the tree, already OpenAPI-bridged via `zodOpenApi.ts`) plus a metadata layer for refs/indexes/defaults. Adapters derive Mongoose schemas or SQLite DDL from the zod definition.
97
+
98
+ - Disruption: **high** (same migration burden as B) and zod lacks natural homes for DB concerns (indexes, refs, hooks), so a wrapper DSL emerges anyway — effectively Option B with zod as the type layer.
99
+
100
+ **Option D — Adopt a multi-dialect ORM (Drizzle/Prisma) as the schema layer.**
101
+ - Disruption: **highest**; abandons the Mongoose ecosystem (plugins, populate, discriminators) and Mongo support in Drizzle is immature. Likely reject, but listed for completeness.
102
+
103
+ A staged path is possible: **A now, B later** — ship the adapter seam with Mongoose-as-DSL (A), and treat a framework-owned schema AST (B) as the eventual next-major direction once the adapter surface is proven.
104
+
105
+ ### Decision 2 — SQLite storage model (if A is chosen)
106
+
107
+ - **A1 — Document store**: one table per model: `_id TEXT PRIMARY KEY, doc JSON`, generated columns + indexes for `queryFields` and sort keys. Arrays, nested docs, and Mixed work unchanged; Mongo-filter translation targets `json_extract`. Closest semantics to today.
108
+ - **A2 — Relational mapping**: schema paths → real columns; arrays/Mixed still fall back to JSON columns. More natural SQL, but introduces a migration story (ALTER TABLE on schema change) and diverges from Mongo semantics.
109
+
110
+ ### Decision 3 — Adapter seam shape
111
+
112
+ Insert a `DatabaseAdapter` (working name) behind the executor/docLoader surface: `create / findById / list(query, sort, skip, limit) / count / save(diff) / delete / populate?`, plus a `translateQuery` boundary where the Mongo-filter subset is either executed natively (Mongo) or compiled to SQL (SQLite). Capabilities beyond CRUD are declared, not assumed:
113
+
114
+ ```
115
+ interface DatabaseAdapter {
116
+ capabilities: {changeStreams: boolean; transactions: boolean; textSearch: boolean; ...};
117
+ }
118
+ ```
119
+
120
+ Mongo-only features (change-stream realtime, syncdb server protocol, feature-flag watchers, Atlas `$search`) check capabilities and either degrade (polling/outbox) or refuse with a clear error.
121
+
122
+ ### Decision 4 — Auth on SQLite
123
+
124
+ - Better Auth: swap the hard-wired `mongodbAdapter` for adapter-provided database handles (Kysely/Drizzle SQLite are first-class upstream).
125
+ - Legacy JWT/passport path: `passport-local-mongoose` is Mongo-only; either implement a portable credential store on the adapter surface, or make Better Auth the required auth provider on SQLite.
126
+
127
+ ### Decision 5 — Packaging
128
+
129
+ - `@terreno/api` keeps the interface + Mongoose adapter (default, zero breaking change); SQLite ships as `@terreno/db-sqlite` (mirrors the comms adapter-package pattern), or everything lives in `api` behind optional peer deps.
130
+
131
+ ## Open questions (blocking)
132
+
133
+ Recorded in the planning thread; the IP will not be written until these are answered:
134
+
135
+ 1. Schema authoring: A (Mongoose-as-DSL, zero disruption), B (new DSL, high one-time disruption), C (zod), or staged A→B?
136
+ 2. SQLite storage model: A1 document-style vs A2 relational?
137
+ 3. v1 capability line: which of realtime/syncdb-server/feature-flag-watch/Atlas-search are Mongo-only at launch?
138
+ 4. Auth on SQLite: Better Auth required, or portable credential store for the JWT path too?
139
+ 5. Query language: keep the Mongo filter subset as the public/adapter contract, or introduce a neutral filter DSL?
140
+ 6. Package scope: `api` + example-backend only for v1, or also `admin-backend`/`comms`/`ai`/`feature-flags`?
141
+ 7. SQLite driver: `bun:sqlite` vs `better-sqlite3` vs `node:sqlite` vs `libsql` (and does Node support matter for v1)?
142
+ 8. Packaging: separate `@terreno/db-sqlite` package vs in-`api` optional dependency?
143
+
144
+ ## References
145
+
146
+ - `api/src/api.ts`, `api/src/sync/executors.ts`, `api/src/docLoader.ts`, `api/src/plugins.ts`, `api/src/populate.ts`, `api/src/openApi.ts`, `api/src/auth.ts`, `api/src/betterAuthSetup.ts`, `api/src/realtime/changeStreamWatcher.ts`, `api/src/realtime/queryMatcher.ts`, `api/src/sync/models.ts`
147
+ - `admin-backend/src/adminApp.ts`, `mcp-server/src/tools.ts`, `test/src/mongo/mongoServer.ts`
148
+ - [syncdb-codegen.md](syncdb-codegen.md), [comms-abstraction.md](comms-abstraction.md), [positioning-django-rails-universal.md](positioning-django-rails-universal.md), [deploy-to-gcp.md](deploy-to-gcp.md) (GC2 future-Postgres note)
149
+ - Better Auth SQLite adapters: https://better-auth.com/docs/adapters/sqlite ; Drizzle adapter: https://better-auth.com/docs/adapters/drizzle
150
+ - Prior art: [PR #145](https://github.com/FlourishHealth/terreno/pull/145) (closed) — an earlier SQL-adapter plan proposing Bun.SQL multi-dialect support and JOIN-based population; superseded by this IP's adapter seam + batched population design
@@ -0,0 +1,247 @@
1
+ # Pluggable database layer and SQLite adapter
2
+
3
+ **Status:** Approved — decisions 2026-08-20
4
+ **Branch:** `cursor/sqlite-pluggable-db-ip-4d52`
5
+ **Owner:** —
6
+ **Created:** 2026-08-20
7
+ **Research:** [pluggable-database-sqlite-research.md](pluggable-database-sqlite-research.md)
8
+ **Roadmap issue:** *(pending `roadmap-item` handoff)*
9
+
10
+ ## Goal
11
+
12
+ Make the persistence layer of `@terreno/api` pluggable: extract a `DatabaseAdapter` seam under `modelRouter`, make the current Mongoose/MongoDB handler the default adapter behind it (zero behavior change), and ship a SQLite adapter (`@terreno/db-sqlite`) that supports the full CRUD + permissions + population + OpenAPI + admin surface for apps that do not need Mongo-only capabilities.
13
+
14
+ Existing codebases keep authoring models exactly as today — `new mongoose.Schema({...})` with the standard plugins — on both databases.
15
+
16
+ ## Non-Goals
17
+
18
+ - Realtime change-stream events, the syncdb server protocol, feature-flag watchers, or Atlas `$search` on SQLite (Mongo-only in v1, gated by capability flags).
19
+ - The legacy passport-local-mongoose JWT auth path on SQLite (Better Auth is the auth provider on SQLite).
20
+ - A Postgres adapter (the seam must not preclude one, but none ships here).
21
+ - Data migration tooling between Mongo and SQLite.
22
+ - Certifying `admin-backend`, `comms`, `ai`, or `feature-flags` on SQLite (they are schema-driven and expected to mostly work; certification is follow-up work).
23
+ - A new schema DSL. Mongoose remains the schema authoring format and source of truth for OpenAPI/admin metadata.
24
+
25
+ ## Decisions
26
+
27
+ | # | Question | Decision |
28
+ |---|----------|----------|
29
+ | D1 | Schema authoring | **Option A** — Mongoose schemas stay the authoring format; adapters interpret the compiled model (`schema.paths`). Mongoose remains a dependency on SQLite as schema DSL + document layer. |
30
+ | D2 | SQLite storage model | **Option A2** — relational column mapping. Schema paths become real columns; arrays/subdocuments/Mixed/Map fall back to JSON columns. |
31
+ | D3 | v1 capability line | CRUD, permissions, population, OpenAPI, validation, and admin metadata work on SQLite. Change streams, syncdb server, feature-flag watch, and Atlas search stay Mongo-only behind adapter capability flags. |
32
+ | D4 | Auth on SQLite | Better Auth, via its built-in SQLite support (Kysely adapter accepts `bun:sqlite` / `better-sqlite3` / `node:sqlite` handles). Legacy passport path requires the Mongo adapter. |
33
+ | D5 | Query language | The existing Mongo-filter subset that modelRouter emits is the adapter contract; the SQLite adapter compiles it to SQL. |
34
+ | D6 | Package scope v1 | `@terreno/api` + `@terreno/db-sqlite` + `example-backend` (SQLite mode) + `@terreno/test` harness support. |
35
+ | D7 | Driver | `bun:sqlite` is the default; the adapter takes an injected driver handle so `better-sqlite3` / `node:sqlite` / `libsql` work too. |
36
+ | D8 | Packaging | New workspace package `@terreno/db-sqlite` (mirrors the comms adapter-package pattern). The interface + Mongoose adapter live in `@terreno/api`. |
37
+ | D9 | Population | First-class requirement. `populatePaths` (single refs, arrays of refs, field selection) must behave identically on both adapters, including populated create/update/read responses and soft-delete filtering of referenced docs. |
38
+
39
+ ## Architecture
40
+
41
+ ### The adapter seam
42
+
43
+ New module `api/src/db/adapter.ts` defining the contract; everything in `api.ts`, `sync/executors.ts`, `docLoader.ts`, and the permissions doc-loading path is refactored to call it instead of Mongoose model APIs directly.
44
+
45
+ ```typescript
46
+ export interface DatabaseAdapter {
47
+ readonly capabilities: AdapterCapabilities;
48
+ collection<T>(model: ModelLike<T>): AdapterCollection<T>;
49
+ // Schema lifecycle: no-op on Mongo; DDL sync on SQLite.
50
+ ensureSchema(models: ModelLike<unknown>[]): Promise<void>;
51
+ // Auth integration: what Better Auth's `database` option receives.
52
+ getBetterAuthDatabase(): unknown;
53
+ }
54
+
55
+ export interface AdapterCapabilities {
56
+ changeStreams: boolean; // realtime + syncdb server + feature-flag watch
57
+ transactions: boolean; // sync seq claiming sessions
58
+ textSearch: boolean; // Atlas $search / $autocomplete
59
+ }
60
+
61
+ export interface AdapterCollection<T> {
62
+ create(data: Partial<T>): Promise<HydratedDoc<T>>;
63
+ findById(id: string, options?: {populate?: PopulatePath[]}): Promise<HydratedDoc<T> | null>;
64
+ list(options: {
65
+ query: MongoFilter; // the validated modelRouter filter subset
66
+ sort?: string | Record<string, SortDirection>;
67
+ skip?: number;
68
+ limit?: number;
69
+ populate?: PopulatePath[];
70
+ }): Promise<HydratedDoc<T>[]>;
71
+ count(query: MongoFilter): Promise<number>;
72
+ save(doc: HydratedDoc<T>): Promise<HydratedDoc<T>>; // runs validation, plugin semantics, version check
73
+ delete(doc: HydratedDoc<T>, options: {soft: boolean}): Promise<void>;
74
+ populate(docs: HydratedDoc<T>[], paths: PopulatePath[]): Promise<void>;
75
+ }
76
+ ```
77
+
78
+ Adapter selection happens at app construction; the default preserves today's behavior with no consumer changes:
79
+
80
+ ```typescript
81
+ // Mongo (default — implicit when `database` is omitted)
82
+ new TerrenoApp({userModel: User});
83
+
84
+ // SQLite
85
+ import {sqliteAdapter} from "@terreno/db-sqlite";
86
+ new TerrenoApp({userModel: User, database: sqliteAdapter({filename: "app.db"})});
87
+ ```
88
+
89
+ ### MongooseAdapter (in `@terreno/api`)
90
+
91
+ Thin delegation to the exact calls the framework makes today (`find/findById/create/doc.save()/deleteOne/countDocuments/.populate()`), including the native `model.collection.findOne` soft-delete probe in `docLoader.ts`. Capabilities: all `true`. `getBetterAuthDatabase()` returns the `mongodbAdapter` client as `betterAuthSetup.ts` builds it today. All existing api tests must pass unchanged against this adapter — the Phase 1 gate.
92
+
93
+ ### SqliteAdapter (`@terreno/db-sqlite`)
94
+
95
+ **Documents.** Rows are hydrated into real Mongoose documents with `model.hydrate(row)` — no Mongo connection required — so virtuals, methods, `toJSON({virtuals: true})`, and consumer typing all work unchanged. Writes go through `adapter.save()`, which runs `doc.validate()`, applies the framework plugin semantics natively, and persists changed paths.
96
+
97
+ **Plugin semantics (behavior spec, implemented natively).**
98
+
99
+ | Plugin behavior | SQLite implementation |
100
+ |---|---|
101
+ | `createdUpdatedPlugin` | Set `created` on insert, `updated` on every save |
102
+ | `isDeletedPlugin` | `deleted` column; every read appends `AND deleted IS NOT 1`; `docLoader` probe reads without the filter |
103
+ | Optimistic concurrency / `__v` | `UPDATE ... WHERE _id = ? AND __v = ?`; zero rows → the same conflict error path as Mongoose `VersionError` |
104
+ | `findExactlyOne` / `findOneOrNone` / `upsertPlugin` statics | Work as-is — they run through `model.find`, which the framework only calls via the adapter surface |
105
+ | Consumer `schema.pre/post("save")` document middleware | Invoked through Mongoose's hook registry around persist (best-effort; verified by conformance tests; documented limitation if a hook depends on Mongo-only internals) |
106
+ | Consumer custom **query** middleware | Not translated to SQL beyond the built-in soft-delete/archive filters — documented limitation |
107
+
108
+ **Schema → DDL mapping (D2, A2).**
109
+
110
+ | Mongoose path | SQLite column |
111
+ |---|---|
112
+ | `_id` | `TEXT PRIMARY KEY` — default `new mongoose.Types.ObjectId().toHexString()` (matches the synced-model String-`_id` precedent) |
113
+ | String | `TEXT` |
114
+ | Number | `NUMERIC` |
115
+ | Boolean | `INTEGER` (0/1) |
116
+ | Date | `TEXT` — ISO 8601 UTC (lexicographic order = chronological; Luxon-friendly) |
117
+ | ObjectId / ref | `TEXT` (no `FOREIGN KEY` constraint — matches Mongo's non-enforcing refs) |
118
+ | enum | `TEXT` + `CHECK (col IN (...))` |
119
+ | Array / subdocument / Mixed / Map | `TEXT` JSON (`CHECK (json_valid(col))`) |
120
+ | `DateOnly` custom type | `TEXT` (`YYYY-MM-DD`) |
121
+ | `__v` | `INTEGER NOT NULL DEFAULT 0` |
122
+ | `required: true` | `NOT NULL` |
123
+ | `default` | Applied in the JS layer (supports function defaults), not DDL |
124
+ | `index` / `unique` / `sparse` | `CREATE [UNIQUE] INDEX`; sparse → partial index `WHERE col IS NOT NULL` |
125
+
126
+ **DDL sync.** `ensureSchema(models)` runs at startup: creates missing tables/columns/indexes when `autoMigrate: "additive"` (default in dev); destructive changes (type change, drop, constraint change) are never auto-applied — the adapter throws a clear `APIError`-style startup error and writes the proposed SQL to a migration file for manual review. Connection defaults: WAL mode, `busy_timeout`, `foreign_keys=OFF`.
127
+
128
+ **Query translation.** The adapter compiles exactly the filter surface modelRouter emits: top-level equality on `queryFields`, `$in/$nin/$ne/$exists/$gt/$gte/$lt/$lte`, `$and`/`$or`, and the `_gte`/`_lte` date-range expansion. Column-backed paths compile to plain SQL predicates; nested/JSON paths use `json_extract`; membership in array columns uses `EXISTS (SELECT 1 FROM json_each(col) ...)`. Any operator outside the contract → 400 `APIError` (same as an invalid query today). Sort/skip/limit/count map directly. `realtime/queryMatcher.ts` documents the operator semantics and seeds the conformance fixtures.
129
+
130
+ ### Population (D9)
131
+
132
+ Population is a first-class, conformance-tested part of the adapter contract — not an afterthought:
133
+
134
+ - **Contract:** `populate(docs, paths)` with `PopulatePath = {path, fields?}`, plus a `populate` option on `findById`/`list`. The executors' post-create/post-update re-read with population goes through the same path, so populated response shapes are identical across adapters.
135
+ - **Ref discovery:** from `schema.path(key).options.ref` and array-element refs — the same metadata `getOpenApiSpecForModel` reads, so the populated schemas in `/openapi.json` stay correct on SQLite with no OpenAPI changes.
136
+ - **SQLite strategy:** batched two-phase loads, mirroring Mongoose's own populate. Per path: collect distinct FK values across the page, run one `SELECT _id, <fields> FROM <ref_table> WHERE _id IN (...)`, attach results (single ref → object, array of refs → array). No N+1, no JOIN row-explosion, and `fields` projection happens in SQL.
137
+ - **Soft-delete parity:** when the referenced model has `isDeletedPlugin`, the batched lookup appends the soft-delete filter — a soft-deleted target populates to `null`/absent exactly as it does through Mongoose query middleware today.
138
+ - **Missing refs:** dangling FK values populate to `null` (Mongoose parity).
139
+ - **v1 scope:** single-level populate with field selection — the full surface `populatePaths` supports today. Nested (multi-hop) populate is future work on both adapters.
140
+ - **Conformance suite:** single ref, array of refs, field selection, dangling ref, soft-deleted target, populate-on-create/update responses — run against both adapters.
141
+
142
+ ### Capability gating (D3)
143
+
144
+ `TerrenoApp` checks `adapter.capabilities` at registration time:
145
+
146
+ - Realtime/change-stream registration, syncdb route registration, and feature-flag watchers require `changeStreams: true`; on SQLite they fail fast at startup with an actionable error naming the capability (not a runtime surprise).
147
+ - Atlas `$search`/`$autocomplete` stubs require `textSearch: true`.
148
+ - Sync seq sessions require `transactions: true`.
149
+
150
+ ### Auth (D4)
151
+
152
+ `betterAuthSetup.ts` stops reaching for the global `mongoose.connection` and instead calls `adapter.getBetterAuthDatabase()`. The Mongo adapter returns the `mongodbAdapter` client (today's behavior); the SQLite adapter returns its driver handle, which Better Auth's built-in Kysely adapter accepts directly for `bun:sqlite`, `better-sqlite3`, and `node:sqlite`. Better Auth schema tables are created via its migration tooling, invoked from `ensureSchema`. `setupAuth`/passport (`auth.ts`) is gated to the Mongo adapter with a clear startup error otherwise.
153
+
154
+ ## Models
155
+
156
+ No new Mongoose models. `@terreno/db-sqlite` maintains one internal metadata table (`_terreno_schema`) recording the applied DDL per model for additive-migration diffing.
157
+
158
+ ## APIs
159
+
160
+ No route surface changes. `modelRouter` options, generated endpoints, pagination envelope (`{data, limit, more, page, total}`), array-field endpoints, permissions, `queryFilter`, and OpenAPI output are identical on both adapters. New public API:
161
+
162
+ - `TerrenoAppOptions.database?: DatabaseAdapter` (default: Mongoose adapter).
163
+ - `@terreno/db-sqlite` exports `sqliteAdapter(options: {filename: string; driver?: SqliteDriverHandle; autoMigrate?: "additive" | "off"})`.
164
+ - `@terreno/api` exports the `DatabaseAdapter` / `AdapterCollection` / `AdapterCapabilities` types and `mongooseAdapter()`.
165
+
166
+ ## Notifications
167
+
168
+ None.
169
+
170
+ ## UI
171
+
172
+ None (backend-only).
173
+
174
+ ## Phases
175
+
176
+ ### Phase 1 — Adapter seam + Mongoose adapter (no behavior change)
177
+
178
+ Define the adapter interfaces; implement `MongooseAdapter`; refactor `api.ts`, `sync/executors.ts`, `docLoader.ts`, and permissions doc loading to consume the adapter; wire `TerrenoAppOptions.database`; add capability checks to realtime/sync/feature-flag/search registration (all pass on Mongo). Gate: entire existing api test suite green with zero test edits.
179
+
180
+ ### Phase 2 — `@terreno/db-sqlite` core
181
+
182
+ Package scaffold; DDL mapping + `ensureSchema` additive migration; CRUD; query translation; sort/pagination/count; plugin semantics (timestamps, soft delete, `__v` conflict); **population** (batched loads, soft-delete parity, field selection); enum checks and indexes; driver injection with `bun:sqlite` default. Adapter conformance suite in `@terreno/api` run against both adapters.
183
+
184
+ ### Phase 3 — Auth + capability hardening
185
+
186
+ `getBetterAuthDatabase()` seam; Better Auth on SQLite end-to-end (signup/login/session → `req.user`); passport path gated to Mongo; startup errors for missing capabilities; `@terreno/test` gains `TERRENO_TEST_DB=sqlite` (temp-file DB per suite, no mongod download).
187
+
188
+ ### Phase 4 — Example, docs, and integration
189
+
190
+ `example-backend` `DB_ADAPTER=sqlite` mode (skips change-stream features via capability flags); verify `/openapi.json` and admin `/admin/config` output parity; docs (how-to: run Terreno on SQLite; explanation: adapter capabilities; update positioning doc's "MongoDB-only" claims); MCP `terreno_generate_model`/bootstrap templates mention adapter choice; changelog + upgrade note (none required for existing apps).
191
+
192
+ ## Feature Flags & Migrations
193
+
194
+ - No feature flag: adapter choice is an explicit constructor option; omitting it preserves today's behavior exactly.
195
+ - No data migrations for existing apps.
196
+ - SQLite DDL migrations: additive-auto by default in dev; destructive changes always manual (generated SQL file + startup error). Documented in the how-to.
197
+
198
+ ## Activity Log & User Updates
199
+
200
+ n/a.
201
+
202
+ ## Not Included / Future Work
203
+
204
+ - Postgres adapter (the seam is designed dialect-neutral; JSON handling and DDL mapping are the SQLite-specific parts).
205
+ - Realtime/outbox-based change feed for SQLite (would unblock syncdb + feature-flag watch off-Mongo).
206
+ - Nested (multi-hop) populate on both adapters.
207
+ - Certifying `admin-backend`, `comms`, `ai`, `feature-flags` on SQLite.
208
+ - Mongo↔SQLite data migration tooling.
209
+ - passport-local credential store on SQLite.
210
+
211
+ ## Files to Create / Modify
212
+
213
+ **Create**
214
+
215
+ - `api/src/db/adapter.ts` — interfaces + capability types
216
+ - `api/src/db/mongooseAdapter.ts` — default adapter
217
+ - `api/src/db/conformance/` — shared adapter conformance suite (CRUD, query, populate, plugins)
218
+ - `db-sqlite/` — new workspace package: `src/sqliteAdapter.ts`, `src/ddl.ts`, `src/queryCompiler.ts`, `src/populate.ts`, `src/drivers.ts`, tests
219
+ - `docs/how-to/run-on-sqlite.md`, `docs/explanation/database-adapters.md`
220
+
221
+ **Modify**
222
+
223
+ - `api/src/api.ts`, `api/src/sync/executors.ts`, `api/src/docLoader.ts`, `api/src/permissions.ts` — consume adapter
224
+ - `api/src/terrenoApp.ts` — `database` option, capability gating
225
+ - `api/src/betterAuthSetup.ts` — `getBetterAuthDatabase()` seam
226
+ - `api/src/auth.ts` — Mongo-gate the passport path
227
+ - `api/src/index.ts` — export adapter types + `mongooseAdapter`
228
+ - `test/src/` — `TERRENO_TEST_DB=sqlite` harness mode
229
+ - `example-backend/src/server.ts` + `utils/database.ts` — `DB_ADAPTER=sqlite` mode
230
+ - `mcp-server/src/tools.ts` / bootstrap templates — adapter-aware scaffolds
231
+ - Root `package.json` catalog + workspaces, CI workflows for the new package
232
+
233
+ ## Task List
234
+
235
+ [docs/tasks/pluggable-database-sqlite.md](../tasks/pluggable-database-sqlite.md)
236
+
237
+ ## Acceptance Criteria
238
+
239
+ - [ ] Existing api test suite passes unchanged with the Mongoose adapter as default (Phase 1 gate).
240
+ - [ ] A modelRouter app (models, permissions, queryFilter, pagination, sorting, array endpoints) runs on SQLite with byte-identical response envelopes.
241
+ - [ ] `populatePaths` behaves identically on both adapters: single ref, array of refs, field selection, dangling ref → `null`, soft-deleted ref filtered, populated create/update responses — verified by the conformance suite.
242
+ - [ ] `/openapi.json` for the same models is identical on Mongo and SQLite.
243
+ - [ ] Better Auth signup/login/session works on SQLite; passport path fails fast with a clear error.
244
+ - [ ] Realtime, syncdb, and feature-flag registration on SQLite fail fast at startup with capability-named errors.
245
+ - [ ] `ensureSchema` creates tables/columns/indexes additively; destructive changes error with generated SQL.
246
+ - [ ] `example-backend` boots and serves CRUD in `DB_ADAPTER=sqlite` mode with no mongod running.
247
+ - [ ] `bun run lint`, `bun run compile`, and api + db-sqlite tests green in CI.
@@ -78,7 +78,7 @@ The Django/Rails claim invites the obvious question. `docs/explanation/positioni
78
78
  | Models + ORM | Mongoose schemas + `@terreno/api` plugins | Document store, not relational; no migrations framework |
79
79
  | `ModelViewSet` / scaffolds | `modelRouter` | REST only; no GraphQL |
80
80
  | Django admin | `@terreno/admin-backend` + `admin-frontend` / `admin-spa` | Younger; see the admin parity IP for the gap list |
81
- | Auth + permissions | Better Auth + `Permissions` | RBAC is in progress (`rbac-permissions.md`) |
81
+ | Auth + permissions | Better Auth + `Permissions` / RBAC | RBAC shipped (`rbac-permissions.md`, Complete) |
82
82
  | Templates / views | `@terreno/ui` components, one codebase for iOS/Android/web | SSR is not shipped yet — see [`web-ssr-and-admin-spa`](web-ssr-and-admin-spa.md) |
83
83
  | `manage.py` / generators | MCP server tools (the *tool* layer) + the `/terreno-*` SDLC pipeline (the *process* layer) | Agent-driven rather than CLI-driven. Django gives you `manage.py startapp`; Terreno gives you a reviewed path from a request to a mergeable PR |
84
84
  | Celery / ActiveJob | Not shipped | On the roadmap |
@@ -147,7 +147,7 @@ None beyond GitHub's own subscriptions. Announcements category posts should be t
147
147
  ## Phases
148
148
 
149
149
  1. **Discussions** — enable, create categories, seed each with a pinned intro post, add the RFC template.
150
- 2. **Project board** — create the Project, fields, and views; backfill items from `PLAN_INDEX.md` and the IPs in this program.
150
+ 2. **Project board** — create the Project, fields, and views; backfill items from the IPs in this program.
151
151
  3. **Labels and triage** — create the label set, delete unused defaults, add `triage.yml`.
152
152
  4. **Roadmap generation** — `ROADMAP.md` + `roadmap-generate.yml`, link from README and docs site.
153
153
  5. **Linear bridge** — configure the integration, document the process in `CONTRIBUTING.md`, and record the rules in a maintainer doc.
@@ -195,7 +195,7 @@ See [`docs/tasks/public-roadmap-github.md`](../tasks/public-roadmap-github.md).
195
195
 
196
196
  - [ ] All seven Discussion categories exist with the specified formats and a pinned intro post each.
197
197
  - [ ] The **Terreno Roadmap** Project exists with the six fields and four views described above.
198
- - [ ] Every IP in `docs/implementationPlans/PLAN_INDEX.md` marked Approved or later has a tracking issue on the board with its `IP` field set to the IP slug.
198
+ - [ ] Every IP whose `**Status:**` header is Approved or later has a tracking issue on the board with its `IP` field set to the IP slug.
199
199
  - [ ] The label set matches the taxonomy table; no unused GitHub default labels remain.
200
200
  - [ ] `bun run scripts/generate-roadmap.ts` produces `ROADMAP.md` grouped by `Target` then `Area`, and `roadmap-generate.yml` commits it only when changed.
201
201
  - [ ] Opening a new issue applies `status:needs-triage` and the correct `area:*` label automatically.
@@ -1,9 +1,36 @@
1
1
  # RBAC Permissions for Terreno — API Design
2
2
 
3
- **Status:** Draft — API design for discussion (full IP to follow)
3
+ **Status:** Complete — phases 1–6 shipped in PR #932 (2026-08-22)
4
+ **Roadmap issue:** https://github.com/FlourishHealth/terreno/issues/1089
4
5
  **Target package:** `@terreno/api` (new `src/rbac/` module), with surfaces in `admin-backend`, `admin-frontend`, `admin-spa`, `rtk`, and the modelRouter MCP work
5
6
  **Depends on:** `better-auth/plugins/access` (already a dependency via the Better Auth provider)
6
7
 
8
+ ## Shipped (PR #932)
9
+
10
+ Phases 1–6 in section 8 are implemented. `RoleManager` emits `RbacAudit` (including
11
+ `permissionDelta`) on create/update/remove/assign/unassign; denied escalation attempts are
12
+ written with `denied: true`. `rbacRouter` is a thin HTTP adapter and does not write a second
13
+ audit row. Apps can attach `createAccess({auditSink})` (one sink or an array) to fan the same
14
+ record into a consuming-app audit log. `persistAudit: false` skips the built-in collection
15
+ and requires at least one sink. Seed upsert lives in `upsertSeededRole`; `RbacRole.seedDefaults({statements,
16
+ extraRoles})` is the single seed entry. `previewRoleChange.affectedUserCount` is the number of
17
+ users currently holding that role.
18
+
19
+ ### Compatibility: no default-connection models
20
+
21
+ Importing `@terreno/api` does **not** register `RbacRole` or `RbacAudit` on
22
+ `mongoose.connection`. Apps must:
23
+
24
+ 1. Call `createAccess({connection, ...})` (registers both models on that connection), or
25
+ 2. Call `createRbacRoleModel(connection)` / `createRbacAuditModel(connection)` in scripts/tests.
26
+
27
+ The previous `RbacRoleModel` / `RbacAuditModel` value singletons are removed. The TypeScript
28
+ type `RbacRoleModel` remains exported. Migrate `import {RbacRoleModel} from "@terreno/api"` to
29
+ `createRbacRoleModel(connection)`.
30
+
31
+ Left out of this close: Redis/pubsub cross-instance cache invalidation beyond in-process
32
+ `invalidateCache`.
33
+
7
34
  ---
8
35
 
9
36
  ## 1. Summary
@@ -408,7 +435,11 @@ export interface RoleManager {
408
435
 
409
436
  // Diff previews for the admin UI (see 4.11)
410
437
  previewRoleChange(args: {roleName: string; permissions: PermissionSet}): Promise<RoleDiff>;
411
- previewAssignment(args: {userId: string; roleNames: string[]}): Promise<UserPermissionDiff>;
438
+ previewAssignment(args: {
439
+ actor: User;
440
+ userId: string;
441
+ roleNames: string[];
442
+ }): Promise<UserPermissionDiff>;
412
443
  }
413
444
 
414
445
  export interface RoleDiff {
@@ -429,12 +460,18 @@ Guardrails baked into `RoleManager` (borrowed from Better Auth's dynamic access
429
460
  - **No escalation**: an actor can only grant (or edit a role to include, or assign) permissions
430
461
  the actor **themselves currently holds**. Holding `rbac:manageRoles` lets an actor *manage*
431
462
  roles but never lets them hand out actions they lack — a role editor cannot bootstrap
432
- privileges they do not have. The **only** exception is `superadmin`, which (via its `"*"`
463
+ privileges they do not have. Role **update** checks `actorPermissions ⊇ (existing ∪ incoming)`
464
+ so emptying a role still requires holding its current grants. Role **remove** checks
465
+ `actorPermissions ⊇ existing`. The **only** exception is `superadmin`, which (via its `"*"`
433
466
  expansion) holds every permission and so can grant anything; this is a property of *what
434
467
  superadmin holds*, not a special "manageRoles can escalate" rule. Concretely: the check is
435
468
  `actorPermissions ⊇ permissionsBeingGranted`, evaluated against the actor's effective set.
436
469
  - **Vocabulary validation**: every `resource:action` in `permissions` must exist in
437
470
  `access.statements` — writes with unknown pairs are 400s.
471
+ - **Preview assignment** takes the same `actor` as assign: it requires `rbac:assignRoles`,
472
+ the target-user privilege-subset check, and `assertNoEscalation` for every previewed role.
473
+ The before/after diff uses uncached permission resolution on both sides so a warm cache
474
+ cannot invent gained/lost grants, and the preview cannot write live permission caches.
438
475
  - **Managing roles requires** `rbac:manageRoles`; assigning requires `rbac:assignRoles`. These
439
476
  gate *access* to the operation and are additive to (not a substitute for) the no-escalation
440
477
  subset check above.
@@ -872,11 +909,11 @@ behavior is per-source `staleOnFailure` (default `"deny"`):
872
909
 
873
910
  | Policy | Behavior |
874
911
  |---|---|
875
- | `"deny"` (default) | Omit this source's grants for the request. Local `user.roles` and other sources still apply. Revoked upstream access cannot linger because a dependency is down. |
912
+ | `"deny"` (default) | Omit this source's additive grants (`roles`, `permissions`) for the request. Local `user.roles` and other sources still apply. If a prior fetch cached a `deny` set, keep applying that last-known deny so IdP/ABAC restrictions do not lift while the source is down. Revoked upstream *elevations* cannot linger because a dependency is down. |
876
913
  | `"use-stale-bounded"` | Reuse the last successful grants only if younger than `staleMaxAgeMs` (hard cap). Suitable for read-only hints, not elevation. |
877
914
  | `"use-stale"` | Reuse last successful grants until TTL expiry. **Not permitted** for sources that can grant roles/permissions beyond what local roles already provide — elevation paths must fail closed. |
878
915
 
879
- `PermissionSource.deny` already fails closed; grant caching matches that posture for elevation.
916
+ `PermissionSource.deny` fails closed on refresh failure by keeping the last cached deny set (additive grants from that source are still omitted). Grant caching matches that posture for elevation.
880
917
  Logged at `warn` with source name and policy applied.
881
918
 
882
919
  ### 4.11 Admin UI (first-class)
@@ -890,7 +927,8 @@ New screens in `admin-frontend` / `admin-spa`, driven entirely by the 4.9 endpoi
890
927
  212 users."
891
928
  - **User role assignment** (embedded in the admin user detail): multi-select of roles with
892
929
  exclusion conflicts surfaced inline; confirmation modal shows `UserPermissionDiff` (total
893
- permissions gained/lost after union with their remaining roles).
930
+ permissions gained/lost after union with their remaining roles). The preview endpoint
931
+ requires an actor with `rbac:assignRoles` and rejects roles the actor cannot grant.
894
932
  - **Effective permissions inspector**: per user, each permission annotated with its source
895
933
  ("via PatientGuide", "via healthie").
896
934
 
@@ -968,15 +1006,14 @@ fall out of the admin UI with zero code — the exact win the RBAC move is for.
968
1006
  6. **Deny semantics**: **Only `PermissionSource.deny`** — roles do not carry negative
969
1007
  permissions (union semantics stay simple; matches Better Auth).
970
1008
 
971
- ## 8. Phasing sketch (full task breakdown in the IP)
1009
+ ## 8. Phasing sketch — shipped
972
1010
 
973
- 1. **Core**: statements/merge, `createAccess`, resolve+cache, `RbacRole` model + seeds,
1011
+ 1. **Core** — shipped: statements/merge, `createAccess`, resolve+cache, `RbacRole` model + seeds,
974
1012
  `rbacUserPlugin`, `IsPermitted`, `requireAccess` middleware. Tests.
975
- 2. **modelRouter**: `access` option, scope-aware permission middleware, filter merging,
1013
+ 2. **modelRouter** — shipped: `access` option, scope-aware permission middleware, filter merging,
976
1014
  field-mask response handling, actions support.
977
- 3. **HTTP surface**: `rbacRouter`, previews, `/auth/me` extension, backfill script.
978
- 4. **Admin**: roles screens, diffs, user assignment, `admin:access` migration.
979
- 5. **Realtime + MCP**: swap internals to `access.can()`/`fieldMask()`.
980
- 6. **Sources + client**: `PermissionSource`, rtk selectors, Better Auth bridge, docs +
981
- example-backend/-frontend demonstration (example app gets a `manager` role and a scoped
982
- todo example).
1015
+ 3. **HTTP surface** — shipped: `rbacRouter`, previews, `/auth/me` extension, backfill script.
1016
+ 4. **Admin** — shipped: roles screens, diffs, user assignment, `admin:access` migration.
1017
+ 5. **Realtime + MCP** — shipped: internals use `access.can()`/`fieldMask()`.
1018
+ 6. **Sources + client** — shipped: `PermissionSource`, rtk selectors, Better Auth bridge,
1019
+ example-backend/-frontend `manager` role and scoped todo example.
@@ -1,6 +1,7 @@
1
1
  # Implementation Plan: RTK Deprecation and SyncDB Migration Docs
2
2
 
3
- **Status:** In progress — syncdb available on `release-56.0.0`; reference + migration guide drafted
3
+ **Status:** Complete — syncdb reference, migration guide, agent surfaces, and deprecation signalling shipped on `master`
4
+ **Roadmap issue:** https://github.com/FlourishHealth/terreno/issues/1007
4
5
  **Priority:** Critical (gates most of Wave 1)
5
6
  **Effort:** Big batch
6
7
  **Owner:** unassigned
@@ -206,14 +207,14 @@ See [`docs/tasks/rtk-to-syncdb-migration-docs.md`](../tasks/rtk-to-syncdb-migrat
206
207
 
207
208
  ## Acceptance Criteria
208
209
 
209
- - [ ] A verified inventory of `@terreno/syncdb`'s public exports exists and every documented API appears in the merged package's `src/index.ts`.
210
- - [ ] `docs/reference/syncdb.md` documents every exported hook, client method, and config option, with a runnable example for each.
211
- - [ ] `docs/how-to/migrate-rtk-to-syncdb.md` covers all eleven steps, and every before/after pair compiles against the merged code.
212
- - [ ] Following the guide, a fresh clone of `example-frontend` can be moved from the RTK path to the syncdb path with no undocumented steps.
213
- - [ ] `docs/reference/rtk.md` no longer exists at its old path; the Legacy page states the support window; a Docusaurus redirect resolves the old URL.
214
- - [ ] `docs/explanation/authentication.md` presents Better Auth first and JWT as legacy.
215
- - [ ] No agent-facing surface (`AGENTS.md`, `CLAUDE*.md`, `.rulesync/rules/`, MCP resources, bootstrap templates) instructs an agent to write RTK Query code for a new app.
216
- - [ ] `terreno_bootstrap_app` output uses syncdb, and a bootstrapped app runs.
217
- - [ ] `terreno_get_upgrade_guide` returns the syncdb upgrade note for the release range that includes #869.
218
- - [ ] `CHANGELOG.md` has a `Deprecated` entry naming `@terreno/rtk`, the support window, and the migration guide link.
219
- - [ ] `bun run lint`, `bun run compile`, `bun run rules:check`, and `bun run website:build` all pass.
210
+ - [x] A verified inventory of `@terreno/syncdb`'s public exports exists and every documented API appears in the merged package's `src/index.ts`.
211
+ - [x] `docs/reference/syncdb.md` documents every exported hook, client method, and config option, with a runnable example for each.
212
+ - [x] `docs/how-to/migrate-rtk-to-syncdb.md` covers all eleven steps, and every before/after pair compiles against the merged code.
213
+ - [x] Following the guide, a fresh clone of `example-frontend` can be moved from the RTK path to the syncdb path with no undocumented steps.
214
+ - [x] `docs/reference/rtk.md` no longer exists at its old path; the Legacy page states the support window; a Docusaurus redirect resolves the old URL.
215
+ - [x] `docs/explanation/authentication.md` presents Better Auth first and JWT as legacy.
216
+ - [x] No agent-facing surface (`AGENTS.md`, `CLAUDE*.md`, `.rulesync/rules/`, MCP resources, bootstrap templates) instructs an agent to write RTK Query code for a new app.
217
+ - [x] `terreno_bootstrap_app` output uses syncdb, and a bootstrapped app runs.
218
+ - [x] `terreno_get_upgrade_guide` returns the syncdb upgrade note for the release range that includes #869.
219
+ - [x] `CHANGELOG.md` has a `Deprecated` entry naming `@terreno/rtk`, the support window, and the migration guide link.
220
+ - [x] `bun run lint`, `bun run compile`, `bun run rules:check`, and `bun run website:build` all pass.