@terreno/mcp 57.2.0 → 57.3.0

package/src/docs/versioned/57.2.0/implementationPlans/oss-launch-program.md

@@ -0,0 +1,204 @@
+# Program: Open Source Launch
+
+**Status:** Draft — decisions recorded (defaults accepted 2026-07-29)
+**Roadmap issue:** https://github.com/FlourishHealth/terreno/issues/1094
+**Priority:** High
+**Effort:** Epic (multiple IPs)
+**Owner:** unassigned
+**Created:** 2026-07-27
+
+Umbrella program for taking Terreno from "internal Flourish monorepo that happens to publish npm packages" to a credible public open-source framework. This document is the index, sequencing plan, and single place where cross-IP decisions get recorded.
+
+Individual IPs live alongside this file; each has a companion task list in `docs/tasks/`.
+
+## Positioning
+
+Every IP in this program must use one consistent positioning statement. The agreed framing:
+
+> **Terreno is Django/Rails for TypeScript — with universal app support.**
+>
+> Batteries-included full-stack framework: models, migrations-free schema conventions, auto-generated REST APIs, permissions, admin panel, auth, and an AI service on the backend; one React Native codebase that ships to iOS, Android, and web on the frontend. Built to be driven by AI coding agents from the first prompt to production deploy.
+
+Three pillars, in priority order:
+
+1. **Batteries included** (the Django/Rails claim) — the undifferentiated 80% of an app is already written: auth, CRUD, admin, permissions, AI, realtime, feature flags, consent.
+2. **Universal by default** (the differentiator vs Django/Rails) — one codebase, three platforms. Not a web framework with a mobile bolt-on.
+3. **AI-native** (the differentiator vs everything) — two layers, and the docs must present them as complementary:
+   - the **tool layer**: an MCP server with codegen, docs search, and runtime introspection (logs, client state, navigation), plus per-package agent guidelines;
+   - the **process layer**: the `/terreno-*` lifecycle shipped as an installable plugin —
+     shape, implement test-first, verify in a fresh context, submit with evidence, then
+     react once per current PR state while an outer loop owns waiting and reinvocation.
+
+   Agents are a first-class client of the framework, not an afterthought. Django gives you `manage.py startapp`; Terreno gives you a reviewed path from a request to a mergeable PR.
+
+Language rules for all docs:
+
+- Say "Django/Rails for TypeScript" in the first two sentences of the README and docs landing page.
+- Say "universal app" (not "cross-platform", not "React Native app") when describing the frontend.
+- Never claim features listed under "Where Terreno is headed" as shipped.
+- Avoid "the best framework for X" superlatives in reference docs; keep those to the landing page.
+
+## RTK deprecation gate
+
+**`@terreno/rtk` is being replaced.** PR [#869](https://github.com/flourishhealth/terreno/pull/869) (`@terreno/syncdb` local-first data layer) makes **Better Auth + `@terreno/syncdb`** the supported frontend platform. RTK Query becomes legacy.
+
+**Program-wide rule: IPs flagged `Blocked` may not be implemented until #869 merges.** Writing docs against RTK now guarantees a rewrite. Planning (this program) proceeds; implementation of `None`-flagged IPs may start before #869; `Partial` IPs proceed except for `[RTK]`-marked tasks.
+
+Each IP carries an explicit flag:
+
+| Flag | Meaning |
+|------|---------|
+| **None** | No RTK/Better Auth surface. Safe to implement before #869. |
+| **Partial** | Mostly RTK-independent, but specific tasks touch RTK. Those tasks are marked `[RTK]` in the task file. |
+| **Blocked** | Core content is RTK-shaped. Do not start until #869 merges. |
+
+Every `[RTK]`-marked task in every task file must be reviewed against the merged #869 surface before implementation. The migration content itself is owned by [`rtk-to-syncdb-migration-docs.md`](rtk-to-syncdb-migration-docs.md), which gates most of Wave 1.
+
+### Known RTK-shaped surfaces to re-verify after #869
+
+| Surface | Location | Post-#869 expectation |
+|---------|----------|-----------------------|
+| Frontend data-fetching narrative | `README.md`, `docs/README.md`, `docs/tutorials/getting-started.md` | syncdb local-first reads/mutations |
+| SDK codegen loop | `.rulesync/skills/generate-sdk/SKILL.md`, `example-frontend/openapi-config.ts` | `@terreno/syncdb-codegen` descriptors |
+| Auth story | `docs/how-to/configure-better-auth.md`, `docs/explanation/authentication.md` | Better Auth primary, JWT legacy |
+| `docs/reference/rtk.md` | reference tree | **Removed from launch reference** (P7 A); migration guide only |
+| `get_rtk_state` MCP tool | `mcp-server/src/local/tools/runtime.ts` (PR #802) | Rename/extend to syncdb store inspection |
+| `installTerrenoDevConsoleLogger` | `rtk/src/devConsoleLogger.ts` (PR #802) | Move to syncdb or a shared client package |
+| Feature-flag client hooks | `docs/reference/feature-flags.md`, `rtk/src/` | OpenFeature provider fed by syncdb |
+| Offline queue | `rtk/src/` offline middleware | Superseded by syncdb outbox |
+
+## Boost dependency (AI story)
+
+PR [#802](https://github.com/flourishhealth/terreno/pull/802) (MCP Boost parity, Phases 4–6) is **assumed merged** by the AI-facing IPs. It delivers the runtime half of the AI story:
+
+- `read_logs` merging backend / app (CDP) / Metro / browser sources
+- `last_error` across sources
+- `get_rtk_state` (→ syncdb state post-#869)
+- `evaluate` and `navigate` over CDP, gated by `TERRENO_MCP_EVAL=1`
+- Dev-only `POST /__terreno/browser-logs` ingestion in `@terreno/api`
+
+Combined with the already-shipped hosted tools (`terreno_search_docs`, `terreno_get_component_docs`, `terreno_bootstrap_app`, generators, `terreno_get_upgrade_guide`), this is the "agent can build, run, observe, and fix a Terreno app without a human reading logs" claim. [`ai-dev-loop-boost.md`](ai-dev-loop-boost.md) owns telling that story.
+
+## The `/terreno-*` SDLC pipeline
+
+The repo already ships the process half of the AI story: `.cursor-plugin/marketplace.json`
+plus `plugins/terreno-planning/` define five bounded transitions —
+**`/terreno-1-grow`** (shape), **`/terreno-2-pick`** (build),
+**`/terreno-3-roast`** (prove), **`/terreno-4-brew`** (submit), and
+**`/terreno-5-taste`** (react once). The outer loop owns persistence, waiting, retry, and
+escalation. Former command names are a documented hard-cut migration — see
+[`agentic-sdlc-plugin.md`](agentic-sdlc-plugin.md).
+
+It encodes real judgment rather than automation: a question-first planning gate that refuses to commit to decisions before they are answered, independent review and test-quality sub-agents spawned in **fresh contexts** after every commit, drift detection against the plan, anti-mocking rules, a hard frontend-evidence gate, and a refusal to push speculative fixes for flaky CI.
+
+Two problems block using it as a launch asset, both owned by [`agentic-sdlc-plugin.md`](agentic-sdlc-plugin.md):
+
+1. **It is invisible.** No `plugins/README.md`, no docs page, no mention in `README.md` or `AGENTS.md`. The audit that produced this program missed it entirely — which is exactly what every prospective user will do.
+2. **It only works inside this monorepo.** Repo-root-relative sibling-skill paths, a dependency on a `verify-ui-changes` skill that lives in `.rulesync/`, hardcoded monorepo package names in the frontend gates, and a hardcoded branch suffix. Publishing it without fixing these ships tooling that breaks on first use in a consumer app.
+
+## IP index
+
+### Wave 0 — unblocked, can start immediately
+
+| IP | Title | RTK flag | Why now |
+|----|-------|----------|---------|
+| [oss-governance-baseline](oss-governance-baseline.md) | License, contributing, security, changelog, templates | None | Legal blocker for any public launch; zero framework surface |
+| [public-roadmap-github](public-roadmap-github.md) | GitHub Discussions + Projects roadmap + Linear bridge | None | Needed before inviting outside contributors; process-only |
+| [deploy-to-gcp](deploy-to-gcp.md) | Generalized GCP deploy docs + `deploy-gcp` skill | None | Backend + static hosting only; no client data layer |
+| [agentic-sdlc-plugin](agentic-sdlc-plugin.md) | Publish, port, and document the `/terreno-*` pipeline | Partial | Already built and shipping; only the frontend path lists touch the data layer |
+
+### Wave 1 — gated on #869 (syncdb) merging
+
+| IP | Title | RTK flag | Depends on |
+|----|-------|----------|------------|
+| [rtk-to-syncdb-migration-docs](rtk-to-syncdb-migration-docs.md) | Deprecation policy, migration guide, reference restructure | Blocked | #869 |
+| [positioning-django-rails-universal](positioning-django-rails-universal.md) | Positioning rewrite across README, docs site, agent rules | Partial | rtk-to-syncdb-migration-docs |
+| [docs-reference-coverage](docs-reference-coverage.md) | `ai.md`, `syncdb.md`, `admin-spa.md`, de-stub package READMEs | Blocked | rtk-to-syncdb-migration-docs |
+| [docs-tutorials-ai-first](docs-tutorials-ai-first.md) | Getting-started rewrite + 4 new tutorials | Blocked | docs-reference-coverage, ai-dev-loop-boost |
+| [deployment-foundation](deployment-foundation.md) | Deployment architecture, env reference, backend Dockerfile | Partial | — |
+| [deploy-to-vercel](deploy-to-vercel.md) | Vercel deploy docs + `deploy-vercel` skill | Partial | deployment-foundation |
+| [upgrade-guides-and-skill](upgrade-guides-and-skill.md) | `upgrading-terreno` skill, backfill 0.22–0.26 notes, release gate | Blocked | rtk-to-syncdb-migration-docs |
+| [ai-dev-loop-boost](ai-dev-loop-boost.md) | AI-native story built on Boost #802 | Partial | #802 |
+| [build-terreno-app-validation](build-terreno-app-validation.md) | `build-terreno-app` skill run + blog post | Blocked | docs-tutorials-ai-first, ai-dev-loop-boost |
+| [examples-demo-coverage](examples-demo-coverage.md) | Demo README, missing stories, CI coverage gates | Partial | — |
+
+### Wave 2 — independent feature work
+
+| IP | Title | RTK flag | Notes |
+|----|-------|----------|-------|
+| [web-ssr-and-admin-spa](web-ssr-and-admin-spa.md) | Server-side rendering for universal web + admin-spa migration | Partial | Can be planned/prototyped in parallel; ships after Wave 1 docs stabilize |
+
+## Suggested sequencing
+
+```mermaid
+flowchart TD
+  subgraph w0 ["Wave 0 — now"]
+    gov["oss-governance-baseline"]
+    road["public-roadmap-github"]
+    gcp["deploy-to-gcp"]
+    plug["agentic-sdlc-plugin"]
+  end
+  syncdb["PR #869 syncdb merges"]
+  boost["PR #802 Boost merges"]
+  subgraph w1 ["Wave 1 — after #869"]
+    mig["rtk-to-syncdb-migration-docs"]
+    pos["positioning-django-rails-universal"]
+    ref["docs-reference-coverage"]
+    ai["ai-dev-loop-boost"]
+    tut["docs-tutorials-ai-first"]
+    upg["upgrade-guides-and-skill"]
+    dep["deployment-foundation"]
+    ver["deploy-to-vercel"]
+    demo["examples-demo-coverage"]
+    build["build-terreno-app-validation"]
+  end
+  ssr["web-ssr-and-admin-spa"]
+  syncdb --> mig
+  boost --> ai
+  mig --> pos
+  mig --> ref
+  mig --> upg
+  ref --> tut
+  ai --> tut
+  dep --> ver
+  tut --> build
+  ai --> build
+  plug --> pos
+  plug --> build
+  ref --> ssr
+```
+
+Launch gate: Wave 0 complete **and** `rtk-to-syncdb-migration-docs`, `positioning-django-rails-universal`, `docs-reference-coverage`, `docs-tutorials-ai-first`, `deploy-to-vercel`, `deploy-to-gcp`, `upgrade-guides-and-skill` complete. `build-terreno-app-validation` is the acceptance test for the whole program — if an agent following only public docs and skills cannot build and deploy the demo app, the launch is not ready.
+
+`agentic-sdlc-plugin` feeds two others: it supplies the process half of the AI-native pillar to `positioning-django-rails-universal`, and once public it becomes something the dogfooding harness in `build-terreno-app-validation` should exercise.
+
+## Blocking questions (program level)
+
+**Recorded 2026-07-29** (defaults accepted for all unspecified items).
+
+| # | Question | Decision |
+|---|----------|----------|
+| P1 | License for the whole monorepo | **MIT everywhere** — relicense `api`/`ui` from Apache-2.0; align `@terreno/mcp` and all published packages |
+| P2 | Copyright / governance model | **B** — Flourish + **DCO** sign-off (`Signed-off-by` on external PRs) |
+| P3 | Public support channel | **A** — GitHub Discussions only at launch |
+| P4 | Hosted MCP URL | **`https://mcp.terreno.app`** — retire `mcp.terreno.flourish.health` before launch |
+| P5 | Flourish-specific infra in the public repo | **C now** — move under `infra/flourish/`; **note to migrate to B** (private repo, generic modules only) post-launch |
+| P6 | Publish `@terreno/rtk` after #869 | **A** — keep publishing with deprecation notice through the current major line; **stop publishing in the next major** |
+| P7 | Launch documentation stack | **A** — **syncdb + Better Auth only** in tutorials, reference, and positioning (no parallel RTK/JWT path in public docs) |
+| P8 | Docs site versioning at launch | **B** — latest + one previous major |
+| P9 | Target npm version for launch | **C** — stay on 0.x; launch on **`0.56.x`** aligned with Expo SDK 56; promise 1.0 after syncdb stabilizes |
+| P10 | Blog post venue | **A** — docs site blog only |
+| P11 | `/terreno-*` SDLC plugin marketplace | **A** — public at launch, gated on portability work in [`agentic-sdlc-plugin.md`](agentic-sdlc-plugin.md) |
+
+**Brand / domain:** public surfaces use **`terreno.app`** (MCP at `mcp.terreno.app`, contacts at `@terreno.app`).
+
+Per-IP blocking questions live in each IP file under `## Blocking questions`.
+
+## Not included / future work
+
+- **Vercel deployment topology spike** — whether `@terreno/api` (Socket.io, change streams, SSE) can run on Vercel in one platform with the web app and file storage, or whether a split host remains required. Tracked as open TODOs in [`deploy-to-vercel.md`](deploy-to-vercel.md); do not document a finalized topology until the spike lands.
+- Actual 1.0 API-stability audit.
+- Trademark/brand review of the "Terreno" name.
+- Paid support, sponsorship, or OpenCollective setup.
+- Conference talks / launch-day marketing beyond the blog post.
+- Migrating internal Flourish apps onto syncdb (tracked separately from the OSS launch).

package/src/docs/versioned/57.2.0/implementationPlans/password-reset-and-email-verification.md

@@ -0,0 +1,147 @@
+# Implementation Plan: Password reset and email verification
+
+**Status:** Draft
+**Roadmap issue:** https://github.com/FlourishHealth/terreno/issues/1023
+**Priority:** High
+**Effort:** Big batch
+**Owner:** unassigned
+**Created:** 2026-08-09
+**Program:** [B2B platform](b2b-platform-program.md)
+**Depends on:** [comms-abstraction](comms-abstraction.md), [comms-adapter-sendgrid](comms-adapter-sendgrid.md)
+**RTK deprecation flag:** None — the RTK client's existing `resetPassword` endpoint gets a
+backend at last; new client work is a thin screen pair noted in UI
+
+## Goal
+
+The JWT auth path has no way to recover an account or verify an email address: the
+`@terreno/rtk` client ships a `resetPassword` mutation with **no backend route**, and
+nothing sets or checks email verification. This IP adds forgot/reset password and email
+verification flows to `@terreno/api`'s JWT auth, wires the Better Auth equivalents to the
+same mail templates, and sends everything through `@terreno/comms`.
+
+## Non-Goals
+
+- MFA / step-up auth (`mfa-step-up-auth` item).
+- Magic-link login.
+- SMS-based recovery (possible later via the Twilio Verify adapter).
+- Account lockout / brute-force throttling beyond token single-use + expiry
+  (`rate-limiting` item hardens the endpoints).
+
+## Decisions
+
+| Question | Decision |
+|----------|----------|
+| Token design | 32-byte crypto-random token, stored **hashed** (sha256) with expiry (`1h` reset / `24h` verify) and single-use consumption; separate `AuthToken` collection rather than fields on User |
+| Enumeration safety | `POST /auth/forgotPassword` always returns 202 regardless of whether the email exists |
+| Password set mechanism | `passport-local-mongoose`'s `setPassword` on the user document, then invalidate all refresh tokens |
+| RTK client compatibility | The existing RTK `resetPassword` endpoint's path/shape is the contract — verify at implementation time and alias if it differs from `/auth/resetPassword` |
+| Verification gating | Opt-in `requireEmailVerification` auth option: unverified users get 403 with code `email-not-verified` on login; default off (non-breaking) |
+| Better Auth path | Enable Better Auth's built-in reset/verification, providing `sendResetPassword` / `sendVerificationEmail` hooks that call the same comms templates so both auth paths send identical mail |
+| Link targets | Deep-link/URL base from a new `authOptions.publicAppUrl`; web route examples in example-frontend |
+
+## Architecture
+
+```
+api/src/authTokens.ts      # AuthToken model + issue/consume helpers
+api/src/authRecovery.ts    # forgot/reset/verify routes + options
+                           # registered by addAuthRoutes when comms is available
+comms templates            # resetPassword + verifyEmail templates (subject/text/html)
+```
+
+Flow (reset): `POST /auth/forgotPassword {email}` → 202 always; if user exists, issue
+token, send mail with `${publicAppUrl}/resetPassword?token=...` → `POST
+/auth/resetPassword {token, password}` → validate + consume token, `setPassword`,
+invalidate refresh tokens, 200.
+
+Flow (verify): token issued on signup (and via `POST /auth/sendVerification` for
+re-sends, authenticated); `POST /auth/verifyEmail {token}` sets `user.emailVerified =
+true`.
+
+## Models
+
+**AuthToken** — `userId` (ref User, required, indexed), `tokenHash` (string, unique),
+`type` ("passwordReset" | "emailVerification"), `expiresAt` (date, TTL-indexed),
+`consumedAt` (date, optional). Statics: `issueFor(user, type)`, `consume(token, type)`
+(atomic findOneAndUpdate guarding single-use).
+
+**User** — adds `emailVerified` (boolean, default false) via a small
+`emailVerificationPlugin` so existing user schemas opt in without breaking.
+
+## APIs
+
+| Method | Path | Permissions | Notes |
+|---|---|---|---|
+| POST | `/auth/forgotPassword` | anonymous | `{email}` → always 202 |
+| POST | `/auth/resetPassword` | anonymous | `{token, password}`; path/shape matched to the RTK client contract |
+| POST | `/auth/sendVerification` | IsAuthenticated | Re-send verification mail |
+| POST | `/auth/verifyEmail` | anonymous | `{token}` |
+
+All registered by `addAuthRoutes` when a mail-capable `CommsApp` is present; without one,
+routes 501 in production and console-send in development (inherited comms behavior).
+
+## Notifications
+
+Two mail templates (`resetPassword`, `verifyEmail`) shipped in `@terreno/comms` templates
+with app-override hooks.
+
+## UI
+
+example-frontend: `/resetPassword` (token from URL, new-password form using the existing
+RTK `resetPassword` mutation) and a "verify your email" banner + re-send button on the
+profile screen. `LoginScreen` gains an optional `onForgotPassword` link prop in
+`@terreno/ui`.
+
+## Phases
+
+1. **Token infrastructure:** AuthToken model, issue/consume, `emailVerificationPlugin`,
+   unit tests.
+2. **JWT routes + mail:** four routes, templates, refresh-token invalidation,
+   enumeration-safety + single-use/expiry test suite.
+3. **Better Auth parity:** wire `sendResetPassword`/`sendVerificationEmail` hooks to the
+   same templates; parity tests.
+4. **Client + docs:** example-frontend screens, `@terreno/ui` login link prop, SDK regen,
+   `docs/how-to/password-reset.md`, env/reference docs.
+
+## Feature Flags & Migrations
+
+`requireEmailVerification` defaults off; enabling it for an existing app is documented
+(backfill `emailVerified: true` for existing users or accept re-verification). No schema
+migration otherwise.
+
+## Activity Log & User Updates
+
+Token issue/consume and verification events logged via `logger` and `CommsMessage`
+rows; no user-facing activity feed.
+
+## Not Included / Future Work
+
+- Password strength policy hooks (exists via `PasswordRequirements` UI; server policy is
+  future work).
+- Admin-triggered forced resets (admin panel action — future).
+- Endpoint rate limiting (`rate-limiting` item).
+
+## Files to Create / Modify
+
+- `api/src/authTokens.ts`, `api/src/authRecovery.ts` (new), `api/src/auth.ts`,
+  `api/src/plugins.ts` (emailVerificationPlugin), `api/src/index.ts`
+- `comms/src/templates.ts` — two templates
+- `example-frontend/app/resetPassword.tsx`, profile banner, SDK regen
+- `ui/src/login/LoginScreen.tsx` — `onForgotPassword` prop
+- `docs/how-to/password-reset.md`, `docs/reference/environment-variables.md`
+
+## Task List
+
+See [docs/tasks/password-reset-and-email-verification.md](../tasks/password-reset-and-email-verification.md).
+
+## Acceptance Criteria
+
+- [ ] `POST /auth/forgotPassword` returns 202 for existing and non-existing emails alike,
+      sending mail only for existing ones.
+- [ ] A reset token works exactly once, fails after expiry, and a successful reset
+      invalidates outstanding refresh tokens.
+- [ ] The unmodified RTK client `resetPassword` mutation completes the flow end to end.
+- [ ] With `requireEmailVerification` on, an unverified user gets 403 code
+      `email-not-verified` on login and can complete verification to proceed.
+- [ ] Better Auth mode sends the same two templates through comms.
+- [ ] example-frontend can complete forgot → email link (console adapter output in dev) →
+      reset → login with the new password; UI verification evidence attached to the PR.

package/src/docs/versioned/57.2.0/implementationPlans/pluggable-database-sqlite-research.md

@@ -0,0 +1,150 @@
+# Research: Pluggable database layer and SQLite support
+
+**Status:** Complete — decisions made, IP written: [pluggable-database-sqlite.md](pluggable-database-sqlite.md)
+**Created:** 2026-08-20
+
+## Scope statement
+
+Evaluate how to make the persistence layer of `@terreno/api` (and the packages that build on it) pluggable so that SQLite can be used as an alternative to MongoDB/Mongoose, with special attention to:
+
+- Making the current Mongoose handler one implementation behind a database-agnostic seam.
+- Options for **schema definition** that minimize disruption to existing Terreno codebases (which all author `new mongoose.Schema(...)` today).
+- Which framework capabilities can realistically be ported to SQLite in a first release versus which stay Mongo-only behind capability flags.
+
+## PRD summary
+
+From the request:
+
+- **Problem:** Terreno is MongoDB-only. Every consumer must run a Mongo replica set (even for small apps and local dev), and the positioning docs explicitly rule out relational databases. This limits adoption (SQLite/libSQL is the default for many small deployments, edge platforms, and quick starts).
+- **Goal:** Support SQLite in `@terreno/api` and any other package that needs it; make the Mongoose handler pluggable rather than hard-wired.
+- **Explicit ask:** Present a few options for how to handle schemas with the minimal amount of disruption to existing codebases.
+
+## Codebase findings
+
+### Coupling depth (where Mongoose lives today)
+
+There is **no storage abstraction seam** in the framework today — no repository, `DatabaseAdapter`, or dialect-neutral query builder. Mongoose is the schema source of truth and the runtime data API.
+
+| Layer | Depth | Key touchpoints |
+|---|---|---|
+| CRUD (`modelRouter` + executors) | Very high | `api/src/api.ts`, `api/src/sync/executors.ts`, `api/src/docLoader.ts` — `find/findById/create/doc.save()/deleteOne/countDocuments/populate`, hydrated `Document`s, soft-delete via query middleware, native `model.collection.findOne` probe |
+| Schema plugins | Very high | `api/src/plugins.ts` — `createdUpdatedPlugin`, `isDeletedPlugin`, `findExactlyOne`/`findOneOrNone`, `upsertPlugin`, custom `DateOnly` `SchemaType`; plus `configurationPlugin`, `syncPlugin`, `errorsPlugin`, `githubUserPlugin` |
+| OpenAPI generation | High | `api/src/populate.ts` `getOpenApiSpecForModel` via `mongoose-to-swagger`; reads `schema.paths`, `options.ref`, virtuals, child schemas, `instance === "Mixed"`; feeds `openApi.ts`, AJV validation (`openApiValidator.ts`), and admin metadata |
+| Auth | High | `api/src/auth.ts` `UserModel extends Model<User>` requiring passport-local-mongoose statics; `betterAuthSetup.ts` hard-wires `mongodbAdapter` off the global `mongoose.connection` |
+| Realtime / sync | Extreme (Mongo-only) | `api/src/realtime/changeStreamWatcher.ts` (`db.watch()`, resume tokens, replica set required), `api/src/sync/models.ts` (`$inc` seq counters, optional `ClientSession`), `feature-flags` `.watch()` |
+| Query language | Public API | modelRouter passes Mongo filter objects through (`$and`/`$or`, `_gte`/`_lte` → `$gte`/`$lte`, `OwnerQueryFilter` returns Mongo filters); `realtime/queryMatcher.ts` re-implements Mongo matching in memory |
+
+`mongoose` is a **peerDependency** of `@terreno/api` (`^8 || ^9`); the catalog pins `9.7.4`. Connection lifecycle is the consumer's job (`example-backend/src/utils/database.ts`); the framework assumes the global `mongoose.connection` for change streams and Better Auth.
+
+### Schema definition surface (what "schemas" means downstream)
+
+Mongoose schemas are consumed by four downstream systems, all of which would need a story under any schema-handling option:
+
+1. **OpenAPI**: `mongoose-to-swagger` reads `required`, `enum`, `default`, `readOnly`, `description`; populate/ref metadata comes from raw `schema.path(key).options.ref`.
+2. **Admin** (`admin-backend/src/adminApp.ts`): builds `/admin/config` field metadata from `getOpenApiSpecForModel` + raw `schema.paths` for refs and allowlists.
+3. **syncdb codegen** ([syncdb-codegen.md](syncdb-codegen.md), approved): generates TypeScript **from the OpenAPI spec**, not from schemas directly — so anything that produces equivalent OpenAPI keeps codegen working unchanged.
+4. **MCP** (`mcp-server/src/tools.ts` `terreno_generate_model`): scaffolds Mongoose schema source code.
+
+### Package-by-package Mongoose exposure
+
+| Package | Exposure |
+|---|---|
+| `api` | Structural (see above) |
+| `admin-backend` | Deep — `Model<any>` config, `schema.paths`, ObjectId search, modelRouter CRUD |
+| `comms` | Deep — own Mongoose models (`PushToken`, `CommsMessage`) + modelRouter + direct queries |
+| `ai` | Deep — own Mongoose models (`AIRequest`, `GptHistory`) + modelRouter + aggregation |
+| `feature-flags` | Deep + Mongo-only `.watch()` |
+| `example-backend` | Canonical consumer (connect lifecycle, models, change streams) |
+| `syncdb` (client) | **None** — transport/protocol only, but the server side of the protocol is Mongo-only today |
+| `rtk`, `ui`, `admin-frontend`, `admin-spa` | None |
+| `mcp-server` | Codegen templates emit Mongoose; local tools connect via mongoose |
+| `test` | mongodb-memory-server harness (`TERRENO_TEST_MONGODB_URI`, replica set option, transaction patches) |
+
+### Useful precedents already in-repo
+
+- **String `_id`**: synced models already use String `_id` (hex string default) — precedent for non-ObjectId primary keys.
+- **In-memory Mongo query matcher** (`realtime/queryMatcher.ts`): the framework already treats "a Mongo filter object" as an evaluable spec independent of the database.
+- **Executor seam** (`executeCreate/Update/Delete`): HTTP and sync share one write path — the natural place to insert an adapter.
+- **`TerrenoPlugin` / comms adapter pattern**: the repo already ships pluggable-provider IPs ([comms-abstraction.md](comms-abstraction.md)) — an established shape for "core interface + adapter packages".
+- **zod** (`^4.4.3`) is already a peer of `@terreno/api` and used for action bodies via `zodOpenApi.ts`.
+
+## External findings
+
+- **Better Auth** supports SQLite first-class: built-in Kysely adapter accepts `better-sqlite3`, `node:sqlite` (Node ≥ 22.5), and `bun:sqlite` instances, plus a Drizzle adapter with `provider: "sqlite"`. The Mongo-only hard-wiring in `betterAuthSetup.ts` is a Terreno choice, not a Better Auth limitation.
+- **SQLite as a document store** is a well-trodden pattern: one table per collection with a JSON column (`TEXT` + `json_valid` check), queried via the JSON1 functions (`json_extract`), with **generated columns + indexes** for hot query fields. This preserves Mongo-style nested/Mixed/array data without a relational schema migration story.
+- **Mongo-filter → SQL translation** for the subset modelRouter actually emits (equality, `$in`, `$ne`, `$gte`/`$lte`, `$and`/`$or`, `$exists`) is small and testable; the existing `queryMatcher.ts` documents the exact operator surface needed.
+- **Change streams have no SQLite equivalent.** SQLite alternatives are `data_version` pragma polling, update hooks (better-sqlite3 exposes none; libSQL/Turso has limited CDC), or an app-level outbox table. Realtime/sync on SQLite therefore needs a different backend (likely outbox-table + in-process emit) or stays Mongo-only initially.
+- **Driver options**: `bun:sqlite` (zero-dep under Bun, not Node), `better-sqlite3` (Node, native build), `node:sqlite` (release candidate, Node ≥ 22.5), `libsql` (Turso-compatible, adds remote/replica options). Drizzle supports all of them if an ORM layer is wanted.
+
+## Candidate options
+
+### Decision 1 — Schema definition (the user's headline question)
+
+**Option A — Mongoose schema stays the authoring format; adapters interpret it.**
+Consumers keep writing `new mongoose.Schema({...})` exactly as today. A `DatabaseAdapter` receives the compiled model; the SQLite adapter reads `schema.paths` (types, refs, required, enum, defaults, descriptions) to derive its storage and to feed OpenAPI/admin unchanged. Mongoose remains a dependency even for SQLite users, but only as a schema DSL + document hydrator (it can define schemas without a connection).
+
+- Disruption: **zero** for existing codebases; plugins, `strict: "throw"`, virtuals, and the model-type pattern all keep working.
+- Risk: Mongoose document semantics (hooks, `doc.save()`, query middleware) must be emulated or bypassed on SQLite; hidden Mongo-isms can leak. Mongoose stays a hard dependency of the "database-agnostic" framework, which is philosophically odd but pragmatic.
+
+**Option B — New Terreno schema DSL (framework-owned AST).**
+`defineModel({fields: {...}})` produces a framework-native schema AST that compiles to a Mongoose schema on Mongo and to SQLite DDL/JSON layout on SQLite, and directly emits OpenAPI (dropping `mongoose-to-swagger`). Existing codebases migrate via codemod; a compatibility shim could wrap existing Mongoose schemas during transition.
+
+- Disruption: **high** (every model file in every consumer changes), but one-time and mostly mechanical.
+- Payoff: the framework owns its schema story end-to-end (OpenAPI, admin, MCP codegen, future Postgres); no per-adapter schema interpretation.
+
+**Option C — zod as the schema source.**
+Models defined as zod objects (already in the tree, already OpenAPI-bridged via `zodOpenApi.ts`) plus a metadata layer for refs/indexes/defaults. Adapters derive Mongoose schemas or SQLite DDL from the zod definition.
+
+- Disruption: **high** (same migration burden as B) and zod lacks natural homes for DB concerns (indexes, refs, hooks), so a wrapper DSL emerges anyway — effectively Option B with zod as the type layer.
+
+**Option D — Adopt a multi-dialect ORM (Drizzle/Prisma) as the schema layer.**
+- Disruption: **highest**; abandons the Mongoose ecosystem (plugins, populate, discriminators) and Mongo support in Drizzle is immature. Likely reject, but listed for completeness.
+
+A staged path is possible: **A now, B later** — ship the adapter seam with Mongoose-as-DSL (A), and treat a framework-owned schema AST (B) as the eventual next-major direction once the adapter surface is proven.
+
+### Decision 2 — SQLite storage model (if A is chosen)
+
+- **A1 — Document store**: one table per model: `_id TEXT PRIMARY KEY, doc JSON`, generated columns + indexes for `queryFields` and sort keys. Arrays, nested docs, and Mixed work unchanged; Mongo-filter translation targets `json_extract`. Closest semantics to today.
+- **A2 — Relational mapping**: schema paths → real columns; arrays/Mixed still fall back to JSON columns. More natural SQL, but introduces a migration story (ALTER TABLE on schema change) and diverges from Mongo semantics.
+
+### Decision 3 — Adapter seam shape
+
+Insert a `DatabaseAdapter` (working name) behind the executor/docLoader surface: `create / findById / list(query, sort, skip, limit) / count / save(diff) / delete / populate?`, plus a `translateQuery` boundary where the Mongo-filter subset is either executed natively (Mongo) or compiled to SQL (SQLite). Capabilities beyond CRUD are declared, not assumed:
+
+```
+interface DatabaseAdapter {
+  capabilities: {changeStreams: boolean; transactions: boolean; textSearch: boolean; ...};
+}
+```
+
+Mongo-only features (change-stream realtime, syncdb server protocol, feature-flag watchers, Atlas `$search`) check capabilities and either degrade (polling/outbox) or refuse with a clear error.
+
+### Decision 4 — Auth on SQLite
+
+- Better Auth: swap the hard-wired `mongodbAdapter` for adapter-provided database handles (Kysely/Drizzle SQLite are first-class upstream).
+- Legacy JWT/passport path: `passport-local-mongoose` is Mongo-only; either implement a portable credential store on the adapter surface, or make Better Auth the required auth provider on SQLite.
+
+### Decision 5 — Packaging
+
+- `@terreno/api` keeps the interface + Mongoose adapter (default, zero breaking change); SQLite ships as `@terreno/db-sqlite` (mirrors the comms adapter-package pattern), or everything lives in `api` behind optional peer deps.
+
+## Open questions (blocking)
+
+Recorded in the planning thread; the IP will not be written until these are answered:
+
+1. Schema authoring: A (Mongoose-as-DSL, zero disruption), B (new DSL, high one-time disruption), C (zod), or staged A→B?
+2. SQLite storage model: A1 document-style vs A2 relational?
+3. v1 capability line: which of realtime/syncdb-server/feature-flag-watch/Atlas-search are Mongo-only at launch?
+4. Auth on SQLite: Better Auth required, or portable credential store for the JWT path too?
+5. Query language: keep the Mongo filter subset as the public/adapter contract, or introduce a neutral filter DSL?
+6. Package scope: `api` + example-backend only for v1, or also `admin-backend`/`comms`/`ai`/`feature-flags`?
+7. SQLite driver: `bun:sqlite` vs `better-sqlite3` vs `node:sqlite` vs `libsql` (and does Node support matter for v1)?
+8. Packaging: separate `@terreno/db-sqlite` package vs in-`api` optional dependency?
+
+## References
+
+- `api/src/api.ts`, `api/src/sync/executors.ts`, `api/src/docLoader.ts`, `api/src/plugins.ts`, `api/src/populate.ts`, `api/src/openApi.ts`, `api/src/auth.ts`, `api/src/betterAuthSetup.ts`, `api/src/realtime/changeStreamWatcher.ts`, `api/src/realtime/queryMatcher.ts`, `api/src/sync/models.ts`
+- `admin-backend/src/adminApp.ts`, `mcp-server/src/tools.ts`, `test/src/mongo/mongoServer.ts`
+- [syncdb-codegen.md](syncdb-codegen.md), [comms-abstraction.md](comms-abstraction.md), [positioning-django-rails-universal.md](positioning-django-rails-universal.md), [deploy-to-gcp.md](deploy-to-gcp.md) (GC2 future-Postgres note)
+- Better Auth SQLite adapters: https://better-auth.com/docs/adapters/sqlite ; Drizzle adapter: https://better-auth.com/docs/adapters/drizzle
+- Prior art: [PR #145](https://github.com/FlourishHealth/terreno/pull/145) (closed) — an earlier SQL-adapter plan proposing Bun.SQL multi-dialect support and JOIN-based population; superseded by this IP's adapter seam + batched population design