@terreno/mcp 0.23.0 → 0.23.1

package/dist/docs/versioned/0.23.0/implementationPlans/model-router-mcp.md

@@ -0,0 +1,186 @@
+# Implementation Plan: modelRouter MCP Tools
+
+*When an engineer is assigned to a project but before you begin coding, you should fill in the implementation plan and get feedback from the engineering team. Once you have finished or you make any changes, tag Josh with the @ symbol so he can review. Also tag anyone else that needs to be notified, has conflicting work, etc.*
+
+## Overview
+
+Add MCP (Model Context Protocol) tool generation to `@terreno/api`'s `modelRouter`. Each modelRouter can opt-in to exposing its CRUD operations as MCP tools via an `mcp` config option. When models with MCP config are added to `TerrenoApp`, their tools are registered automatically. On server start, if any models have MCP config, a `/mcp` endpoint is auto-mounted — zero extra setup.
+
+This makes every Terreno app instantly AI-native: any model becomes callable by LLMs with the same permission, filtering, and population guarantees as REST.
+
+**Key design decisions:**
+- Safe defaults: `methods` defaults to `['list', 'read']` (read-only), `maxLimit` defaults to 50
+- `excludeFields` allows hiding sensitive/regulated fields from MCP tool schemas and responses
+- MCP config lives on each `modelRouter` call (per-model opt-in)
+- `TerrenoApp` aggregates all MCP-enabled models into a single `/mcp` endpoint
+- `modelRouter` return type is unchanged — MCP registration is a side effect (same pattern as OpenAPI)
+- Auth-agnostic: works with both JWT and Better Auth
+- Tool naming: `{prefix}_list`, `{prefix}_read`, `{prefix}_create`, `{prefix}_update`, `{prefix}_delete`
+- Populate is an option on the read tool, not a separate tool
+- Also exports `getMCPTools()` returning Vercel `ai` SDK `CoreTool` objects for direct in-process use
+
+**Related work:** The `ai-multimodal-tools-mcp` branch has `MCPService` (MCP client) and `AIService` in `@terreno/ai` — that's the consumer of what this plan builds.
+
+## Models
+
+No new Mongoose models or database changes. New TypeScript interfaces added to `@terreno/api`:
+
+### MCPConfig (on ModelRouterOptions)
+
+```typescript
+// api/src/mcp/types.ts
+
+export interface MCPConfig {
+  methods?: Array<'create' | 'list' | 'read' | 'update' | 'delete'>; // Default: ['list', 'read']
+  description?: string;            // Override auto-generated model description
+  toolPrefix?: string;             // Override tool name prefix (default: pluralized model name)
+  excludeFields?: string[];        // Fields to hide from MCP tool schemas and responses
+                                   // e.g. ['internalNote', '__v', 'metadata.secretKey']
+  maxLimit?: number;               // Max items returned by list tool (default: 50)
+  mcpResponseHandler?: (          // MCP-specific serialization (separate from REST responseHandler)
+    value: any,
+    method: 'create' | 'list' | 'read' | 'update' | 'delete',
+    user?: User
+  ) => Promise<JSONValue>;
+}
+
+// Added to existing ModelRouterOptions<T>
+export interface ModelRouterOptions<T> {
+  // ... all existing fields unchanged ...
+  mcp?: MCPConfig;  // NEW — opt-in MCP tool generation
+}
+```
+
+### Internal Registry
+
+```typescript
+// api/src/mcp/registry.ts
+
+interface MCPRegistryEntry {
+  modelName: string;
+  model: Model<any>;
+  config: MCPConfig;
+  options: ModelRouterOptions<any>;  // full router options for permissions, population, etc.
+}
+
+// Global registry — modelRouter() pushes to this when mcp config is present
+// TerrenoApp reads from this on startup
+const mcpRegistry: MCPRegistryEntry[] = [];
+```
+
+## APIs
+
+No new REST endpoints. One MCP protocol endpoint and auto-generated tools.
+
+### MCP Endpoint
+
+| Method | Path | Description |
+|--------|------|-------------|
+| POST | `/mcp` | MCP JSON-RPC 2.0 handler via `StreamableHTTPServerTransport` |
+
+Auto-mounted by `TerrenoApp` on startup if any registered models have `mcp` config.
+
+### Auto-generated MCP Tools
+
+For each model with `mcp` config, tools are generated based on `mcp.methods`:
+
+| Tool Name | Generates When | Input Schema | Output |
+|-----------|---------------|--------------|--------|
+| `{prefix}_list` | `'list'` in methods | `{ query?, page?, limit?, sort?, populate? }` | `{ data, total, page, more }` (limit capped at `maxLimit`, default 50) |
+| `{prefix}_read` | `'read'` in methods | `{ id, populate? }` | `{ data }` |
+| `{prefix}_create` | `'create'` in methods | Model fields from Mongoose schema | `{ data }` |
+| `{prefix}_update` | `'update'` in methods | `{ id, ...partial model fields }` | `{ data }` |
+| `{prefix}_delete` | `'delete'` in methods | `{ id }` | `{ success: true }` |
+
+### Auth Flow (per tool call)
+
+1. Extract headers from MCP transport
+2. Call auth provider (JWT or Better Auth) to get user — same logic as `authenticateMiddleware`
+3. Run `checkPermissions()` with the user and method
+4. For read/update/delete: load document, run object-level permission check
+5. Apply `queryFilter` for list operations
+6. Apply `populatePaths` (default) + optional extra `populate` param from tool input
+7. Strip `excludeFields` from response data
+8. Serialize through `mcpResponseHandler` if provided, otherwise default serialization
+
+### Tool Description Auto-generation
+
+Generated from model name, field names/types, and `queryFields`. Example:
+> "List todo items. Filterable by: completed, status, ownerId. Sortable. Paginated (default 50, max controlled by maxLimit)."
+
+`mcp.description` overrides the auto-generated description.
+
+### CoreTool Export
+
+`getMCPTools()` exported from `@terreno/api` returns all registered MCP tools as `Record<string, CoreTool>` for direct use with the Vercel `ai` SDK's `streamText()` / `generateText()`.
+
+## Notifications
+
+No notifications required for this feature. This is framework-level infrastructure.
+
+## UI
+
+### RTK Query Hooks (`@terreno/rtk`)
+
+- `useMCPTools()` — returns all available MCP tools from the connected backend via MCP client tool discovery
+- Handles Bearer token injection from auth state (JWT or Better Auth)
+
+### ai SDK React Integration (`@terreno/rtk`)
+
+- `useTerrenoChat({ baseURL, tools? })` — wraps `@ai-sdk/react`'s `useChat()` pre-configured for the Terreno backend's MCP endpoint
+- Streams tool calls and results to the UI
+- Works with React Native (Expo) via existing RTK/Redux store for auth state
+
+### Example App
+
+- Demo chat screen in `example-frontend` showing `useTerrenoChat()` connected to MCP-enabled models
+
+## Phases
+
+### Phase 1: Core MCP Infrastructure (`@terreno/api`)
+- MCP types and config (`MCPConfig`, registry)
+- Tool generation from modelRouter options + Mongoose schema (Zod schemas from model)
+- MCP server setup with `@modelcontextprotocol/sdk` in TerrenoApp
+- Auth-agnostic user extraction from MCP transport headers
+- Permission checking, query filtering, population in tool handlers
+- `getMCPTools()` export for direct CoreTool usage
+- Tests for tool generation, permissions, CRUD operations over MCP
+- **Deliverable:** Any Terreno app can add `mcp` to a modelRouter and get a working `/mcp` endpoint
+
+### Phase 2: Frontend Integration (`@terreno/rtk`)
+- `useTerrenoChat()` hook wrapping `@ai-sdk/react` `useChat()`
+- Auth token injection (JWT + Better Auth)
+- `useMCPTools()` hook for tool discovery
+- **Deliverable:** Frontend can connect to MCP-enabled backend with typed hooks
+
+### Phase 3: Examples & Polish
+- Example backend: add `mcp` config to existing Todo/Food models
+- Example frontend: demo chat screen using `useTerrenoChat()`
+- Auto-generated tool descriptions refinement
+- Documentation in example apps
+- **Deliverable:** Working full-stack demo of AI-native Terreno app
+
+## Feature Flags & Migrations
+
+**Feature flags:** None. MCP is opt-in per model via `mcp` config. No models have it = nothing is mounted. Zero impact on existing apps.
+
+**Data migrations:** None. No schema changes, no new collections.
+
+**Rollout:** Minor version bump of `@terreno/api`. Existing apps upgrade and add `mcp` configs at their own pace. No breaking changes.
+
+## Activity Log & User Updates
+
+No activity logging specific to this feature. MCP tool calls go through the same permission/hook pipeline as REST — any existing `postCreate`/`postUpdate`/`postDelete` hooks fire normally for write operations.
+
+## Not Included / Future Work
+
+- **Rate limiting** — structured for future hooks but not implemented
+- **MCP resources/prompts** — only tools generated from modelRouter
+- **Custom MCP tools** — use modelRouter's `endpoints` for custom routes; MCP-specific custom tools deferred
+- **Array operation tools** — handled via update, no dedicated push/patch/delete array tools
+- **WebSocket transport** — HTTP/SSE only
+- **Admin dashboard** — use `getMCPTools()` programmatically or MCP client tool listing
+- **Streaming list results** — pagination only
+- **Per-tool rate limiting or cost tracking**
+- **MCP tool versioning**
+- **Auto-generated RTK hooks per model** — codegen from OpenAPI (stretch goal)

package/dist/docs/versioned/0.23.0/implementationPlans/offline-mode.md

@@ -0,0 +1,311 @@
+# Implementation Plan: Offline Mode
+
+**Status:** Approved for implementation tasking
+**Priority:** High
+**Effort:** Medium
+
+## Context
+
+Terreno already has offline primitives in `@terreno/rtk`, including `offlineSlice`, `offlineMiddleware`, `offlineGate`, `useOfflineStatus`, and `useServerStatus`. The offline-mode plan should harden those primitives into an opt-in offline-first framework that starts with generated `modelRouter` CRUD endpoints, keeps local data through token refresh failures, and gives app teams reusable UI without blocking custom sync logic later.
+
+## Core Concept
+
+Offline mode v1 is a configurable client-side sync layer for modelRouter endpoints. Apps opt in per model/endpoint, RTK Query caches receive optimistic updates, mutations are queued in persisted Redux state while offline or auth-blocked, and queued work replays when connection and auth are healthy.
+
+The backend remains the existing modelRouter API. Backend work is limited to formalizing two modelRouter capabilities that are already present or aligned with Mongoose:
+
+- optimistic create with client-provided ObjectId-compatible `_id` values,
+- update conflict detection with `If-Unmodified-Since` / `X-Unmodified-Since-ISO` and a `409 Conflict` server document response.
+
+## Models
+
+No new server-side Mongoose models are required for v1. Offline queue and conflict records are local client state in `@terreno/rtk` and should be persisted through the consuming app's Redux persistence configuration.
+
+### Client state: offline slice
+
+```typescript
+interface OfflineState {
+  connectionQuality: "online" | "spotty" | "offline";
+  queue: QueuedMutation[];
+  conflicts: ConflictRecord[];
+  isSyncing: boolean;
+  isReplayPausedForAuth: boolean;
+  lastHealthCheck?: HealthCheckSnapshot;
+}
+
+interface QueuedMutation {
+  id: string;
+  endpointName: string;
+  modelName: string;
+  operation: "create" | "update" | "delete" | "arrayPush" | "arrayUpdate" | "arrayRemove";
+  args: unknown;
+  body?: Record<string, unknown>;
+  optimisticId?: string;
+  serverId?: string;
+  idempotencyKey: string;
+  createdAt: string;
+  lastAttemptAt?: string;
+  attemptCount: number;
+  baseUpdatedAt?: string;
+  status: "queued" | "replaying" | "authBlocked" | "conflicted" | "failed";
+  error?: string;
+}
+
+interface ConflictRecord {
+  id: string;
+  queueId: string;
+  endpointName: string;
+  modelName: string;
+  operation: "update" | "delete" | "arrayUpdate" | "arrayRemove";
+  localArgs: unknown;
+  localBody?: Record<string, unknown>;
+  serverValue: unknown;
+  baseUpdatedAt?: string;
+  serverUpdatedAt?: string;
+  dismissed: boolean;
+  createdAt: string;
+}
+
+interface HealthCheckSnapshot {
+  checkedAt: string;
+  latencyMs?: number;
+  consecutiveFailures: number;
+  recentFailureRate: number;
+}
+```
+
+### Configuration model
+
+```typescript
+interface OfflineModelRouterConfig {
+  enabled: boolean;
+  models: OfflineModelConfig[];
+  connectionQuality?: ConnectionQualityConfig;
+  auth?: OfflineAuthConfig;
+}
+
+interface OfflineModelConfig {
+  modelName: string;
+  tagType: string;
+  endpoints: {
+    create?: OfflineEndpointConfig;
+    update?: OfflineEndpointConfig;
+    delete?: OfflineEndpointConfig;
+    arrayPush?: OfflineEndpointConfig;
+    arrayUpdate?: OfflineEndpointConfig;
+    arrayRemove?: OfflineEndpointConfig;
+  };
+  idStrategy?: OfflineIdStrategy;
+  conflictStrategy?: "manual" | "keepMine" | "useServer";
+}
+
+interface OfflineEndpointConfig {
+  endpointName: string;
+  enabled?: boolean;
+  optimisticUpdate?: OfflineOptimisticUpdate;
+}
+
+interface OfflineIdStrategy {
+  generateId?: () => string;
+  requestField?: string;
+  reconcile?: "assumeClientId" | "mapServerId";
+}
+
+interface ConnectionQualityConfig {
+  healthUrl?: string;
+  pollIntervalMs?: number;
+  timeoutMs?: number;
+  spottyLatencyMs?: number;
+  offlineFailureCount?: number;
+  spottyFailureRate?: number;
+}
+
+interface OfflineAuthConfig {
+  pauseReplayWhileRefreshing?: boolean;
+  pauseReplayOnRefreshFailure?: boolean;
+  clearCacheOnLogoutOnly?: boolean;
+}
+```
+
+### ID strategy
+
+Default behavior:
+
+- generate an ObjectId-compatible string for optimistic creates,
+- send it as `_id` in the modelRouter create body,
+- assume the server returns the same ID,
+- keep cache keys stable from optimistic create through replay confirmation.
+
+Overrides:
+
+- apps can provide `generateId`,
+- apps can choose another request field,
+- apps can reconcile a server-returned ID if the backend does not preserve the client ID.
+
+## APIs
+
+### Server API surface
+
+No custom sync endpoints for v1. Offline mode targets standard modelRouter endpoints only:
+
+| Method | Path | modelRouter operation | Offline behavior |
+| --- | --- | --- | --- |
+| `POST` | `/{resource}` | create | Queue and optimistically insert a local item when offline or spotty; default `_id` comes from client ID strategy. |
+| `PATCH` | `/{resource}/:id` | update | Queue and optimistically patch cache; replay with `If-Unmodified-Since` / `X-Unmodified-Since-ISO` when a base timestamp is known. |
+| `DELETE` | `/{resource}/:id` | delete | Queue and optimistically remove/mark local item based on app strategy. |
+| `POST` | `/{resource}/:id/:field` | array push | Queue and optimistically append item to cached field. |
+| `PATCH` | `/{resource}/:id/:field/:itemId` | array update | Queue and optimistically patch array item. |
+| `DELETE` | `/{resource}/:id/:field/:itemId` | array remove | Queue and optimistically remove array item. |
+
+### Backend requirements
+
+- modelRouter create should accept client-provided ObjectId-compatible `_id` values when the schema allows them.
+- modelRouter update should keep returning `409 Conflict` when precondition headers are older than the current server `updated` timestamp.
+- The conflict response should include the current server document in a stable response field.
+- No server-side queue, sync cursor, or batch replay endpoint is included in v1.
+
+### Client API surface
+
+```typescript
+const offlineMiddleware = createOfflineMiddleware({
+  api,
+  offline: {
+    enabled: true,
+    models: [
+      {
+        modelName: "Todo",
+        tagType: "todos",
+        endpoints: {
+          create: {endpointName: "postTodos"},
+          update: {endpointName: "patchTodosById"},
+          delete: {endpointName: "deleteTodosById"},
+        },
+      },
+    ],
+  },
+});
+```
+
+New or hardened exports from `@terreno/rtk`:
+
+- `createOfflineMiddleware(options)` - opt-in middleware with modelRouter config.
+- `offlineReducer` - persisted local queue/conflict state.
+- `useOfflineStatus()` - queue, sync, conflict, and auth-blocked state.
+- `useServerStatus(config)` - computes `online`, `spotty`, or `offline`.
+- `resolveConflict({conflictId, resolution})` - supports `resolution: "keepMine" | "useServer"`.
+- `selectConnectionQuality`, `selectQueuedMutations`, `selectConflicts`.
+
+### Auth behavior
+
+- Token refresh attempts continue using the existing mutex in `emptyApi`.
+- If refresh cannot complete because the network/server is unavailable, replay pauses and queued mutations move to `authBlocked` instead of failing permanently.
+- Cached RTK Query data, queued mutations, conflicts, and optimistic local records remain intact.
+- Cache/queue clearing only happens after an explicit logout action.
+
+### Conflict resolution behavior
+
+`useServer`:
+
+- remove the queued mutation,
+- patch the RTK Query cache with the server document,
+- mark the conflict resolved/dismissed.
+
+`keepMine`:
+
+- update the queued mutation's `baseUpdatedAt` to the server `updated` timestamp,
+- keep or reapply the local optimistic cache patch,
+- replay the mutation again with the new precondition headers.
+
+## Notifications
+
+No push, email, or SMS notifications are required for v1.
+
+In-app notifications:
+
+- offline/spotty/syncing banner,
+- pending mutation count,
+- auth-blocked sync message when replay needs re-auth,
+- conflict notification with actions for "keep mine" and "use server".
+
+## UI
+
+Reusable `@terreno/ui` components and hooks:
+
+- `OfflineBanner` evolves to render `online`, `spotty`, `offline`, `syncing`, and `authBlocked` states.
+- `OfflineConflictList` renders unresolved conflicts.
+- `OfflineConflictCard` shows local/server values and resolution buttons.
+- `useOfflineStatus` remains the low-level hook for custom app UI.
+- Example frontend wires the status monitor and banner at the root layout and shows conflicts on the todos screen.
+
+## Feature Flags & Migrations
+
+- Offline mode is default off.
+- Apps enable it in store setup by mounting `offlineReducer` and adding `createOfflineMiddleware`.
+- Existing consumers are unaffected unless they opt in.
+- Queue persistence should include a version number so future changes can migrate or discard incompatible queue records safely.
+- No backend data migration is required.
+
+## Activity Log & User Updates
+
+No new activity log system is required for v1. Replayed modelRouter mutations should behave like normal API mutations on the backend. A future version can add audit metadata such as `X-Terreno-Offline-Replay` if consuming apps need to distinguish offline-originated changes.
+
+## Phases
+
+1. **RTK core hardening**
+   - Formalize modelRouter endpoint config.
+   - Add create ID strategy defaults and overrides.
+   - Add auth-blocked replay state.
+   - Add conflict resolution actions.
+   - Add queue persistence versioning.
+
+2. **Connection status and UI**
+   - Upgrade `useServerStatus` to emit `online`, `spotty`, and `offline`.
+   - Update `OfflineBanner`.
+   - Add conflict resolution UI components.
+
+3. **Backend formalization and tests**
+   - Add/verify modelRouter tests for client-provided IDs.
+   - Add/verify conflict response contract tests.
+   - Document modelRouter offline requirements.
+
+4. **Example app integration and docs**
+   - Configure todos for offline mode.
+   - Demonstrate optimistic create/update/delete.
+   - Demonstrate offline, spotty, auth-blocked, and conflict states.
+
+## Not Included / Future Work
+
+- Custom route sync strategies.
+- Server-side mutation queues.
+- Batch sync/cursor endpoints.
+- Background sync while the app is closed.
+- Binary/file upload offline queues.
+- Multi-user collaborative merge UI beyond "keep mine" / "use server".
+- Local relational database adapters.
+
+## Risks & Mitigations
+
+- **Generic optimistic updates can become too magical:** Keep modelRouter defaults simple and expose per-endpoint custom optimistic update functions.
+- **Create ID reconciliation can break cache keys:** Default to client-provided ObjectId values and only support server ID remapping through an explicit strategy.
+- **Auth refresh failures can look like logout:** Introduce a separate auth-blocked replay state and reserve destructive clearing for explicit logout.
+- **Spotty detection can be noisy:** Make thresholds configurable and expose raw health metrics for app tuning.
+- **Conflict UI can balloon:** Limit v1 to "keep mine" and "use server"; defer field-level merge tooling.
+
+## Acceptance Criteria
+
+- Offline mode is opt-in and disabled by default.
+- A modelRouter-backed model can opt into offline create/update/delete through `@terreno/rtk` configuration.
+- Offline create uses a stable default client ID and supports a custom ID strategy.
+- Queued mutations persist through app reload when the consuming app persists the offline slice.
+- Failed token refresh pauses replay without clearing RTK Query cache or queued mutations.
+- Explicit logout clears auth and configured offline/cache state.
+- Conflicts from modelRouter `409` responses can be resolved with "keep mine" or "use server".
+- Connection status can represent `online`, `spotty`, and `offline` with configurable thresholds.
+- Reusable UI surfaces show offline/spotty/syncing/auth-blocked/conflict states.
+- Example frontend demonstrates offline todos with optimistic create/update/delete and conflict resolution.
+
+---
+
+## Approval Notes
+
+The modelRouter-only client/API foundation has been approved. Implementation tasks live in `docs/tasks/offline-mode.md`.