@agent-native/core 0.39.1 → 0.40.0

package/src/templates/default/.agents/skills/storing-data/SKILL.md

@@ -1,128 +1,161 @@
 ---
 name: storing-data
 description: >-
-  How and where to store application data. Use when adding new data models,
-  deciding between settings vs Drizzle tables, reading/writing app config,
-  or working with application state.
+  How to store application data in agent-native apps. All data lives in SQL.
+  Use when adding data models, deciding where to store data, or reading/writing
+  application data.
+metadata:
+  internal: true
 ---
 
-# Storing Data
+# Storing Data — SQL is the Source of Truth
 
-## Where Data Goes
+## Rule
 
-All data lives in one SQLite database (`data/app.db`). In production, set `DATABASE_URL` to point to Turso, Neon, Supabase, or D1 — same code, no changes needed.
+All application data lives in **SQL** (SQLite locally, persistent database in production). The agent and UI share the same database. Do not store durable app data in the filesystem.
 
-There are three storage layers, each for a different kind of data:
+## How It Works
 
-### 1. Settings — app configuration
+Agent-native apps use Drizzle ORM over the configured SQL backend. Local development works out of the box with a SQLite file at `data/app.db`; production and shared preview deploys need a persistent `DATABASE_URL` because container/serverless filesystems can reset. The code should behave the same across backends, but the local SQLite file is not durable once deployed.
 
-Key-value store for persistent non-secret config that the user or agent can change. Theme, preferences, integration settings, availability schedules.
+For app code, use Drizzle's schema/query DSL by default. Raw SQL is an escape hatch for additive migrations, health checks, or one-off maintenance, not the normal way to build features.
 
-```ts
-import { getSetting, putSetting } from "@agent-native/core/settings";
+### Core SQL Stores (auto-created, available in all templates)
 
-// Read (returns null if not set)
-const prefs = await getSetting("user-preferences");
+| Store               | Purpose                                              | Access                                     |
+| ------------------- | ---------------------------------------------------- | ------------------------------------------ |
+| `application_state` | Ephemeral UI state (compose windows, navigation)     | `readAppState()` / `writeAppState()`       |
+| `settings`          | Persistent KV config (preferences, app settings)     | `getSetting()` / `putSetting()`            |
+| `oauth_tokens`      | OAuth credentials                                    | `@agent-native/core/oauth-tokens`          |
+| `sessions`          | Auth sessions                                        | `@agent-native/core/server`               |
 
-// Write (creates or replaces)
-await putSetting("user-preferences", { theme: "dark", density: "comfortable" });
-```
+### Domain Data (per-template)
+
+Define schema with the framework Drizzle helpers in `server/db/schema.ts`. Get a database instance with `const db = getDb()` from `server/db/index.ts`. All queries are async.
 
-From scripts:
 ```ts
-import { readSetting, writeSetting } from "@agent-native/core/settings";
-const prefs = await readSetting("user-preferences");
-```
+import { eq } from "drizzle-orm";
+import { table, text, integer, now } from "@agent-native/core/db/schema";
+
+export const tasks = table("tasks", {
+  id: text("id").primaryKey(),
+  title: text("title").notNull(),
+  completed: integer("completed", { mode: "boolean" })
+    .notNull()
+    .default(false),
+  createdAt: text("created_at").notNull().default(now()),
+});
 
-SSE: writes automatically notify the UI via `{ source: "settings", type: "change", key }`.
+const rows = await db.select().from(tasks).where(eq(tasks.id, taskId));
+```
 
-### 2. Application State — ephemeral UI state
+Never import `sqliteTable` / `pgTable` or column helpers from `drizzle-orm/sqlite-core` or `drizzle-orm/pg-core` in app templates. Use `@agent-native/core/db/schema` so the same schema can run against SQLite, Postgres, libSQL/Turso, D1, and other supported backends.
 
-For state the agent and UI share in real-time: what the user is looking at, compose drafts, navigation commands. Scoped by session — cleared between sessions.
+| Template     | Tables                                        |
+| ------------ | --------------------------------------------- |
+| **Mail**     | emails, labels (+ Gmail API when connected)   |
+| **Calendar** | events, bookings                              |
+| **Forms**    | forms, responses                              |
+| **Content**  | documents                                     |
+| **Slides**   | decks (JSON stored in SQL)                    |
+| **Videos**   | compositions in registry + localStorage       |
 
-```ts
-import { readAppState, writeAppState, deleteAppState, listAppState } from "@agent-native/core/application-state";
+### Agent Access
 
-// Write state (UI updates instantly via SSE)
-await writeAppState("navigate", { view: "inbox", threadId: "t-123" });
+The agent uses app-specific actions to read/write the database. Core DB scripts are for inspection and maintenance, not for implementing normal product behavior:
 
-// Read state
-const nav = await readAppState("navigation");
+- `pnpm action db-schema` — Show all tables, columns, types
+- `pnpm action db-query --sql "SELECT * FROM forms"` — Run SELECT queries
+- `pnpm action db-exec --sql "UPDATE ..."` — Last-resort ad-hoc maintenance for short columns, multi-column writes, or computed updates when no domain action exists. For several related writes, prefer `--statements '[{"sql":"...","args":[...]}]'` so they run sequentially in one transaction. Schema changes are blocked; use reviewed additive migrations/startup code instead.
+- `pnpm action db-patch --table <t> --column <c> --where "<clause>" --find "<old>" --replace "<new>"` — **Surgical search/replace on a large text column.** Sends the diff instead of re-transmitting the whole value, so it's dramatically more token-efficient than `db-exec UPDATE` when editing multi-kilobyte documents, slide HTML, dashboard/form JSON, etc. Targets exactly one row per call — narrow `--where` by primary key. Supports `--edits '[{find,replace},...]'` for batch edits and `--all` to replace every occurrence.
+- App-specific actions for domain operations — **always prefer these over raw SQL when one exists.** They encode business rules, power the client action hooks, and for editor-backed tables (documents, slides) also push live Yjs updates to open collaborative editors. `db-patch` is the generic fallback for tables without a dedicated edit action.
 
-// List by prefix (e.g., all compose drafts)
-const drafts = await listAppState("compose-");
+**For one-off maintenance, how to choose between `db-exec UPDATE` and `db-patch`:**
 
-// Delete (one-shot commands: UI reads, then agent or UI deletes)
-await deleteAppState("navigate");
-```
+| Scenario                                                       | Use          |
+| -------------------------------------------------------------- | ------------ |
+| `SET status = 'published'` on one row                          | `db-exec`    |
+| `SET calories = calories + 50`                                 | `db-exec`    |
+| Updating several columns at once                               | `db-exec`    |
+| Inserting/updating several rows as one logical operation        | `db-exec --statements` |
+| Fixing a typo in a 50KB markdown document's `content` column   | `db-patch`   |
+| Changing a single key in a dashboard's JSON blob               | `db-patch`   |
+| Tweaking one paragraph of slide HTML stored in `decks.data`    | `db-patch`   |
+| Any edit where you'd otherwise re-send thousands of characters | `db-patch`   |
 
-SSE: writes automatically notify the UI via `{ source: "app-state", type: "change", key }`.
+All of these honor the per-user / per-org data scoping — you can't read or write rows outside the current user's data, regardless of which tool you choose.
 
-### 3. Drizzle Tables — structured domain data
+### Frontend Access
 
-For data with schemas, relationships, and queries: forms, bookings, emails, compositions. Define tables in `server/db/schema.ts` using Drizzle ORM.
+The frontend calls actions using React Query hooks from the client API. The framework owns the HTTP transport behind these hooks, so components should not call action routes with raw `fetch`.
 
 ```ts
-import { table, text, integer } from "@agent-native/core/db/schema";
+import { useActionQuery, useActionMutation } from "@agent-native/core/client";
 
-export const bookings = table("bookings", {
-  id: text("id").primaryKey(),
-  name: text("name").notNull(),
-  email: text("email").notNull(),
-  startTime: integer("start_time").notNull(),
-  endTime: integer("end_time").notNull(),
-});
+// Read data
+const { data } = useActionQuery("list-meals", { date: "2025-01-01" });
+
+// Write data
+const { mutate } = useActionMutation("log-meal");
 ```
 
-Query via `getDb()` singleton from `server/db/index.ts`.
+Actions are the **preferred way** for the frontend to access data. You rarely need custom `/api/` routes — only for file uploads, streaming, webhooks, or OAuth callbacks.
 
-### 4. OAuth Tokens — credentials
+### Production / Cloud Deployment
 
-For OAuth tokens acquired at runtime (Google, etc.). Never store these in settings — use the dedicated store.
+Local SQLite works out of the box for development. To deploy to production or any environment where data must survive restarts:
 
-```ts
-import { saveOAuthTokens, getOAuthTokens, listOAuthAccounts } from "@agent-native/core/oauth-tokens";
+1. Set `DATABASE_URL` to a persistent SQL database.
+2. Set `DATABASE_AUTH_TOKEN` only when the provider requires a separate token, such as Turso/libSQL.
+3. No code changes should be needed when the schema and queries stay portable.
 
-await saveOAuthTokens("google", "user@gmail.com", { access_token: "...", refresh_token: "..." });
-const tokens = await getOAuthTokens("google", "user@gmail.com");
-const accounts = await listOAuthAccounts("google");
-```
+Turso is one valid option, not the required option. Common choices include Neon or Supabase Postgres, Turso/libSQL, plain Postgres, durable SQLite, Cloudflare D1 bindings, and managed platform SQL environments when available.
+
+### Real-time Sync
+
+Polling streams database changes to the UI. When the agent writes to the database via scripts, the UI updates automatically via `useDbSync()` which invalidates React Query caches.
+
+## Do
+
+- Use Drizzle ORM for structured domain data (forms, bookings, documents)
+- Use Drizzle query builder methods (`select`, `insert`, `update`, `delete`) and portable operators from `drizzle-orm` (`eq`, `and`, `or`, `inArray`, `desc`, etc.) for app reads/writes
+- Use framework schema helpers from `@agent-native/core/db/schema`, not dialect-specific Drizzle imports
+- Use the `settings` store for app configuration and user preferences
+- Use `application-state` for ephemeral UI state that the agent and UI share
+- Use `oauth-tokens` for OAuth credentials
+- Use core DB scripts (`db-schema`, `db-query`, `db-exec`, `db-patch`) for ad-hoc database operations
+- Use `db-exec --statements` instead of several separate `db-exec` calls for related writes; it is faster and rolls back the whole batch if one statement fails
+- Reach for `db-patch` instead of `db-exec UPDATE` whenever you're making a small change to a large text/JSON column — it's much cheaper on tokens
 
-### 5. Secrets / Credentials — encrypted values
+## Don't
 
-For API keys, service tokens, webhook secrets, and user/org/workspace
-credentials. Register user-facing secrets with the secrets registry and read
-them server-side with `readAppSecret`, or use `saveCredential` /
-`resolveCredential` for scoped credential lookup. Never store these values in
-settings, application state, source code, docs, examples, logs, or action
-responses.
+- Don't store structured app data as JSON files
+- Don't store app state in localStorage, sessionStorage, or cookies (except for UI-only preferences like sidebar width)
+- Don't keep state only in memory (server variables, global stores)
+- Don't use Redis or any external state store for app data
+- Don't implement product features with raw SQL or `getDbExec()` when Drizzle can express the query
+- Don't write SQLite-only or Postgres-only SQL in app code
+- Don't interpolate user input directly into SQL queries — use Drizzle ORM's query builder
 
-## Which Layer to Use
+## Security
 
-| Data | Layer | Why |
-|------|-------|-----|
-| User preferences, theme, config | Settings | Persistent KV, SSE notifications, simple read/write |
-| What the user sees on screen | Application State | Ephemeral, real-time sync, agent ↔ UI bridge |
-| Compose drafts, wizard steps | Application State | Temporary, deleted when done |
-| Domain records (forms, bookings) | Drizzle table | Needs schema, queries, relationships |
-| API keys, service tokens, webhook secrets | Secrets / credentials | Encrypted and scoped; never client-readable |
-| OAuth refresh tokens | OAuth Tokens | Secure, per-provider, per-account |
+- **SQL injection** — Use Drizzle ORM's query builder, never raw string interpolation for SQL queries
+- **Validate before writing** — Check data shape before writing, especially for user-submitted data
 
-## Environment Variables
+## Application State and Context Awareness
 
-Infrastructure config stays in `.env` — these differ per deployment:
+When storing app-state, include **navigation state** — the agent needs to know what the user is looking at. The `application_state` table holds ephemeral UI state that both the agent and UI share. Key patterns:
 
-- `DATABASE_URL` — database connection (default: `file:./data/app.db`)
-- `DATABASE_AUTH_TOKEN` — for remote databases
-- `GOOGLE_CLIENT_ID`, `GOOGLE_CLIENT_SECRET` — OAuth app credentials
-- `ACCESS_TOKEN` — production auth token
+- **`navigation` key** — the UI writes current view and selection on every route change. The agent reads this before acting.
+- **`navigate` key** — the agent writes one-shot commands to navigate the UI. The UI processes and deletes them.
+- **Domain-specific keys** (e.g., `compose-{id}`) — bidirectional state for features like email drafts.
 
-Everything else (user settings, app state, domain data) goes in SQL through the
-appropriate store. User/org/workspace credential values go in the encrypted
-secrets/credential stores, not plain settings rows.
+When adding a new data model or feature, also consider what navigation and selection state needs to be exposed via application-state. See the **context-awareness** skill for the full pattern.
 
-## Security Rules
+## Related Skills
 
-- **Never store API keys or secrets in Settings or Application State** — use the secrets registry / vault or `saveCredential` / `resolveCredential` for API keys and service tokens, deploy env vars only for deploy-level secrets, and `oauth_tokens` for OAuth credentials. Settings and application state are readable by the client.
-- **Every Drizzle table with user data must have `owner_email`** — the framework auto-scopes queries in production so users only see their own data. Run `pnpm action db-check-scoping` to verify. See the `security` skill for the full model.
-- **Never return secrets in action responses** — action responses may be visible in the agent chat or sent to the client. Keep credentials server-side only.
+- **context-awareness** — How to expose navigation and selection state via application-state
+- **real-time-sync** — Set up polling so the UI updates when the database changes
+- **actions** — Create actions with `defineAction` to query the database
+- **client-methods** — Keep route details behind named client helpers/hooks
+- **self-modifying-code** — The agent can also modify the app's source code

package/src/templates/default/DEVELOPING.md

@@ -6,7 +6,7 @@ This guide is for development-mode agents editing this app's source code. For ap
 
 **Client-side-first rendering:** This app uses React Router v7 framework mode with `ssr: true`, but all app content renders **client-side only**. The server renders only the HTML shell (meta tags, styles, scripts) plus a loading spinner. This is enforced by the `ClientOnly` wrapper in `root.tsx` — never remove it. Browser APIs (`window`, `localStorage`, `new Date()`) are safe to use anywhere in app code because components never run on the server.
 
-**Do NOT fetch data server-side** in route loaders. The standard pattern is: server renders a spinner, client hydrates, React Query hooks fetch from `/api/*`.
+**Do NOT fetch data server-side** in route loaders unless the page genuinely needs SEO/OG content. The standard pattern is: SSR renders the shell, client hydrates, and React reads/writes normal app data through actions with `useActionQuery` / `useActionMutation`.
 
 ## Adding a Page
 
@@ -30,7 +30,7 @@ In a workspace, this app can be mounted under `/<app-id>`. React Router already
 | `app/routes/review.tsx` | `/review`          | `/<app-id>/review`  |
 | `app/routes/$id.tsx`    | `/:id`             | `/<app-id>/:id`     |
 
-Use `<Link to="/review">` and `navigate("/review")` inside this app. Do not prefix React Router paths with `/<app-id>` or the URL can double-prefix, e.g. `/<app-id>/<app-id>/review`. Use `appPath()` for raw `href`s/static assets, `appApiPath()` for `/api/*`, and `agentNativePath()` for `/_agent-native/*`.
+Use `<Link to="/review">` and `navigate("/review")` inside this app. Do not prefix React Router paths with `/<app-id>` or the URL can double-prefix, e.g. `/<app-id>/<app-id>/review`. Use `appPath()` for raw `href`s/static assets, `agentNativePath()` for `/_agent-native/*`, and `appApiPath()` only for legitimate route-only `/api/*` endpoints.
 
 Each route file exports a default component and optional `meta()`:
 
@@ -46,17 +46,15 @@ export default function MyPageRoute() {
 }
 ```
 
-## Adding an API Route
+## Adding App Data
 
-Create a file in `server/routes/api/`. The filename determines the URL path and HTTP method:
+Normal app data starts as an action, not a custom route. Add `actions/<verb>-<resource>.ts` with `defineAction`, mark reads with `http: { method: "GET" }`, and call reads/writes from React with `useActionQuery` / `useActionMutation` from `@agent-native/core/client`. This keeps the UI and agent on one contract and lets mutating actions refresh action-backed queries automatically.
 
-```
-server/routes/api/items/index.get.ts    → GET  /api/items
-server/routes/api/items/[id].get.ts     → GET  /api/items/:id
-server/routes/api/items/[id].patch.ts   → PATCH /api/items/:id
-```
+## Adding a Route-Only Endpoint
+
+Use `server/routes/api/` only for protocols that cannot be modeled as JSON actions: multipart uploads, streaming/SSE/WebSocket, webhooks, OAuth callbacks/redirects, public SEO/OG endpoints, or binary/static asset serving. Do not add `/api/*` routes for normal CRUD, data queries, or pass-through wrappers around actions; the action endpoint already exists at `/_agent-native/actions/:name`.
 
-Each file exports a default `defineEventHandler`.
+Each route-only endpoint still exports a default `defineEventHandler`, but keep shared app logic in actions or server libraries so agent and UI behavior do not fork.
 
 ## Server Plugins
 
@@ -84,10 +82,9 @@ export default defineNitroPlugin(async (nitroApp) => {
 | Agent chat context state helpers             | Optional advanced helpers for two-way sync with staged context chips       |
 | `agentChat`                                  | Send messages to agent from scripts (server-side)                          |
 
-## Adding a Script
+## Adding an Action
 
-Create `actions/my-script.ts` exporting `default async function(args: string[])`.
-Run with: `pnpm action my-script --arg value`
+Create `actions/<verb>-<resource>.ts` with `defineAction`. Run with `pnpm action <name> --id value`; React callers should use `useActionQuery` for GET actions and `useActionMutation` for mutating actions, not a matching `/api/*` wrapper.
 
 ## Sending to Agent Chat
 

package/src/templates/workspace-core/.agents/skills/client-methods/references/legacy-client-fetch-audit-2026-06-03.md

@@ -7,6 +7,11 @@ contracts; some need new actions or helper modules first.
 
 ## Highest Priority
 
+- 2026-06-07 follow-up: the same high-priority route-first clusters are still
+  present. The biggest migrations remain Analytics, Calendar, Mail, Slides, and
+  Content. Do not copy these patterns into new work; when editing the relevant
+  area, add or reuse actions first, then call them with `useActionQuery`,
+  `useActionMutation`, or `callAction`.
 - `templates/analytics/app/pages/analyses/AnalysesList.tsx`,
   `templates/analytics/app/pages/analyses/AnalysisDetail.tsx`,
   `templates/analytics/app/components/layout/Sidebar.tsx`, and
@@ -43,6 +48,10 @@ contracts; some need new actions or helper modules first.
 - Content comments and versions are partially migrated. Add missing actions such
   as `resolve-comment`, `delete-comment`, `list-document-versions`, and
   `restore-document-version`.
+- Plans version history is the model to copy for new history/rollback work:
+  `list-plan-versions`, `get-plan-version`, and `restore-plan-version` are
+  action-native, and the UI calls them through action hooks. Do not copy
+  Content's legacy document-version `/api/*` helpers into new version panels.
 
 ## Acceptable Exceptions
 

package/src/templates/workspace-core/.agents/skills/writing-agent-instructions/SKILL.md

@@ -74,6 +74,33 @@ Keep one canonical instructions file: `AGENTS.md`. If a client expects
 hand-maintained files drift, and the agent ends up with contradictory rules.
 One source of truth, linked where needed.
 
+## Keep generated guidance in sync
+
+Framework guidance is authored once in this repo and copied outward. Treat
+`.agents/skills/` as the canonical source for shared skills. Generated
+workspace skills in `packages/core/src/templates/workspace-core/.agents/skills/`
+and first-party template copies of shared skills must stay byte-for-byte in
+sync; run `pnpm sync:workspace-skills` after editing a shared skill, and
+`pnpm guard:workspace-skills` before calling the guidance done.
+
+Generated app and workspace instructions must teach the same action-first data
+contract:
+
+- Normal app data goes through `defineAction` files in `actions/`.
+- React calls actions with `useActionQuery`, `useActionMutation`, or
+  `callAction`; route paths are a transport detail hidden behind helpers.
+- Custom `/api/*` routes are only for route-shaped protocols such as uploads,
+  streaming, webhooks, OAuth callbacks, public SEO/OG endpoints, or binary
+  assets.
+- Do not create pass-through routes whose main job is to call, repackage, or
+  re-export an action.
+
+When documenting version history, restore, or audit trails, use actions for
+full restorable snapshots (`list-<resource>-versions`,
+`get-<resource>-version`, `restore-<resource>-version`). Do not copy legacy
+raw-route version panels, such as document-version `/api/*` helpers, into new
+features. The Plans version-history pattern is the preferred model.
+
 ## SKILL.md frontmatter must say what AND when
 
 The `description` is the only thing the agent sees when deciding whether to read