@async/db 0.2.0

package/docs/README.md

@@ -0,0 +1,42 @@
+# @async/db Docs
+
+This folder is the durable markdown manual for @async/db. The root [README](../README.md) is the mini guide; these pages hold the task and contract details.
+
+## Start
+
+- [Getting Started](./getting-started.md): install, first fixture, sync, serve, viewer, REST call, and first schema.
+- [Concepts](./concepts.md): data-first fixtures, schema-first fixtures, mixed resources, runtime stores, source writebacks, and product boundaries.
+- [Examples](../examples/basic/README.md): runnable example READMEs are the authority for example-specific commands.
+
+## Build Local Data
+
+- [Fixtures And Schemas](./fixtures-and-schemas.md): JSON, JSONC, CSV, schema files, `.schema.js`, Standard Schema validators, computed fields, source readers, nested folders, inference, and validation.
+- [TypeScript Schema Sources](./typescript-schema-sources.md): JavaScript ESM schemas and TypeScript-authored schemas compiled to supported runtime files.
+- [Generated Files](./generated-files.md): `.db/`, state, generated TypeScript, committed generated outputs, schema manifests, and cleanup rules.
+- [Configuration](./configuration.md): `db.config.mjs`, fixture folders, resource naming, strictness, registered operations, mock delay/errors, server options, and runtime fork boundaries.
+- [Production JSON Database](./json-production.md): scoped production use for `@async/db/json`, hard limits, operation boundaries, and mixed-store graduation.
+- [Resource Graduation And Mixed Stores](./store-graduation.md): move one resource from JSON to SQLite/Postgres/custom stores while preserving app-facing operations.
+- [Fork And Branch Workflows](./fork-branch-workflows.md): tenants, debug copies, snapshots, branches, and resource migrations as app-owned workflows.
+- [CMS Storage Patterns](./cms-storage-patterns.md): app-owned CMS draft/publish helpers over JSON files, SQLite indexes, Postgres, and static JSON outputs.
+- [Schema UI example](../examples/schema-ui/README.md): manifest-driven CMS HTML with **`serve.mjs`** SSR from live mirror rows (`node ./examples/schema-ui/serve.mjs`).
+
+## Serve And Integrate
+
+- [Server And Viewer](./server-and-viewer.md): REST routes, registered operations, GraphQL boundary, viewer, CSV import, watch behavior, batching, response formats, and local trust boundaries.
+- [Prototype To Production REST Guide](./prototype-to-production.md): move `/db/*` prototypes to `/api/db/*` or `/api/*`, registered operation refs, and route lockdown.
+- [Package API](./package-api.md): CLI commands, runtime API, HTTP client operations, schema/config helpers, and package export map.
+- [Integrations](./integrations.md): Vite plugin, Hono route registration, SQLite starter generation, and optional dependency boundaries.
+
+## Maintain The Repo
+
+- [Architecture](./architecture.md): source-to-runtime flow, implementation boundaries, generated outputs, and where to start for code changes.
+- [CI And Release](./ci-and-release.md): verification commands, Node versions, package `files`, pack dry-run expectations, and release hygiene.
+- [Product Spec](../SPEC.md): full product model and acceptance criteria.
+
+## Documentation Rules
+
+- Keep the root README short enough to scan.
+- Keep deep behavior near the contract it belongs to.
+- Keep examples runnable and focused.
+- Prefer exact commands and repo-relative links.
+- When adding generated output examples, state whether the output is normally committed.

package/docs/architecture.md

@@ -0,0 +1,112 @@
+# @async/db Architecture
+
+@async/db is a dependency-light Node.js ESM package. Fixture files become generated schema metadata, TypeScript types, runtime JSON state, and local API/viewer routes.
+
+Use this page when deciding where behavior belongs in the implementation. Use [SPEC.md](../SPEC.md) for the full product and acceptance model.
+
+## Main Flow
+
+```txt
+db/*.json, *.jsonc, *.csv, *.schema.json(c), *.schema.mjs, *.schema.js
+  -> source readers
+  -> resource schemas and diagnostics
+  -> sync output
+  -> .db/schema.generated.json
+  -> .db/types/index.d.ts and optional committed generated files
+  -> selected runtime store
+  -> package API, REST, GraphQL, viewer, client, and generators
+```
+
+## Public Surfaces
+
+| Surface | Entry point | Notes |
+| --- | --- | --- |
+| CLI | `src/cli.ts`, `src/cli/index.ts`, `src/cli/commands/` | `sync`, `types`, `schema`, `doctor`, `check`, `create`, `serve`, `generate hono`. |
+| Runtime API | `src/index.ts`, `src/features/runtime/` | Collections, singleton documents, validation, and storage adapters. |
+| Config API | `src/config-public.ts`, `src/config.d.ts` | `defineConfig`, manifest helpers, and user config shape. |
+| Schema helpers | `src/schema-builders.ts`, `src/schema.d.ts` | `.schema.mjs` and `.schema.js` authoring helpers. |
+| HTTP client | `src/client.ts` | REST, GraphQL, registered query operations, direct batching, automatic batching, fork support. |
+| REST server | `src/server.ts`, `src/rest/` | Dependency-free local routes and response shaping. |
+| GraphQL | `src/graphql/` | Dependency-free subset parser, executor, and HTTP handler. |
+| Viewer | `src/web/` | Built-in UI served at `server.apiBase`, defaulting to `/__db`. |
+| Vite integration | `src/vite.ts`, `src/integrations/` | Optional dev server plugin and virtual client. |
+| JSON/Hono/SQLite/Postgres/KV | `src/json.ts`, `src/hono.ts`, `src/sqlite.ts`, `src/postgres.ts`, `src/kv.ts`, `src/redis.ts`, `src/generate/hono.ts` | First-party JSON file database surface, optional runtime integrations, injected-client stores, and generated starter output. |
+
+## Core Boundaries
+
+- Source discovery and loading live under `src/features/schema/sources.ts`. Built-in readers handle JSON, JSONC, CSV, and schema files; custom readers normalize into the same data/schema source shape.
+- Field normalization, inference, relations, and resource construction live under `src/features/schema/`.
+- Validation and diagnostics live under `src/features/schema/validation.ts` and nearby schema feature modules.
+- Sync lives under `src/features/sync/`. It writes generated schema, generated types, optional schema manifests, source metadata, and hydrates runtime store state.
+- Runtime storage lives under `src/features/storage/` and `src/features/runtime/`. The default and first-party file database store is JSON files under `.db/state`; memory, static, sourceFile, SQLite, Postgres, KV, Redis-like, and custom stores fit behind the same store boundary.
+- HTTP serving starts in `src/server.ts`. REST routing lives in `src/rest/`, GraphQL lives in `src/graphql/`, and built-in viewer HTML/JS lives in `src/web/`.
+- Optional graduation paths are separate from the core. Hono/SQLite starter generation lives in `src/generate/hono.ts` and `src/generate/hono/`; optional integrations live in `src/integrations/`.
+
+## Start Here When Changing Behavior
+
+| Change | Start with |
+| --- | --- |
+| Source discovery or custom readers | `src/features/schema/sources.ts` |
+| Schema inference or field normalization | `src/features/schema/fields.ts`, `src/features/schema/resource.ts` |
+| Schema validation or diagnostics | `src/features/schema/validation.ts` |
+| Runtime store hydration or generated outputs | `src/features/sync/index.ts`, `src/types.ts`, `src/schema-manifest.ts` |
+| Package runtime APIs | `src/features/runtime/collection.ts`, `src/features/runtime/document.ts`, `src/features/runtime/db.ts` |
+| REST routes or response shaping | `src/rest/handler.ts`, `src/rest/shape.ts` |
+| GraphQL parsing or execution | `src/graphql/parser.ts`, `src/graphql/execute.ts` |
+| Built-in viewer behavior | `src/web/viewer.ts` |
+| CLI behavior | `src/cli/index.ts`, `src/cli/commands/` |
+| Hono/SQLite starter generation | `src/generate/hono.ts`, `src/generate/hono/` |
+| Optional Hono integration | `src/hono.ts` |
+| Optional SQLite adapter | `src/sqlite.ts` |
+| Optional Postgres/KV/Redis-like stores | `src/postgres.ts`, `src/kv.ts`, `src/redis.ts`, `src/integrations/` |
+
+## Generated Outputs
+
+Default generated runtime output:
+
+```txt
+.db/schema.generated.json
+.db/types/index.d.ts
+.db/state/*.json
+```
+
+Normally `.db/` is uncommitted. Committed generated outputs are allowed only when configured, such as `outputs.committedTypes` or `outputs.schemaManifest`.
+
+Examples that intentionally commit generated outputs:
+
+```txt
+examples/advanced/src/generated/db.types.d.ts
+examples/basic/src/generated/db.types.d.ts
+examples/computed-fields/src/generated/db.types.d.ts
+examples/content-collections/src/generated/db.types.d.ts
+examples/production-json/src/generated/db.types.d.ts
+examples/schema-first/src/generated/db.types.d.ts
+examples/schema-manifest/src/generated/db.schema.json
+examples/schema-manifest/src/generated/db.types.d.ts
+examples/schema-ui/src/generated/db.schema.json
+examples/schema-ui/src/generated/db.types.d.ts
+```
+
+See [Generated Files](./generated-files.md).
+
+## Local Trust Model
+
+- `async-db serve` binds to `127.0.0.1` by default and is intended for local development.
+- `.schema.mjs` and `.schema.js` files execute as local project JavaScript. Treat them like source code, not untrusted data.
+- `db.config.mjs`, source readers, format renderers, and manifest hooks also execute as local project code.
+- The viewer CSV import endpoint writes CSV files into the configured `dbDir`.
+- The default `json` store keeps source fixtures clean and writes app changes to `.db/state`.
+- The `sourceFile` store may write generated ids back to plain `.json` source fixtures when configured intentionally.
+- `.db/` is generated runtime output and should normally stay uncommitted.
+
+## Verification
+
+Before handing off changes:
+
+```bash
+npm run check
+npm test
+npm --cache /private/tmp/db-npm-cache pack --dry-run
+```
+
+See [CI And Release](./ci-and-release.md) for package and docs checks.

package/docs/ci-and-release.md

@@ -0,0 +1,177 @@
+# CI And Release
+
+This page documents the verification and package checks that keep docs, examples, and generated-file policy honest.
+
+## Supported Node Versions
+
+The package supports Node.js 20 and newer.
+
+CI runs on Node.js 20, 22, and 24 through `.github/workflows/ci.yml`.
+
+Generated Hono/SQLite standalone apps may require newer Node versions because SQLite output uses `node:sqlite`.
+
+## Required Verification
+
+Run these before handing off changes:
+
+```bash
+npm run release:check
+```
+
+If the default npm cache has ownership or permission issues on this machine, use a temp cache for the pack check:
+
+```bash
+npm --cache /private/tmp/async-db-npm-cache pack --dry-run
+```
+
+`release:check` expands to:
+
+```bash
+npm run check
+npm test
+npm pack --dry-run
+```
+
+## Useful Smoke Commands
+
+Use the dev loop while editing package code:
+
+```bash
+npm run dev          # watch src and relaunch all examples
+npm run examples     # one-shot all examples server for smoke checks
+npm run watch        # compile dist and test-build in watch mode only
+npm run cli -- sync --cwd ./examples/basic
+```
+
+Use `npm run dev` for active package work. Use `npm run examples` for CI-like example smoke checks.
+Npm task entrypoints live under `scripts/tasks/`; reusable helper scripts live directly under `scripts/`.
+
+```bash
+npm run db -- sync --cwd ./examples/basic
+npm run db -- schema validate --cwd ./examples/basic
+npm run db -- create users '{"id":"u_2","name":"Grace Hopper","email":"grace@example.com"}' --cwd ./examples/basic
+npm run examples
+```
+
+Use `npm run examples -- --tailscale-serve` only for local tailnet previews.
+The default examples command remains loopback-only and does not configure
+Tailscale.
+
+The local REST server binds a loopback port. In sandboxed environments this may require explicit approval:
+
+```bash
+npm run db -- serve --cwd ./examples/basic
+```
+
+## Package Files Allowlist
+
+`package.json` includes:
+
+```txt
+src/**/*.js
+src/**/*.d.ts
+scripts
+examples/*/README.md
+examples/*/example.json
+examples/*/package.json
+examples/*/db/**
+examples/*/db.config.mjs
+examples/*/src/**
+docs/**/*.md
+!docs/goals/**
+db.config.example.mjs
+CHANGELOG.md
+README.md
+SPEC.md
+```
+
+Docs restructuring affects the published package because `docs/**/*.md` is included, except local GoalBuddy boards under `docs/goals/`. Use `npm pack --dry-run` to confirm the tarball contains the expected docs and does not include generated runtime output.
+
+## Generated Runtime Output
+
+Generated `.db/` output should normally stay uncommitted.
+
+Committed generated files are allowed when configured:
+
+```txt
+examples/advanced/src/generated/db.types.d.ts
+examples/basic/src/generated/db.types.d.ts
+examples/computed-fields/src/generated/db.types.d.ts
+examples/content-collections/src/generated/db.types.d.ts
+examples/production-json/src/generated/db.types.d.ts
+examples/schema-first/src/generated/db.types.d.ts
+examples/schema-manifest/src/generated/db.schema.json
+examples/schema-manifest/src/generated/db.types.d.ts
+examples/schema-ui/src/generated/db.schema.json
+examples/schema-ui/src/generated/db.types.d.ts
+```
+
+If a smoke command writes `.db/` inside an example, remove that generated runtime state before finalizing unless the task explicitly asks to commit it.
+
+## Docs Link Checks
+
+Use lightweight checks after docs edits:
+
+```bash
+wc -l README.md
+rg -n "\\]\\(#" README.md docs examples
+rg -n "docs/|SPEC.md|architecture.md" README.md docs examples -g "*.md"
+```
+
+The first check confirms the README stayed compact. The second highlights same-page anchors that may need review after moving sections. The third makes docs and source-of-truth links easy to inspect.
+
+## Release Hygiene
+
+- Keep `CHANGELOG.md` focused on release history, not docs planning notes.
+- Keep root `SPEC.md` as the product and acceptance source of truth.
+- Keep implementation ownership and source maps in [Architecture](./architecture.md) and `AGENTS.md`, not duplicated across every doc page.
+
+## Release Automation
+
+Release pull requests and npm publication are handled by `.github/workflows/release.yml`.
+
+- Every push to `main` runs Release Please.
+- After `0.1.0`, Release Please opens or updates a release PR with the next version, `package.json`, `.release-please-manifest.json`, and `CHANGELOG.md`.
+- Merging a release PR creates the GitHub release.
+- When a Release Please release is created, the same workflow checks out the release commit, runs `npm run release:check`, packs the package, publishes `@async/db` to npm, and uploads the tarball to the GitHub release.
+- The first `0.1.0` release is seeded in `.release-please-manifest.json`; publish it by pushing the existing `v0.1.0` tag, or by running the `Release` workflow manually with `v0.1.0` after the tag exists.
+
+The release workflow uses npm Trusted Publishing through GitHub Actions OIDC. Before the first automated publish, configure npm for:
+
+```txt
+package: @async/db
+owner/repo: async-framework/async-db
+workflow: release.yml
+environment: none
+```
+
+Keep the package public through `publishConfig.access: "public"` and the workflow publish command:
+
+```bash
+npm publish --access public
+```
+
+If Trusted Publishing is not configured yet, the release workflow can create the release PR and GitHub release, but npm publish will fail until npm trusts this repository and workflow.
+
+## First Release
+
+The first public package version is already recorded as `0.1.0` in `package.json`, `CHANGELOG.md`, and `.release-please-manifest.json`. After npm Trusted Publishing is configured:
+
+```bash
+git tag v0.1.0
+git push origin v0.1.0
+```
+
+The tag publish path validates that the tag matches `package.json`, runs `npm run release:check`, publishes the tarball, creates the GitHub release if needed, and uploads the tarball as a release asset.
+
+## Manual Release Checks
+
+Use local scripts for preflight and emergency manual publish work:
+
+```bash
+npm run release:check
+npm run release:pack
+npm run release:publish
+```
+
+Prefer the GitHub workflow for normal releases so npm provenance is tied to the release commit.

package/docs/cms-storage-patterns.md

@@ -0,0 +1,108 @@
+# CMS Storage Patterns
+
+CMS workflows are app code on top of `@async/db` primitives. The package should not know what `draft`, `review`, `published`, or `unpublished` mean.
+
+## App Code Boundary
+
+```js
+export function createCms(db, { tenantId }) {
+  return {
+    async saveDraft(pageId, changes) {
+      const tenant = await db.forks.open(tenantId);
+      const draft = await tenant.branches.open('draft');
+      return draft.collection('pages').patch(pageId, {
+        ...changes,
+        status: 'draft',
+      });
+    },
+
+    async publish() {
+      const tenant = await db.forks.open(tenantId);
+      const draft = await tenant.branches.open('draft');
+      const published = await tenant.branches.open('published');
+      const pages = await draft.collection('pages').all();
+
+      await published.collection('pages').replaceAll(
+        pages
+          .filter((page) => page.status === 'published')
+          .map(({ id, slug, title, status, summary, bodyMarkdown }) => ({
+            id,
+            slug,
+            title,
+            status,
+            summary,
+            bodyMarkdown,
+          })),
+      );
+    },
+  };
+}
+```
+
+`@async/db` owns forks, branches, snapshots, resources, operations, migrations, and routing. `@async/db/json` owns file/object storage, durability, encryption hooks, corruption checks, and layout.
+
+## Pattern 1: One JSON File Per Resource
+
+Use this for simple content:
+
+```txt
+forks/tenant_acme/branches/draft/resources/pages.json
+forks/tenant_acme/branches/published/resources/pages.json
+```
+
+The app filters draft records and writes the public result into the `published` branch.
+
+## Pattern 2: App-Owned Static JSON Outputs
+
+When static hosting, S3/R2 reads, or page-level cache invalidation matter, keep that materialization in app code instead of configuring the JSON store layout:
+
+```js
+const tenant = await db.forks.open(tenantId);
+const publishedBranch = await tenant.branches.open('published');
+const published = await publishedBranch.collection('pages').all();
+
+for (const page of published) {
+  await writePublicJson(`pages/${page.slug}.json`, page);
+}
+```
+
+The package owns resource reads/writes and branch isolation. The app owns any extra files it wants to publish for CDN or public API access.
+
+## Pattern 3: JSON Files With SQLite Metadata Index
+
+Use JSON files as documents and SQLite as an app-owned metadata index:
+
+```txt
+content/pages/home.json
+content/pages/about.json
+content-index.sqlite
+```
+
+The app writes the JSON document and updates SQLite metadata such as `slug`, `status`, `published_at`, and `file_path`. Queries use SQLite to find files, then read JSON documents.
+
+## Pattern 4: Draft JSON To Published Postgres
+
+Keep editorial drafts as JSON, then materialize published records into Postgres:
+
+```js
+await cms.publish({
+  from: { branch: 'draft', store: 'json' },
+  to: { branch: 'published', store: 'postgres' },
+});
+```
+
+This is useful when the public site needs database indexes but editorial workflow still benefits from file-like JSON.
+
+## Pattern 5: Static JSON API Materialization
+
+Publishing can write multiple public JSON outputs:
+
+```txt
+published/pages.json
+published/pages/home.json
+published/navigation.json
+published/search-index.json
+published/sitemap.json
+```
+
+That materialization is app-owned. The package provides resource reads/writes, branches, snapshots, and JSON storage helpers.

package/docs/concepts.md

@@ -0,0 +1,141 @@
+# Concepts
+
+@async/db turns local fixture sources into generated contracts, runtime state, local APIs, and type metadata. The default path is intentionally small: write fixture files first, then add schema only when the app contract needs it.
+
+## Product Boundary
+
+@async/db is:
+
+- local development and test infrastructure
+- a simple JSON file database for scoped low-write production resources
+- data-first by default
+- REST-first by default
+- dependency-light in the core package
+- useful before the real database or backend contract is settled
+- a stable API/data layer while selected resources graduate to SQLite, Postgres, or custom stores
+- a way to isolate data in forks, branches, snapshots, and resource-by-resource migrations
+
+@async/db is not:
+
+- a hosted production database service
+- a multi-writer transactional database
+- an auth or permission system
+- an ORM
+- broad JSON Schema compatibility
+- a replacement for application-owned validation in production services
+
+The JSON store is appropriate in production only for small, low-write, file-suitable resources such as settings, feature flags, content, templates, policy rules, and seed data. Use SQLite, Postgres, or another app-owned store for high-write records, transactional data, analytics/events, chat/messages, ledgers, inventory counters, multi-instance writes, and compliance-heavy data.
+
+Forks and branches are generic database lifecycle primitives. App code can use them for tenants, CMS draft/publish workflows, feature-flag previews, settings rollback, prompt experiments, debug snapshots, and forked test environments. @async/db does not know what "paid", "published", or "approved" means; those decisions belong to the app.
+
+## Data-First
+
+Start with `db/*.json`, `db/*.jsonc`, or `db/*.csv` when you already have sample records.
+
+```json
+[
+  {
+    "id": "u_1",
+    "name": "Ada Lovelace",
+    "active": true
+  }
+]
+```
+
+@async/db infers useful local contracts from the fixture shape. It uses those contracts for generated types, REST metadata, GraphQL metadata, viewer metadata, and write validation.
+
+When inference is ambiguous, @async/db should emit diagnostics and `doctor` suggestions instead of guessing too hard.
+
+## Schema-First
+
+Use schema-first fixtures when you know the contract before you have useful records.
+
+```json
+{
+  "kind": "collection",
+  "idField": "id",
+  "fields": {
+    "id": { "type": "string", "required": true },
+    "role": {
+      "type": "enum",
+      "values": ["admin", "user"],
+      "default": "user"
+    }
+  },
+  "seed": []
+}
+```
+
+Schema-first resources can start empty and still generate types, REST metadata, GraphQL metadata, and viewer metadata.
+
+## Mixed Mode
+
+Mixed mode means a resource has both schema and data sources:
+
+```txt
+db/users.schema.json
+db/users.json
+```
+
+The schema file is authoritative. The data file provides seed records. If a schema file still contains embedded `seed` while a data fixture exists, @async/db ignores the embedded seed and warns.
+
+Useful commands:
+
+```bash
+npm run db -- schema unbundle users
+npm run db -- schema bundle users --out artifacts/users.bundle.schema.json
+```
+
+Keep bundled schema-plus-seed artifacts outside `db/` unless you intentionally use `--force`.
+
+## Runtime Stores
+
+The default `json` store keeps source fixtures clean and writes app changes to generated runtime state:
+
+```txt
+db/users.json              source fixture
+.db/state/users.json   writable runtime JSON store
+```
+
+This is the safest default for local development because tests, demos, and UI prototyping do not rewrite committed fixtures.
+
+It is also the first-party JSON file database for production resources that should remain small, reviewable, and low-write. Keep production app traffic behind @async/db resources or registered operations so a resource can later move from JSON to SQLite, Postgres, or a custom store without rewriting frontend data access.
+
+Bind a resource to a different store when runtime state belongs somewhere else:
+
+```js
+export default {
+  stores: {
+    default: 'json',
+  },
+  resources: {
+    settings: { store: 'json' },
+    activityEvents: { store: 'sqlite' },
+  },
+};
+```
+
+## Source File Store
+
+Use the `sourceFile` store only when you intentionally want @async/db to write supported changes back to source files.
+
+The main source writeback case is generated ids for plain `.json` collection fixtures that omit `id`. JSONC and CSV sources remain parsed source inputs; generated runtime state still lives under `.db/`.
+
+## Diagnostics
+
+Diagnostics are part of the workflow:
+
+- schema errors should block invalid resources
+- warnings should surface schema/data drift without breaking unrelated resources
+- `doctor` should suggest helpful schema or fixture improvements
+- `check --strict` should make warnings fail in CI
+
+Commands:
+
+```bash
+npm run db -- doctor
+npm run db -- doctor --json
+npm run db -- check --strict
+```
+
+See [Fixtures And Schemas](./fixtures-and-schemas.md) for authoring details and [Server And Viewer](./server-and-viewer.md) for how diagnostics appear while serving.