@odla-ai/chapter 0.20.0 → 0.20.2

package/runbooks/adopt-existing.md

@@ -0,0 +1,505 @@
+# Adoption runbook: convert an existing site to Chapter
+
+Audience: an LLM or coding agent converting an existing or deployed website,
+Worker, database, auth flow, or operating membership product.
+
+Read this file completely before changing the application. Also read the
+installed package `README.md`, `package.json` exports, and `dist/*.d.ts`. The
+existing product stays live until measured Chapter parity and cutover are
+approved.
+
+If the repository is empty and no operating state exists, use
+[`greenfield.md`](./greenfield.md).
+
+## Completion means conversion, not installation
+
+The following are canary evidence only:
+
+- installing `@odla-ai/chapter`;
+- writing `defineChapter()` or rendering `ChapterAdmin`;
+- provisioning an ODLA tenant;
+- serving a login screen or one Chapter API route;
+- deploying a parallel Worker with copied sample data.
+
+Adoption is complete only when the real product routes and journeys use Chapter
+as the primary implementation, live data is reconciled and continuously
+handled, production reads and writes have crossed explicit gates, and both
+runtime and data rollback remain proven.
+
+## Non-negotiable boundaries
+
+1. Never mutate, reset, delete, detach, or repoint current production resources
+   during development adoption. Preserve Workers, Pages projects, domains,
+   databases, R2/KV, queues, Workflows, cron, webhooks, Clerk, Stripe, email,
+   Google calendars, source repositories, and immutable history.
+2. Never print or commit secrets. Dev and production use separate credentials
+   and explicitly targeted Worker environments.
+3. Preserve existing public URLs, content, SEO, header/navigation, responsive
+   layout, theme behavior, accessibility, and branded assets unless the human
+   approves a product change. Framework replacement is not redesign authority.
+4. Preserve data semantics before deleting code. Chapter schema, routes, sends,
+   auth, account, payment, and scheduling defaults are hypotheses until parity
+   tests prove them.
+5. Runtime rollback and data rollback are different. Restoring an old Worker
+   cannot recover writes accepted after Chapter became authoritative.
+6. No silent fallback. A Chapter-primary failure is explicit and measured; a
+   legacy fallback, if approved, is named, time-bounded, and visible.
+7. Keep ODLA rules default-deny and every admin/provider mutation
+   server-mediated.
+
+## Phase 0 — Freeze the operating baseline
+
+Create or extend root `MIGRATION.md`; never replace its history. Record state
+`inventoried` only after the human approves the baseline.
+
+### Inventory the product
+
+- every public, member, admin, API, webhook, health, asset, feed, and 404 route;
+- current production/dev origins, Worker/Pages names and version ids, routes,
+  DNS, compatibility settings, build automation, and rollback versions;
+- D1/SQL/ODLA/KV/R2/files and every reader/writer;
+- queues, Workflow definitions, cron, provider webhooks, retry/outbox behavior;
+- Clerk app/instances, JWT claims, roles/allowlists, redirects, and first-admin
+  process;
+- Stripe products/prices/webhooks/idempotency, Google calendars, Email Service;
+- current schema/rules, row counts, earliest/latest timestamps, natural keys,
+  relations, projections, and owner-edited fields;
+- public/private content boundaries and every raw HTML/Markdown/inline-JSON
+  render sink;
+- telemetry, freshness/error alerts, deploy commands, and CI auto-deploys.
+
+### Freeze visual and behavior evidence
+
+Capture representative production screenshots and measured layouts at 390,
+768, 1280, and 1440 pixels in every supported theme. Record:
+
+- header height, wordmark/nav alignment, responsive breakpoints, page geometry;
+- form fields/labels/errors, Clerk containment, tables/cards/rails, loading,
+  empty, error, unauthorized, and 404 states;
+- full public route/link inventory, titles, descriptions, canonical URLs,
+  structured data, robots/sitemap/feed behavior;
+- join, payment, booking, member, admin, CRM, email, and refund journeys.
+
+Do not rely on visual memory. These fixtures are acceptance authority until a
+human explicitly approves a redesign.
+
+### Define migration budgets
+
+Record accepted count/hash differences, maximum sync lag/newest-data age, route
+and status parity, latency/error budget, visual tolerances, identity merge
+policy, deterministic synthetic user/application ids, and an observation
+window. Record both rollback plans before any Chapter write.
+
+Maintain a migration matrix:
+
+| Track | Legacy authority | Chapter/ODLA dev target | Current reads | Current writes | Evidence/freshness | Rollback | State |
+| --- | --- | --- | --- | --- | --- | --- | --- |
+| runtime/bindings | | | | | | | |
+| membership data/CRM | | | | | | | |
+| jobs/webhooks/outboxes | | | | | | | |
+| Clerk/auth | | | | | | | |
+| Stripe/payments | | | | | | | |
+| calendar/booking | | | | | | | |
+| email | | | | | | | |
+| public/member/admin UI | | | | | | | |
+| o11y/alerts | | | | | | | |
+
+Allowed states: `not-started`, `canary`, `parallel`, `verified`,
+`primary-dev`, `primary-prod`, `retired`, or `n/a` with a reason.
+
+## Phase 1 — Model Chapter without deleting anything
+
+Install the version-matched agent guidance and inspect the automation boundary:
+
+```sh
+npx @odla-ai/cli@0.17.1 setup
+npx @odla-ai/cli@0.17.1 capabilities --json
+```
+
+Pin the exact known-good package matrix from the installed Chapter README.
+For this runbook, first require
+`npm view @odla-ai/chapter@0.20.2 version` to succeed. Install every runtime,
+toolchain, CLI, and security package with the exact versions in that README;
+never use `latest`, a range, or `--legacy-peer-deps`.
+
+Write `src/chapter.config.mjs` from the inventory. Make every behavior decision
+explicit:
+
+- `mode`, `services`, complete light/dark brand tokens;
+- price/currency/interval and owner-approved policy copy;
+- email addresses/templates/debug redirect and send trigger;
+- scheduling/timezone and pipeline stages/bookable/approvable subsets;
+- exact application fields, caps, disclaimer requirement, Clerk-visible
+  `profileFields`, and CRM-projected `crmFields`;
+- claim/table auth, ladder/super-admin tier;
+- account side effect (`none`, `create`, or emailed `invite`);
+- approve/refund policy and any network targets/field allowlists.
+
+### Schema/rule parity gate
+
+Before deleting old schema/rules/provisioning:
+
+1. freeze the legacy schema and rules as reviewed test fixtures;
+2. normalize only approved representational differences;
+3. compare `createChapterIntegration(chapter)` namespaces, attrs, links, rules,
+   and seeds to the fixtures;
+4. fail on missing fields, widened rules, cardinality changes, renamed natural
+   keys, new client-readable data, or unapproved seed differences;
+5. record every intentional delta and migration mapping in `MIGRATION.md`.
+
+`createChapterIntegration()` inserts the group seed only when absent. Existing
+runtime owner edits do not update from later config changes. Compare the current
+group row field by field, classify ownership, and write an explicit migration
+for approved differences.
+
+### Behavior-delta gate
+
+At minimum audit:
+
+- services default (`db`, `calendar`, `o11y`);
+- per-field and request-body caps;
+- account default `none` and outbound effects of `create`/`invite`;
+- stock `JoinIsland` generates a fresh `submissionId` on each invocation, so an
+  ambiguous retry after a lost response is not idempotent even though a
+  concurrent click is disabled;
+- Clerk-client-readable `public_metadata.profile`;
+- application-to-CRM projection fields;
+- required disclaimer acknowledgement and `disclaimerAckAt`;
+- initial/bookable/approvable stages and never-backwards transitions;
+- when each lifecycle email fires;
+- Chapter recognizes only the literal `prod` environment as live mail; the
+  CLI-valid alias `production` follows the non-production fail-safe;
+- approve promotion/email and refund/subscription behavior;
+- Chapter route names and member/admin routing;
+- Chapter's insert-only group seed and runtime-edited fields;
+- payment readiness versus actual webhook readiness;
+- calendar's HTTP-200 `schedulingReady: false` degradation.
+
+The stock `applications` namespace is fixed. Its configurable form strings are
+limited to `firstName`, `lastName`, `email`, `referral`, `referralName`,
+`whoYouAre`, `linkedin`, `message`, `phone`, and `state`; `focus` and
+`disclaimerAck` are special. An unknown configured field can pass config
+validation and then fail at the database. Preserve a host schema/submit route
+for every legacy field that cannot be mapped without loss.
+
+The current `prices.currency` and `prices.interval` values do not control
+storage, join config, formatting, or the Stripe subscription. Treat the live
+Stripe Price as authoritative and make provider-side amount/currency/interval
+equality a cutover gate.
+
+Keep one test per accepted behavior decision so a package upgrade cannot change
+the site silently.
+
+### Make the dev descriptor executable
+
+Before Phase 2, create or adapt `odla.config.mjs`. A repository with no ODLA
+config may use the installed exact CLI's `init` command as a starting point,
+but the reviewed result must declare:
+
+- the existing product's one app id/name and only `envs: ["dev"]`;
+- explicit `chapter.services`;
+- `integrations: [createChapterIntegration(chapter)]` plus any compatible,
+  non-overlapping host integrations;
+- the intended dev Clerk publishable key, calendar ids, o11y service, local
+  credential paths, and a null/off-route dev link.
+
+Do not provision both a legacy inline schema and the Chapter integration for the
+same namespaces. After the schema/rule parity gate passes, freeze the legacy
+descriptor as a test fixture and make the reviewed Chapter integration the
+active dev descriptor. Production configuration remains absent.
+
+Create an off-route dev Worker entry using
+`chapterWorker({ chapter, routes: legacyRoutes })`, wrapped with o11y, and an
+explicit `wrangler ... --env dev` target. Keeping all legacy handlers as host
+routes at this point changes no route ownership; it only makes the canary and
+Chapter data surface executable.
+
+### Add a fail-closed migration readiness route
+
+The built-in `/api/health` is only `{ "ok": true }`; it does not touch the
+database. Add an admin-only host route or executable command that returns `503`
+until all of these match the acceptance manifest:
+
+- literal tenant/app/env and Chapter/package release;
+- expected schema/rules digest and required group/config rows;
+- reconciled counts, natural-key collisions, relation integrity, and newest
+  source timestamp within the freshness budget;
+- zero unhandled quarantines and bounded/zero pending outbox work;
+- healthy continuous sync and the Chapter-primary-to-legacy rollback mirror.
+
+Return only redacted counts/digests. This gate is allowed to fail during initial
+canary work; it must be mechanically green before switching either reads or
+writes, and after every rollback recovery.
+
+## Phase 2 — Create an isolated dev canary
+
+Run:
+
+```sh
+npx odla-ai doctor
+npx odla-ai provision --dry-run
+```
+
+The dev topology must be distinct from production:
+
+- `envs: ["dev"]`, tenant `<app-id>--dev`;
+- explicit dev Worker name and `wrangler ... --env dev`;
+- no production route, DNS, database binding, queue, Workflow, cron, webhook,
+  Git writer, Stripe live resource, email recipient, or Google write authority;
+- `debugEmail` enabled and Stripe test mode only;
+- dev Clerk `pk_test_...` from the intended workspace/application;
+- o11y wrapped around the real canary handler with safe low-cardinality labels.
+
+Show the dry-run target, schema/rules, services, link, and secret recipients.
+After the human approves the device code:
+
+```sh
+npx odla-ai provision --email <existing-odla-account> --write-dev-vars --push-secrets
+npm run build
+npx wrangler deploy --env dev
+npx odla-ai smoke --env dev
+```
+
+If credential transfer alone must be retried, use the non-rotating
+`npx odla-ai secrets push --env dev`; do not reprovision or rotate by default.
+Verify the deployed Chapter health marker, a database-backed read, and an
+unauthenticated private-route rejection. Record the URL and Cloudflare version
+id. A canary advances matrix rows only to `canary`; the migration-readiness
+route is expected to remain `503` until Phase 3 evidence passes.
+
+## Phase 3 — Adopt data with replay and reconciliation
+
+Skip historical backfill only when the human-approved inventory proves the
+site has no operating data. Otherwise:
+
+1. Write a checked-in mapping for every legacy table/collection/file and
+   relation: natural key, target namespace, attrs, omissions/nulls, edge
+   direction, ownership, tombstone/deletion behavior, and provenance.
+2. Freeze a source snapshot or bounded high-water mark. Record counts,
+   earliest/latest timestamps, and a digest/immutable id.
+3. Backfill in bounded deterministic chunks using stable mutation ids.
+   Persist checkpoints and a rejection ledger; each input is written, skipped
+   with reason, or failed/retryable.
+4. Replay from an earlier checkpoint and prove no duplicate applications,
+   people, companies, meetings, email log entries, relations, or side effects.
+5. Generate a redacted reconciliation report: counts, required attrs, duplicate
+   natural keys, dangling relations, identity collisions, status/price/role
+   differences, and newest timestamps.
+6. Preserve human-owned fields. Automated projection omits fields it does not
+   own rather than reading and writing stale copies.
+
+### Continuous operation
+
+Choose one durable event boundary for production-shaped changes. New legacy
+applications, payments, role changes, meetings, CRM edits, and emails must reach
+the dev Chapter projection through a measured outbox/queue/replay process.
+Record oldest pending age, attempts, quarantine, and source-to-target lag.
+
+Backfill alone is not conversion. Advance to `continuously-synced` only after
+normal schedules/webhooks are observed and failures replay from durable state.
+
+### Shadow reads
+
+For list/detail/search/admin dashboard/member status/join config, read both
+legacy and Chapter in dev while still serving the legacy response. Compare
+normalized status, ids, ordering, counts, fields, relations, price/policy, and
+freshness. Log bounded diffs, never private row bodies.
+
+At separate human checkpoints:
+
+1. switch dev reads to Chapter and make failures explicit;
+2. switch accepted dev writes to Chapter;
+3. durably mirror Chapter-primary writes back to the legacy rollback authority;
+4. simulate ODLA unavailable and rollback-mirror unavailable.
+
+Every authenticated/admin mutator must use this authority contract or be
+disabled. A separate Chapter demo route does not satisfy primary-dev.
+
+## Phase 4 — Adopt the Worker and routes in safe order
+
+Do not replace schema, worker, routes, and UI in one unreviewable change.
+
+1. Re-assert that the active dev descriptor is the reviewed
+   `createChapterIntegration(chapter)` from Phase 1 and that the legacy
+   provisioner remains only as a parity fixture.
+2. Start from the off-route `chapterWorker({ chapter, routes })` canary. Host
+   routes run before built-ins and receive Chapter's existing context
+   (`verifyUser`, `makeDb`, `roleFor`, `isAdmin`); do not verify JWTs twice.
+3. Initially keep every bespoke endpoint as a host route. Alias legacy URLs to
+   Chapter behavior instead of breaking clients.
+4. Compare built-in and legacy behavior route by route; only then remove a host
+   route that Chapter fully owns.
+5. Keep static fallback and API handling explicit so unknown APIs cannot fall
+   through to a successful SPA document.
+
+Test public/private method policies, body limits, idempotency, auth 401 vs 403,
+safe redirects, webhook signatures/replay, scheduling conflict,
+replay-deduplicated and concurrent email sends, and unavailable-provider
+degradation. Chapter's email log prevents a later replay after success; it does
+not serialize two concurrent sends, so preserve a stricter existing outbox or
+provider-idempotency contract.
+
+## Phase 5 — Adopt UI without losing the product
+
+### Preserve first
+
+Render the Chapter component beside the current implementation in a dev-only
+fixture. Compare markup/behavior and the Phase 0 screenshots before deletion.
+
+- Import `@odla-ai/ui/themes/salt/app.css` before host CSS.
+- Map the legacy design tokens into `brand.palette` and `paletteDark`; do not
+  carry an unbounded parallel theme system.
+- Public pages remain site-owned. Preserve their content hierarchy, imagery,
+  URLs, SEO, and responsive geometry.
+- Replace join orchestration with `JoinIsland`, member behavior with
+  `MembersArea`, and admin with `<ChapterAdmin chapter={chapter}/>` only after
+  field/auth/action parity.
+- The host still supplies member CSS and the Clerk wrapper around
+  `MembersArea`; Chapter does not ship a whole public stylesheet.
+- Scope host form styles. Bare `input`, `button`, `select`, `textarea`, `label`,
+  or global box-model rules must not corrupt Clerk/vendor DOM.
+
+### Required visual acceptance
+
+At 390, 768, 1280, and 1440 pixels in every theme, compare:
+
+- identical product header/navigation position between public/member/admin;
+- wordmark, text baseline, theme control, active state, and hover/focus;
+- join fields/errors, payment and slot picker, Clerk sign-in containment;
+- member cards/actions and every admin section/table/drawer;
+- no clipped labels, card text escape, accidental document scroll, or
+  horizontal overflow;
+- loading, empty, error, unauthorized, 404, and reduced-motion states.
+
+The human approves deployed screenshots/browser behavior. Passing CSS tests is
+not visual parity.
+
+## Phase 6 — Provider and authorization parity
+
+### Clerk
+
+Use the same Clerk application/workspace. Chapter claim mode requires session
+claims for both email and role; hub table mode uses lowercase `admins` rows.
+Prove existing allowed/forbidden users, role ladder, super-admin behavior,
+malformed/expired sessions, sign-out, and return targets.
+
+If the legacy verifier enforces JWT `aud` or `azp`, keep that host verifier in
+front of Chapter. This Chapter release verifies issuer but does not expose or
+enforce the CLI-provisioned Clerk audience. Add wrong-audience and
+wrong-authorized-party fixtures; do not cut over auth until the legacy contract
+still rejects them.
+
+When Chapter needs server-side account/role operations, set the named
+app-readable `clerk_secret_key` through:
+
+```sh
+npx odla-ai secrets set clerk_secret_key --env dev --stdin
+```
+
+Do not substitute `secrets set-clerk-key`; it writes the platform-reserved
+`$clerk_secret`, which Chapter cannot read under the named key. The generic
+command does not validate Clerk instance prefixes: independently confirm dev
+uses `sk_test_`. Never put `sk_live_` in dev.
+
+### Stripe
+
+Use test resources and an isolated dev webhook. Store `stripe_secret_key` and
+`stripe_webhook_secret` write-only; reconcile group `stripePublishableKey` and
+`stripePriceId`. Read that exact Stripe Price from Stripe and prove its first
+charge, recurring amount, currency, and interval match the rendered and
+approved contract. `paymentsReady: true` proves neither that equality nor
+webhook readiness; Chapter's built-in money formatter is USD/dollar-specific in
+this release. Preserve a host UI/route for non-USD or mismatched pricing. Test
+successful payment, duplicate webhook, failure, refund, and subscription
+cancellation before switching the UI.
+
+### Calendar and email
+
+Complete dev Google booking consent, reconnecting any grant from the retired
+read-only mirror, then prove actual slots, booking, rebooking, cancellation,
+and invitation behavior. `schedulingReady: false` with HTTP 200 is a failed
+readiness gate, not a pass.
+
+Use a verified dev sender plus `debugEmail`; prove every lifecycle template,
+recipient redirect, replay deduplication, concurrent-send behavior, and the
+selected account invitation effect. Never let dev send to imported members.
+
+## Phase 7 — Full deployed-dev journey
+
+Check in an executable acceptance manifest with:
+
+- actual dev URLs and Worker version ids;
+- deterministic synthetic member/application ids;
+- expected routes/statuses, schema/counts/freshness, and Chapter backend marker;
+- expected migration-readiness inputs and a `200` only when each is green;
+- expected account, payment, booking, email, CRM, and admin outcomes;
+- expected o11y service/release and safe trace;
+- replay and failure-injection outcomes;
+- visual viewport/theme fixtures.
+
+Run the real journey:
+
+1. browse every public route/deep link and verify SEO/404/method behavior;
+2. submit the configured join form, including consent;
+3. execute Stripe test payment and authoritative webhook;
+4. book/rebook the intro call and receive the debug-routed email;
+5. sign in as provisional/member/admin and use the real member/admin pages;
+6. perform every admin mutation family and verify Chapter-primary audit plus
+   legacy rollback mirror;
+7. replay the event and prove no duplicate row, charge, booking, account, or
+   email;
+8. inject ODLA/provider/mirror failure, observe explicit failure/retry, recover;
+9. verify freshness/continuous-sync budget and visual parity;
+10. require the fail-closed migration-readiness gate to return `200`;
+11. follow safe o11y evidence across the real Worker.
+
+`doctor`, unit tests, build, CLI smoke, a login page, and a human-looking admin
+screen are necessary but not sufficient.
+
+## Phase 8 — Production parallel run and cutover
+
+Do not add production until the human approves the full dev report and both
+rollback plans.
+
+1. Add only the literal `prod` environment using the production instance of the
+   same Clerk app, Stripe live resources, approved email sender, calendars,
+   links, and distinct secret targets. Do not use the CLI-valid alias
+   `production`: Chapter would suppress or debug-redirect lifecycle mail.
+2. Run production build/tests/security scan, `doctor`, and
+   `provision --dry-run`; inspect CI/Workers Builds so merging cannot
+   auto-deploy the legacy production target unexpectedly.
+3. After explicit approval, provision with `--yes` and deploy a production
+   Chapter candidate off-route. Record its version id.
+4. Repeat historical backfill, reconciliation, continuous sync, shadow reads,
+   provider checks, and a production-safe synthetic journey against production
+   source data. Dev data is not a production backfill.
+5. At one checkpoint, require the fail-closed readiness gate to return `200`,
+   then switch production reads. Verify freshness, auth, visual behavior, and
+   explicit failure handling while legacy writes continue.
+6. At a second checkpoint, record a write high-water mark, require readiness
+   again, and make Chapter authoritative for writes. Keep the durable legacy
+   rollback mirror and continuously reconcile it.
+7. Abort when write loss, unbounded lag, unexplained diff, auth failure,
+   freshness breach, or accepted error budget is crossed.
+
+### Runtime rollback
+
+Restore the recorded known-good Worker version plus routes, bindings, triggers,
+and domains. Verify representative routes and health. Keep Chapter resources
+and telemetry for investigation; do not delete them.
+
+### Data rollback
+
+Pause/drain new Chapter-primary work; enumerate accepted writes since the
+checkpoint; reverse-project missing writes with stable idempotency keys;
+reconcile queues, counts, timestamps, identities, status, meetings, charges,
+and email; only then report legacy reads healthy. Never reset either store.
+
+Keep both rollback capabilities through the approved observation window
+(normally at least 72 hours). Retire each legacy reader, writer, binding,
+webhook, job, and secret only in a separate human-approved change with a fresh
+journey afterward.
+
+Close `MIGRATION.md` only when every applicable track is `primary-prod` or
+`retired`, the observation window passes, and the human confirms the existing
+product—not merely a Chapter canary—has been converted.