@rehearsal-db/core 0.1.0-beta.7 → 0.1.0-beta.9

package/docs/configuration.md

@@ -44,17 +44,45 @@ export default defineRehearsalConfig({
     artifactDirectory: ".rehearsal",
     sanitizationPolicy: "infrastructure/rehearsal/sanitization-policy.json",
   },
+  // Optional, separately approved production-shaped preparation.
+  // preparation: {
+  //   sourcePolicy: "infrastructure/rehearsal/source-access-policy.json",
+  //   privacyKey: ".rehearsal/secrets/privacy.key",
+  //   batchRows: 500,
+  //   maximumRows: 1000000,
+  //   maximumBytes: 2147483648,
+  //   diskHeadroomBytes: 67108864,
+  // },
+  // runtimePolicy: "infrastructure/rehearsal/runtime-policy.json",
+  // Optional signup defaults, role/reference transfer, Storage paths, and token checks.
+  // identityPolicy: "infrastructure/rehearsal/identity-policy.json",
   containerRuntime: {
     autoStartColima: true,
   },
   cleanup: {
     retainBaselineGenerations: 2,
   },
+  // Optional. Each dependent database uses another complete Rehearsal config.
+  // dependentTargets: [
+  //   {
+  //     name: "publication-api",
+  //     configPath: "rehearsal.publication.config.mjs",
+  //     prepareCommand: "npm run rehearsal:prepare-publication",
+  //   },
+  // ],
   application: {
     // CHECK: replace these when the detected package scripts are not correct.
     startCommand: "npm run dev:rehearsal",
     proofCommand: "npm run test:rehearsal",
     environmentFile: ".rehearsal/runtime.env",
+    // environmentVariables: { DATABASE_URL: "primary:DATABASE_URL" },
+    readiness: {
+      url: "http://localhost:5175/health",
+      expectedStatus: 200,
+      timeoutSeconds: 30,
+    },
+    // Add at least one real positive and one negative before enabling.
+    // httpProofs: [],
   },
   runtime: {
     target: "supabase",
@@ -86,6 +114,31 @@ All paths resolve inside the consuming project. The artifact directory must be n
 Rehearsal never overwrites this config. To start over, move the existing file somewhere
 safe, run setup again, and compare the two files before deleting either one.
 
+`preparation.privacyKey` must stay inside `.rehearsal`; it is ignored and owner-readable
+only. Source, runtime, and identity policy files are declarations that may be reviewed in
+source control. They must never contain passwords, tokens, production URLs, raw owner
+identifiers, SQL, or executable code.
+
+## Optional source preparation
+
+`preparation` enables the separate `source` and `baseline refresh` commands. It does not
+change ordinary `run`, `reset`, `migrate`, or `verify` behavior.
+
+- `sourcePolicy` declares the hashed target identity, temporary roles, exact export
+  columns, public/approved-owner row scope, migration ledger, and optional Storage scope.
+- `privacyKey` stores the local keyed-pseudonym secret.
+- `batchRows`, `maximumRows`, and `maximumBytes` bound extraction.
+- `diskHeadroomBytes` prevents activation when the copy cannot be built safely.
+
+See [Production source](production-source.md) for complete examples and the exact
+approval sequence.
+
+`runtimePolicy` replaces common restore adapters with reviewed schemas, allowlisted
+extensions, managed triggers, local-only singleton rows, and structural expectations.
+`identityPolicy` declares a hashed verified email, copied placeholder, safe signup
+defaults, required role/reference transfers, Storage path rewrites, and token checks. See
+[Runtime and identity policies](runtime-policies.md).
+
 ## Container runtime and cleanup
 
 ```ts
@@ -102,9 +155,67 @@ is running and `autoStartColima` is `true`, Rehearsal may run `colima start`. It
 the user's Colima CPU, memory, and disk settings and never changes them. Set the field to
 `false` when you prefer to start Docker or Colima yourself.
 
-`retainBaselineGenerations` controls how many immutable baseline generations survive
-`rehearsal cleanup`; it must be at least 1. Runtime deletion and shared-image inspection
-are command choices, not automatic retention settings. See [CLI commands](commands.md).
+`retainBaselineGenerations` controls how many immutable baseline generations survive an
+approved `rehearsal refresh` or `rehearsal cleanup`; it must be at least 1. Refresh shows
+the exact old generations before confirmation and never removes the active baseline
+before its replacement is verified. Runtime deletion outside refresh and shared-image
+inspection remain explicit command choices. See [CLI commands](commands.md).
+
+## Dependent databases
+
+Use `dependentTargets` when one local application needs another database, such as a
+publication or read-model database:
+
+```ts
+dependentTargets: [
+  {
+    name: "publication-api",
+    configPath: "rehearsal.publication.config.mjs",
+    prepareCommand: "npm run rehearsal:prepare-publication",
+  },
+],
+```
+
+The referenced file is an ordinary complete Rehearsal config. Give it its own runtime
+project ID, ports, application environment file, and artifact folder ending in
+`.rehearsal`, for example `infrastructure/rehearsal/publication/.rehearsal`. Create and
+review its baseline separately:
+
+The easiest starting point is to copy the generated primary config, then change the
+project name, migration folder, artifact folder, environment file, runtime project ID,
+ports, and proof command. Keep the same fail-closed safety settings. Omit
+`prepareCommand` when the dependent baseline is already ready to test. Rehearsal never
+copies rows between databases automatically. A dependent Supabase target also needs its
+own local `rehearsalConfig` file with matching dedicated ports. Preparation must be safe
+to rerun because `run`, `reset`, and `migrate` each invoke it after the databases are
+ready.
+
+```bash
+npx rehearsal baseline create \
+  --config=rehearsal.publication.config.mjs \
+  --records=path/to/publication-data.ndjson \
+  --ledger=path/to/publication-ledger.json
+```
+
+The primary `doctor`, `explain`, `candidates`, lifecycle, and cleanup commands then cover
+the whole stack. Rehearsal restores and migrates the primary target first, followed by
+dependents in the listed order. For `run`, `reset`, and `migrate`, it next runs each
+`prepareCommand`. On `run`, it executes each dependent config's
+`application.proofCommand` before the primary proof. `stop` and `discard` run in reverse
+order. Cleanup verifies every target's exact preview before removing any selected
+resource.
+
+Legacy preparation commands receive paths—not credentials—in environment variables:
+`REHEARSAL_PRIMARY_ENV_FILE` and
+`REHEARSAL_DEPENDENT_<UPPERCASE_NAME>_ENV_FILE`. New integrations should prefer an
+already prepared dependent baseline or the package's declarative source/privacy
+workflow. `prepareCommand` remains compatible for existing projects but is ordinary
+project code and must be reviewed. Nested dependencies are rejected.
+
+The dependent proof must include at least one known successful lookup and appropriate
+negative controls, such as unauthorized, malformed, withheld, or missing records. A
+response that merely avoids a server error does not prove that eligible data was
+published or can be found.
 
 ## Supabase service environment
 
@@ -166,20 +277,40 @@ the exactly named container and volume carrying the matching project label.
 Plain PostgreSQL does not restore Supabase Storage assets or synthesize Supabase Auth
 users.
 
-Project-owned runtime adapters remain the place for application-specific setup and
-checks; they do not replace the database driver.
+Declarative runtime policies are preferred for supported schema prerequisites and
+checks. Existing project-owned adapters remain compatible but do not replace the
+database driver.
 
 ## Application proof
 
-`proofCommand` is mandatory and project-owned. It should test restored relationships,
-authentication shape, critical reads, and candidate-migration behavior. A command that
-only checks whether the home page returns 200 is usually too weak.
+`startCommand` is the ordinary application command. When `readiness` is present,
+`rehearsal run` launches it with a minimal environment, waits for the exact local status,
+runs proofs, and stops only that child process group. `environmentVariables` explicitly
+maps generated runtime values such as `primary:DATABASE_URL`; undeclared inherited
+hosted secrets are not passed through.
+
+Your application framework may still load `.env` files from the project directory on
+its own. Rehearsal cannot intercept that file loading. If a local proof could otherwise
+pick up a hosted URL, token, CAPTCHA key, or similar value, set an explicit safe local
+value in the configured `startCommand` or the framework's test-mode configuration.
+Environment filtering is a process boundary, not an operating-system or file-access
+firewall.
+
+Supabase targets write validated loopback-only `DATABASE_URL`, `PGHOST`, `PGPORT`,
+`PGDATABASE`, `PGUSER`, and `PGPASSWORD` values into the ignored runtime environment.
+Package-owned identity commands use that same local connection; an adapter is not
+required to provide it.
+
+`proofCommand` remains an ordinary project test command. Optional `httpProofs` add
+package-owned checks and must include real positive and negative expectations. A 200,
+404, 401, empty result, or non-5xx response is not a positive unless it matches the
+declared status and body assertion.
 
 ## Runtime adapters
 
-Most projects do not need an adapter. Use one only for schema-specific restore setup,
-synthetic local identities, explicit generated environment variables, or post-restore
-invariants. See [adapters](adapters.md).
+Existing adapters remain compatible, but new projects should first use `runtimePolicy`,
+`identityPolicy`, environment mappings, and HTTP proofs. Use an adapter only for a shape
+the documented declarations explicitly reject. See [adapters](adapters.md).
 
 ## Advanced library entry points
 
@@ -191,7 +322,9 @@ may use these explicit subpaths:
 - `@rehearsal-db/core/schema`
 - `@rehearsal-db/core/diagnostics`
 - `@rehearsal-db/core/process-environment`
+- `@rehearsal-db/core/privacy`
 - `@rehearsal-db/core/service-environment`
+- `@rehearsal-db/core/source-access`
 
 These advanced entry points are ESM JavaScript APIs in the first beta. The root
 configuration and sanitization API has TypeScript declarations; the advanced subpaths do

package/docs/getting-started.md

@@ -1,7 +1,7 @@
 # Getting started
 
 This guide sets up Rehearsal in an existing project. It creates a disposable database on
-your computer and never connects to a hosted database.
+your computer. Normal setup and rehearsal commands never connect to a hosted database.
 
 Want to experiment first? Use the [safe hands-on tutorial](tutorial.md).
 
@@ -111,10 +111,20 @@ A successful run:
 1. restores the baseline;
 2. applies only the migrations you approved;
 3. verifies the local database;
-4. runs your application proof.
+4. launches the ordinary application when readiness is configured;
+5. runs the ordinary test command and declared positive/negative HTTP proofs;
+6. stops only the application process it launched.
 
 Your original baseline remains unchanged.
 
+If a source owner later approves a production-shaped copy, use the separate workflow in
+[Optional production-shaped preparation](production-source.md). Do not add source
+credentials to the normal config or runtime environment.
+
+Once temporary source access is ready, `npx rehearsal refresh` previews the safe
+replacement workflow. It means “get a newly verified copy.” By contrast, `reset` means
+“discard local edits and restore the copy I already have.”
+
 ## 6. Test and clean up
 
 Use the guide for normal runtime tasks:

package/docs/glossary.md

@@ -6,6 +6,10 @@
 
 **Candidate digest** — checksum binding the exact candidate filenames and bytes.
 
+**Dependent target** — another isolated local database managed with the primary runtime.
+
+**Combined digest** — one checksum binding candidate or cleanup sets across every target.
+
 **Project-owned** — application-specific policy or code that remains outside the package.
 
 **Represented migration** — historical migration whose exact bytes are bound into the

package/docs/production-source.md

@@ -1,33 +1,176 @@
-# Designing a production source boundary
-
-The npm package does not connect to production and does not ship an extraction command.
-That separation is a security boundary: each project must authorize, sanitize, and
-audit its own source.
-
-## Recommended flow
-
-1. Define versioned read-only export views containing only approved columns.
-2. Grant a temporary role `SELECT` on those views and nothing else.
-3. Use a short-lived credential outside shell history and source control.
-4. Stream records through the project-owned sanitization policy.
-5. Write into a private building generation below `.rehearsal`.
-6. Verify counts, checksums, policy coverage, migration history, and secret canaries.
-7. Atomically activate the completed generation.
-8. Revoke and verify removal of the temporary credential.
-9. Delete raw intermediate files and retain only the sanitized artifact.
-
-Use server-side cursors or another bounded streaming mechanism. A full in-memory dump
-is not acceptable for large sources. Extraction logs should contain counts and hashes,
-never row bodies.
-
-## What not to do
-
-- Do not point `rehearsal.config.mjs` at a hosted URL.
-- Do not put a production connection string in `.rehearsal/runtime.env`.
-- Do not grant table-wide access merely because a view is inconvenient.
-- Do not commit sanitized baselines; sanitized data is still data.
-- Do not let the reusable package infer which fields are safe.
-- Do not leave the export role provisioned after refresh.
-
-The local execution engine remains useful with synthetic or manually approved baselines.
-Production-shaped onboarding is a separate project security exercise.
+# Optional production-shaped preparation
+
+Start with synthetic data. Use this workflow only when the source owner has approved the
+exact tables, columns, owner scope, assets, and privacy policy.
+
+Ordinary Rehearsal commands remain local-only. Source preparation is separate and cannot
+be triggered by `run`, `reset`, `migrate`, `verify`, or application launch.
+
+## 1. Add the config paths
+
+```js
+preparation: {
+  sourcePolicy: "infrastructure/rehearsal/source-access-policy.json",
+  privacyKey: ".rehearsal/secrets/privacy.key",
+  batchRows: 500,
+  maximumRows: 1000000,
+  maximumBytes: 2147483648,
+  diskHeadroomBytes: 67108864,
+},
+```
+
+The privacy key stays ignored under `.rehearsal`. The policy is safe to review in source
+control because it contains hashes and environment-variable names, not credentials,
+URLs, or raw owner identifiers.
+
+## 2. Declare the source boundary
+
+`source-access-policy.json` uses this shape:
+
+```json
+{
+  "accessVersion": 1,
+  "targetFingerprint": "<sha256 of the approved PostgreSQL host, port, and database>",
+  "administratorEnvironmentVariable": "REHEARSAL_SOURCE_ADMIN_URL",
+  "reader": {
+    "role": "rehearsal_example_reader",
+    "ownerRole": "rehearsal_example_owner",
+    "credentialFile": ".rehearsal/secrets/source-reader.env",
+    "validForMinutes": 30
+  },
+  "exportSchema": "rehearsal_export_example",
+  "migrationLedger": {
+    "schema": "supabase_migrations",
+    "table": "schema_migrations",
+    "versionColumn": "version",
+    "nameColumn": "name",
+    "statementsColumn": "statements"
+  },
+  "relations": [
+    {
+      "source": { "schema": "public", "table": "widgets" },
+      "view": "widgets",
+      "targetSchema": "public",
+      "targetTable": "widgets",
+      "columns": ["id", "owner_id", "name"],
+      "orderBy": ["id"],
+      "rowScope": { "kind": "approved-public" }
+    },
+    {
+      "source": { "schema": "public", "table": "profiles" },
+      "view": "approved_owner_profile",
+      "targetSchema": "app_api",
+      "targetTable": "profiles",
+      "columns": ["id", "email"],
+      "orderBy": ["id"],
+      "rowScope": {
+        "kind": "approved-owner",
+        "column": "id",
+        "valueEnvironmentVariable": "REHEARSAL_APPROVED_OWNER_ID"
+      }
+    }
+  ],
+  "assets": []
+}
+```
+
+Calculate fingerprints with the exported helpers from `@rehearsal-db/core/source-access`
+or a reviewed local script. A fingerprint binds the destination without recording its
+address. Changing the host, port, or database invalidates the plan.
+
+`approved-public` means every selected row is licensed or otherwise approved for this
+use. `approved-owner` requires one explicit owner value from the named environment
+variable. Rehearsal never chooses an owner by finding an administrator.
+
+`targetSchema` is optional and defaults to `public`. Set it when the restored relation
+belongs elsewhere, and use the same schema in privacy policy version 2. This is relation
+identity, not a request to search every schema.
+
+For Storage, add an `assetReader` block and approved `assets` entries. The reader URL and
+token come from named environment variables. Objects outside the exact bucket/prefix are
+not listed or downloaded. Size, object-count, path, ETag, and exact-byte checks apply.
+
+## 3. Use executable privacy policy version 2
+
+Every selected column has one action and, where required, a bounded recipe:
+
+```json
+{
+  "policyVersion": 2,
+  "migrationCutoff": "20260101000000",
+  "tables": [
+    {
+      "schema": "app_api",
+      "name": "profiles",
+      "sourceRows": "STREAM AND SANITIZE",
+      "columns": [
+        {
+          "name": "id",
+          "action": "PSEUDONYMIZE",
+          "recipe": { "format": "uuid", "namespace": "account-id" },
+          "generated": "NEVER",
+          "identity": "NO",
+          "foreignKey": null
+        },
+        {
+          "name": "email",
+          "action": "PSEUDONYMIZE",
+          "recipe": { "format": "email", "namespace": "account-email" },
+          "generated": "NEVER",
+          "identity": "NO",
+          "foreignKey": null
+        }
+      ]
+    }
+  ]
+}
+```
+
+Supported recipes are intentionally small: keyed UUID/email/text/integer pseudonyms,
+explicit constants, bounded date shifts, and fully classified JSON objects. Unknown
+tables, columns, JSON keys, formats, and recipes stop the refresh. There is no JavaScript
+or SQL escape hatch.
+
+## 4. Preview, apply, refresh, and retire
+
+Keep credentials out of shell history by loading them from an owner-only environment
+manager or file, then run:
+
+```bash
+npx rehearsal privacy key --write
+npx rehearsal source plan
+npx rehearsal source apply --confirm-source-access=<full-sha256>
+npx rehearsal refresh
+npx rehearsal refresh --confirm-refresh=<full-sha256>
+npx rehearsal source retire
+npx rehearsal source retire --confirm-source-retirement=<full-sha256>
+```
+
+`source apply` creates new time-limited roles and export views only when every exact name
+is unused. It verifies the reader cannot read raw tables, write, escalate roles, or call
+an exposed security-definer network function. Externally provisioned readers are not yet
+supported; use the managed reader so Rehearsal can prove its scope and retirement.
+
+`refresh` previews the complete replacement first. After exact confirmation, its baseline
+step uses one read-only repeatable-read transaction for the schema shape,
+migration ledger, and rows. It streams in bounded batches, records only counts and
+hashes, stages privately, rechecks schema/ledger drift, and activates atomically. It then
+resets the local runtime stack and applies configured baseline retention. A failed new
+runtime is rolled back to the previous baseline and runtime. The lower-level `baseline
+refresh` command remains available when automation should activate a baseline without
+resetting the runtime or pruning old generations.
+
+Storage cannot share PostgreSQL's transaction snapshot. Rehearsal instead verifies each
+object's inventory version and byte count during transfer, then records its SHA-256 in
+the immutable baseline.
+
+`source retire` inventories the exact recorded views before removal, then removes only
+those views, the two recorded roles, and local credential/receipt files. Unrelated roles,
+grants, objects, baselines, runtimes, and Docker resources are preserved.
+
+## Limits
+
+Rehearsal does not decide consent, licensing, retention rights, privileged-role meaning,
+or which owner is yours. It does not provide an OS firewall, backup, disaster recovery,
+or universal database support. Real Google account selection and provider callbacks
+still require direct human observation; a synthetic redirect cannot prove them.

package/docs/roadmap.md

@@ -1,39 +1,41 @@
 # Next steps
 
-The current Rehearsal beta supports guided Supabase and ordinary PostgreSQL rehearsals.
-The next work should be driven by real project use before adding more database targets.
+Rehearsal supports guided Supabase and ordinary PostgreSQL migration rehearsals. The
+current development track removes the need for consumer-owned Rehearsal engines while
+keeping application meaning and privacy decisions project-owned.
 
-## 1. Complete the first real-project acceptance run
+## Standalone workflow
 
-Update an existing Supabase application to `@rehearsal-db/core@beta` on its own clean
-branch, then prove:
+The implementation sequence is dependency-driven:
 
-- `doctor` reports `READY`;
-- the intended candidate migrations are the only candidates;
-- a complete rehearsal and the application proof pass;
-- verify, reset, stop, and restart behave as expected.
+1. exact temporary source access and retirement;
+2. bounded coherent extraction and executable privacy rules;
+3. approved Storage assets and declarative restore prerequisites;
+4. verified local identity association;
+5. ordinary application launch and meaningful positive/negative proofs;
+6. onboarding, upgrade safety, and exact packaged-artifact acceptance.
 
-Record any confusing instruction or unnecessary manual step. Those findings should guide
-the next usability changes.
+These capabilities stay behind separate review boundaries. Normal runtime commands never
+receive source credentials. Source-side writes, package publication, consumer cleanup,
+and real provider observation each require their own authorization.
 
-## 2. Simplify baseline onboarding
+Track the full breakdown in
+[standalone roadmap issue #38](https://github.com/Ddupasquier/rehearsal-db/issues/38).
 
-Make safe records, migration evidence, and sanitization-policy review easier for a new
-developer. Reduce manual file preparation without weakening review, checksum, or
-local-only safety rules.
+## Acceptance still required
 
-## 3. Validate another ordinary PostgreSQL project
+- Install the exact packed candidate in clean Supabase and PostgreSQL consumers.
+- Exercise two unrelated synthetic schemas, bounded large data, interruption, drift,
+  invalid migrations, identities, images, persistence, reset, and cleanup.
+- Run the released package in the real reporting consumer and record positives and
+  negatives.
+- Directly observe any claimed Google account-picker/callback and unfamiliar-developer
+  onboarding behavior; automation cannot invent that evidence.
+- Remove consumer integration code only after its replacement passes and that deletion
+  is separately approved.
 
-Use a non-Supabase application with real migration history and a synthetic baseline.
-Fix general PostgreSQL problems before adding provider-specific behavior.
+## Later database targets
 
-## 4. Consider PostgreSQL service compatibility
-
-Only after the ordinary PostgreSQL workflow is reliable, evaluate compatibility needs
-for individual PostgreSQL services. Keep rehearsals disposable and local; hosted database
-execution remains outside the current safety model.
-
-## Later
-
-MySQL, MongoDB, and unrelated database families require different migration and restore
-behavior. They remain out of scope until the PostgreSQL experience is proven and stable.
+After the PostgreSQL family is stable in real projects, evaluate individual PostgreSQL
+services. MySQL, MongoDB, and unrelated database families require different migration,
+restore, and safety behavior and remain out of scope.