@rehearsal-db/core 0.1.0-beta.12 → 0.1.0-beta.14

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (38) hide show
  1. package/CHANGELOG.md +116 -0
  2. package/COMPATIBILITY.md +6 -0
  3. package/README.md +5 -3
  4. package/docs/commands.md +15 -11
  5. package/docs/configuration.md +40 -5
  6. package/docs/getting-started.md +32 -3
  7. package/docs/production-source.md +175 -17
  8. package/docs/releasing.md +5 -3
  9. package/docs/roadmap.md +3 -2
  10. package/docs/sanitization.md +479 -12
  11. package/docs/security-model.md +8 -0
  12. package/docs/standalone-workflow.md +34 -13
  13. package/docs/troubleshooting.md +17 -6
  14. package/docs/tutorial.md +7 -1
  15. package/package.json +2 -2
  16. package/scripts/install/create_root_config.mjs +5 -22
  17. package/src/baseline/artifact.mjs +27 -4
  18. package/src/baseline/privacy_engine.mjs +865 -207
  19. package/src/baseline/privacy_structured_json.mjs +554 -0
  20. package/src/cli/arguments.mjs +3 -0
  21. package/src/cli/guided.mjs +8 -6
  22. package/src/cli/rehearsal.mjs +23 -3
  23. package/src/cli/renderers.mjs +29 -4
  24. package/src/cli/runtime_commands.mjs +92 -9
  25. package/src/cli/source_commands.mjs +37 -8
  26. package/src/identity/claim.mjs +23 -21
  27. package/src/project/install_scaffold.mjs +35 -1
  28. package/src/project/setup.mjs +14 -24
  29. package/src/runtime/plan.mjs +7 -3
  30. package/src/runtime/restore.mjs +13 -5
  31. package/src/shared/diagnostics.mjs +7 -0
  32. package/src/shared/node_runtime.mjs +31 -0
  33. package/src/source/access.mjs +271 -51
  34. package/src/source/asset_transfer.mjs +156 -74
  35. package/src/source/baseline.mjs +42 -6
  36. package/src/source/external_reader.mjs +368 -0
  37. package/src/source/postgresql_access.mjs +213 -55
  38. package/src/source/privacy_paths.mjs +38 -0
package/CHANGELOG.md CHANGED
@@ -5,6 +5,122 @@ a Changelog, and versions will follow Semantic Versioning after the package exis
5
5
 
6
6
  ## Unreleased
7
7
 
8
+ ## [0.1.0-beta.14] - 2026-10-05
9
+
10
+ ### Added
11
+
12
+ - Packed-package onboarding is now a required CI job. It covers fresh Supabase and
13
+ PostgreSQL installs, beta.7 configuration preservation, missing credentials, occupied
14
+ ports, malformed configuration, and non-destructive partial-setup recovery.
15
+ - The independent test lab can run the same first-install boundary against an exact
16
+ candidate tarball before either database lifecycle.
17
+ - Installation and setup now report the exact active Rehearsal configuration path and
18
+ explain when a supported existing nested config prevents root-file generation.
19
+ - Packed-package onboarding covers fresh, nested, existing-root, and repeated installs
20
+ while proving reviewed configuration bytes are preserved.
21
+ - Privacy policy version 2 can explicitly treat nullable owners as non-matches and match
22
+ an approved owner through any of a bounded list of pseudonymized identity columns.
23
+ - Bounded JSON unions classify each permitted root type, while JSON dictionaries require
24
+ reviewed runtime-key formats, key actions, value recipes, and item/depth/byte limits.
25
+ - JSON dictionaries can restrict runtime keys to canonical signed integers within an
26
+ explicit safe-integer range, preserving positive, negative, and reviewed zero keys
27
+ without widening numeric maps to arbitrary text.
28
+ - Date shifting can explicitly preserve numeric epoch-millisecond representation,
29
+ including inside structured JSON, while sharing a stable grouped offset with related
30
+ epoch or ISO-string dates.
31
+ - Bounded scalar enums retain only explicitly reviewed structural strings, finite
32
+ numbers, or booleans inside ordinary fields and nested JSON declarations, rejecting
33
+ out-of-domain values without echoing them in diagnostics.
34
+ - Validated strings preserve bounded, case-sensitive structural codes from open domains
35
+ using a package-owned portable alphabet, refusing credential-shaped and malformed
36
+ text without consumer regexes, callbacks, or static identifier lists.
37
+ - Keyed hexadecimal and GTIN pseudonyms preserve reviewed database constraints, while
38
+ fixed-width digest recipes derive fingerprints only from explicitly named sanitized
39
+ columns and reject missing, excluded, duplicate, or cyclic inputs.
40
+ - A single GTIN recipe can declare reviewed `allowedLengths`, preserving each source
41
+ value's 8-, 12-, 13-, or 14-digit shape and check digit even inside structured JSON.
42
+ - Keyed URL pseudonyms replace private HTTPS URLs with deterministic, bounded HTTPS
43
+ loopback paths without retaining source hosts, paths, credentials, queries, or tokens.
44
+ - Bounded binding substitution can retain permitted surrounding text while replacing
45
+ exact occurrences of a reviewed UUID with its corresponding keyed UUID pseudonym.
46
+ - External PostgreSQL readers can declare a bounded exact allowlist of provider-owned
47
+ permission groups. Rehearsal verifies safe membership options, group attributes,
48
+ nesting, view ownership, and the complete effective privilege surface.
49
+
50
+ ### Documentation
51
+
52
+ - First-time users now have a short recovery table for interrupted setup, failed
53
+ upgrades, port conflicts, missing OAuth credentials, and low Docker capacity.
54
+ - The configuration guide documents a safe manual move to the project root, separating
55
+ config-file-relative JavaScript imports from project-root-relative Rehearsal paths.
56
+ - The sanitization guide documents nullable and multi-column ownership, heterogeneous
57
+ JSON values, runtime-keyed dictionaries, keyed HTTPS replacements, bounded identity
58
+ substitution, and their fail-closed boundaries.
59
+
60
+ ### Fixed
61
+
62
+ - Source preparation now stores and reads each target's access receipt and owner-only
63
+ reader credential under its configured baseline artifact directory. Nested and
64
+ multi-target state roots no longer collide, and retirement refuses a receipt from a
65
+ different target root.
66
+ - Supabase Storage source preparation now recursively traverses real folder placeholder
67
+ entries while keeping the approved prefix, path, depth, object-count, per-object, and
68
+ total-byte boundaries fail closed.
69
+ - Approved-owner selection now rejects non-finite numeric values in every declared
70
+ owner column before choosing a privacy branch, including when another column already
71
+ matches the approved owner.
72
+ - Pseudonymized canonical integer dictionary keys now share the same keyed identity
73
+ mapping as numeric columns and nested numeric values using the same namespace, so
74
+ runtime maps keep their reviewed cross-representation joins.
75
+ - External-reader membership mismatches now return a redacted, actionable source
76
+ authorization refusal instead of an internal-failure diagnostic.
77
+ - All external-reader privilege, expiration, readable-surface, and security-definer
78
+ denials now use the same `SOURCE_AUTHORIZATION_REFUSED` diagnostic and exit code 3.
79
+ - Identity claims now sequence PostgreSQL reads on their single transaction client,
80
+ avoiding deprecated overlapping queries and remaining compatible with pg 9.
81
+
82
+ ## [0.1.0-beta.13] - 2026-10-05
83
+
84
+ ### Added
85
+
86
+ - Runtime and CI compatibility now cover maintained Node.js 22, 24, and 26 releases;
87
+ Node.js 24 LTS remains the recommended default and release-build runtime.
88
+ - Source access policies can use an externally provisioned, database-expiring
89
+ PostgreSQL reader without administrator credentials. Rehearsal binds canonical export
90
+ view definitions, verifies the complete approved read surface and deny controls, and
91
+ never changes or retires provider-owned source objects.
92
+ - `rehearsal --version`, its conventional `-V` alias, and `rehearsal version` now
93
+ report the installed package version without requiring project configuration.
94
+ - Privacy policy version 2 can use one reviewed identity-aware path mapping for both
95
+ database references and physical Storage destinations. Owner prefixes stay in the
96
+ environment, review plans retain only their hashes, and unsafe or mismatched paths
97
+ fail closed.
98
+ - Related timestamps can share a named deterministic date-shift group, preserving their
99
+ ordering and duration without consumer callbacks.
100
+ - The baseline verification corpus now exercises two unrelated target schemas, a
101
+ 10,003-row packed-package stream, concurrent row/schema/ledger changes, malformed and
102
+ truncated results, connection loss, low disk, interruption, and deterministic
103
+ fixed-key output.
104
+
105
+ ### Fixed
106
+
107
+ - Baseline preparation now rejects missing, negative, overflowing, malformed, or
108
+ oversized PostgreSQL estimate and cursor results before activation. Every preparation
109
+ failure preserves the active baseline, removes private staging, and leaves an edited
110
+ runtime untouched.
111
+ - Approved-owner UUID columns are no longer confused with PostgreSQL sequence-backed
112
+ identities. Restore checks the actual local schema before resetting a sequence, while
113
+ genuine numeric identity columns retain correct next-value behavior.
114
+ - A failed Storage transfer now recursively restores write permission only within its
115
+ owned staging tree, removes that incomplete generation, and preserves the original
116
+ safe preparation error instead of replacing it with a cleanup error.
117
+ - Storage preparation now records a canonical media type from reviewed object metadata,
118
+ removing response-only parameters such as a text charset that local bucket MIME
119
+ allowlists reject during restore.
120
+ - Container startup and health failures are now classified as runtime dependency
121
+ failures and include safe Docker/Colima capacity context instead of being mislabeled
122
+ as migration verification failures.
123
+
8
124
  ## [0.1.0-beta.12] - 2026-10-04
9
125
 
10
126
  ### Added
package/COMPATIBILITY.md CHANGED
@@ -21,6 +21,12 @@ The first stable `1.0.0` requires external beta evidence, a support policy, and
21
21
  public API. Supporting additional database families, package managers, or operating
22
22
  systems is not implied by the 0.x contract.
23
23
 
24
+ Rehearsal supports maintained even-numbered Node.js release lines that pass its CI
25
+ matrix. Beta.13 supports Node.js 22, 24, and 26; Node.js 24 LTS is the recommended
26
+ default and the release-build runtime. End-of-life and odd-numbered Node.js releases are
27
+ not supported. A future release may remove a Node.js line after its upstream end-of-life
28
+ date, with that change called out in the changelog.
29
+
24
30
  The npm install hook may create project scaffolding only when it proves one application
25
31
  root. Ordinary npm application installs are covered. A workspace is automatic only when
26
32
  one declaring consumer is already visible in its manifest or lockfile; npm can save a
package/README.md CHANGED
@@ -22,7 +22,7 @@ production deployment tool.
22
22
 
23
23
  ## What you need
24
24
 
25
- - Node.js 24 and npm
25
+ - Node.js 22, 24, or 26 and npm (Node.js 24 LTS recommended)
26
26
  - Docker Desktop, Colima, or another Docker-compatible engine
27
27
  - timestamped `.sql` migration files
28
28
  - a project test command that can prove the migrated application works
@@ -45,7 +45,9 @@ For an ordinary npm application install, Rehearsal safely creates a commented
45
45
  `rehearsal.config.mjs` in the project root, adds only its local-artifact entries to
46
46
  `.gitignore`, and prepares the isolated local runtime configuration. It detects the
47
47
  project name, database target, migration paths, commands, and free ports. Review lines
48
- marked `CHECK`, then open the guide. Existing files are never overwritten.
48
+ marked `CHECK`, then open the guide. Existing files are never overwritten. If a supported
49
+ root or nested config already exists, Rehearsal reports that active path and does not
50
+ create a competing file; see [Configuration](docs/configuration.md#workspaces-and-custom-paths).
49
51
 
50
52
  If install scripts are disabled or the workspace root is ambiguous, installation makes
51
53
  no guess. Run `npx rehearsal` inside the application and choose **Set the stage** for the
@@ -169,7 +171,7 @@ problems belong in the private process described in [SECURITY.md](SECURITY.md).
169
171
 
170
172
  ## For contributors
171
173
 
172
- Use Node.js 24, run `npm ci`, then run:
174
+ Use a supported Node.js release (22, 24, or 26), run `npm ci`, then run:
173
175
 
174
176
  ```bash
175
177
  npm run check
package/docs/commands.md CHANGED
@@ -43,17 +43,17 @@ existing config.
43
43
  These commands are separate from ordinary rehearsals. Configure `preparation` only after
44
44
  reviewing [the standalone security contract](standalone-workflow.md).
45
45
 
46
- | Command | What it does |
47
- | ------------------------------------------------------------- | -------------------------------------------------------------------- |
48
- | `npx rehearsal privacy key` | Preview the owner-only pseudonym-key path. |
49
- | `npx rehearsal privacy key --write` | Create the key once; refuses to replace an existing key. |
50
- | `npx rehearsal source plan` | Preview the exact temporary reader, views, columns, and asset scope. |
51
- | `npx rehearsal source apply --confirm-source-access=<sha256>` | Create only the reviewed, time-limited source access. |
52
- | `npx rehearsal baseline refresh` | Replace only the baseline from a reviewed source. |
53
- | `npx rehearsal refresh` | Preview a complete safe refresh of the baseline and local runtime. |
54
- | Add `--confirm-refresh=<sha256>` | Create the replacement, reset locally, and prune listed old copies. |
55
- | `npx rehearsal source retire` | Preview exact reader/view/credential retirement. |
56
- | Add `--confirm-source-retirement=<sha256>` | Apply that retirement plan and verify the exact object inventory. |
46
+ | Command | What it does |
47
+ | ------------------------------------------------------------- | ----------------------------------------------------------------------- |
48
+ | `npx rehearsal privacy key` | Preview the owner-only pseudonym-key path. |
49
+ | `npx rehearsal privacy key --write` | Create the key once; refuses to replace an existing key. |
50
+ | `npx rehearsal source plan` | Preview the exact temporary reader, views, columns, and asset scope. |
51
+ | `npx rehearsal source apply --confirm-source-access=<sha256>` | Create managed access, or verify external access without source writes. |
52
+ | `npx rehearsal baseline refresh` | Replace only the baseline from a reviewed source. |
53
+ | `npx rehearsal refresh` | Preview a complete safe refresh of the baseline and local runtime. |
54
+ | Add `--confirm-refresh=<sha256>` | Create the replacement, reset locally, and prune listed old copies. |
55
+ | `npx rehearsal source retire` | Preview managed retirement or external local-secret cleanup. |
56
+ | Add `--confirm-source-retirement=<sha256>` | Apply that retirement plan and verify the exact object inventory. |
57
57
 
58
58
  Source credentials are read from environment variables named in the reviewed source
59
59
  policy. Never paste a connection string into an argument. `baseline refresh` leaves the
@@ -176,6 +176,8 @@ belonging to other projects are never included. This follows
176
176
 
177
177
  | Option | Meaning |
178
178
  | -------------------------------------- | ---------------------------------------------------------------- |
179
+ | `--help`, `-h` | Show the installed command reference. |
180
+ | `--version`, `-V` | Print the installed package version and exit. |
179
181
  | `--json` | Return the versioned machine-readable result. |
180
182
  | `--verbose` | Show more safe detail. |
181
183
  | `--debug` | Show the most diagnostic detail; still review it before sharing. |
@@ -193,6 +195,8 @@ belonging to other projects are never included. This follows
193
195
  | `--write` | Apply a setup, policy, or cleanup preview. |
194
196
 
195
197
  Run `npx rehearsal --help` to print the command list available in your installed version.
198
+ Run `npx rehearsal --version` to print only its version number. Add `--json` when a
199
+ script also needs the package name.
196
200
 
197
201
  When a project has more than one config, pass `--config=<path>` on every command for the
198
202
  non-default one. Rehearsal keeps that exact config selected through status, run, reset,
@@ -130,15 +130,20 @@ All paths resolve inside the consuming project. The artifact directory must be n
130
130
  `.rehearsal`; this is an intentional deletion guard. `runtimeWorkdir` must be its
131
131
  `runtime` child, and the generated application environment file must remain inside it.
132
132
 
133
- Rehearsal never overwrites this config. To start over, move the existing file somewhere
134
- safe, run setup again, and compare the two files before deleting either one.
135
-
133
+ Rehearsal never overwrites this config. Do not move it merely to inspect a newer template.
136
134
  After upgrading, run `npx rehearsal init` to print the current release's generated
137
135
  template beside the path of your existing config. This comparison is read-only; adding a
138
136
  new optional key remains your decision. `npx rehearsal init --write` still refuses to
139
137
  replace an existing config. Rehearsal lists important optional settings that are
140
138
  available but not enabled before printing the template, so they are easier to find.
141
139
 
140
+ If setup was interrupted after creating only some files, rerun
141
+ `npx rehearsal setup --write`. It recreates missing generated files while preserving every
142
+ existing config and local runtime file. When a config itself is invalid, keep it as a
143
+ backup, generate a fresh file, and manually carry over only reviewed settings. See
144
+ [Getting started](getting-started.md#if-setup-or-an-upgrade-is-interrupted) for the short
145
+ recovery table.
146
+
142
147
  ### Workspaces and custom paths
143
148
 
144
149
  For an npm workspace, automatic creation proceeds only when exactly one application
@@ -151,8 +156,38 @@ checkout never receive project files.
151
156
 
152
157
  Supported existing config paths—including
153
158
  `infrastructure/rehearsal/rehearsal.config.mjs`—are preserved, and a second root config
154
- is not created. Use `--config=<project-relative-path>` for other explicit paths when
155
- running commands.
159
+ is not created. The install hook reports the exact active path when npm displays lifecycle
160
+ output, and setup always prints it. `npx rehearsal init --json` is the package-manager-
161
+ independent way to check the `destination`. Rehearsal checks the supported root filenames
162
+ first and then the supported nested filenames; finding any one of them prevents automatic
163
+ creation of another. Use
164
+ `--config=<project-relative-path>` for other explicit paths when running commands.
165
+
166
+ ### Move an existing config to the project root
167
+
168
+ Relocation is deliberately manual because the config may contain reviewed settings and
169
+ JavaScript imports. Rehearsal will never move it or create a competing root file.
170
+
171
+ From the directory containing `package.json`:
172
+
173
+ 1. Stop active Rehearsal commands and confirm the current path with
174
+ `npx rehearsal init --json`.
175
+ 2. Make a backup outside every supported discovery path, such as
176
+ `.rehearsal/config-backups/rehearsal.config.mjs`.
177
+ 3. Move—do not copy—the active file to `./rehearsal.config.mjs`. Use `git mv` when it is
178
+ tracked so review history remains clear.
179
+ 4. Update relative JavaScript `import` specifiers inside the moved file. Imports resolve
180
+ from the config file's own folder, so a nested `../../something.mjs` import usually
181
+ changes after moving to the root.
182
+ 5. Leave config values such as `migrationDirectory`, `sanitizationPolicy`,
183
+ `runtimePolicy`, `identityPolicy`, and `dependentTargets[].configPath` unchanged unless
184
+ the referenced project file also moved. Rehearsal resolves those values from the
185
+ project root, not from the config file's folder.
186
+ 6. Run `npx rehearsal init --json`, confirm its `destination` is
187
+ `rehearsal.config.mjs`, then run `npx rehearsal doctor` and review the Git diff.
188
+
189
+ If validation fails, restore the backup to its original path. Do not run setup while the
190
+ move is half-finished; setup may correctly conclude that no supported config exists.
156
191
 
157
192
  `preparation.privacyKey` must stay inside `.rehearsal`; it is ignored and owner-readable
158
193
  only. Source, runtime, and identity policy files are declarations that may be reviewed in
@@ -7,7 +7,8 @@ Want to experiment first? Use the [safe hands-on tutorial](tutorial.md).
7
7
 
8
8
  ## 1. Check the required tools
9
9
 
10
- You need Node.js 24, npm, and a running Docker-compatible engine:
10
+ You need Node.js 22, 24, or 26, npm, and a running Docker-compatible engine. Node.js 24
11
+ LTS is the recommended default:
11
12
 
12
13
  ```bash
13
14
  node --version
@@ -15,8 +16,8 @@ npm --version
15
16
  docker info
16
17
  ```
17
18
 
18
- `node --version` must begin with `v24`. If `docker info` fails, start Docker Desktop,
19
- Colima, or your usual Docker engine.
19
+ `node --version` must begin with `v22`, `v24`, or `v26`. If `docker info` fails, start
20
+ Docker Desktop, Colima, or your usual Docker engine.
20
21
 
21
22
  Choose the extra requirement for your project:
22
23
 
@@ -47,6 +48,11 @@ especially:
47
48
  - the application start and proof commands;
48
49
  - the local ports.
49
50
 
51
+ If a supported configuration already exists elsewhere—such as
52
+ `infrastructure/rehearsal/rehearsal.config.mjs`—the installer reports that exact active
53
+ path and does not generate a competing root file. Keep using it, or follow the explicit
54
+ [root relocation guide](configuration.md#move-an-existing-config-to-the-project-root).
55
+
50
56
  If lifecycle scripts were disabled, a global/cache install was used, or a workspace has
51
57
  more than one possible application, no project files are created automatically. Run
52
58
  `npx rehearsal` inside the intended application, choose **Set the stage**, select
@@ -179,6 +185,29 @@ config during an upgrade. After upgrading, run `npx rehearsal init` to see impor
179
185
  optional settings you have not enabled and compare your file with the installed
180
186
  release's current template.
181
187
 
188
+ ### If setup or an upgrade is interrupted
189
+
190
+ Do not delete `.rehearsal/` or your existing config. Start with:
191
+
192
+ ```bash
193
+ npx rehearsal doctor
194
+ npx rehearsal init
195
+ ```
196
+
197
+ Use the reported problem to choose the smallest recovery:
198
+
199
+ | Problem | Safe recovery |
200
+ | ----------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
201
+ | A generated runtime file is missing | Run `npx rehearsal setup --write`. Existing files stay unchanged; only missing files are created. |
202
+ | A port is already in use | Stop the conflicting local process or choose new dedicated ports, then rerun setup. |
203
+ | The config is malformed | Move it to a backup name, run `npx rehearsal setup --write`, then copy reviewed choices into the new file. |
204
+ | OAuth credentials are missing | Create the ignored file named by `supabase.authentication.environmentFile`, add only the declared keys, and run `chmod 600` on it. Ambient shell credentials are not used. |
205
+ | An npm upgrade fails | Keep the existing config, baseline, and lockfile. Restore dependencies with your package manager, then run `npx rehearsal init` before retrying. |
206
+ | Docker storage is low | Run `npx rehearsal cleanup` for an exact preview. Rehearsal never performs a global Docker prune or silently removes volumes. |
207
+
208
+ `setup --write` and package installation never replace a config. If an existing file is
209
+ invalid, Rehearsal fails closed and leaves it available for comparison.
210
+
182
211
  ## Files Rehearsal creates
183
212
 
184
213
  - `rehearsal.config.mjs` — reviewed project configuration
@@ -23,6 +23,15 @@ The privacy key stays ignored under `.rehearsal`. The policy is safe to review i
23
23
  control because it contains hashes and environment-variable names, not credentials,
24
24
  URLs, or raw owner identifiers.
25
25
 
26
+ `credentialFile` must be inside the selected target's
27
+ `baseline.artifactDirectory`, under its `secrets` folder. With the default artifact
28
+ directory, use `.rehearsal/secrets/source-reader.env`. For a target whose artifact
29
+ directory is `targets/primary/.rehearsal`, use
30
+ `targets/primary/.rehearsal/secrets/source-reader.env` and place `privacyKey` there as
31
+ well. Rehearsal stores that target's access receipt beside its baseline at
32
+ `targets/primary/.rehearsal/source-access-receipt.json`; separate targets never share
33
+ the receipt or credential.
34
+
26
35
  ## 2. Declare the source boundary
27
36
 
28
37
  `source-access-policy.json` uses this shape:
@@ -33,6 +42,7 @@ URLs, or raw owner identifiers.
33
42
  "targetFingerprint": "<sha256 of the approved PostgreSQL host, port, and database>",
34
43
  "administratorEnvironmentVariable": "REHEARSAL_SOURCE_ADMIN_URL",
35
44
  "reader": {
45
+ "mode": "managed",
36
46
  "role": "rehearsal_example_reader",
37
47
  "ownerRole": "rehearsal_example_owner",
38
48
  "credentialFile": ".rehearsal/secrets/source-reader.env",
@@ -61,7 +71,7 @@ URLs, or raw owner identifiers.
61
71
  "view": "approved_owner_profile",
62
72
  "targetSchema": "app_api",
63
73
  "targetTable": "profiles",
64
- "columns": ["id", "email"],
74
+ "columns": ["id", "email", "avatar_path"],
65
75
  "orderBy": ["id"],
66
76
  "rowScope": {
67
77
  "kind": "approved-owner",
@@ -70,10 +80,28 @@ URLs, or raw owner identifiers.
70
80
  }
71
81
  }
72
82
  ],
73
- "assets": []
83
+ "assetReader": {
84
+ "endpointFingerprint": "<sha256 of the approved Storage endpoint>",
85
+ "baseUrlEnvironmentVariable": "REHEARSAL_SOURCE_STORAGE_URL",
86
+ "tokenEnvironmentVariable": "REHEARSAL_SOURCE_STORAGE_TOKEN",
87
+ "maximumObjects": 1000,
88
+ "maximumObjectBytes": 52428800,
89
+ "maximumTotalBytes": 2147483648
90
+ },
91
+ "assets": [
92
+ {
93
+ "bucket": "avatars",
94
+ "prefixEnvironmentVariable": "REHEARSAL_APPROVED_OWNER_ID",
95
+ "rights": "approved-owner",
96
+ "pathMapping": "owner-storage"
97
+ }
98
+ ]
74
99
  }
75
100
  ```
76
101
 
102
+ `managed` is the default for existing policies, but writing it explicitly makes the
103
+ credential boundary easier to review.
104
+
77
105
  Calculate fingerprints with the exported helpers from `@rehearsal-db/core/source-access`
78
106
  or a reviewed local script. A fingerprint binds the destination without recording its
79
107
  address. Changing the host, port, or database invalidates the plan.
@@ -82,13 +110,92 @@ address. Changing the host, port, or database invalidates the plan.
82
110
  use. `approved-owner` requires one explicit owner value from the named environment
83
111
  variable. Rehearsal never chooses an owner by finding an administrator.
84
112
 
113
+ ### Use a provider-managed reader
114
+
115
+ If a database owner already supplies the restricted export views and reader, use
116
+ `mode: "external"`. Omit `administratorEnvironmentVariable`; Rehearsal will not request
117
+ an administrator credential or change the source:
118
+
119
+ ```json
120
+ {
121
+ "reader": {
122
+ "mode": "external",
123
+ "role": "provided_rehearsal_reader",
124
+ "allowedMemberships": ["provided_rehearsal_readers"],
125
+ "connectionEnvironmentVariable": "REHEARSAL_SOURCE_READER_URL",
126
+ "credentialFile": ".rehearsal/secrets/source-reader.env",
127
+ "maximumValidForMinutes": 30
128
+ },
129
+ "exportSchema": "provided_rehearsal_export",
130
+ "relations": [
131
+ {
132
+ "source": { "schema": "private", "table": "profiles" },
133
+ "view": "approved_owner_profile",
134
+ "targetSchema": "app_api",
135
+ "targetTable": "profiles",
136
+ "columns": ["id", "email", "avatar_path"],
137
+ "orderBy": ["id"],
138
+ "rowScope": {
139
+ "kind": "approved-owner",
140
+ "column": "id",
141
+ "valueEnvironmentVariable": "REHEARSAL_APPROVED_OWNER_ID"
142
+ },
143
+ "viewDefinitionSha256": "<sha256 from externalViewDefinitionFingerprint>"
144
+ }
145
+ ]
146
+ }
147
+ ```
148
+
149
+ The database owner must create a `security_barrier` view owned by a role other than the
150
+ reader. Record PostgreSQL's exact canonical definition, not handwritten SQL:
151
+
152
+ ```js
153
+ import { externalViewDefinitionFingerprint } from "@rehearsal-db/core/source-access";
154
+
155
+ console.log(externalViewDefinitionFingerprint(canonicalViewDefinition));
156
+ ```
157
+
158
+ Obtain `canonicalViewDefinition` with
159
+ `select pg_get_viewdef('provided_rehearsal_export.approved_owner_profile'::regclass, false)`.
160
+ Rehearsal verifies that fingerprint before saving any local access receipt.
161
+ The resulting receipt also binds the normalized source-policy fingerprint. Editing the
162
+ view hash, columns, scopes, or reader declaration requires a new verification.
163
+
164
+ External mode also requires all non-system readable columns to match only the declared
165
+ views and migration-ledger columns. It refuses database, schema, relation, or column
166
+ writes; privileged role attributes; executable non-system security-definer functions;
167
+ a reader-owned view; or a database-enforced expiration beyond
168
+ `maximumValidForMinutes`.
169
+
170
+ This audit is intentionally conservative: an object-level table, column, sequence, or
171
+ function grant remains disallowed even when its schema currently lacks `USAGE`. Do not
172
+ use schema visibility to hide stale grants. Revoke the object grant or move the approved
173
+ surface behind the reviewed export views; this also avoids relying on behavior of an
174
+ already-open or prepared database session.
175
+
176
+ An external login may inherit its narrow permissions from provider-owned group roles.
177
+ List every direct group in `allowedMemberships`; omit the field or use `[]` when the
178
+ login has direct grants only. Rehearsal requires the observed membership set to match
179
+ that list exactly. Every allowed group must be non-login and non-privileged, grant
180
+ inheritance to the login, disable both `SET ROLE` and membership delegation, inherit no
181
+ other role, and own none of the export views. PostgreSQL versions that cannot prove the
182
+ per-membership role-switch setting are refused. Rehearsal still evaluates the login's
183
+ complete effective read, write, sequence, and executable-function surface after those
184
+ checks. Adding, removing, or changing a group changes the reviewed plan digest.
185
+
85
186
  `targetSchema` is optional and defaults to `public`. Set it when the restored relation
86
187
  belongs elsewhere, and use the same schema in privacy policy version 2. This is relation
87
188
  identity, not a request to search every schema.
88
189
 
89
- For Storage, add an `assetReader` block and approved `assets` entries. The reader URL and
90
- token come from named environment variables. Objects outside the exact bucket/prefix are
91
- not listed or downloaded. Size, object-count, path, ETag, and exact-byte checks apply.
190
+ For Storage, the reader URL and token come from named environment variables. A public,
191
+ non-identity prefix may use `prefix`. A private owner prefix uses
192
+ `prefixEnvironmentVariable`, `rights: "approved-owner"`, and a `pathMapping` declared in
193
+ the privacy policy below. Its raw value stays out of tracked files and the review plan
194
+ records only its SHA-256. Objects outside the exact bucket/prefix are not listed or
195
+ downloaded. Size, object-count, path, ETag, and exact-byte checks apply.
196
+ Supabase folder entries are traversed recursively beneath that exact prefix. Folder
197
+ entries count toward `maximumObjects`, so the same reviewed limit bounds both the
198
+ directory walk and transferred files. Empty files remain files and are copied normally.
92
199
 
93
200
  ## 3. Use executable privacy policy version 2
94
201
 
@@ -98,6 +205,21 @@ Every selected column has one action and, where required, a bounded recipe:
98
205
  {
99
206
  "policyVersion": 2,
100
207
  "migrationCutoff": "20260101000000",
208
+ "bindings": {
209
+ "approved-owner": {
210
+ "environmentVariable": "REHEARSAL_APPROVED_OWNER_ID",
211
+ "approvedValueSha256": "<sha256 of the exact approved owner ID>"
212
+ }
213
+ },
214
+ "pathMappings": {
215
+ "owner-storage": {
216
+ "binding": "approved-owner",
217
+ "format": "uuid",
218
+ "namespace": "account-id",
219
+ "maximumBytes": 512,
220
+ "maximumSegments": 8
221
+ }
222
+ },
101
223
  "tables": [
102
224
  {
103
225
  "schema": "app_api",
@@ -119,6 +241,14 @@ Every selected column has one action and, where required, a bounded recipe:
119
241
  "generated": "NEVER",
120
242
  "identity": "NO",
121
243
  "foreignKey": null
244
+ },
245
+ {
246
+ "name": "avatar_path",
247
+ "action": "DERIVE",
248
+ "recipe": { "kind": "path-map", "mapping": "owner-storage" },
249
+ "generated": "NEVER",
250
+ "identity": "NO",
251
+ "foreignKey": null
122
252
  }
123
253
  ]
124
254
  }
@@ -126,10 +256,13 @@ Every selected column has one action and, where required, a bounded recipe:
126
256
  }
127
257
  ```
128
258
 
129
- Supported recipes are intentionally small: keyed UUID/email/text/integer pseudonyms,
130
- explicit constants, bounded date shifts, and fully classified JSON objects. Unknown
131
- tables, columns, JSON keys, formats, and recipes stop the refresh. There is no JavaScript
132
- or SQL escape hatch.
259
+ Supported recipes are intentionally small: keyed UUID/email/text/integer/hex/GTIN/URL
260
+ pseudonyms, hexadecimal digests of explicitly named sanitized columns, explicit
261
+ constants, bounded or grouped date shifts, identity-aware path mappings, bounded
262
+ reviewed-identity substitution inside retained text, and fully classified JSON
263
+ structures. Unknown tables, columns, digest inputs, JSON keys, formats, and recipes stop
264
+ the refresh. There is no JavaScript or SQL escape hatch. See
265
+ [Sanitization policy](sanitization.md) for complete path and date examples.
133
266
 
134
267
  ## 4. Preview, apply, refresh, and retire
135
268
 
@@ -146,10 +279,12 @@ npx rehearsal source retire
146
279
  npx rehearsal source retire --confirm-source-retirement=<full-sha256>
147
280
  ```
148
281
 
149
- `source apply` creates new time-limited roles and export views only when every exact name
150
- is unused. It verifies the reader cannot read raw tables, write, escalate roles, or call
151
- an exposed security-definer network function. Externally provisioned readers are not yet
152
- supported; use the managed reader so Rehearsal can prove its scope and retirement.
282
+ In managed mode, `source apply` creates new time-limited roles and export views only when
283
+ every exact name is unused. It verifies the reader cannot read raw tables, write,
284
+ escalate roles, or call an exposed security-definer network function. In external mode,
285
+ the same command performs only the read-only checks above and stores the already supplied
286
+ credential in the ignored owner-only local file. It never creates, grants, alters, or
287
+ drops a source object.
153
288
 
154
289
  `refresh` previews the complete replacement first. After exact confirmation, its baseline
155
290
  step uses one read-only repeatable-read transaction for the schema shape,
@@ -160,13 +295,36 @@ runtime is rolled back to the previous baseline and runtime. The lower-level `ba
160
295
  refresh` command remains available when automation should activate a baseline without
161
296
  resetting the runtime or pruning old generations.
162
297
 
298
+ Rows committed after the repeatable-read transaction begins belong to the next refresh;
299
+ they cannot leak into the current snapshot midway through streaming. A schema or ledger
300
+ change during preparation stops activation. Malformed batches, network loss, interruption,
301
+ limit overruns, and low disk also leave the previous baseline active. Because `baseline
302
+ refresh` never resets the runtime, local runtime edits remain in place whether that
303
+ baseline-only preparation succeeds or fails.
304
+
305
+ Privacy `identity` metadata describes PostgreSQL `GENERATED ... AS IDENTITY`, not a
306
+ person or account identifier. UUID account keys normally use `identity: "NO"` while
307
+ remaining explicitly pseudonymized. During restore, Rehearsal consults the actual local
308
+ table for a backing sequence before resetting it; UUID columns never receive numeric
309
+ `max` or `setval` operations.
310
+
163
311
  Storage cannot share PostgreSQL's transaction snapshot. Rehearsal instead verifies each
164
312
  object's inventory version and byte count during transfer, then records its SHA-256 in
165
- the immutable baseline.
313
+ the immutable baseline. For an identity-aware path, the database reference and physical
314
+ destination use the same package-owned pseudonym; the source suffix is preserved.
315
+
316
+ In managed mode, `source retire` inventories the exact recorded views before removal,
317
+ then removes only those views, the two recorded roles, and local credential/receipt
318
+ files. In external mode, it removes only Rehearsal's local credential and receipt; the
319
+ provider-owned role and views remain untouched. The database-enforced expiration still
320
+ limits access, and the provider may revoke it sooner. Rehearsal does not claim that local
321
+ cleanup revoked a provider-owned credential. Unrelated roles, grants, objects, baselines,
322
+ runtimes, and Docker resources are preserved.
166
323
 
167
- `source retire` inventories the exact recorded views before removal, then removes only
168
- those views, the two recorded roles, and local credential/receipt files. Unrelated roles,
169
- grants, objects, baselines, runtimes, and Docker resources are preserved.
324
+ Apply, refresh, and retire always resolve those local files from the active config's
325
+ `baseline.artifactDirectory`. If you switch configs or target roots, Rehearsal refuses
326
+ to retire a receipt from the other target. It also refuses an existing receipt or
327
+ credential rather than overwriting or adopting it.
170
328
 
171
329
  ## Limits
172
330
 
package/docs/releasing.md CHANGED
@@ -32,9 +32,11 @@ publication must change and prove the workflow before narrowing that permission.
32
32
  1. Start from protected `main` on a dedicated ticketed release branch.
33
33
  2. Verify the exact version, changelog, compatibility notes, and packed file list.
34
34
  3. Run unit and contract tests, formatting, package-content and secret audits,
35
- dependency audit, clean tarball installation, and the installed Docker fixture.
36
- 4. Have a developer unfamiliar with the implementing project follow the clean-project
37
- onboarding. Correct and retest the first confusing, missing, or wrong instruction.
35
+ dependency audit, packed onboarding, clean tarball installation, and the installed
36
+ Docker fixtures.
37
+ 4. When an unfamiliar developer is available, have them follow clean-project onboarding.
38
+ Correct and retest the first confusing, missing, or wrong instruction. Record when
39
+ this human observation was unavailable; automated checks must not impersonate it.
38
40
  5. Change `private` to `false` only in the reviewed release change.
39
41
  6. Record the exact tarball filename, SHA-1, SHA-256, allowlisted files, unpacked size,
40
42
  executable, declared runtime dependency inventory, and dependency audit result.
package/docs/roadmap.md CHANGED
@@ -29,8 +29,9 @@ Track the full breakdown in
29
29
  invalid migrations, identities, images, persistence, reset, and cleanup.
30
30
  - Run the released package in the real reporting consumer and record positives and
31
31
  negatives.
32
- - Directly observe any claimed Google account-picker/callback and unfamiliar-developer
33
- onboarding behavior; automation cannot invent that evidence.
32
+ - Directly observe any claimed Google account-picker/callback. Record
33
+ unfamiliar-developer onboarding behavior when a tester is available; automation cannot
34
+ invent either kind of human evidence.
34
35
  - Remove consumer integration code only after its replacement passes and that deletion
35
36
  is separately approved.
36
37