@rehearsal-db/core 0.1.0-beta.12 → 0.1.0-beta.14
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +116 -0
- package/COMPATIBILITY.md +6 -0
- package/README.md +5 -3
- package/docs/commands.md +15 -11
- package/docs/configuration.md +40 -5
- package/docs/getting-started.md +32 -3
- package/docs/production-source.md +175 -17
- package/docs/releasing.md +5 -3
- package/docs/roadmap.md +3 -2
- package/docs/sanitization.md +479 -12
- package/docs/security-model.md +8 -0
- package/docs/standalone-workflow.md +34 -13
- package/docs/troubleshooting.md +17 -6
- package/docs/tutorial.md +7 -1
- package/package.json +2 -2
- package/scripts/install/create_root_config.mjs +5 -22
- package/src/baseline/artifact.mjs +27 -4
- package/src/baseline/privacy_engine.mjs +865 -207
- package/src/baseline/privacy_structured_json.mjs +554 -0
- package/src/cli/arguments.mjs +3 -0
- package/src/cli/guided.mjs +8 -6
- package/src/cli/rehearsal.mjs +23 -3
- package/src/cli/renderers.mjs +29 -4
- package/src/cli/runtime_commands.mjs +92 -9
- package/src/cli/source_commands.mjs +37 -8
- package/src/identity/claim.mjs +23 -21
- package/src/project/install_scaffold.mjs +35 -1
- package/src/project/setup.mjs +14 -24
- package/src/runtime/plan.mjs +7 -3
- package/src/runtime/restore.mjs +13 -5
- package/src/shared/diagnostics.mjs +7 -0
- package/src/shared/node_runtime.mjs +31 -0
- package/src/source/access.mjs +271 -51
- package/src/source/asset_transfer.mjs +156 -74
- package/src/source/baseline.mjs +42 -6
- package/src/source/external_reader.mjs +368 -0
- package/src/source/postgresql_access.mjs +213 -55
- package/src/source/privacy_paths.mjs +38 -0
package/docs/sanitization.md
CHANGED
|
@@ -39,11 +39,13 @@ Two policy versions exist:
|
|
|
39
39
|
- Version 2 is required by `baseline refresh`. It adds bounded declarative recipes so
|
|
40
40
|
Rehearsal can sanitize without project callbacks.
|
|
41
41
|
|
|
42
|
-
Version 2 supports keyed UUID, email, text,
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
42
|
+
Version 2 supports keyed UUID, email, text, integer, fixed-width hexadecimal, and GTIN
|
|
43
|
+
pseudonyms; digests derived from named sanitized columns; explicit constant replacement;
|
|
44
|
+
bounded and grouped date shifting; identity-aware path mapping;
|
|
45
|
+
approved-owner conditionals; exhaustively classified JSON objects; bounded JSON arrays;
|
|
46
|
+
reviewed JSON type unions; and runtime-keyed JSON dictionaries. Unknown tables, columns,
|
|
47
|
+
nested keys, formats, recipes, variants, or missing required fields fail closed. It never
|
|
48
|
+
evaluates JavaScript, regular expressions, or SQL from the policy.
|
|
47
49
|
|
|
48
50
|
## Approved-owner values
|
|
49
51
|
|
|
@@ -69,7 +71,7 @@ environment-variable name and SHA-256 receipt:
|
|
|
69
71
|
"action": "PSEUDONYMIZE",
|
|
70
72
|
"recipe": { "format": "uuid", "namespace": "account-id" },
|
|
71
73
|
"generated": "NEVER",
|
|
72
|
-
"identity": "
|
|
74
|
+
"identity": "NO",
|
|
73
75
|
"foreignKey": null
|
|
74
76
|
},
|
|
75
77
|
{
|
|
@@ -100,12 +102,278 @@ node -e 'const {createHash}=require("node:crypto"); process.stdout.write(createH
|
|
|
100
102
|
```
|
|
101
103
|
|
|
102
104
|
Every conditional table names its owner column. Missing bindings, hash mismatches,
|
|
103
|
-
missing owner columns,
|
|
104
|
-
binding are refused. The owner column must
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
105
|
+
missing owner columns, unsupported owner values, and conditionals without an owner
|
|
106
|
+
binding are refused. The owner column must use `PSEUDONYMIZE`, preventing a conditional
|
|
107
|
+
policy from retaining raw account IDs.
|
|
108
|
+
|
|
109
|
+
When ownership may come through several relationships, replace `column` with an explicit
|
|
110
|
+
bounded any-of declaration:
|
|
111
|
+
|
|
112
|
+
```json
|
|
113
|
+
{
|
|
114
|
+
"ownerBinding": {
|
|
115
|
+
"binding": "approved-owner",
|
|
116
|
+
"columns": ["target_user_id", "actor_user_id"],
|
|
117
|
+
"match": "any",
|
|
118
|
+
"nullBehavior": "otherwise"
|
|
119
|
+
}
|
|
120
|
+
}
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
Every listed column must exist and be pseudonymized. One matching value selects the
|
|
124
|
+
`approved` branch. `nullBehavior: "otherwise"` explicitly treats a null value as a
|
|
125
|
+
non-match; it never grants owner access. Without that declaration, encountering null is
|
|
126
|
+
an error. Empty strings, non-finite numbers (`NaN`, positive infinity, or negative
|
|
127
|
+
infinity), objects, arrays, booleans, missing columns, duplicate declarations, and
|
|
128
|
+
unsupported matching modes remain errors. Rehearsal validates every declared owner
|
|
129
|
+
column before selecting either branch, even when another owner column matches. A
|
|
130
|
+
single-column binding may also declare `nullBehavior: "otherwise"` when that column is
|
|
131
|
+
reviewed as nullable.
|
|
132
|
+
|
|
133
|
+
`identity` describes PostgreSQL's generated SQL identity metadata, not whether a value
|
|
134
|
+
identifies a person: ordinary UUID primary keys use `"NO"`, while a genuine
|
|
135
|
+
`GENERATED ... AS IDENTITY` column uses `"YES"`. Rehearsal checks the restored PostgreSQL
|
|
136
|
+
schema for a real backing sequence before attempting a sequence reset, so UUID identity
|
|
137
|
+
values never receive numeric sequence operations. Other owners take the declared
|
|
138
|
+
`otherwise` path; every other identity-bearing field still needs its own pseudonymization
|
|
139
|
+
declaration. Supply the binding only to the local server-side Rehearsal process; do not
|
|
140
|
+
put it in the policy, baseline, or tracked files.
|
|
141
|
+
|
|
142
|
+
## Identity-aware Storage paths
|
|
143
|
+
|
|
144
|
+
Use one named mapping when a database column and a physical Storage object both begin
|
|
145
|
+
with the approved owner's ID. Rehearsal pseudonymizes only that first segment and keeps
|
|
146
|
+
the remaining path unchanged:
|
|
147
|
+
|
|
148
|
+
```json
|
|
149
|
+
{
|
|
150
|
+
"pathMappings": {
|
|
151
|
+
"owner-storage": {
|
|
152
|
+
"binding": "approved-owner",
|
|
153
|
+
"format": "uuid",
|
|
154
|
+
"namespace": "account-id",
|
|
155
|
+
"maximumBytes": 512,
|
|
156
|
+
"maximumSegments": 8
|
|
157
|
+
}
|
|
158
|
+
},
|
|
159
|
+
"tables": [
|
|
160
|
+
{
|
|
161
|
+
"name": "profiles",
|
|
162
|
+
"sourceRows": "STREAM AND SANITIZE",
|
|
163
|
+
"columns": [
|
|
164
|
+
{
|
|
165
|
+
"name": "avatar_path",
|
|
166
|
+
"action": "DERIVE",
|
|
167
|
+
"recipe": { "kind": "path-map", "mapping": "owner-storage" },
|
|
168
|
+
"generated": "NEVER",
|
|
169
|
+
"identity": "NO",
|
|
170
|
+
"foreignKey": null
|
|
171
|
+
}
|
|
172
|
+
]
|
|
173
|
+
}
|
|
174
|
+
]
|
|
175
|
+
}
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
The source policy uses the same mapping name and names the binding's environment
|
|
179
|
+
variable as `prefixEnvironmentVariable`; it never contains the raw owner ID. During
|
|
180
|
+
`rehearsal refresh`, the package uses the mapping for both database values and physical
|
|
181
|
+
object destinations. A missing mapping, a different environment variable, a path whose
|
|
182
|
+
first segment is not the exact reviewed owner, traversal, or a size/segment limit breach
|
|
183
|
+
stops the refresh.
|
|
184
|
+
|
|
185
|
+
## Related dates
|
|
186
|
+
|
|
187
|
+
Give related timestamps the same `group` and `days` value to shift them by one stable
|
|
188
|
+
offset. This preserves their order and duration:
|
|
189
|
+
|
|
190
|
+
```json
|
|
191
|
+
{
|
|
192
|
+
"name": "starts_at",
|
|
193
|
+
"action": "DERIVE",
|
|
194
|
+
"recipe": { "kind": "date-shift", "days": 30, "group": "event-window" },
|
|
195
|
+
"generated": "NEVER",
|
|
196
|
+
"identity": "NO",
|
|
197
|
+
"foreignKey": null
|
|
198
|
+
}
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
Apply the same recipe to `ends_at`. Rehearsal refuses a group that declares conflicting
|
|
202
|
+
`days` values. Leave out `group` when a date should shift independently.
|
|
203
|
+
|
|
204
|
+
`date-shift` returns an ISO string by default, preserving the existing behavior. When
|
|
205
|
+
the reviewed application model stores a date as numeric epoch milliseconds, declare
|
|
206
|
+
that representation explicitly:
|
|
207
|
+
|
|
208
|
+
```json
|
|
209
|
+
{
|
|
210
|
+
"action": "DERIVE",
|
|
211
|
+
"recipe": {
|
|
212
|
+
"kind": "date-shift",
|
|
213
|
+
"days": 30,
|
|
214
|
+
"group": "saved-recipe-window",
|
|
215
|
+
"representation": "epoch-milliseconds"
|
|
216
|
+
}
|
|
217
|
+
}
|
|
218
|
+
```
|
|
219
|
+
|
|
220
|
+
This form accepts only finite safe integers that JavaScript can represent as dates and
|
|
221
|
+
returns a numeric epoch-millisecond value. It preserves `null`, refuses strings,
|
|
222
|
+
fractions, non-finite numbers, unsafe integers and values outside the supported date
|
|
223
|
+
range. ISO-string and epoch-millisecond declarations may share a group; they receive the
|
|
224
|
+
same stable offset while each retains its declared output representation. The only
|
|
225
|
+
supported representation names are `iso-string` and `epoch-milliseconds`.
|
|
226
|
+
|
|
227
|
+
## Constraint-preserving identifiers and fingerprints
|
|
228
|
+
|
|
229
|
+
Use a keyed `hex` pseudonym when one source value must remain a lowercase 32- or
|
|
230
|
+
64-character hexadecimal value:
|
|
231
|
+
|
|
232
|
+
```json
|
|
233
|
+
{
|
|
234
|
+
"name": "source_fingerprint",
|
|
235
|
+
"action": "PSEUDONYMIZE",
|
|
236
|
+
"recipe": {
|
|
237
|
+
"format": "hex",
|
|
238
|
+
"namespace": "source-fingerprint",
|
|
239
|
+
"length": 64
|
|
240
|
+
},
|
|
241
|
+
"generated": "NEVER",
|
|
242
|
+
"identity": "NO",
|
|
243
|
+
"foreignKey": null
|
|
244
|
+
}
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
Use a `digest` derivation when the fingerprint must describe other sanitized fields.
|
|
248
|
+
Inputs are column names, evaluated in the listed order after their own privacy recipes.
|
|
249
|
+
The existing raw fingerprint is ignored:
|
|
250
|
+
|
|
251
|
+
```json
|
|
252
|
+
{
|
|
253
|
+
"name": "evidence_fingerprint",
|
|
254
|
+
"action": "DERIVE",
|
|
255
|
+
"recipe": {
|
|
256
|
+
"kind": "digest",
|
|
257
|
+
"format": "hex",
|
|
258
|
+
"namespace": "evidence-fingerprint",
|
|
259
|
+
"length": 32,
|
|
260
|
+
"inputs": ["sanitized_owner_id", "evidence_state"]
|
|
261
|
+
},
|
|
262
|
+
"generated": "NEVER",
|
|
263
|
+
"identity": "NO",
|
|
264
|
+
"foreignKey": null
|
|
265
|
+
}
|
|
266
|
+
```
|
|
267
|
+
|
|
268
|
+
Digest inputs must exist, must not be excluded, and cannot form a dependency cycle.
|
|
269
|
+
Rehearsal supports only 32- and 64-character hexadecimal output. The namespace and local
|
|
270
|
+
privacy key separate otherwise identical declarations.
|
|
271
|
+
|
|
272
|
+
Use the `gtin` pseudonym format for a normalized GTIN whose length and check digit must
|
|
273
|
+
remain valid:
|
|
274
|
+
|
|
275
|
+
```json
|
|
276
|
+
{
|
|
277
|
+
"name": "barcode",
|
|
278
|
+
"action": "PSEUDONYMIZE",
|
|
279
|
+
"recipe": { "format": "gtin", "namespace": "product-barcode", "length": 14 },
|
|
280
|
+
"generated": "NEVER",
|
|
281
|
+
"identity": "NO",
|
|
282
|
+
"foreignKey": null
|
|
283
|
+
}
|
|
284
|
+
```
|
|
285
|
+
|
|
286
|
+
Supported GTIN lengths are 8, 12, 13, and 14. Non-null input must be a normalized string
|
|
287
|
+
of exactly that length with a valid check digit; malformed input stops preparation. Null
|
|
288
|
+
stays null. Rehearsal creates a deterministic keyed payload and recalculates the check
|
|
289
|
+
digit, so related fields using the same namespace, length, source value, and key remain
|
|
290
|
+
aligned.
|
|
291
|
+
|
|
292
|
+
When one reviewed field legitimately contains more than one GTIN length, replace
|
|
293
|
+
`"length": 14` with an explicit allowlist such as `"allowedLengths": [12, 14]`. The two
|
|
294
|
+
keys are mutually exclusive. Each source value must match one allowed length, and its
|
|
295
|
+
result retains that exact length. Empty, duplicated, or unsupported allowlist entries
|
|
296
|
+
are refused. This also works inside structured JSON declarations; a JSON union is not
|
|
297
|
+
needed merely to distinguish GTIN string lengths.
|
|
298
|
+
|
|
299
|
+
Use the `url` pseudonym format when a sanitized field must remain an HTTPS URL:
|
|
300
|
+
|
|
301
|
+
```json
|
|
302
|
+
{
|
|
303
|
+
"name": "evidence_reference",
|
|
304
|
+
"action": "PSEUDONYMIZE",
|
|
305
|
+
"recipe": {
|
|
306
|
+
"format": "url",
|
|
307
|
+
"namespace": "evidence-url",
|
|
308
|
+
"origin": "https://127.0.0.1:58432",
|
|
309
|
+
"maxLength": 500
|
|
310
|
+
},
|
|
311
|
+
"generated": "NEVER",
|
|
312
|
+
"identity": "NO",
|
|
313
|
+
"foreignKey": null
|
|
314
|
+
}
|
|
315
|
+
```
|
|
316
|
+
|
|
317
|
+
`origin` is required and must be an HTTPS loopback origin using `127.0.0.1`, `::1`, or
|
|
318
|
+
`localhost`. Credentials, paths, queries, fragments, non-HTTPS schemes, and non-local
|
|
319
|
+
hosts are refused. The value must be the canonical origin itself, optionally followed by
|
|
320
|
+
one `/`; dot segments and encoded dot segments are rejected before URL normalization.
|
|
321
|
+
`maxLength` is also required, cannot exceed 2,048 characters, and must fit the complete
|
|
322
|
+
generated URL.
|
|
323
|
+
|
|
324
|
+
Non-null input must be a valid HTTPS URL without surrounding or embedded whitespace.
|
|
325
|
+
Rehearsal hashes the entire source value with the local privacy key and namespace, then
|
|
326
|
+
emits only the reviewed origin and a 32-character pseudonymous path. It never copies the
|
|
327
|
+
source host, credentials, path, query, token, or fragment. Null remains null. Equal
|
|
328
|
+
inputs using the same namespace, key, and origin correspond across ordinary columns and
|
|
329
|
+
structured JSON; different inputs and rotated keys produce different paths.
|
|
330
|
+
|
|
331
|
+
The same declaration can be used for items inside bounded `json-array` recipes, which
|
|
332
|
+
covers PostgreSQL text-array projections represented by the source pipeline. Each array
|
|
333
|
+
and object still needs its ordinary item, depth, and byte bounds.
|
|
334
|
+
|
|
335
|
+
### Retained text containing a reviewed identity
|
|
336
|
+
|
|
337
|
+
Use `binding-substitute` when permitted text may contain the exact value of a reviewed
|
|
338
|
+
UUID binding. This keeps the surrounding text but replaces every occurrence of that UUID
|
|
339
|
+
with the same keyed pseudonym used by a `uuid` declaration sharing its namespace:
|
|
340
|
+
|
|
341
|
+
```json
|
|
342
|
+
{
|
|
343
|
+
"action": "DERIVE",
|
|
344
|
+
"recipe": {
|
|
345
|
+
"kind": "approved-owner",
|
|
346
|
+
"approved": {
|
|
347
|
+
"action": "DERIVE",
|
|
348
|
+
"recipe": {
|
|
349
|
+
"kind": "binding-substitute",
|
|
350
|
+
"binding": "approved-owner",
|
|
351
|
+
"format": "uuid",
|
|
352
|
+
"namespace": "account-id",
|
|
353
|
+
"maximumBytes": 256
|
|
354
|
+
}
|
|
355
|
+
},
|
|
356
|
+
"otherwise": {
|
|
357
|
+
"action": "PSEUDONYMIZE",
|
|
358
|
+
"recipe": {
|
|
359
|
+
"format": "text",
|
|
360
|
+
"namespace": "food-description",
|
|
361
|
+
"maxLength": 128
|
|
362
|
+
}
|
|
363
|
+
}
|
|
364
|
+
}
|
|
365
|
+
}
|
|
366
|
+
```
|
|
367
|
+
|
|
368
|
+
The named binding must already exist in `policy.bindings`, and its environment value must
|
|
369
|
+
match the tracked SHA-256 review receipt. For this recipe the reviewed value must be a
|
|
370
|
+
UUID. `maximumBytes` is required and applies before and after replacement. Null remains
|
|
371
|
+
null; other input must be text. Ordinary text is unchanged, literal UUID occurrences are
|
|
372
|
+
all replaced, and the raw identity is never stored in policy.
|
|
373
|
+
|
|
374
|
+
This is intentionally not a general search-and-replace facility: there are no regular
|
|
375
|
+
expressions, arbitrary patterns, replacement strings, or callbacks. Pair it with an
|
|
376
|
+
`approved-owner` branch when only the reviewed owner's surrounding text may be retained.
|
|
109
377
|
|
|
110
378
|
## Structured JSON
|
|
111
379
|
|
|
@@ -145,6 +413,205 @@ the same declarations, and every nested identity uses an explicit pseudonym reci
|
|
|
145
413
|
}
|
|
146
414
|
```
|
|
147
415
|
|
|
416
|
+
### JSON values with several reviewed types
|
|
417
|
+
|
|
418
|
+
Use `json-union` when one JSON column legitimately stores more than one root type. Every
|
|
419
|
+
accepted type has its own declaration. Null is rejected unless `allowNull` is explicitly
|
|
420
|
+
true. Arrays and objects cannot use whole-value `KEEP`; their contents must be classified
|
|
421
|
+
through `json-array`, `json-object`, `json-dictionary`, another bounded union, or a safe
|
|
422
|
+
replacement or pseudonymization.
|
|
423
|
+
|
|
424
|
+
```json
|
|
425
|
+
{
|
|
426
|
+
"action": "DERIVE",
|
|
427
|
+
"recipe": {
|
|
428
|
+
"kind": "json-union",
|
|
429
|
+
"allowNull": true,
|
|
430
|
+
"maximumItems": 25,
|
|
431
|
+
"maximumDepth": 5,
|
|
432
|
+
"maximumBytes": 8192,
|
|
433
|
+
"variants": {
|
|
434
|
+
"string": { "action": "KEEP" },
|
|
435
|
+
"number": { "action": "KEEP" },
|
|
436
|
+
"array": {
|
|
437
|
+
"action": "DERIVE",
|
|
438
|
+
"recipe": {
|
|
439
|
+
"kind": "json-array",
|
|
440
|
+
"maximumItems": 25,
|
|
441
|
+
"items": {
|
|
442
|
+
"action": "REPLACE",
|
|
443
|
+
"recipe": { "kind": "constant", "value": "Synthetic item" }
|
|
444
|
+
}
|
|
445
|
+
}
|
|
446
|
+
},
|
|
447
|
+
"object": {
|
|
448
|
+
"action": "DERIVE",
|
|
449
|
+
"recipe": {
|
|
450
|
+
"kind": "json-object",
|
|
451
|
+
"fields": {
|
|
452
|
+
"status": { "action": "KEEP" },
|
|
453
|
+
"note": {
|
|
454
|
+
"action": "REPLACE",
|
|
455
|
+
"required": false,
|
|
456
|
+
"recipe": { "kind": "constant", "value": "Synthetic note" }
|
|
457
|
+
}
|
|
458
|
+
}
|
|
459
|
+
}
|
|
460
|
+
}
|
|
461
|
+
}
|
|
462
|
+
}
|
|
463
|
+
}
|
|
464
|
+
```
|
|
465
|
+
|
|
466
|
+
Supported variant names are `string`, `number`, `boolean`, `array`, and `object`.
|
|
467
|
+
Undeclared types, non-finite numbers, excessive bytes, depth or items, and unclassified
|
|
468
|
+
object fields fail closed.
|
|
469
|
+
|
|
470
|
+
### Reviewed scalar domains
|
|
471
|
+
|
|
472
|
+
Use `enum` when a structural string, number, or boolean must remain exact, but only a
|
|
473
|
+
small reviewed set is valid. For example, an image mode can retain its application
|
|
474
|
+
meaning without allowing arbitrary text through:
|
|
475
|
+
|
|
476
|
+
```json
|
|
477
|
+
{
|
|
478
|
+
"action": "DERIVE",
|
|
479
|
+
"recipe": {
|
|
480
|
+
"kind": "enum",
|
|
481
|
+
"values": ["contain", "cover", "custom"],
|
|
482
|
+
"allowNull": true
|
|
483
|
+
}
|
|
484
|
+
}
|
|
485
|
+
```
|
|
486
|
+
|
|
487
|
+
The recipe accepts one through 128 distinct values. Each value must be a JSON string,
|
|
488
|
+
finite number, or boolean; strings are limited to 256 UTF-8 bytes. `null` is accepted
|
|
489
|
+
only when `allowNull` is true. Runtime values match by both type and exact value, so the
|
|
490
|
+
number `1` does not authorize the string `"1"`. Rejected values are never included in
|
|
491
|
+
the error message.
|
|
492
|
+
|
|
493
|
+
An enum can be used directly or inside reviewed objects, arrays, dictionary values, and
|
|
494
|
+
`approved-owner` branches. It validates structure; it is not a place to list private
|
|
495
|
+
data, credentials, tokens, user-entered prose, or identifiers.
|
|
496
|
+
|
|
497
|
+
### Open structural codes
|
|
498
|
+
|
|
499
|
+
Use `validated-string` when an open set of structural codes must remain exact and an
|
|
500
|
+
enum would incorrectly freeze the domain. The built-in `portable-code` format accepts
|
|
501
|
+
case-preserving ASCII letters, digits, `_`, and `-`, starting with a letter or digit:
|
|
502
|
+
|
|
503
|
+
```json
|
|
504
|
+
{
|
|
505
|
+
"action": "DERIVE",
|
|
506
|
+
"recipe": {
|
|
507
|
+
"kind": "validated-string",
|
|
508
|
+
"format": "portable-code",
|
|
509
|
+
"maximumBytes": 64,
|
|
510
|
+
"allowNull": true
|
|
511
|
+
}
|
|
512
|
+
}
|
|
513
|
+
```
|
|
514
|
+
|
|
515
|
+
`maximumBytes` is required and may be 1 through 1,024. Dots, whitespace, slashes,
|
|
516
|
+
colons, control characters, non-ASCII text, wrong JSON types, and over-limit strings
|
|
517
|
+
are rejected without printing the input. This preserves codes such as `PROCNT`, `203`,
|
|
518
|
+
or `energy-kcal_100g`, while refusing JWT-shaped strings. Arbitrary regular expressions
|
|
519
|
+
are not supported.
|
|
520
|
+
|
|
521
|
+
Like enums, validated strings compose inside reviewed objects, arrays, dictionary
|
|
522
|
+
values, and `approved-owner` branches. Use them only for documented structural code
|
|
523
|
+
domains—not prose, URLs, paths, credentials, secrets, personal data, or identifiers
|
|
524
|
+
whose legitimate alphabet requires other characters.
|
|
525
|
+
|
|
526
|
+
### JSON dictionaries with runtime keys
|
|
527
|
+
|
|
528
|
+
Use `json-dictionary` for a reviewed object whose keys come from runtime data. This is
|
|
529
|
+
different from `json-object`, where every field name is known in advance. A dictionary
|
|
530
|
+
requires a key format, an explicit key action, one value declaration, and item, depth and
|
|
531
|
+
byte bounds:
|
|
532
|
+
|
|
533
|
+
```json
|
|
534
|
+
{
|
|
535
|
+
"action": "DERIVE",
|
|
536
|
+
"recipe": {
|
|
537
|
+
"kind": "json-dictionary",
|
|
538
|
+
"maximumItems": 100,
|
|
539
|
+
"maximumDepth": 3,
|
|
540
|
+
"maximumBytes": 8192,
|
|
541
|
+
"keys": { "format": "positive-integer", "action": "KEEP" },
|
|
542
|
+
"values": {
|
|
543
|
+
"action": "DERIVE",
|
|
544
|
+
"recipe": {
|
|
545
|
+
"kind": "json-union",
|
|
546
|
+
"maximumItems": 1,
|
|
547
|
+
"variants": { "number": { "action": "KEEP" } }
|
|
548
|
+
}
|
|
549
|
+
}
|
|
550
|
+
}
|
|
551
|
+
}
|
|
552
|
+
```
|
|
553
|
+
|
|
554
|
+
Key formats are `integer`, `positive-integer`, `uuid`, `identifier`, and bounded `text`.
|
|
555
|
+
Use `integer` when runtime keys may be positive, negative, or zero. It requires inclusive
|
|
556
|
+
safe-integer `minimum` and `maximum` bounds, accepts only canonical decimal keys, and
|
|
557
|
+
preserves their exact spelling:
|
|
558
|
+
|
|
559
|
+
```json
|
|
560
|
+
{
|
|
561
|
+
"keys": {
|
|
562
|
+
"format": "integer",
|
|
563
|
+
"action": "KEEP",
|
|
564
|
+
"minimum": -9007199254740991,
|
|
565
|
+
"maximum": 9007199254740991
|
|
566
|
+
}
|
|
567
|
+
}
|
|
568
|
+
```
|
|
569
|
+
|
|
570
|
+
The reviewed range decides whether zero is allowed. Leading zeroes, `-0`, a leading `+`,
|
|
571
|
+
fractions, exponents, unsafe integers, and unrelated text are rejected. Use
|
|
572
|
+
`positive-integer` for canonical decimal keys starting at 1. Text keys must declare
|
|
573
|
+
`maximumLength`. Every key explicitly uses either `KEEP` or `PSEUDONYMIZE`.
|
|
574
|
+
Identity-bearing keys should use `PSEUDONYMIZE` with the ordinary keyed pseudonym recipe:
|
|
575
|
+
|
|
576
|
+
```json
|
|
577
|
+
{
|
|
578
|
+
"keys": {
|
|
579
|
+
"format": "uuid",
|
|
580
|
+
"action": "PSEUDONYMIZE",
|
|
581
|
+
"recipe": { "format": "uuid", "namespace": "account-id" }
|
|
582
|
+
}
|
|
583
|
+
}
|
|
584
|
+
```
|
|
585
|
+
|
|
586
|
+
When a dictionary is keyed by the same numeric identity stored in a column or nested
|
|
587
|
+
numeric field, use `format: "integer"` for both pseudonym recipes and reuse the exact
|
|
588
|
+
namespace:
|
|
589
|
+
|
|
590
|
+
```json
|
|
591
|
+
{
|
|
592
|
+
"keys": {
|
|
593
|
+
"format": "integer",
|
|
594
|
+
"action": "PSEUDONYMIZE",
|
|
595
|
+
"minimum": -9007199254740991,
|
|
596
|
+
"maximum": 9007199254740991,
|
|
597
|
+
"recipe": { "format": "integer", "namespace": "external:food-id" }
|
|
598
|
+
}
|
|
599
|
+
}
|
|
600
|
+
```
|
|
601
|
+
|
|
602
|
+
Rehearsal validates the object key as canonical decimal text, converts it to its safe
|
|
603
|
+
integer identity, and then applies the keyed mapping. Therefore the source number `203`
|
|
604
|
+
and reviewed dictionary key `"203"` receive the same pseudonym. Positive, negative, and
|
|
605
|
+
reviewed zero identities correspond; namespace changes and privacy-key rotation still
|
|
606
|
+
produce separate mappings. This normalization applies only to explicitly typed
|
|
607
|
+
`integer` dictionary keys. It does not weaken the key syntax or range checks.
|
|
608
|
+
|
|
609
|
+
Invalid keys, duplicate transformed keys, excessive entries, unclassified surrounding
|
|
610
|
+
object fields, and values rejected by the declared value recipe stop preparation. A
|
|
611
|
+
dictionary never infers that runtime keys are safe. Its value declaration also cannot
|
|
612
|
+
retain an unknown JSON shape wholesale; use a bounded union when a scalar value should be
|
|
613
|
+
kept exactly.
|
|
614
|
+
|
|
148
615
|
The package still exports `validateSanitizationCoverage` and
|
|
149
616
|
`applySanitizationAction` for existing version 1 preparation tools:
|
|
150
617
|
|
package/docs/security-model.md
CHANGED
|
@@ -26,6 +26,9 @@ runtime.
|
|
|
26
26
|
- successful receipts written only after verification.
|
|
27
27
|
- separate exact-digest approval for source setup, source retirement, and identity claim;
|
|
28
28
|
- short-lived source readers limited to reviewed views/columns and deny checks;
|
|
29
|
+
- externally provisioned readers require canonical view-definition receipts,
|
|
30
|
+
`security_barrier`, independent ownership, an exact non-system readable-column surface,
|
|
31
|
+
no write or role-escalation paths, and database-enforced expiration;
|
|
29
32
|
- bounded read-only extraction with no raw dump or row-value logging;
|
|
30
33
|
- exhaustive privacy policy version 2 with an owner-only local pseudonym key;
|
|
31
34
|
- application children receive only explicit local mappings and stay in an owned process
|
|
@@ -39,6 +42,11 @@ No single environment variable or config edit can redirect runtime commands to
|
|
|
39
42
|
production. Optional source preparation is a separate boundary and never provides a
|
|
40
43
|
hosted execution mode.
|
|
41
44
|
|
|
45
|
+
External source mode verifies access but does not own it. Rehearsal makes no source-side
|
|
46
|
+
changes and removes only its local credential and receipt. The provider owns early
|
|
47
|
+
revocation and the database enforces the reviewed short expiration; Rehearsal never
|
|
48
|
+
reports local cleanup as proof that a provider-owned role was deleted.
|
|
49
|
+
|
|
42
50
|
## Identity providers
|
|
43
51
|
|
|
44
52
|
An optional external identity handshake is distinct from database access. Only declared
|
|
@@ -8,16 +8,18 @@ The optional preparation workflow is a separate security boundary. It has
|
|
|
8
8
|
three deliberately separate phases:
|
|
9
9
|
|
|
10
10
|
1. `source plan` reads declarations and produces a redacted, exact plan.
|
|
11
|
-
2. `source apply` requires the plan digest
|
|
12
|
-
named environment variable
|
|
13
|
-
|
|
11
|
+
2. `source apply` requires the plan digest. Managed mode uses source-owner credentials
|
|
12
|
+
from a named environment variable and creates only the declared, time-limited reader
|
|
13
|
+
and export surface. External mode verifies an already restricted reader without an
|
|
14
|
+
administrator credential or source-side writes.
|
|
14
15
|
3. `refresh` uses that reader in a read-only repeatable-read transaction, builds
|
|
15
16
|
privately, activates only after verification, resets the local runtime, and applies
|
|
16
17
|
reviewed retention. `baseline refresh` is the lower-level baseline-only form.
|
|
17
18
|
|
|
18
|
-
`source retire` has its own preview and digest.
|
|
19
|
-
reader and export objects Rehearsal recorded.
|
|
20
|
-
|
|
19
|
+
`source retire` has its own preview and digest. Managed mode removes only the exact
|
|
20
|
+
reader and export objects Rehearsal recorded. External mode removes only local Rehearsal
|
|
21
|
+
credentials and receipts, preserving provider-owned objects. It never searches for
|
|
22
|
+
similarly named roles or removes unrelated grants.
|
|
21
23
|
|
|
22
24
|
## What belongs in declarations
|
|
23
25
|
|
|
@@ -46,9 +48,16 @@ separate. A declaration contains only the name of an environment variable or
|
|
|
46
48
|
owner-only secret file. Rehearsal never puts credential values in command-line
|
|
47
49
|
arguments, reports, baselines, receipts, or a launched application.
|
|
48
50
|
|
|
49
|
-
The
|
|
50
|
-
|
|
51
|
-
|
|
51
|
+
The default managed workflow creates and verifies its own temporary PostgreSQL reader.
|
|
52
|
+
External mode accepts a provider-managed reader only when its complete non-system
|
|
53
|
+
readable-column surface exactly matches the reviewed export views and migration ledger.
|
|
54
|
+
It also verifies canonical view-definition fingerprints, `security_barrier`, ownership,
|
|
55
|
+
write denial, security-definer denial, and a short database-enforced expiration. An
|
|
56
|
+
external login can declare a bounded exact list of provider-owned permission groups;
|
|
57
|
+
each must be non-login, non-privileged, directly inherited, unable to be assumed or
|
|
58
|
+
delegated, and free of nested membership. Undeclared or unsafe role membership fails as
|
|
59
|
+
an actionable source-authorization refusal. Rehearsal never claims ownership or
|
|
60
|
+
revocation of provider-managed objects.
|
|
52
61
|
|
|
53
62
|
## Snapshot and privacy guarantees
|
|
54
63
|
|
|
@@ -57,16 +66,28 @@ read-only repeatable-read PostgreSQL transaction. Rehearsal rechecks the source
|
|
|
57
66
|
identity and migration evidence before commit. Rows stream in bounded batches;
|
|
58
67
|
record bodies never appear in progress or error output.
|
|
59
68
|
|
|
69
|
+
The snapshot boundary is deliberate: row commits made after extraction starts are left
|
|
70
|
+
for the next baseline, while concurrent schema or migration-ledger changes block the
|
|
71
|
+
current activation. Invalid estimates, malformed batches, connection loss, interruption,
|
|
72
|
+
and disk or configured-limit failures remove private staging and preserve the previously
|
|
73
|
+
active baseline.
|
|
74
|
+
|
|
75
|
+
Cleanup recursively makes only Rehearsal's private staging tree removable. A failed
|
|
76
|
+
Storage transfer therefore reports the original safe transfer error, removes incomplete
|
|
77
|
+
files, and does not alter the active generation.
|
|
78
|
+
|
|
60
79
|
Storage objects do not share the database transaction guarantee. Their inventory records
|
|
61
80
|
size and version metadata; transfer requires that version, verifies the exact byte count,
|
|
62
81
|
and records a SHA-256 in the immutable baseline. Transfer refuses an object that changes,
|
|
63
82
|
exceeds its declared limit, or falls outside an approved bucket and prefix.
|
|
64
83
|
|
|
65
84
|
Privacy policy version 2 is executable without project sanitizer code. It
|
|
66
|
-
supports exact keep, exclusion, keyed
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
85
|
+
supports exact keep, exclusion, keyed and constraint-preserving pseudonyms (including
|
|
86
|
+
safe loopback HTTPS replacements), bounded reviewed-identity substitution inside
|
|
87
|
+
retained text,
|
|
88
|
+
constants, named sanitized-input digests, and a small set of documented
|
|
89
|
+
derivations. Policies classify every selected field and supported nested JSON
|
|
90
|
+
key. A new or unknown field blocks refresh until the policy is reviewed again.
|
|
70
91
|
|
|
71
92
|
The pseudonym key is an owner-only local secret. It is not stored in the
|
|
72
93
|
baseline or source control. Receipts bind the policy bytes, schema shape,
|
package/docs/troubleshooting.md
CHANGED
|
@@ -32,16 +32,21 @@ npx rehearsal
|
|
|
32
32
|
|
|
33
33
|
Choose **Set the stage** and approve the preview. For pnpm and Yarn, use this guided path;
|
|
34
34
|
automatic root generation is currently tested with npm. An existing supported config is
|
|
35
|
-
kept in place, including `infrastructure/rehearsal/rehearsal.config.mjs`.
|
|
35
|
+
kept in place, including `infrastructure/rehearsal/rehearsal.config.mjs`. When npm shows
|
|
36
|
+
lifecycle output, the install message names the active path and explains when that
|
|
37
|
+
existing file prevented `rehearsal.config.mjs` from being generated. You can always run
|
|
38
|
+
`npx rehearsal init --json` and read `data.destination`. To relocate it safely, follow
|
|
39
|
+
[Move an existing config to the project root](configuration.md#move-an-existing-config-to-the-project-root).
|
|
36
40
|
|
|
37
|
-
## Setup says Node.js
|
|
41
|
+
## Setup says this Node.js version is unsupported
|
|
38
42
|
|
|
39
|
-
Rehearsal
|
|
40
|
-
the current shell before installing or
|
|
43
|
+
Rehearsal supports the maintained Node.js 22, 24, and 26 release lines. Node.js 20 and
|
|
44
|
+
odd-numbered releases are end-of-life. Switch the current shell before installing or
|
|
45
|
+
running Rehearsal. The latest LTS is the recommended default:
|
|
41
46
|
|
|
42
47
|
```bash
|
|
43
|
-
nvm install
|
|
44
|
-
nvm use
|
|
48
|
+
nvm install --lts
|
|
49
|
+
nvm use --lts
|
|
45
50
|
node --version
|
|
46
51
|
```
|
|
47
52
|
|
|
@@ -53,6 +58,12 @@ writes setup files.
|
|
|
53
58
|
Start Docker Desktop or Colima, confirm `docker info`, then rerun `rehearsal doctor`.
|
|
54
59
|
Restarting the computer is rarely necessary.
|
|
55
60
|
|
|
61
|
+
When a container stays in `starting` or becomes `unhealthy`, Rehearsal reports a runtime
|
|
62
|
+
dependency failure rather than a migration failure. The safe diagnostic includes Docker
|
|
63
|
+
CPU, memory, and container counts; Colima also includes available VM memory when it can
|
|
64
|
+
be read. Stop unrelated local stacks or increase the engine's resources before retrying.
|
|
65
|
+
Rehearsal never changes engine settings or bypasses service health checks.
|
|
66
|
+
|
|
56
67
|
## Docker or Colima is out of disk space
|
|
57
68
|
|
|
58
69
|
Preview what Rehearsal can safely remove:
|
package/docs/tutorial.md
CHANGED
|
@@ -3,7 +3,8 @@
|
|
|
3
3
|
This tutorial runs a complete PostgreSQL rehearsal in a temporary fictional project. It
|
|
4
4
|
does not use your application, production data, or a hosted database.
|
|
5
5
|
|
|
6
|
-
Allow about five minutes. You need Node.js 24 and a running
|
|
6
|
+
Allow about five minutes. You need Node.js 22, 24, or 26 and a running
|
|
7
|
+
Docker-compatible engine. Node.js 24 LTS is recommended.
|
|
7
8
|
|
|
8
9
|
## 1. Prepare Rehearsal
|
|
9
10
|
|
|
@@ -105,3 +106,8 @@ npm run test:fixture
|
|
|
105
106
|
```
|
|
106
107
|
|
|
107
108
|
The PostgreSQL equivalent is `npm run test:fixture:postgresql`.
|
|
109
|
+
|
|
110
|
+
The release workflow also runs `npm run test:fixture:onboarding` from the packed package.
|
|
111
|
+
That check creates fresh Supabase and PostgreSQL consumers, verifies beta configuration
|
|
112
|
+
preservation, and exercises safe refusal and recovery for partial setup, occupied ports,
|
|
113
|
+
malformed configuration, and missing credentials.
|