@classytic/repo-core 0.13.0 → 0.17.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -4,6 +4,152 @@ All notable changes to `@classytic/repo-core` are documented here.
4
4
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
5
5
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6
6
 
7
+ ## [0.17.0] - 2026-07-25
8
+
9
+ ### Added — `./cleanup`: framework-free cleanup provider step contract
10
+
11
+ - **`@classytic/repo-core/cleanup`** — new subpath (pure types, no runtime).
12
+ Domain kernels (`@classytic/flow`, `@classytic/order`, `@classytic/facts`,
13
+ `@classytic/ledger`, …) export `CleanupStep[]` for the data they own; a host
14
+ framework (`@classytic/arc/cleanup`) folds them into a Cleanup Center recipe.
15
+ The contract lives in repo-core so kernels stay framework-free: kernels already
16
+ depend on repo-core (the chunked-purge mechanics) and MUST NOT depend on arc.
17
+ - **`CleanupStep`** — the core provider interface: `id`, `resource`, `destructive`,
18
+ optional `rebuildActions`; and three lifecycle methods: `estimate(ctx)` (preview
19
+ without mutating), `execute(ctx)` (chunked, idempotent, cancellation-aware),
20
+ and optional `verify(ctx)` (post-checks — a delete count alone is never success).
21
+ - **Supporting types**: `CleanupStepContext` (injected `now`, `signal`, `ambient`,
22
+ `parameters`, `logger`), `CleanupStepExecuteContext` (adds `onProgress` +
23
+ `throwIfCancelled`), `CleanupStepEstimate` (row count, `retained`, `blockers`,
24
+ `warnings`), `CleanupStepOutcome` (`processed`, `ok`, `error`, `cursor`),
25
+ `CleanupStepCheck` (`name`, `ok`, `detail`), `CleanupStepProgress`, `CleanupStepLogger`.
26
+
27
+ Purely additive — zero runtime code, type-only subpath export.
28
+
29
+ ## [0.16.0] - 2026-07-24
30
+
31
+ ### Added — `runPurgeConformance`: cross-kit chunked-purge contract suite
32
+
33
+ - **`runPurgeConformance(harness)`** (from `@classytic/repo-core/testing`) —
34
+ proves a kit's purge port makes **stable progress for every strategy** when
35
+ the match set exceeds `batchSize`. Every scenario seeds more rows than one
36
+ batch; a chunk budget converts a non-progressing port's infinite loop into a
37
+ crisp assertion failure. Scenarios: hard drain, soft multi-batch WITHOUT
38
+ caller-supplied exclusion predicates, anonymize static + function-form,
39
+ exact-batch boundary, skip, empty scope, abort-between-chunks.
40
+ - **`PurgePort` progression contract (documented, mandatory)** — successive
41
+ `purgeChunk` calls MUST advance through the match set for every strategy via
42
+ stable keyset progression (`pk > lastSeen`, advanced only after the chunk's
43
+ write succeeds). Re-running the bare predicate only self-advances for
44
+ `hard`; `soft`/`anonymize` re-select the same first chunk forever. Offsets
45
+ are explicitly ruled out. A port instance is single-run state.
46
+
47
+ ### Fixed
48
+
49
+ - `purgeByField` docstring no longer claims soft/anonymized rows "simply don't
50
+ match the next pass" — they generally DO still match; idempotency holds by
51
+ outcome convergence, and within-run progression is the port's keyset
52
+ responsibility.
53
+
54
+ ## [0.15.0] - 2026-07-24
55
+
56
+ ### Added — `purgeByFilter`: range/filter-scoped purge + anonymize
57
+
58
+ - **`StandardRepo.purgeByFilter?(filter, strategy, options)`** — the
59
+ range/filter-scoped sibling of `purgeByField`. Where `purgeByField` matches
60
+ a single `field = value` equality, this optional method takes the full
61
+ portable `FilterInput` (Filter IR or a plain kit-native record) and runs any
62
+ `TenantPurgeStrategy` (`hard` / `soft` / `anonymize` / `skip`) over the
63
+ matched slice. THE compliance primitive for "purge/anonymize a dimension
64
+ across a RANGE while RETAINING measures" — redact a PII column across a
65
+ `civilDate` window, hard-delete rows past a retention cutoff, soft-delete a
66
+ compound cohort. Returns the same `TenantPurgeResult` envelope; chunking,
67
+ index requirement, idempotency, plugin composition, and narrowed-write
68
+ re-assertion are identical to `purgeByField`. Gate on the new
69
+ `capabilities.purgeByFilter`.
70
+ - **`RepoCapabilities.purgeByFilter?: boolean`** — feature-detection flag,
71
+ mirroring `purgeByField`.
72
+ - **`PurgePort` doc** clarifies the two bound-predicate forms (equality-bound
73
+ vs filter-bound) both satisfy the single port interface, so `runChunkedPurge`
74
+ drives both unchanged.
75
+
76
+ Strictly additive: `purgeByField`, `TenantPurgeStrategy`, and `runChunkedPurge`
77
+ are unchanged; both new members are optional.
78
+
79
+ ## [0.14.0] - 2026-07-16
80
+
81
+ ### Added — canonical `matchesRecordFilter` (the `DataAdapter.matchesFilter` home)
82
+
83
+ - **`matchesRecordFilter(item, record)`** + **`policyRecordToFilter(record)`**
84
+ (`@classytic/repo-core/filter`) — THE single, shared implementation of the
85
+ `DataAdapter.matchesFilter` seam. Evaluates arc's Mongo-record `_policyFilters`
86
+ (`{ ownerId }`, `{ organizationId }`, `{ $or: [{ ownerId }, { _id: { $in } }] }`)
87
+ against an already-fetched document IN PROCESS, by converting to the portable
88
+ `Filter` IR and delegating to `matchFilter` — the SAME IR kits compile to
89
+ SQL/Mongo, so in-memory and DB enforcement agree by construction. Every kit's
90
+ adapter now delegates here; **no per-kit matcher, one contract, one IR.**
91
+ Operator scope: `$or`/`$and`/`$nor`/`$not`, `$eq`/`$ne`/`$gt`/`$gte`/`$lt`/`$lte`,
92
+ `$in`/`$nin`, `$exists`, implicit-eq; fails LOUD on anything else. (Distinct from
93
+ `recordToFilter`, the bare-operator query normalizer that does not accept
94
+ `$`-prefixed or logical operators.)
95
+ - **`matchFilter` is now fully array + id + date aware** (additive superset;
96
+ primitive/Date-vs-Date behavior unchanged — strict cases still short-circuit):
97
+ - **id coercion** — an id-like object with a meaningful `toString` (Mongo
98
+ `ObjectId`, `Buffer`, `Decimal128`) matches its string form, so one shared
99
+ matcher serves Mongo (`ObjectId` `_id`) and SQL (primitive ids) alike.
100
+ - **array semantics** — dot-paths fan out over subdocument arrays
101
+ (`items.sku` on `[{sku},{sku}]`), and a scalar condition on a leaf array
102
+ field matches when ANY element satisfies it (`{ tags: 'x' }`,
103
+ `{ scores: { $gt: 5 } }`, regex on array elements). All array unwrapping is
104
+ concentrated in one helper; the comparators stay pure scalar.
105
+ - **Date⇄ISO-string range** — `compare` coerces the string side to a date
106
+ instant when the other side is a genuine `Date` (mirrors `equals`), and
107
+ never claims ordering across a number/string type boundary (fails closed
108
+ instead of matching spuriously).
109
+ - **prototype-pollution-safe reads** — path resolution uses `Object.hasOwn`,
110
+ so a crafted `{ '__proto__.x': … }` / `{ constructor.name: … }` filter can
111
+ never traverse the prototype chain.
112
+ - **Ability parity across kits**: mongokit's earlier standalone matcher
113
+ (subdocument-array fan-out, array-contains, `$regex`, ObjectId coercion) is now
114
+ fully covered by the shared engine — nothing was lost in consolidation.
115
+ - **MongoDB-parity hardening** (validated against the MongoDB manual + sift.js +
116
+ mingo — the two industry-standard in-memory matchers). The authorization-critical
117
+ rule "an absent field participates in comparisons as null/undefined" is now
118
+ fully honored:
119
+ - `{ field: null }`, `$ne`, `$nin`, and a `null` MEMBER of `$in`/`$nin` all
120
+ match a MISSING field (a policy filter `{ status: { $ne: 'archived' } }`
121
+ correctly returns docs that lack the field, exactly as MongoDB does — the top
122
+ silent-authorization-divergence trap). `$ne: null` remains the exception
123
+ (requires present + non-null).
124
+ - `$in` accepts RegExp-literal members (`{ name: { $in: [/^a/] } }`).
125
+ - Comparison ops are type-bracketed (no cross-type ordering; `$gt: null` matches
126
+ nothing); NaN equals NaN for `$eq` (via `Object.is`, not `===`).
127
+ - Numeric dot-path segments resolve as positional array indices (`items.0.sku`).
128
+ - Documented DELIBERATE divergences: `$exists` = present-and-non-null (matches
129
+ the IR `exists` op + SQL `IS NOT NULL` + sift; Mongo/mingo count present-null
130
+ as existing); `Date`⇄ISO-string range leniency; array-literal operands are
131
+ element-matched not exact-matched.
132
+ - New: 41-case matcher suite incl. the 11 researched MongoDB gotchas + adversarial
133
+ (prototype pollution, NaN/Infinity, boolean/zero/empty-string, empty
134
+ `$in`/`$or`/`$and`/`$nor`, fail-loud on unsupported operators) + a 100k-doc
135
+ performance smoke (linear, cached regex).
136
+ - **Security hardening** (from a review against sift.js/mingo CVEs + the OWASP
137
+ NoSQL-injection / ReDoS / prototype-pollution literature):
138
+ - **Prototype-key denylist** on path segments — `__proto__` / `constructor` /
139
+ `prototype` resolve to nothing (fail-closed), string-normalized (the
140
+ object-path CVE-2021-23434 lesson: an array-typed segment bypassed a `===`
141
+ check). Closes the "match an inherited member → wrong auth answer" case and
142
+ the `JSON.parse('{"__proto__":…}')` own-property vector, on top of the
143
+ already-`Object.hasOwn` reads.
144
+ - **`$regex` input-length cap** (64 KiB) — ReDoS is `pattern × input`; a field
145
+ value longer than the cap is treated as no-match (fail-closed) so one slow
146
+ match can't stall the event loop and amplify across a realtime fan-out
147
+ (matcher runs once per subscriber per record). Pattern-side ReDoS is a
148
+ non-issue for framework-supplied (trusted) patterns.
149
+ - Reaffirmed **fail-closed** posture: comparisons never coerce across a
150
+ number/string type boundary (return no-match rather than JS-coerced nonsense
151
+ — the classic over-visibility leak), and unsupported operators throw.
152
+
7
153
  ## [0.13.0] - 2026-07-15
8
154
 
9
155
  ### Added — `StandardRepo.applyTransition?()` contract (state-machine CAS with history)
package/README.md CHANGED
@@ -1,5 +1,7 @@
1
1
  # @classytic/repo-core
2
2
 
3
+ [![Sponsor](https://img.shields.io/github/sponsors/classytic?style=flat-square&label=Sponsor&logo=GitHub&color=EA4AAA)](https://github.com/sponsors/classytic)
4
+
3
5
  **Driver-agnostic repository primitives.** Hooks, Filter IR, operations registry, pagination, URL query parsing, cache contract — the shared foundation for `@classytic/mongokit`, `@classytic/sqlitekit`, and future `@classytic/pgkit` / `@classytic/prismakit`.
4
6
 
5
7
  Repo-core is **infrastructure for kit authors.** End-users install a kit (mongokit / sqlitekit) and import their full API from that one namespace. Repo-core is what each kit's runtime is built on — you typically won't import it directly in application code.
@@ -0,0 +1,2 @@
1
+ import { CleanupStep, CleanupStepCheck, CleanupStepContext, CleanupStepEstimate, CleanupStepExecuteContext, CleanupStepLogger, CleanupStepOutcome, CleanupStepProgress } from "./types.mjs";
2
+ export type { CleanupStep, CleanupStepCheck, CleanupStepContext, CleanupStepEstimate, CleanupStepExecuteContext, CleanupStepLogger, CleanupStepOutcome, CleanupStepProgress };
File without changes
@@ -0,0 +1,167 @@
1
+ //#region src/cleanup/types.d.ts
2
+ /**
3
+ * `@classytic/repo-core/cleanup` — the framework-free **cleanup provider step**
4
+ * contract (data-cleanup / retention design §6.3, §6.6).
5
+ *
6
+ * A domain kernel (`@classytic/flow`, `@classytic/order`, `@classytic/facts`,
7
+ * `@classytic/ledger`, …) exports one or more `CleanupStep`s for the data it
8
+ * owns — the unit retention.md §6.6 calls "cleanup recipe steps." A host
9
+ * (`be-prod`) composes ordered steps into a full Cleanup Center recipe.
10
+ *
11
+ * This contract lives in repo-core — NOT in `@classytic/arc` — on purpose:
12
+ *
13
+ * - Kernels already depend on repo-core (it owns the chunked purge mechanics,
14
+ * §6.3) and MUST stay free of any host-framework dependency. A kernel that
15
+ * imported `@classytic/arc/cleanup` would invert the layering.
16
+ * - Arc's Cleanup Center framework (`@classytic/arc/cleanup`) imports THIS
17
+ * contract and folds `CleanupStep[]` into an arc `CleanupRecipe` via its
18
+ * `recipeFromSteps()` composer. One shape, two consumers, zero cycle.
19
+ *
20
+ * A step is a PURE PROVIDER: it knows only how to estimate / execute / verify a
21
+ * slice of its own data. It never authorizes a superadmin, reads go-live state,
22
+ * speaks HTTP, or opens a Mongo transaction the host didn't hand it. It reuses
23
+ * repo-core's own chunked-purge envelope (`TenantPurgeResult` / progress) under
24
+ * the hood; this contract is the composable wrapper around that envelope.
25
+ */
26
+ /**
27
+ * Framework-free ambient a cleanup step reads. A structural SUBSET of any host
28
+ * cleanup context (e.g. arc's `CleanupContext`), so passing the host context
29
+ * straight through type-checks — no adapter object required.
30
+ */
31
+ interface CleanupStepContext {
32
+ /** Injected clock — steps never call `new Date()` directly (testability). */
33
+ readonly now: Date;
34
+ /** Cooperative cancellation — observed between chunks. */
35
+ readonly signal?: AbortSignal | undefined;
36
+ /**
37
+ * Opaque host-provided ambient scope (resolved company/branch, feature
38
+ * gates, …). MUST be JSON-serializable — a host may persist it on a durable
39
+ * run so a worker in another process rebuilds the exact operation context.
40
+ * A step reads only what it declared it needs; nobody else inspects it.
41
+ */
42
+ readonly ambient?: Readonly<Record<string, unknown>> | undefined;
43
+ /**
44
+ * Operator-supplied recipe parameters (branch id, created-before date, module
45
+ * set, …). Opaque + host-validated at the edge; a step reads/parses the keys
46
+ * it declares. Recipes are boot-time singletons, so parameters arrive HERE at
47
+ * plan/execute time rather than being closed over at construction. The host
48
+ * composer threads the plan's sealed parameters through unchanged, so a
49
+ * worker replays the exact same op.
50
+ */
51
+ readonly parameters?: Readonly<Record<string, unknown>> | undefined;
52
+ /** Optional structured logger. */
53
+ readonly logger?: CleanupStepLogger | undefined;
54
+ }
55
+ interface CleanupStepLogger {
56
+ info(msg: string, meta?: Record<string, unknown>): void;
57
+ warn(msg: string, meta?: Record<string, unknown>): void;
58
+ error(msg: string, meta?: Record<string, unknown>): void;
59
+ }
60
+ /** One committed chunk's progress — folded into the host run's bounded summary. */
61
+ interface CleanupStepProgress {
62
+ readonly resource: string;
63
+ /** Cumulative rows processed by THIS step so far. */
64
+ readonly processed: number;
65
+ /** Opaque resume cursor (keyset position) for observability. */
66
+ readonly cursor?: string | undefined;
67
+ }
68
+ /**
69
+ * Execution-time context — adds progress reporting + a durable-cancellation
70
+ * probe. Both are optional so a step is trivially runnable in a unit test with
71
+ * a bare `CleanupStepContext`.
72
+ */
73
+ interface CleanupStepExecuteContext extends CleanupStepContext {
74
+ /** Report one committed chunk. Call AFTER the chunk's write commits. */
75
+ onProgress?(update: CleanupStepProgress): void | Promise<void>;
76
+ /**
77
+ * Throw the host's cancellation error if a cancel was requested for this run.
78
+ * Cheap to call between chunks; backed by the host's durable `cancelRequested`
79
+ * flag (source of truth) plus the in-process `signal`. Absent in a bare test
80
+ * context — treat as "never cancelled."
81
+ */
82
+ throwIfCancelled?(): void | Promise<void>;
83
+ }
84
+ /**
85
+ * One preview line — maps 1:1 onto a host plan item. A business record class
86
+ * (`'sales facts'`, `'journal entries'`), never a collection name.
87
+ */
88
+ interface CleanupStepEstimate {
89
+ readonly resource: string;
90
+ /** Estimated records this step would affect. */
91
+ readonly estimated: number;
92
+ /** What this step RETAINS (e.g. `'measures kept, PII redacted'`). */
93
+ readonly retained?: string | undefined;
94
+ /**
95
+ * Domain blockers preventing this step (e.g. `'POSTED_BOOKS_IMMUTABLE'`,
96
+ * `'OPEN_TRANSFER'`). A non-empty list is a HARD STOP — the host refuses to
97
+ * execute until the operator resolves it. Blockers are DOMAIN facts, never a
98
+ * permission decision (that is the host's job).
99
+ */
100
+ readonly blockers?: readonly string[] | undefined;
101
+ /**
102
+ * Non-blocking warnings surfaced to the operator (e.g. "rebuild may take
103
+ * several minutes on this generation").
104
+ */
105
+ readonly warnings?: readonly string[] | undefined;
106
+ }
107
+ /** One executed step's outcome — maps onto a host step result. */
108
+ interface CleanupStepOutcome {
109
+ readonly resource: string;
110
+ /** Rows this step actually processed (deleted / redacted / rebuilt). */
111
+ readonly processed: number;
112
+ /**
113
+ * `false` iff the step failed. The host composer STOPS the recipe on the
114
+ * first `ok: false` (retention §8: never return success on a provider
115
+ * failure) and marks the run `failed`/`partial`. A step MUST NOT swallow a
116
+ * failure and report `ok: true`.
117
+ */
118
+ readonly ok: boolean;
119
+ /** Failure message when `ok: false`. */
120
+ readonly error?: string | undefined;
121
+ /** Opaque resume cursor for observability. */
122
+ readonly cursor?: string | undefined;
123
+ }
124
+ /** One post-check — maps onto a host verification check (§9). */
125
+ interface CleanupStepCheck {
126
+ readonly name: string;
127
+ readonly ok: boolean;
128
+ readonly detail?: string | undefined;
129
+ }
130
+ /**
131
+ * A cleanup PROVIDER STEP — the unit a domain kernel exports for data it owns
132
+ * (retention design §6.6). Composed by the host into an ordered recipe.
133
+ *
134
+ * Idempotency + chunking are the step's responsibility: `execute` must be safe
135
+ * to re-run after a crash/retry (keyset progression, dedupe-by-occurrence, or a
136
+ * naturally-idempotent rebuild), and must observe `signal` / `throwIfCancelled`
137
+ * between chunks so a cancel lands between committed batches, never mid-write.
138
+ */
139
+ interface CleanupStep {
140
+ /** Stable machine id, unique within a recipe. */
141
+ readonly id: string;
142
+ /** Logical resource label for the preview (e.g. `'sales facts'`). */
143
+ readonly resource: string;
144
+ /**
145
+ * `true` if the step deletes/redacts data; `false` for a pure rebuild
146
+ * (§4.3 — rebuilding a projection is not destructive to source data). A
147
+ * recipe is destructive iff ANY of its steps is.
148
+ */
149
+ readonly destructive: boolean;
150
+ /**
151
+ * Projection / scaffolding rebuilds this step performs AFTER its cleanup —
152
+ * surfaced in the preview's `rebuildActions` (e.g. `'rebuild sales rollup'`).
153
+ */
154
+ readonly rebuildActions?: readonly string[] | undefined;
155
+ /** Preview WITHOUT mutating. Idempotent + side-effect-free. */
156
+ estimate(ctx: CleanupStepContext): Promise<CleanupStepEstimate>;
157
+ /** Do the work. Chunked, idempotent, cancellation-aware. */
158
+ execute(ctx: CleanupStepExecuteContext): Promise<CleanupStepOutcome>;
159
+ /**
160
+ * Post-checks — the step owns what "clean" means for its data (§9: a delete
161
+ * count alone is never success). Optional: a pure-rebuild step may skip it,
162
+ * though verifying the watermark/counts is strongly encouraged.
163
+ */
164
+ verify?(ctx: CleanupStepContext): Promise<readonly CleanupStepCheck[]>;
165
+ }
166
+ //#endregion
167
+ export { CleanupStep, CleanupStepCheck, CleanupStepContext, CleanupStepEstimate, CleanupStepExecuteContext, CleanupStepLogger, CleanupStepOutcome, CleanupStepProgress };
@@ -3,6 +3,7 @@ import { FALSE, TRUE, and, anyOf as in_, between, contains, endsWith, eq, exists
3
3
  import { recordToFilter } from "./from-record.mjs";
4
4
  import { isFilter } from "./guard.mjs";
5
5
  import { asPredicate, matchFilter } from "./match.mjs";
6
+ import { matchesRecordFilter, policyRecordToFilter } from "./match-record.mjs";
6
7
  import { SCOPE_ANY, buildTenantScope, mergeScope } from "./scope.mjs";
7
8
  import { collectFields, mapFilter, walkFilter } from "./walk.mjs";
8
- export { FALSE, type Filter, type FilterAnd, type FilterEq, type FilterExists, type FilterFalse, type FilterGt, type FilterGte, type FilterIn, type FilterLike, type FilterLt, type FilterLte, type FilterNe, type FilterNin, type FilterNot, type FilterOp, type FilterOr, type FilterRaw, type FilterRegex, type FilterTrue, SCOPE_ANY, TRUE, and, in_ as anyOf, asPredicate, between, buildTenantScope, collectFields, contains, endsWith, eq, exists, gt, gte, iEq, in_, not as invert, isFilter, isNotNull, isNull, like, lt, lte, mapFilter, matchFilter, mergeScope, ne, nin, nin as noneOf, not, or, raw, recordToFilter, regex, startsWith, walkFilter };
9
+ export { FALSE, type Filter, type FilterAnd, type FilterEq, type FilterExists, type FilterFalse, type FilterGt, type FilterGte, type FilterIn, type FilterLike, type FilterLt, type FilterLte, type FilterNe, type FilterNin, type FilterNot, type FilterOp, type FilterOr, type FilterRaw, type FilterRegex, type FilterTrue, SCOPE_ANY, TRUE, and, in_ as anyOf, asPredicate, between, buildTenantScope, collectFields, contains, endsWith, eq, exists, gt, gte, iEq, in_, not as invert, isFilter, isNotNull, isNull, like, lt, lte, mapFilter, matchFilter, matchesRecordFilter, mergeScope, ne, nin, nin as noneOf, not, or, policyRecordToFilter, raw, recordToFilter, regex, startsWith, walkFilter };
@@ -3,5 +3,6 @@ import { collectFields, mapFilter, walkFilter } from "./walk.mjs";
3
3
  import { FALSE, TRUE, and, anyOf as in_, between, contains, endsWith, eq, exists, gt, gte, iEq, invert as not, isNotNull, isNull, like, lt, lte, ne, nin, or, raw, regex, startsWith } from "./builders.mjs";
4
4
  import { recordToFilter } from "./from-record.mjs";
5
5
  import { asPredicate, matchFilter } from "./match.mjs";
6
+ import { matchesRecordFilter, policyRecordToFilter } from "./match-record.mjs";
6
7
  import { SCOPE_ANY, buildTenantScope, mergeScope } from "./scope.mjs";
7
- export { FALSE, SCOPE_ANY, TRUE, and, in_ as anyOf, asPredicate, between, buildTenantScope, collectFields, contains, endsWith, eq, exists, gt, gte, iEq, in_, not as invert, isFilter, isNotNull, isNull, like, lt, lte, mapFilter, matchFilter, mergeScope, ne, nin, nin as noneOf, not, or, raw, recordToFilter, regex, startsWith, walkFilter };
8
+ export { FALSE, SCOPE_ANY, TRUE, and, in_ as anyOf, asPredicate, between, buildTenantScope, collectFields, contains, endsWith, eq, exists, gt, gte, iEq, in_, not as invert, isFilter, isNotNull, isNull, like, lt, lte, mapFilter, matchFilter, matchesRecordFilter, mergeScope, ne, nin, nin as noneOf, not, or, policyRecordToFilter, raw, recordToFilter, regex, startsWith, walkFilter };
@@ -0,0 +1,19 @@
1
+ import { Filter } from "./types.mjs";
2
+ //#region src/filter/match-record.d.ts
3
+ /**
4
+ * Convert an arc Mongo-record `_policyFilters` object into Filter IR.
5
+ * `{}` → `TRUE`. Throws on unsupported top-level operators.
6
+ */
7
+ declare function policyRecordToFilter(record: Record<string, unknown>): Filter;
8
+ /**
9
+ * Evaluate an arc Mongo-record `_policyFilters` object against a document —
10
+ * converts to Filter IR, then delegates to the shared {@link matchFilter}
11
+ * engine. THE canonical `DataAdapter.matchesFilter` implementation; every
12
+ * kit's adapter delegates here.
13
+ *
14
+ * @param item The already-fetched document / row.
15
+ * @param filters Arc's `_policyFilters` in Mongo record syntax.
16
+ */
17
+ declare function matchesRecordFilter(item: unknown, filters: Record<string, unknown>): boolean;
18
+ //#endregion
19
+ export { matchesRecordFilter, policyRecordToFilter };
@@ -0,0 +1,187 @@
1
+ import { FALSE, TRUE, and, anyOf as in_, eq, gt, gte, invert as not, isNotNull, isNull, lt, lte, ne, or, regex } from "./builders.mjs";
2
+ import { matchFilter } from "./match.mjs";
3
+ //#region src/filter/match-record.ts
4
+ /**
5
+ * Mongo-record `_policyFilters` → Filter IR conversion + in-memory match.
6
+ *
7
+ * The CANONICAL, single home for evaluating arc's row-level policy filters
8
+ * against an already-fetched document IN PROCESS (no DB round-trip). Every
9
+ * kit's `DataAdapter.matchesFilter` delegates here — one contract, one IR,
10
+ * no per-kit matcher.
11
+ *
12
+ * Arc's permission helpers emit policy filters in Mongo record syntax,
13
+ * kit-agnostic (`requireOwnership` → `{ ownerId }`, multiTenant →
14
+ * `{ organizationId }`, `requireGrant` list resolutions →
15
+ * `{ $or: [{ ownerId }, { _id: { $in } }] }`). This module converts that
16
+ * record into the portable {@link Filter} IR and evaluates it with the
17
+ * shared {@link matchFilter} engine — the SAME IR kits compile to SQL /
18
+ * Mongo, so in-memory and DB-level enforcement agree by construction.
19
+ *
20
+ * `matchFilter` is id-coercion aware (Mongo `ObjectId` `_id` matches its
21
+ * string form — no kit-specific coercion) and array-aware (dot-paths fan
22
+ * out over subdocument arrays; scalar conditions on array fields match any
23
+ * element). See `match.ts`.
24
+ *
25
+ * SCOPE — the operators arc's policy filters emit. Fails LOUD on anything
26
+ * else so a silent mismatch never masquerades as a denial:
27
+ *
28
+ * logical: $or, $and, $nor, $not
29
+ * comparison: implicit-eq, $eq, $ne, $gt, $gte, $lt, $lte
30
+ * membership: $in, $nin
31
+ * existence: $exists (see divergence note below)
32
+ * pattern: $regex (+ $options; RegExp literal accepted)
33
+ *
34
+ * MongoDB parity (validated against the MongoDB manual + sift/mingo):
35
+ * - Missing field ≡ null for `{field: null}`, `$ne`/`$nin`, and a `null`
36
+ * member of `$in`/`$nin` — the authorization-critical rule (a policy
37
+ * filter `{ status: { $ne: 'archived' } }` MUST return docs lacking the
38
+ * field, exactly as MongoDB does).
39
+ * - `$in` accepts RegExp-literal members (Mongo allows `/re/` in `$in`).
40
+ * - Comparison ops are TYPE-BRACKETED: no cross-type ordering
41
+ * (`{ n: { $gt: 5 } }` never matches a string `n`); `$gt: null` matches
42
+ * nothing. NaN equals NaN for `$eq`.
43
+ * - Dot-paths fan out over arrays AND resolve numeric segments as
44
+ * positional indices (`items.0.sku`).
45
+ *
46
+ * DELIBERATE divergences (documented, not bugs):
47
+ * - `$exists` = present-AND-non-null (a null value reads as absent),
48
+ * matching the shared IR `exists` op + SQL `IS NOT NULL` + sift.js.
49
+ * MongoDB/mingo treat present-null as existing; that would require a
50
+ * separate key-presence IR op threaded through every kit's SQL/Mongo
51
+ * compiler. Arc's built-in policy helpers never emit `$exists`; a
52
+ * custom filter that needs Mongo key-presence should use
53
+ * `{ field: { $ne: null } }` (present + non-null) or `{ field: null }`
54
+ * (null OR missing) instead.
55
+ * - `$gt`/`$lt` allow ONE cross-type leniency: a `Date` field compares
56
+ * against an ISO-string operand (JSON policy filters carry dates as
57
+ * strings). Consistent with `$eq`'s Date⇄string coercion.
58
+ * - An array-literal operand (`{ tags: ['a','b'] }`) is element-matched,
59
+ * not exact-array-matched — policy filters never assert whole-array
60
+ * equality.
61
+ *
62
+ * Distinct from {@link recordToFilter}, which is the ergonomic
63
+ * record→IR normalizer for BARE-operator query shorthand (`{ price:
64
+ * { gte } }`) and deliberately does NOT accept `$`-prefixed operators or
65
+ * logical `$or`/`$and`. This function is the arc-policy-filter dialect
66
+ * (`$`-prefixed, with logical operators).
67
+ */
68
+ /** Field operators understood inside a `{ field: { ... } }` condition. */
69
+ const FIELD_OPS = [
70
+ "$eq",
71
+ "$ne",
72
+ "$gt",
73
+ "$gte",
74
+ "$lt",
75
+ "$lte",
76
+ "$in",
77
+ "$nin",
78
+ "$exists",
79
+ "$regex"
80
+ ];
81
+ function isOperatorObject(value) {
82
+ if (value === null || typeof value !== "object") return false;
83
+ if (Array.isArray(value) || value instanceof Date) return false;
84
+ const keys = Object.keys(value);
85
+ return keys.length > 0 && keys.every((k) => k.startsWith("$"));
86
+ }
87
+ /**
88
+ * `$in` with MongoDB parity: a `null` member also matches a MISSING field
89
+ * (inherits `{field: null}` semantics), and RegExp-literal members match by
90
+ * pattern (Mongo allows `/re/` inside `$in`). Split members into
91
+ * null / regex / scalar and OR the branches.
92
+ */
93
+ function buildIn(field, members) {
94
+ const branches = [];
95
+ const scalars = [];
96
+ let hasNull = false;
97
+ for (const m of members) if (m === null || m === void 0) hasNull = true;
98
+ else if (m instanceof RegExp) branches.push(regex(field, m.source, m.flags));
99
+ else scalars.push(m);
100
+ if (hasNull) branches.push(isNull(field));
101
+ if (scalars.length > 0) branches.push(in_(field, scalars));
102
+ if (branches.length === 0) return FALSE;
103
+ return branches.length === 1 ? branches[0] : or(...branches);
104
+ }
105
+ /** `$nin` is the negation of `$in` — none of the members may match. */
106
+ function buildNin(field, members) {
107
+ const inFilter = buildIn(field, members);
108
+ return inFilter.op === "false" ? TRUE : not(inFilter);
109
+ }
110
+ /** Convert a single `{ field: condition }` entry into a Filter IR node. */
111
+ function fieldFilter(field, condition) {
112
+ if (!isOperatorObject(condition)) return condition === null ? isNull(field) : eq(field, condition);
113
+ const parts = [];
114
+ const options = typeof condition["$options"] === "string" ? condition["$options"] : void 0;
115
+ for (const [op, operand] of Object.entries(condition)) switch (op) {
116
+ case "$options": break;
117
+ case "$eq":
118
+ parts.push(operand === null ? isNull(field) : eq(field, operand));
119
+ break;
120
+ case "$ne":
121
+ parts.push(operand === null ? isNotNull(field) : ne(field, operand));
122
+ break;
123
+ case "$gt":
124
+ parts.push(gt(field, operand));
125
+ break;
126
+ case "$gte":
127
+ parts.push(gte(field, operand));
128
+ break;
129
+ case "$lt":
130
+ parts.push(lt(field, operand));
131
+ break;
132
+ case "$lte":
133
+ parts.push(lte(field, operand));
134
+ break;
135
+ case "$in":
136
+ parts.push(buildIn(field, operand ?? []));
137
+ break;
138
+ case "$nin":
139
+ parts.push(buildNin(field, operand ?? []));
140
+ break;
141
+ case "$exists":
142
+ parts.push(operand ? isNotNull(field) : isNull(field));
143
+ break;
144
+ case "$regex": {
145
+ const pattern = operand instanceof RegExp ? operand.source : String(operand);
146
+ const flags = operand instanceof RegExp ? operand.flags : options;
147
+ parts.push(flags ? regex(field, pattern, flags) : regex(field, pattern));
148
+ break;
149
+ }
150
+ default: throw new Error(`[repo-core] matchesRecordFilter: unsupported field operator '${op}'. Supported: ${FIELD_OPS.join(", ")}.`);
151
+ }
152
+ return parts.length === 1 ? parts[0] : and(...parts);
153
+ }
154
+ /**
155
+ * Convert an arc Mongo-record `_policyFilters` object into Filter IR.
156
+ * `{}` → `TRUE`. Throws on unsupported top-level operators.
157
+ */
158
+ function policyRecordToFilter(record) {
159
+ const parts = [];
160
+ for (const [key, value] of Object.entries(record)) if (key === "$or") parts.push(or(...asFilterArray(value)));
161
+ else if (key === "$and") parts.push(and(...asFilterArray(value)));
162
+ else if (key === "$nor") parts.push(not(or(...asFilterArray(value))));
163
+ else if (key === "$not") parts.push(not(policyRecordToFilter(value)));
164
+ else if (key.startsWith("$")) throw new Error(`[repo-core] matchesRecordFilter: unsupported top-level operator '${key}'. Supported: $and, $or, $nor, $not.`);
165
+ else parts.push(fieldFilter(key, value));
166
+ if (parts.length === 0) return TRUE;
167
+ return parts.length === 1 ? parts[0] : and(...parts);
168
+ }
169
+ function asFilterArray(value) {
170
+ if (!Array.isArray(value)) throw new Error("[repo-core] matchesRecordFilter: $or/$and/$nor operand must be an array");
171
+ return value.map((entry) => policyRecordToFilter(entry));
172
+ }
173
+ /**
174
+ * Evaluate an arc Mongo-record `_policyFilters` object against a document —
175
+ * converts to Filter IR, then delegates to the shared {@link matchFilter}
176
+ * engine. THE canonical `DataAdapter.matchesFilter` implementation; every
177
+ * kit's adapter delegates here.
178
+ *
179
+ * @param item The already-fetched document / row.
180
+ * @param filters Arc's `_policyFilters` in Mongo record syntax.
181
+ */
182
+ function matchesRecordFilter(item, filters) {
183
+ if (item === null || typeof item !== "object") return false;
184
+ return matchFilter(item, policyRecordToFilter(filters));
185
+ }
186
+ //#endregion
187
+ export { matchesRecordFilter, policyRecordToFilter };
@@ -7,35 +7,26 @@ function matchFilter(doc, filter) {
7
7
  case "and": return filter.children.every((child) => matchFilter(doc, child));
8
8
  case "or": return filter.children.some((child) => matchFilter(doc, child));
9
9
  case "not": return !matchFilter(doc, filter.child);
10
- case "eq": return equals(getField(doc, filter.field), filter.value);
11
- case "ne": return !equals(getField(doc, filter.field), filter.value);
12
- case "gt": return compare(getField(doc, filter.field), filter.value) > 0;
13
- case "gte": return compare(getField(doc, filter.field), filter.value) >= 0;
14
- case "lt": return compare(getField(doc, filter.field), filter.value) < 0;
15
- case "lte": return compare(getField(doc, filter.field), filter.value) <= 0;
16
- case "in": {
17
- const v = getField(doc, filter.field);
18
- return filter.values.some((candidate) => equals(v, candidate));
19
- }
20
- case "nin": {
21
- const v = getField(doc, filter.field);
22
- return !filter.values.some((candidate) => equals(v, candidate));
23
- }
10
+ case "eq": return someValue(resolve(doc, filter.field), (v) => equals(v, filter.value));
11
+ case "ne": return !someValue(resolve(doc, filter.field), (v) => equals(v, filter.value));
12
+ case "gt": return someValue(resolve(doc, filter.field), (v) => compare(v, filter.value) > 0);
13
+ case "gte": return someValue(resolve(doc, filter.field), (v) => compare(v, filter.value) >= 0);
14
+ case "lt": return someValue(resolve(doc, filter.field), (v) => compare(v, filter.value) < 0);
15
+ case "lte": return someValue(resolve(doc, filter.field), (v) => compare(v, filter.value) <= 0);
16
+ case "in": return someValue(resolve(doc, filter.field), (v) => filter.values.some((candidate) => equals(v, candidate)));
17
+ case "nin": return !someValue(resolve(doc, filter.field), (v) => filter.values.some((candidate) => equals(v, candidate)));
24
18
  case "exists": {
25
- const v = getField(doc, filter.field);
26
- const present = v !== void 0 && v !== null;
19
+ const present = fieldPresent(doc, filter.field);
27
20
  return filter.exists ? present : !present;
28
21
  }
29
22
  case "like": {
30
- const v = getField(doc, filter.field);
31
- if (typeof v !== "string") return false;
32
23
  const flags = filter.caseSensitivity === "sensitive" ? "" : "i";
33
- return getOrCompileLike(filter.pattern, flags).test(v);
24
+ const re = getOrCompileLike(filter.pattern, flags);
25
+ return someValue(resolve(doc, filter.field), (v) => regexTest(re, v));
34
26
  }
35
27
  case "regex": {
36
- const v = getField(doc, filter.field);
37
- if (typeof v !== "string") return false;
38
- return getOrCompileRegex(filter.pattern, filter.flags).test(v);
28
+ const re = getOrCompileRegex(filter.pattern, filter.flags);
29
+ return someValue(resolve(doc, filter.field), (v) => regexTest(re, v));
39
30
  }
40
31
  case "raw": return false;
41
32
  }
@@ -47,27 +38,92 @@ function matchFilter(doc, filter) {
47
38
  function asPredicate(filter) {
48
39
  return (doc) => matchFilter(doc, filter);
49
40
  }
50
- function getField(doc, path) {
51
- if (!doc || typeof doc !== "object") return void 0;
52
- const segments = path.split(".");
53
- let cursor = doc;
54
- for (const segment of segments) {
55
- if (cursor === null || cursor === void 0) return void 0;
56
- if (typeof cursor !== "object") return void 0;
57
- cursor = cursor[segment];
41
+ /**
42
+ * Resolve a dot-path to the SET of values it reaches, fanning out over
43
+ * arrays on intermediate segments (Mongo/JSON-path array semantics):
44
+ * `items.sku` on `{ items: [{ sku: 1 }, { sku: 2 }] }` → `[1, 2]`. A path
45
+ * with no array yields a single-element list, so scalar leaf ops behave
46
+ * exactly as before. A leaf array field (`tags`) is returned as one value
47
+ * (the array) so `equals`'s array-contains handles it.
48
+ */
49
+ /**
50
+ * Path segments that must never be resolved — reading them can surface an
51
+ * inherited member (or, for a JSON-parsed doc where `JSON.parse('{"__proto__":
52
+ * …}')` created a real OWN `__proto__`, a crafted value) and produce a WRONG
53
+ * authorization answer. Denied string-normalized (the object-path CVE-2021-23434
54
+ * lesson: an array-typed segment bypassed a `===` check). Fail closed: any
55
+ * path touching one of these resolves to no values → no match.
56
+ */
57
+ const DANGEROUS_SEGMENTS = /* @__PURE__ */ new Set([
58
+ "__proto__",
59
+ "constructor",
60
+ "prototype"
61
+ ]);
62
+ function resolve(doc, path) {
63
+ let frontier = [doc];
64
+ for (const segment of path.split(".")) {
65
+ if (DANGEROUS_SEGMENTS.has(String(segment))) return [];
66
+ const next = [];
67
+ for (const node of frontier) {
68
+ if (node === null || node === void 0 || typeof node !== "object") continue;
69
+ if (Array.isArray(node)) {
70
+ const idx = Number(segment);
71
+ if (Number.isInteger(idx) && idx >= 0 && idx < node.length) next.push(node[idx]);
72
+ for (const el of node) if (el && typeof el === "object" && Object.hasOwn(el, segment)) next.push(el[segment]);
73
+ } else if (Object.hasOwn(node, segment)) next.push(node[segment]);
74
+ }
75
+ if (next.length === 0) return [];
76
+ frontier = next;
58
77
  }
59
- return cursor;
78
+ return frontier;
79
+ }
80
+ /** Is any value reachable at `path` present (defined + non-null)? */
81
+ function fieldPresent(doc, path) {
82
+ return resolve(doc, path).some((v) => v !== void 0 && v !== null);
83
+ }
84
+ /**
85
+ * Apply a scalar predicate to a resolved value SET, unwrapping leaf array
86
+ * values so a scalar condition on an array field (`tags`, `scores`)
87
+ * matches when ANY element satisfies it — Mongo + SQL array semantics,
88
+ * concentrated in ONE place so `equals`/`compare`/regex stay pure scalar.
89
+ */
90
+ function someValue(values, pred) {
91
+ for (const v of values) if (Array.isArray(v)) {
92
+ if (v.some(pred)) return true;
93
+ } else if (pred(v)) return true;
94
+ return false;
60
95
  }
61
96
  function equals(a, b) {
97
+ if (a === b) return true;
98
+ if (typeof a === "number" && typeof b === "number") return Number.isNaN(a) && Number.isNaN(b);
62
99
  if (a instanceof Date && b instanceof Date) return a.getTime() === b.getTime();
63
100
  if (a instanceof Date && typeof b === "string") return a.toISOString() === b;
64
101
  if (b instanceof Date && typeof a === "string") return b.toISOString() === a;
65
- return a === b;
102
+ const as = idString(a);
103
+ const bs = idString(b);
104
+ if (as !== void 0 && bs !== void 0) return as === bs;
105
+ return false;
106
+ }
107
+ /** String form of an id-like value for coercing comparison; else undefined. */
108
+ function idString(value) {
109
+ if (typeof value === "string") return value;
110
+ if (value === null || value === void 0 || typeof value !== "object") return void 0;
111
+ if (Array.isArray(value)) return void 0;
112
+ const s = String(value);
113
+ return s === "[object Object]" ? void 0 : s;
66
114
  }
67
115
  function compare(a, b) {
116
+ if (a instanceof Date && typeof b === "string") {
117
+ const t = Date.parse(b);
118
+ if (!Number.isNaN(t)) b = new Date(t);
119
+ } else if (b instanceof Date && typeof a === "string") {
120
+ const t = Date.parse(a);
121
+ if (!Number.isNaN(t)) a = new Date(t);
122
+ }
68
123
  const aNum = toComparable(a);
69
124
  const bNum = toComparable(b);
70
125
  if (aNum === void 0 || bNum === void 0) return NaN;
126
+ if (typeof aNum !== typeof bNum) return NaN;
71
127
  if (aNum < bNum) return -1;
72
128
  if (aNum > bNum) return 1;
73
129
  return 0;
@@ -114,6 +170,21 @@ function getOrCompileRegex(pattern, flags) {
114
170
  regexCache.set(key, re);
115
171
  return re;
116
172
  }
173
+ /**
174
+ * Max string length fed to a regex `.test()`. ReDoS is `pattern × input`;
175
+ * even a benign developer-written pattern can go quadratic on a pathological
176
+ * INPUT string — and in a realtime fan-out the matcher runs once per
177
+ * subscriber per record, so one slow match blocks the event loop and
178
+ * amplifies across the whole subscriber set. A field value longer than this
179
+ * is treated as NO MATCH (fail closed) rather than risking a stall; policy
180
+ * filters never regex-test megabyte fields. (Trusted-source patterns make
181
+ * pattern-side ReDoS a non-issue; this caps the input side.)
182
+ */
183
+ const MAX_REGEX_INPUT = 64 * 1024;
184
+ /** Guarded regex test: string-only, input-length-capped (see MAX_REGEX_INPUT). */
185
+ function regexTest(re, v) {
186
+ return typeof v === "string" && v.length <= MAX_REGEX_INPUT && re.test(v);
187
+ }
117
188
  /** SQL `LIKE` pattern → JS regex body. Escapes regex metachars; `%` → `.*`, `_` → `.`. */
118
189
  function likeToRegex(pattern) {
119
190
  let out = "";
@@ -126,6 +126,15 @@ interface RepoCapabilities {
126
126
  * tenant cleanup primitive.
127
127
  */
128
128
  purgeByField?: boolean;
129
+ /**
130
+ * `purgeByFilter(filter, strategy, options)` — range/filter-scoped
131
+ * variant of `purgeByField`. Processes rows matching an arbitrary
132
+ * compiled filter (a `civilDate` window, a retention cutoff, a compound
133
+ * cohort) rather than a single `field = value` equality — the GDPR /
134
+ * retention "anonymize a slice across a RANGE while retaining measures"
135
+ * primitive.
136
+ */
137
+ purgeByFilter?: boolean;
129
138
  /**
130
139
  * `archiveByFilter(filter, sink, options)` — chunked cold-storage
131
140
  * extraction (write-before-delete, at-least-once). The data-lifecycle
@@ -12,11 +12,30 @@ type WritingPurgeStrategy = Exclude<TenantPurgeStrategy, {
12
12
  * Driver-facing port the orchestrator drives. Each kit implements one
13
13
  * closure over its driver primitives + the purge predicate.
14
14
  *
15
+ * **Bound-predicate forms.** The port binds its selection predicate
16
+ * internally — the orchestrator never sees it. Two factory shapes both
17
+ * satisfy this single interface:
18
+ *
19
+ * - **Equality-bound** (`purgeByField`): the predicate is
20
+ * `{ [field]: value }` — the classic tenant-scoped cleanup where an
21
+ * organization id equals the deleted tenant.
22
+ * - **Filter-bound** (`purgeByFilter`): the predicate is a compiled
23
+ * range/compound filter (e.g. a `civilDate` window `gte/lte`,
24
+ * `{ status: 'archived', createdAt: { $lt: cutoff } }`). This is the
25
+ * GDPR/retention "anonymize a PII dimension across a range while
26
+ * RETAINING measures" op — no native equality analogue. Mirrors how
27
+ * the archive port's factory (`createMongoArchivePort`) binds a
28
+ * pre-compiled Filter rather than a `field/value` pair.
29
+ *
30
+ * Because `runChunkedPurge` drives ANY `PurgePort`, both factories reuse
31
+ * the same orchestrator unchanged — only the bound base predicate differs.
32
+ *
15
33
  * **Plugin-bypass invariant.** Implementations MUST bypass tenant
16
- * scoping in plugin hooks — the caller's `field = value` predicate IS
17
- * the authoritative scope; a tenant-injecting hook would narrow to the
18
- * wrong tenant. Pass `bypassTenant: true` on inner Repository calls
19
- * (which keeps audit / cache hooks active but disables tenant injection).
34
+ * scoping in plugin hooks — the caller's bound predicate (`field = value`
35
+ * OR the compiled filter) IS the authoritative scope; a tenant-injecting
36
+ * hook would narrow to the wrong tenant. Pass `bypassTenant: true` on
37
+ * inner Repository calls (which keeps audit / cache hooks active but
38
+ * disables tenant injection).
20
39
  *
21
40
  * **Throughput contract.** Implementations should issue the minimum
22
41
  * number of round-trips a chunk requires:
@@ -38,6 +57,23 @@ interface PurgePort {
38
57
  * Returning `0` signals "no more matching rows"; the orchestrator
39
58
  * exits. Returning a partial batch (`< limit`) is also a terminal
40
59
  * signal — saves one round-trip on the last chunk.
60
+ *
61
+ * **PROGRESSION CONTRACT (mandatory).** Successive calls MUST advance
62
+ * through the match set for EVERY strategy. `hard` advances naturally
63
+ * (deleted rows leave the predicate's match set) — but `soft` and
64
+ * `anonymize` mutate rows that usually STILL satisfy the base
65
+ * predicate, so a port that re-runs `find(filter).limit(n)` re-selects
66
+ * the same first chunk forever and the orchestrator never terminates.
67
+ * Implementations must use **stable keyset progression**: order by the
68
+ * primary key and keep an internal `pk > lastSeen` cursor across calls
69
+ * (advance it only after the chunk's write succeeds, so a retried
70
+ * chunk re-selects the same rows). Offsets (`skip`) are not acceptable
71
+ * — they shift under concurrent writes and re-scan the head.
72
+ *
73
+ * A port instance is single-run state: build a fresh port per
74
+ * `runChunkedPurge` invocation, never share one across runs.
75
+ * `runPurgeConformance` (from `@classytic/repo-core/testing`) proves a
76
+ * kit satisfies this contract with match sets larger than `batchSize`.
41
77
  */
42
78
  purgeChunk(strategy: WritingPurgeStrategy, limit: number): Promise<number>;
43
79
  }
@@ -1598,10 +1598,13 @@ interface StandardRepo<TDoc> extends MinimalRepo<TDoc> {
1598
1598
  * - sqlite: `EXPLAIN QUERY PLAN SELECT … WHERE field = ?` shows
1599
1599
  * `SEARCH … USING INDEX`, never `SCAN`.
1600
1600
  *
1601
- * **Idempotent.** Re-running with the same arguments is safe — rows
1602
- * already deleted/anonymized simply don't match the next pass.
1603
- * Crucial for at-least-once cascade workers that may retry after
1604
- * partial failure.
1601
+ * **Idempotent by outcome.** Re-running with the same arguments is
1602
+ * safe: `hard` rows are gone, and `soft`/`anonymize` rows converge to
1603
+ * the same terminal field values on a second pass (the rows generally
1604
+ * DO still match the predicate — which is exactly why ports must use
1605
+ * keyset progression, not re-selection, to advance WITHIN a run; see
1606
+ * the {@link PurgePort} progression contract). Crucial for
1607
+ * at-least-once cascade workers that may retry after partial failure.
1605
1608
  *
1606
1609
  * **Plugin composition.** Kits route the underlying chunked ops
1607
1610
  * through their standard `before:deleteMany` / `before:updateMany`
@@ -1619,6 +1622,41 @@ interface StandardRepo<TDoc> extends MinimalRepo<TDoc> {
1619
1622
  * @param options Chunking, session, progress, abort signal.
1620
1623
  */
1621
1624
  purgeByField?(field: string, value: unknown, strategy: TenantPurgeStrategy, options?: TenantPurgeOptions): Promise<TenantPurgeResult>;
1625
+ /**
1626
+ * Range/filter-scoped variant of {@link purgeByField} — processes every
1627
+ * row matching an arbitrary `filter` (not just `field = value`) under the
1628
+ * given {@link TenantPurgeStrategy}. The compliance primitive for
1629
+ * "purge/anonymize a slice across a RANGE while retaining measures":
1630
+ * redact a PII dimension across a `civilDate` window, hard-delete rows
1631
+ * older than a retention cutoff, soft-delete a compound-predicate cohort.
1632
+ *
1633
+ * `purgeByField(field, value, ...)` is the equality special case
1634
+ * (`{ [field]: value }`); this method takes the full portable
1635
+ * {@link FilterInput} (Filter IR or a plain kit-native record), compiled
1636
+ * once by the kit before the chunk loop — the same dual-dialect rule
1637
+ * every other verb follows.
1638
+ *
1639
+ * Everything else is identical to `purgeByField`:
1640
+ * - Strategy → kit-native primitive (`hard`/`soft`/`anonymize`/`skip`).
1641
+ * - **Chunking mandatory** — implementations MUST honor `batchSize`.
1642
+ * - **Index requirement** — the filter's leading field(s) MUST be indexed
1643
+ * or every chunk re-scans the collection / table.
1644
+ * - **Idempotent** — re-running with the same arguments is safe; already
1645
+ * purged rows simply don't match the next pass.
1646
+ * - **Plugin composition** — the chunked ops route through the kit's
1647
+ * `before:deleteMany` / `before:updateMany` hooks so audit /
1648
+ * cache-invalidation / observability plugins fire naturally.
1649
+ * - **Narrowed-write re-assertion** — kits re-assert the base filter on
1650
+ * the `{ _id: { $in: ids }, ...filter }` write, defending against a row
1651
+ * that left the matching set between the id select and the write.
1652
+ *
1653
+ * Optional method; gate on `capabilities.purgeByFilter`.
1654
+ *
1655
+ * @param filter Predicate selecting rows to purge (Filter IR or record).
1656
+ * @param strategy Strategy declaration — see {@link TenantPurgeStrategy}.
1657
+ * @param options Chunking, session, progress, abort signal.
1658
+ */
1659
+ purgeByFilter?(filter: FilterInput, strategy: TenantPurgeStrategy, options?: TenantPurgeOptions): Promise<TenantPurgeResult>;
1622
1660
  /**
1623
1661
  * Chunked cold-storage extraction — move every row matching `filter`
1624
1662
  * into a host-provided {@link ArchiveSink}, then remove it from the hot
@@ -3,4 +3,5 @@ import { ConformanceContext, ConformanceDoc, ConformanceFeatures, ConformanceHar
3
3
  import { runStandardRepoConformance } from "./conformance.mjs";
4
4
  import { LockConformanceHarness, runLockAdapterConformance } from "./lock-conformance.mjs";
5
5
  import { UsageConformanceHarness, runUsageStoreContract } from "./usage-conformance.mjs";
6
- export { type AggregateOpsSupport, type ConformanceContext, type ConformanceDoc, type ConformanceFeatures, type ConformanceHarness, type LockConformanceHarness, type UsageConformanceHarness, runLockAdapterConformance, runStandardRepoConformance, runUsageStoreContract };
6
+ import { PurgeConformanceContext, PurgeConformanceHarness, runPurgeConformance } from "./purge-conformance.mjs";
7
+ export { type AggregateOpsSupport, type ConformanceContext, type ConformanceDoc, type ConformanceFeatures, type ConformanceHarness, type LockConformanceHarness, type PurgeConformanceContext, type PurgeConformanceHarness, type UsageConformanceHarness, runLockAdapterConformance, runPurgeConformance, runStandardRepoConformance, runUsageStoreContract };
@@ -1,4 +1,5 @@
1
1
  import { runStandardRepoConformance } from "./conformance.mjs";
2
2
  import { runLockAdapterConformance } from "./lock-conformance.mjs";
3
3
  import { runUsageStoreContract } from "./usage-conformance.mjs";
4
- export { runLockAdapterConformance, runStandardRepoConformance, runUsageStoreContract };
4
+ import { runPurgeConformance } from "./purge-conformance.mjs";
5
+ export { runLockAdapterConformance, runPurgeConformance, runStandardRepoConformance, runUsageStoreContract };
@@ -0,0 +1,33 @@
1
+ import { TenantPurgeOptions, TenantPurgeResult, TenantPurgeStrategy } from "../repository/types.mjs";
2
+ //#region src/testing/purge-conformance.d.ts
3
+ /** Everything the shared scenarios need from a kit. */
4
+ interface PurgeConformanceContext {
5
+ /**
6
+ * Seed `inScope` docs matching the purge scope and `outOfScope` docs
7
+ * outside it. Every in-scope doc must carry:
8
+ * - a string field `email` set to `'user-<i>@test.local'` (anonymize target);
9
+ * - a numeric field `amount` (a measure that must SURVIVE soft/anonymize).
10
+ */
11
+ seed(inScope: number, outOfScope: number): Promise<void>;
12
+ /** Run the kit's chunked purge over the bound scope. */
13
+ purge(strategy: TenantPurgeStrategy, options?: TenantPurgeOptions): Promise<TenantPurgeResult>;
14
+ /** RAW physical count of in-scope rows — MUST bypass soft-delete query filters. */
15
+ countRaw(): Promise<number>;
16
+ /** RAW count of in-scope rows carrying the soft-deleted flag. */
17
+ countSoftFlagged(): Promise<number>;
18
+ /** RAW count of in-scope rows whose `email` equals `value`. */
19
+ countEmail(value: string): Promise<number>;
20
+ /** RAW sum of `amount` across in-scope rows (proves measures retained). */
21
+ sumAmount(): Promise<number>;
22
+ /** RAW physical count of OUT-of-scope rows — must never change. */
23
+ countOutOfScope(): Promise<number>;
24
+ }
25
+ interface PurgeConformanceHarness {
26
+ /** Kit name — the top-level describe() label. */
27
+ name: string;
28
+ /** Fresh isolated context per test (own collection/table). */
29
+ setup(): Promise<PurgeConformanceContext>;
30
+ }
31
+ declare function runPurgeConformance(harness: PurgeConformanceHarness): void;
32
+ //#endregion
33
+ export { PurgeConformanceContext, PurgeConformanceHarness, runPurgeConformance };
@@ -0,0 +1,160 @@
1
+ import { beforeEach, describe, expect, it } from "vitest";
2
+ //#region src/testing/purge-conformance.ts
3
+ /**
4
+ * `runPurgeConformance` — cross-kit chunked-purge contract suite.
5
+ *
6
+ * Proves a kit's `purgeByField`/`purgeByFilter` port makes **stable
7
+ * progress for EVERY strategy** when the match set exceeds `batchSize`
8
+ * — the exact property the naive "re-query the same predicate" port
9
+ * shape violates: `hard` self-advances (deleted rows leave the match
10
+ * set) but `soft`/`anonymize` re-select the same first chunk forever
11
+ * because the mutated rows still satisfy the base predicate.
12
+ *
13
+ * Every scenario seeds MORE rows than `batchSize`, so a port without
14
+ * keyset progression fails here instead of hanging production. A chunk
15
+ * budget (via `onProgress` + abort) converts the would-be infinite loop
16
+ * into a crisp assertion failure.
17
+ *
18
+ * ## Usage from a kit
19
+ *
20
+ * import { runPurgeConformance } from '@classytic/repo-core/testing';
21
+ *
22
+ * describe('mongokit purge conformance', () => {
23
+ * runPurgeConformance({
24
+ * name: 'mongokit',
25
+ * async setup() { …return a PurgeConformanceContext… },
26
+ * });
27
+ * });
28
+ */
29
+ const IN_SCOPE = 25;
30
+ const OUT_SCOPE = 5;
31
+ const BATCH = 10;
32
+ const AMOUNT_EACH = 7;
33
+ /**
34
+ * Wrap TenantPurgeOptions with a chunk budget: when the port stops
35
+ * progressing, the loop would otherwise run forever — the budget aborts
36
+ * it after `limit` chunks so the suite fails with a readable assertion
37
+ * instead of a vitest timeout.
38
+ */
39
+ function budgeted(limit, chunks = []) {
40
+ const controller = new AbortController();
41
+ return {
42
+ batchSize: BATCH,
43
+ signal: controller.signal,
44
+ onProgress({ chunkSize }) {
45
+ chunks.push(chunkSize);
46
+ if (chunks.length >= limit) controller.abort();
47
+ }
48
+ };
49
+ }
50
+ /** ceil(25/10) + 1 slack — a progressing port never needs more. */
51
+ const CHUNK_BUDGET = Math.ceil(IN_SCOPE / BATCH) + 1;
52
+ function runPurgeConformance(harness) {
53
+ describe(`${harness.name} — chunked purge conformance`, () => {
54
+ let ctx;
55
+ beforeEach(async () => {
56
+ ctx = await harness.setup();
57
+ });
58
+ it("hard: drains a multi-batch scope exactly once per row", async () => {
59
+ await ctx.seed(IN_SCOPE, OUT_SCOPE);
60
+ const chunks = [];
61
+ const res = await ctx.purge({ type: "hard" }, budgeted(CHUNK_BUDGET, chunks));
62
+ expect(res.ok).toBe(true);
63
+ expect(res.processed).toBe(IN_SCOPE);
64
+ expect(chunks).toEqual([
65
+ 10,
66
+ 10,
67
+ 5
68
+ ]);
69
+ expect(await ctx.countRaw()).toBe(0);
70
+ expect(await ctx.countOutOfScope()).toBe(OUT_SCOPE);
71
+ });
72
+ it("soft: progresses across batches WITHOUT caller-supplied exclusion predicates", async () => {
73
+ await ctx.seed(IN_SCOPE, OUT_SCOPE);
74
+ const chunks = [];
75
+ const res = await ctx.purge({ type: "soft" }, budgeted(CHUNK_BUDGET, chunks));
76
+ expect(res.ok).toBe(true);
77
+ expect(res.processed).toBe(IN_SCOPE);
78
+ expect(chunks).toEqual([
79
+ 10,
80
+ 10,
81
+ 5
82
+ ]);
83
+ expect(await ctx.countRaw()).toBe(IN_SCOPE);
84
+ expect(await ctx.countSoftFlagged()).toBe(IN_SCOPE);
85
+ expect(await ctx.sumAmount()).toBe(IN_SCOPE * AMOUNT_EACH);
86
+ expect(await ctx.countOutOfScope()).toBe(OUT_SCOPE);
87
+ });
88
+ it("anonymize (static): progresses across batches and retains measures", async () => {
89
+ await ctx.seed(IN_SCOPE, OUT_SCOPE);
90
+ const chunks = [];
91
+ const res = await ctx.purge({
92
+ type: "anonymize",
93
+ fields: { email: "redacted@example.invalid" }
94
+ }, budgeted(CHUNK_BUDGET, chunks));
95
+ expect(res.ok).toBe(true);
96
+ expect(res.processed).toBe(IN_SCOPE);
97
+ expect(chunks).toEqual([
98
+ 10,
99
+ 10,
100
+ 5
101
+ ]);
102
+ expect(await ctx.countEmail("redacted@example.invalid")).toBe(IN_SCOPE);
103
+ expect(await ctx.sumAmount()).toBe(IN_SCOPE * AMOUNT_EACH);
104
+ expect(await ctx.countRaw()).toBe(IN_SCOPE);
105
+ expect(await ctx.countOutOfScope()).toBe(OUT_SCOPE);
106
+ });
107
+ it("anonymize (function-form): progresses across batches", async () => {
108
+ await ctx.seed(IN_SCOPE, OUT_SCOPE);
109
+ const res = await ctx.purge({
110
+ type: "anonymize",
111
+ fields: { email: () => "fn-redacted@example.invalid" }
112
+ }, budgeted(CHUNK_BUDGET));
113
+ expect(res.ok).toBe(true);
114
+ expect(res.processed).toBe(IN_SCOPE);
115
+ expect(await ctx.countEmail("fn-redacted@example.invalid")).toBe(IN_SCOPE);
116
+ expect(await ctx.sumAmount()).toBe(IN_SCOPE * AMOUNT_EACH);
117
+ });
118
+ it("exact-batch boundary: inScope === batchSize processes each row once", async () => {
119
+ await ctx.seed(BATCH, OUT_SCOPE);
120
+ const res = await ctx.purge({ type: "soft" }, budgeted(CHUNK_BUDGET));
121
+ expect(res.ok).toBe(true);
122
+ expect(res.processed).toBe(BATCH);
123
+ expect(await ctx.countSoftFlagged()).toBe(BATCH);
124
+ });
125
+ it("skip: declared no-op reaches no rows", async () => {
126
+ await ctx.seed(3, 0);
127
+ const res = await ctx.purge({
128
+ type: "skip",
129
+ reason: "retention-owned"
130
+ });
131
+ expect(res.ok).toBe(true);
132
+ expect(res.processed).toBe(0);
133
+ expect(res.skipReason).toBe("retention-owned");
134
+ expect(await ctx.countRaw()).toBe(3);
135
+ });
136
+ it("empty scope: terminates immediately with zero processed", async () => {
137
+ await ctx.seed(0, OUT_SCOPE);
138
+ const res = await ctx.purge({ type: "hard" }, budgeted(CHUNK_BUDGET));
139
+ expect(res.ok).toBe(true);
140
+ expect(res.processed).toBe(0);
141
+ expect(await ctx.countOutOfScope()).toBe(OUT_SCOPE);
142
+ });
143
+ it("abort between chunks: committed chunks stay, result is ok:false", async () => {
144
+ await ctx.seed(IN_SCOPE, OUT_SCOPE);
145
+ const controller = new AbortController();
146
+ const res = await ctx.purge({ type: "hard" }, {
147
+ batchSize: BATCH,
148
+ signal: controller.signal,
149
+ onProgress() {
150
+ controller.abort();
151
+ }
152
+ });
153
+ expect(res.ok).toBe(false);
154
+ expect(res.processed).toBe(BATCH);
155
+ expect(await ctx.countRaw()).toBe(IN_SCOPE - BATCH);
156
+ });
157
+ });
158
+ }
159
+ //#endregion
160
+ export { runPurgeConformance };
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@classytic/repo-core",
3
- "version": "0.13.0",
3
+ "version": "0.17.0",
4
4
  "description": "Driver-agnostic repository primitives: hooks, Filter IR, operations, pagination, cache contract. Foundation for mongokit, sqlitekit, pgkit, and prismakit. Lean by design — no plugins ship here; each kit owns its own.",
5
5
  "type": "module",
6
6
  "sideEffects": false,
@@ -102,6 +102,10 @@
102
102
  "./sync": {
103
103
  "types": "./dist/sync/index.d.mts",
104
104
  "default": "./dist/sync/index.mjs"
105
+ },
106
+ "./cleanup": {
107
+ "types": "./dist/cleanup/index.d.mts",
108
+ "default": "./dist/cleanup/index.mjs"
105
109
  }
106
110
  },
107
111
  "keywords": [