@stll/anonymize 2.0.0 → 2.0.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -6,7 +6,7 @@
6
6
 
7
7
  Runtime package for multi-layer PII detection and anonymization.
8
8
 
9
- It combines regex detectors, trigger phrases, deny-list matching, coreference handling, and NER into a single pipeline that works in native Node.js and in browser builds through the WASM entrypoint.
9
+ It combines regex detectors, trigger phrases, deny-list matching, and coreference handling in a single deterministic pipeline that works in native Node.js and in browser builds through the WASM entrypoint.
10
10
 
11
11
  ## Install
12
12
 
@@ -68,6 +68,130 @@ const warmDiagnosticsJson = anonymizer.warmLazyRegexDiagnosticsJson();
68
68
  const result = anonymizer.redact_text(text, { redactString: "***" });
69
69
  ```
70
70
 
71
+ For related documents, create an explicit in-memory session from the prepared
72
+ anonymizer. The session reuses placeholders for the same normalized entity while
73
+ keeping its mutable mapping state isolated from other sessions:
74
+
75
+ ```ts
76
+ const session = anonymizer.createRedactionSession("opaque_case_1");
77
+ const first = session.redact_text(firstDocument);
78
+ const second = session.redact_text(secondDocument);
79
+ const restoredText = session.restoreText(first.redaction.redactedText);
80
+ ```
81
+
82
+ `restoreText()` replaces only complete placeholders owned by that active
83
+ session. It performs one non-cascading pass, leaves other session namespaces
84
+ unchanged, and rejects unknown placeholders in its own namespace. Pass the
85
+ caller-supplied observation time as the second argument for lifecycle sessions.
86
+
87
+ `session.toPlaintextJson()` supports deterministic in-memory transfer between
88
+ runtime instances. Its output contains original personal data in plaintext: do
89
+ not log it or persist it without an application-owned protection layer. Restore
90
+ validated transfer state with `anonymizer.restoreRedactionSession(json)`.
91
+
92
+ For persistence or transfer, prefer an authenticated encrypted archive. Supply
93
+ a caller-owned 32-byte key; key generation, storage, rotation, and access
94
+ control remain application responsibilities. Restoring also requires the
95
+ expected session ID so an archive cannot be substituted across records:
96
+
97
+ ```ts
98
+ const key = crypto.getRandomValues(new Uint8Array(32));
99
+ const archive = session.toEncryptedArchive(key);
100
+ const restored = anonymizer.restoreEncryptedRedactionSession({
101
+ archive,
102
+ key,
103
+ expectedSessionId: "opaque_case_1",
104
+ });
105
+ ```
106
+
107
+ The archive contains personal data as ciphertext. Do not log the key or derive
108
+ it directly from a password; use a key-management boundary appropriate to the
109
+ deployment.
110
+
111
+ Sessions can carry explicit lifecycle bounds. The engine never reads the system
112
+ clock; supply the UTC epoch-second observation time for each lifecycle-aware
113
+ operation:
114
+
115
+ ```ts
116
+ const key = crypto.getRandomValues(new Uint8Array(32));
117
+ const session = anonymizer.createRedactionSessionWithLifecycle({
118
+ sessionId: "opaque_case_2",
119
+ createdAtEpochSeconds: 1_800_000_000,
120
+ expiresAtEpochSeconds: 1_800_086_400,
121
+ });
122
+ const result = session.redactTextAt({
123
+ fullText: document,
124
+ observedAtEpochSeconds: 1_800_000_100,
125
+ });
126
+ const metadata = session.inspect(1_800_000_100); // contains no entity values
127
+ const archive = session.toEncryptedArchiveAt(key, 1_800_000_100);
128
+ const restored = anonymizer.restoreEncryptedRedactionSession({
129
+ archive,
130
+ key,
131
+ expectedSessionId: "opaque_case_2",
132
+ observedAtEpochSeconds: 1_800_000_100,
133
+ });
134
+ const deletion = session.delete();
135
+ ```
136
+
137
+ Expiry is fail-closed at its exact boundary. `delete()` performs logical
138
+ deletion: it clears the session mappings and prevents future use, but does not
139
+ revoke earlier exported copies or claim physical erasure of process memory.
140
+
141
+ Per-label operators support `replace`, `redact`, `keep`, and tagged `mask`
142
+ configuration. `keep` records
143
+ that an entity was processed while leaving its source text unchanged; it
144
+ creates no reversible redaction-key entry:
145
+
146
+ ```ts
147
+ const result = anonymizer.redactText(text, {
148
+ operators: { organization: "keep" },
149
+ });
150
+ ```
151
+
152
+ `mask` replaces a configured number of visible Unicode grapheme clusters from
153
+ the start or end. A masking character must itself be exactly one grapheme:
154
+
155
+ ```ts
156
+ const result = anonymizer.redactText(text, {
157
+ operators: {
158
+ "email address": {
159
+ type: "mask",
160
+ maskingCharacter: "*",
161
+ charactersToMask: 6,
162
+ direction: "start",
163
+ },
164
+ },
165
+ });
166
+ ```
167
+
168
+ Caller-produced spans enter the same resolution and redaction pipeline. Node
169
+ and browser offsets use JavaScript UTF-16 string indexes; matched text is
170
+ derived from the input:
171
+
172
+ ```ts
173
+ const result = anonymizer.redactTextWithCallerDetections("😀Alice signed.", {
174
+ detections: [
175
+ {
176
+ start: 2,
177
+ end: 7,
178
+ label: "person",
179
+ score: 0.95,
180
+ providerId: "example-ner",
181
+ detectionId: "person-1",
182
+ },
183
+ ],
184
+ });
185
+ ```
186
+
187
+ `providerId` and `detectionId` are required provenance identifiers. They must
188
+ be 1–128 ASCII characters, start with an alphanumeric character, and otherwise
189
+ contain only alphanumerics, `.`, `_`, `:`, or `-`; do not encode personal data
190
+ in them. Retained result entities preserve both IDs. Use
191
+ `redactTextWithCallerDetectionsDiagnosticsJson()` for audit-safe input and
192
+ retained counts. Diagnostic events include provenance, labels, offsets, and
193
+ scores, but never matched text.
194
+
71
195
  The config module may export a `PipelineConfig` directly or `{ config, gazetteerEntries }`. Include `@stll/anonymize-data` dictionaries there if your runtime config uses the deny-list or name-corpus layers; keep the corresponding layers enabled for caller-owned `customDenyList`, `customRegexes`, and gazetteers. Those inputs are part of the prepared package and should be regenerated when they change.
72
196
 
73
197
  ## Python SDK
@@ -81,10 +205,40 @@ prepared = anonymize.preload_default_native_pipeline(
81
205
  )
82
206
  result = prepared.redact_text(text, redact_string="***")
83
207
 
208
+ session = prepared.create_redaction_session("opaque_case_1")
209
+ redacted = session.redact_text(text)
210
+ restored_text = session.restore_text(redacted.redaction.redacted_text)
211
+ archive = session.to_encrypted_archive(application_owned_32_byte_key)
212
+ restored = prepared.restore_encrypted_redaction_session(
213
+ archive,
214
+ application_owned_32_byte_key,
215
+ "opaque_case_1",
216
+ )
217
+
84
218
  print(result.redaction.redacted_text)
85
219
  ```
86
220
 
87
- The Python SDK uses the same Rust core and prepared-package contract as the Node SDK. Prefer `get_default_native_pipeline()`, `preload_default_native_pipeline()`, `load_prepared_package()`, or `load_prepared_package_file()` for repeated calls; top-level `redact_text()` and `redact_text_json()` prepare from config on each call.
221
+ Python caller detections use Python character indexes:
222
+
223
+ ```py
224
+ result = prepared.redact_text_with_caller_detections(
225
+ "😀Alice signed.",
226
+ [{"start": 1, "end": 6, "label": "person", "score": 0.95,
227
+ "provider_id": "example-ner", "detection_id": "person-1"}],
228
+ )
229
+ ```
230
+
231
+ Python preserves `provider_id` and `detection_id` on retained entities. Use
232
+ `redact_text_with_caller_detections_diagnostics_json()` for the same audit-safe
233
+ diagnostics contract.
234
+
235
+ The Python SDK uses the same Rust core, encrypted session archive format, and
236
+ prepared-package contract as the Node SDK. The application owns archive-key
237
+ generation, storage, rotation, and authorization. Prefer
238
+ `get_default_native_pipeline()`, `preload_default_native_pipeline()`,
239
+ `load_prepared_package()`, or `load_prepared_package_file()` for repeated calls;
240
+ top-level `redact_text()` and `redact_text_json()` prepare from config on each
241
+ call.
88
242
 
89
243
  ## Caller-Owned Deny Lists and Regexes
90
244
 
@@ -143,7 +297,10 @@ export default {
143
297
 
144
298
  - Native architecture and extension guidance:
145
299
  [`ARCHITECTURE.md`](ARCHITECTURE.md).
146
- - `labels: []` disables deterministic label filtering; when NER is enabled it falls back to the default label set.
300
+ - `labels: []` disables deterministic label filtering.
301
+ - `enableNer` is a compatibility field for the removed TypeScript pipeline. The
302
+ native pipeline rejects `true`; model-produced spans will use a separate
303
+ caller-detection API.
147
304
  - `enableNameCorpus` also controls whether first names, surnames, and titles are injected into deny-list matching when `enableDenyList` is enabled.
148
305
  - The optional `@stll/anonymize-data` package carries the published dictionary and trigger data used when building prepared packages.
149
306
  - `customDenyList` and `customRegexes` are part of the prepared package input and should be regenerated when they change.
@@ -0,0 +1,123 @@
1
+ import { o as ENTITY_CAPABILITIES } from "./constants2.mjs";
2
+ //#region src/capabilities.d.ts
3
+ declare const CAPABILITY_MANIFEST_SCHEMA_VERSION: 1;
4
+ declare const CAPABILITY_RUNTIMES: readonly ["node", "python", "wasm"];
5
+ type CapabilityRuntime = (typeof CAPABILITY_RUNTIMES)[number];
6
+ type CapabilityManifest = {
7
+ schemaVersion: typeof CAPABILITY_MANIFEST_SCHEMA_VERSION;
8
+ runtimes: readonly CapabilityRuntime[];
9
+ entities: typeof ENTITY_CAPABILITIES;
10
+ };
11
+ /**
12
+ * Versioned, runtime-free discovery contract for the deterministic pipeline.
13
+ * The manifest contains no accuracy claims; it describes available labels,
14
+ * activation, provenance sources, and runtime parity only.
15
+ */
16
+ declare const CAPABILITY_MANIFEST: {
17
+ readonly schemaVersion: 1;
18
+ readonly runtimes: readonly ["node", "python", "wasm"];
19
+ readonly entities: readonly [{
20
+ readonly label: "person";
21
+ readonly selection: "default";
22
+ readonly detectionSources: readonly ["trigger", "regex", "deny-list", "coreference"];
23
+ }, {
24
+ readonly label: "organization";
25
+ readonly selection: "default";
26
+ readonly detectionSources: readonly ["trigger", "deny-list", "legal-form", "gazetteer", "coreference"];
27
+ }, {
28
+ readonly label: "phone number";
29
+ readonly selection: "default";
30
+ readonly detectionSources: readonly ["regex", "trigger"];
31
+ }, {
32
+ readonly label: "address";
33
+ readonly selection: "default";
34
+ readonly detectionSources: readonly ["regex", "trigger", "deny-list"];
35
+ }, {
36
+ readonly label: "country";
37
+ readonly selection: "default";
38
+ readonly detectionSources: readonly ["country"];
39
+ }, {
40
+ readonly label: "email address";
41
+ readonly selection: "default";
42
+ readonly detectionSources: readonly ["regex"];
43
+ }, {
44
+ readonly label: "date";
45
+ readonly selection: "default";
46
+ readonly detectionSources: readonly ["regex", "trigger"];
47
+ }, {
48
+ readonly label: "date of birth";
49
+ readonly selection: "default";
50
+ readonly detectionSources: readonly ["trigger"];
51
+ }, {
52
+ readonly label: "bank account number";
53
+ readonly selection: "default";
54
+ readonly detectionSources: readonly ["regex", "trigger"];
55
+ }, {
56
+ readonly label: "iban";
57
+ readonly selection: "default";
58
+ readonly detectionSources: readonly ["regex", "trigger"];
59
+ }, {
60
+ readonly label: "tax identification number";
61
+ readonly selection: "default";
62
+ readonly detectionSources: readonly ["regex", "trigger"];
63
+ }, {
64
+ readonly label: "identity card number";
65
+ readonly selection: "default";
66
+ readonly detectionSources: readonly ["regex", "trigger"];
67
+ }, {
68
+ readonly label: "birth number";
69
+ readonly selection: "default";
70
+ readonly detectionSources: readonly ["regex", "trigger"];
71
+ }, {
72
+ readonly label: "national identification number";
73
+ readonly selection: "default";
74
+ readonly detectionSources: readonly ["regex", "trigger"];
75
+ }, {
76
+ readonly label: "social security number";
77
+ readonly selection: "default";
78
+ readonly detectionSources: readonly ["regex", "trigger"];
79
+ }, {
80
+ readonly label: "registration number";
81
+ readonly selection: "default";
82
+ readonly detectionSources: readonly ["regex", "trigger"];
83
+ }, {
84
+ readonly label: "credit card number";
85
+ readonly selection: "default";
86
+ readonly detectionSources: readonly ["regex"];
87
+ }, {
88
+ readonly label: "passport number";
89
+ readonly selection: "default";
90
+ readonly detectionSources: readonly ["regex"];
91
+ }, {
92
+ readonly label: "crypto";
93
+ readonly selection: "default";
94
+ readonly detectionSources: readonly ["regex"];
95
+ }, {
96
+ readonly label: "monetary amount";
97
+ readonly selection: "default";
98
+ readonly detectionSources: readonly ["regex", "trigger"];
99
+ }, {
100
+ readonly label: "land parcel";
101
+ readonly selection: "default";
102
+ readonly detectionSources: readonly ["trigger"];
103
+ }, {
104
+ readonly label: "misc";
105
+ readonly selection: "default";
106
+ readonly detectionSources: readonly ["regex", "deny-list"];
107
+ }, {
108
+ readonly label: "ip address";
109
+ readonly selection: "opt-in";
110
+ readonly detectionSources: readonly ["regex"];
111
+ }, {
112
+ readonly label: "mac address";
113
+ readonly selection: "opt-in";
114
+ readonly detectionSources: readonly ["regex"];
115
+ }, {
116
+ readonly label: "url";
117
+ readonly selection: "opt-in";
118
+ readonly detectionSources: readonly ["regex"];
119
+ }];
120
+ };
121
+ //#endregion
122
+ export { CAPABILITY_MANIFEST, CAPABILITY_MANIFEST_SCHEMA_VERSION, CAPABILITY_RUNTIMES, CapabilityManifest, CapabilityRuntime };
123
+ //# sourceMappingURL=capabilities.d.mts.map
@@ -0,0 +1,22 @@
1
+ import { ENTITY_CAPABILITIES } from "./constants.mjs";
2
+ //#region src/capabilities.ts
3
+ const CAPABILITY_MANIFEST_SCHEMA_VERSION = 1;
4
+ const CAPABILITY_RUNTIMES = [
5
+ "node",
6
+ "python",
7
+ "wasm"
8
+ ];
9
+ /**
10
+ * Versioned, runtime-free discovery contract for the deterministic pipeline.
11
+ * The manifest contains no accuracy claims; it describes available labels,
12
+ * activation, provenance sources, and runtime parity only.
13
+ */
14
+ const CAPABILITY_MANIFEST = {
15
+ schemaVersion: 1,
16
+ runtimes: CAPABILITY_RUNTIMES,
17
+ entities: ENTITY_CAPABILITIES
18
+ };
19
+ //#endregion
20
+ export { CAPABILITY_MANIFEST, CAPABILITY_MANIFEST_SCHEMA_VERSION, CAPABILITY_RUNTIMES };
21
+
22
+ //# sourceMappingURL=capabilities.mjs.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"capabilities.mjs","names":[],"sources":["../src/capabilities.ts"],"sourcesContent":["import { ENTITY_CAPABILITIES } from \"./constants\";\n\nexport const CAPABILITY_MANIFEST_SCHEMA_VERSION = 1 as const;\n\nexport const CAPABILITY_RUNTIMES = [\"node\", \"python\", \"wasm\"] as const;\n\nexport type CapabilityRuntime = (typeof CAPABILITY_RUNTIMES)[number];\n\nexport type CapabilityManifest = {\n schemaVersion: typeof CAPABILITY_MANIFEST_SCHEMA_VERSION;\n runtimes: readonly CapabilityRuntime[];\n entities: typeof ENTITY_CAPABILITIES;\n};\n\n/**\n * Versioned, runtime-free discovery contract for the deterministic pipeline.\n * The manifest contains no accuracy claims; it describes available labels,\n * activation, provenance sources, and runtime parity only.\n */\nexport const CAPABILITY_MANIFEST = {\n schemaVersion: CAPABILITY_MANIFEST_SCHEMA_VERSION,\n runtimes: CAPABILITY_RUNTIMES,\n entities: ENTITY_CAPABILITIES,\n} as const satisfies CapabilityManifest;\n"],"mappings":";;AAEA,MAAa,qCAAqC;AAElD,MAAa,sBAAsB;CAAC;CAAQ;CAAU;AAAM;;;;;;AAe5D,MAAa,sBAAsB;CACjC,eAAA;CACA,UAAU;CACV,UAAU;AACZ"}
@@ -1,2 +1,2 @@
1
- import { a as OPERATOR_TYPES, i as DetectionSource, n as DETECTION_SOURCES, o as OperatorType, r as DETECTOR_PRIORITY, t as DEFAULT_ENTITY_LABELS } from "./constants2.mjs";
2
- export { DEFAULT_ENTITY_LABELS, DETECTION_SOURCES, DETECTOR_PRIORITY, DetectionSource, OPERATOR_TYPES, OperatorType };
1
+ import { a as DetectionSource, c as ENTITY_SELECTIONS, d as EntitySelection, f as OPERATOR_TYPES, i as DefaultEntityLabel, l as EntityCapability, n as DETECTION_SOURCES, o as ENTITY_CAPABILITIES, p as OperatorType, r as DETECTOR_PRIORITY, s as ENTITY_LABELS, t as DEFAULT_ENTITY_LABELS, u as EntityLabel } from "./constants2.mjs";
2
+ export { DEFAULT_ENTITY_LABELS, DETECTION_SOURCES, DETECTOR_PRIORITY, DefaultEntityLabel, DetectionSource, ENTITY_CAPABILITIES, ENTITY_LABELS, ENTITY_SELECTIONS, EntityCapability, EntityLabel, EntitySelection, OPERATOR_TYPES, OperatorType };
@@ -49,41 +49,171 @@ const DETECTOR_PRIORITY = {
49
49
  * Anonymization operator types. Each operator defines
50
50
  * how a confirmed entity is replaced in the output.
51
51
  */
52
- const OPERATOR_TYPES = ["replace", "redact"];
52
+ const OPERATOR_TYPES = [
53
+ "replace",
54
+ "redact",
55
+ "keep",
56
+ "mask"
57
+ ];
58
+ const ENTITY_SELECTIONS = {
59
+ DEFAULT: "default",
60
+ OPT_IN: "opt-in"
61
+ };
53
62
  /**
54
- * Canonical entity labels used across the pipeline.
55
- * NER models may use different native labels; the bench
56
- * NER wrapper maps model output to these canonical names.
63
+ * Canonical entity capabilities exposed by the deterministic native pipeline.
64
+ * `selection` describes whether the default package requests the label; opt-in
65
+ * labels have built-in detection rules but must be requested explicitly.
57
66
  *
58
67
  * These labels are ephemeral: entities are regenerated on
59
68
  * every pipeline run and never persisted to the database.
60
69
  * Renaming a label here requires no migration.
61
70
  */
62
- const DEFAULT_ENTITY_LABELS = [
63
- "person",
64
- "organization",
65
- "phone number",
66
- "address",
67
- "country",
68
- "email address",
69
- "date",
70
- "date of birth",
71
- "bank account number",
72
- "iban",
73
- "tax identification number",
74
- "identity card number",
75
- "birth number",
76
- "national identification number",
77
- "social security number",
78
- "registration number",
79
- "credit card number",
80
- "passport number",
81
- "crypto",
82
- "monetary amount",
83
- "land parcel",
84
- "misc"
71
+ const ENTITY_CAPABILITIES = [
72
+ {
73
+ label: "person",
74
+ selection: ENTITY_SELECTIONS.DEFAULT,
75
+ detectionSources: [
76
+ DETECTION_SOURCES.TRIGGER,
77
+ DETECTION_SOURCES.REGEX,
78
+ DETECTION_SOURCES.DENY_LIST,
79
+ DETECTION_SOURCES.COREFERENCE
80
+ ]
81
+ },
82
+ {
83
+ label: "organization",
84
+ selection: ENTITY_SELECTIONS.DEFAULT,
85
+ detectionSources: [
86
+ DETECTION_SOURCES.TRIGGER,
87
+ DETECTION_SOURCES.DENY_LIST,
88
+ DETECTION_SOURCES.LEGAL_FORM,
89
+ DETECTION_SOURCES.GAZETTEER,
90
+ DETECTION_SOURCES.COREFERENCE
91
+ ]
92
+ },
93
+ {
94
+ label: "phone number",
95
+ selection: ENTITY_SELECTIONS.DEFAULT,
96
+ detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER]
97
+ },
98
+ {
99
+ label: "address",
100
+ selection: ENTITY_SELECTIONS.DEFAULT,
101
+ detectionSources: [
102
+ DETECTION_SOURCES.REGEX,
103
+ DETECTION_SOURCES.TRIGGER,
104
+ DETECTION_SOURCES.DENY_LIST
105
+ ]
106
+ },
107
+ {
108
+ label: "country",
109
+ selection: ENTITY_SELECTIONS.DEFAULT,
110
+ detectionSources: [DETECTION_SOURCES.COUNTRY]
111
+ },
112
+ {
113
+ label: "email address",
114
+ selection: ENTITY_SELECTIONS.DEFAULT,
115
+ detectionSources: [DETECTION_SOURCES.REGEX]
116
+ },
117
+ {
118
+ label: "date",
119
+ selection: ENTITY_SELECTIONS.DEFAULT,
120
+ detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER]
121
+ },
122
+ {
123
+ label: "date of birth",
124
+ selection: ENTITY_SELECTIONS.DEFAULT,
125
+ detectionSources: [DETECTION_SOURCES.TRIGGER]
126
+ },
127
+ {
128
+ label: "bank account number",
129
+ selection: ENTITY_SELECTIONS.DEFAULT,
130
+ detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER]
131
+ },
132
+ {
133
+ label: "iban",
134
+ selection: ENTITY_SELECTIONS.DEFAULT,
135
+ detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER]
136
+ },
137
+ {
138
+ label: "tax identification number",
139
+ selection: ENTITY_SELECTIONS.DEFAULT,
140
+ detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER]
141
+ },
142
+ {
143
+ label: "identity card number",
144
+ selection: ENTITY_SELECTIONS.DEFAULT,
145
+ detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER]
146
+ },
147
+ {
148
+ label: "birth number",
149
+ selection: ENTITY_SELECTIONS.DEFAULT,
150
+ detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER]
151
+ },
152
+ {
153
+ label: "national identification number",
154
+ selection: ENTITY_SELECTIONS.DEFAULT,
155
+ detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER]
156
+ },
157
+ {
158
+ label: "social security number",
159
+ selection: ENTITY_SELECTIONS.DEFAULT,
160
+ detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER]
161
+ },
162
+ {
163
+ label: "registration number",
164
+ selection: ENTITY_SELECTIONS.DEFAULT,
165
+ detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER]
166
+ },
167
+ {
168
+ label: "credit card number",
169
+ selection: ENTITY_SELECTIONS.DEFAULT,
170
+ detectionSources: [DETECTION_SOURCES.REGEX]
171
+ },
172
+ {
173
+ label: "passport number",
174
+ selection: ENTITY_SELECTIONS.DEFAULT,
175
+ detectionSources: [DETECTION_SOURCES.REGEX]
176
+ },
177
+ {
178
+ label: "crypto",
179
+ selection: ENTITY_SELECTIONS.DEFAULT,
180
+ detectionSources: [DETECTION_SOURCES.REGEX]
181
+ },
182
+ {
183
+ label: "monetary amount",
184
+ selection: ENTITY_SELECTIONS.DEFAULT,
185
+ detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER]
186
+ },
187
+ {
188
+ label: "land parcel",
189
+ selection: ENTITY_SELECTIONS.DEFAULT,
190
+ detectionSources: [DETECTION_SOURCES.TRIGGER]
191
+ },
192
+ {
193
+ label: "misc",
194
+ selection: ENTITY_SELECTIONS.DEFAULT,
195
+ detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.DENY_LIST]
196
+ },
197
+ {
198
+ label: "ip address",
199
+ selection: ENTITY_SELECTIONS.OPT_IN,
200
+ detectionSources: [DETECTION_SOURCES.REGEX]
201
+ },
202
+ {
203
+ label: "mac address",
204
+ selection: ENTITY_SELECTIONS.OPT_IN,
205
+ detectionSources: [DETECTION_SOURCES.REGEX]
206
+ },
207
+ {
208
+ label: "url",
209
+ selection: ENTITY_SELECTIONS.OPT_IN,
210
+ detectionSources: [DETECTION_SOURCES.REGEX]
211
+ }
85
212
  ];
213
+ const ENTITY_LABELS = ENTITY_CAPABILITIES.map(({ label }) => label);
214
+ const isDefaultEntityCapability = (capability) => capability.selection === ENTITY_SELECTIONS.DEFAULT;
215
+ const DEFAULT_ENTITY_LABELS = ENTITY_CAPABILITIES.filter(isDefaultEntityCapability).map(({ label }) => label);
86
216
  //#endregion
87
- export { DEFAULT_ENTITY_LABELS, DETECTION_SOURCES, DETECTOR_PRIORITY, OPERATOR_TYPES };
217
+ export { DEFAULT_ENTITY_LABELS, DETECTION_SOURCES, DETECTOR_PRIORITY, ENTITY_CAPABILITIES, ENTITY_LABELS, ENTITY_SELECTIONS, OPERATOR_TYPES };
88
218
 
89
219
  //# sourceMappingURL=constants.mjs.map
@@ -1 +1 @@
1
- {"version":3,"file":"constants.mjs","names":[],"sources":["../src/constants.ts"],"sourcesContent":["/**\n * Runtime-free constants for the anonymization pipeline.\n *\n * This module is the SSR-safe / browser-safe entrypoint:\n * importing it must not pull in `@stll/text-search`,\n * `@stll/anonymize-wasm`, or any other runtime-bearing\n * module. Consumers that only need the static label list,\n * detection-source identifiers, or operator names import\n * from `@stll/anonymize/constants` (or\n * `@stll/anonymize-wasm/constants`) without paying the\n * wasm / regex-set startup cost.\n *\n * `types.ts` re-exports these for back-compat, so existing\n * `import { DEFAULT_ENTITY_LABELS } from \"@stll/anonymize\"`\n * call sites keep working.\n */\n\n/**\n * Source of a detected entity span.\n * Ordered by detection layer in the pipeline.\n */\nexport const DETECTION_SOURCES = {\n TRIGGER: \"trigger\",\n REGEX: \"regex\",\n DENY_LIST: \"deny-list\",\n LEGAL_FORM: \"legal-form\",\n GAZETTEER: \"gazetteer\",\n COUNTRY: \"country\",\n NER: \"ner\",\n COREFERENCE: \"coreference\",\n} as const;\n\nexport type DetectionSource =\n (typeof DETECTION_SOURCES)[keyof typeof DETECTION_SOURCES];\n\n/**\n * Priority levels for detection sources.\n * Higher = more structurally reliable. Used during\n * overlap resolution so deterministic detectors beat\n * probabilistic ones regardless of raw score.\n */\nexport const DETECTOR_PRIORITY = {\n [DETECTION_SOURCES.GAZETTEER]: 5,\n [DETECTION_SOURCES.TRIGGER]: 4,\n [DETECTION_SOURCES.LEGAL_FORM]: 3,\n [DETECTION_SOURCES.REGEX]: 3,\n [DETECTION_SOURCES.COUNTRY]: 3,\n [DETECTION_SOURCES.DENY_LIST]: 2,\n [DETECTION_SOURCES.COREFERENCE]: 2,\n [DETECTION_SOURCES.NER]: 1,\n} as const satisfies Record<DetectionSource, number>;\n\n/**\n * Anonymization operator types. Each operator defines\n * how a confirmed entity is replaced in the output.\n */\nexport const OPERATOR_TYPES = [\"replace\", \"redact\"] as const;\n\nexport type OperatorType = (typeof OPERATOR_TYPES)[number];\n\n/**\n * Canonical entity labels used across the pipeline.\n * NER models may use different native labels; the bench\n * NER wrapper maps model output to these canonical names.\n *\n * These labels are ephemeral: entities are regenerated on\n * every pipeline run and never persisted to the database.\n * Renaming a label here requires no migration.\n */\nexport const DEFAULT_ENTITY_LABELS = [\n \"person\",\n \"organization\",\n \"phone number\",\n \"address\",\n \"country\",\n \"email address\",\n \"date\",\n \"date of birth\",\n \"bank account number\",\n \"iban\",\n \"tax identification number\",\n \"identity card number\",\n \"birth number\",\n \"national identification number\",\n \"social security number\",\n \"registration number\",\n \"credit card number\",\n \"passport number\",\n \"crypto\",\n \"monetary amount\",\n \"land parcel\",\n \"misc\",\n] as const;\n"],"mappings":";;;;;;;;;;;;;;;;;;;;;AAqBA,MAAa,oBAAoB;CAC/B,SAAS;CACT,OAAO;CACP,WAAW;CACX,YAAY;CACZ,WAAW;CACX,SAAS;CACT,KAAK;CACL,aAAa;AACf;;;;;;;AAWA,MAAa,oBAAoB;EAC9B,kBAAkB,YAAY;EAC9B,kBAAkB,UAAU;EAC5B,kBAAkB,aAAa;EAC/B,kBAAkB,QAAQ;EAC1B,kBAAkB,UAAU;EAC5B,kBAAkB,YAAY;EAC9B,kBAAkB,cAAc;EAChC,kBAAkB,MAAM;AAC3B;;;;;AAMA,MAAa,iBAAiB,CAAC,WAAW,QAAQ;;;;;;;;;;AAalD,MAAa,wBAAwB;CACnC;CACA;CACA;CACA;CACA;CACA;CACA;CACA;CACA;CACA;CACA;CACA;CACA;CACA;CACA;CACA;CACA;CACA;CACA;CACA;CACA;CACA;AACF"}
1
+ {"version":3,"file":"constants.mjs","names":[],"sources":["../src/constants.ts"],"sourcesContent":["/**\n * Runtime-free constants for the anonymization pipeline.\n *\n * This module is the SSR-safe / browser-safe entrypoint:\n * importing it must not pull in `@stll/text-search`,\n * `@stll/anonymize-wasm`, or any other runtime-bearing\n * module. Consumers that only need the static label list,\n * detection-source identifiers, or operator names import\n * from `@stll/anonymize/constants` (or\n * `@stll/anonymize-wasm/constants`) without paying the\n * wasm / regex-set startup cost.\n *\n * `types.ts` re-exports these for back-compat, so existing\n * `import { DEFAULT_ENTITY_LABELS } from \"@stll/anonymize\"`\n * call sites keep working.\n */\n\n/**\n * Source of a detected entity span.\n * Ordered by detection layer in the pipeline.\n */\nexport const DETECTION_SOURCES = {\n TRIGGER: \"trigger\",\n REGEX: \"regex\",\n DENY_LIST: \"deny-list\",\n LEGAL_FORM: \"legal-form\",\n GAZETTEER: \"gazetteer\",\n COUNTRY: \"country\",\n NER: \"ner\",\n COREFERENCE: \"coreference\",\n} as const;\n\nexport type DetectionSource =\n (typeof DETECTION_SOURCES)[keyof typeof DETECTION_SOURCES];\n\n/**\n * Priority levels for detection sources.\n * Higher = more structurally reliable. Used during\n * overlap resolution so deterministic detectors beat\n * probabilistic ones regardless of raw score.\n */\nexport const DETECTOR_PRIORITY = {\n [DETECTION_SOURCES.GAZETTEER]: 5,\n [DETECTION_SOURCES.TRIGGER]: 4,\n [DETECTION_SOURCES.LEGAL_FORM]: 3,\n [DETECTION_SOURCES.REGEX]: 3,\n [DETECTION_SOURCES.COUNTRY]: 3,\n [DETECTION_SOURCES.DENY_LIST]: 2,\n [DETECTION_SOURCES.COREFERENCE]: 2,\n [DETECTION_SOURCES.NER]: 1,\n} as const satisfies Record<DetectionSource, number>;\n\n/**\n * Anonymization operator types. Each operator defines\n * how a confirmed entity is replaced in the output.\n */\nexport const OPERATOR_TYPES = [\"replace\", \"redact\", \"keep\", \"mask\"] as const;\n\nexport type OperatorType = (typeof OPERATOR_TYPES)[number];\n\nexport const ENTITY_SELECTIONS = {\n DEFAULT: \"default\",\n OPT_IN: \"opt-in\",\n} as const;\n\nexport type EntitySelection =\n (typeof ENTITY_SELECTIONS)[keyof typeof ENTITY_SELECTIONS];\n\nexport type EntityCapability = {\n label: string;\n selection: EntitySelection;\n detectionSources: readonly DetectionSource[];\n};\n\n/**\n * Canonical entity capabilities exposed by the deterministic native pipeline.\n * `selection` describes whether the default package requests the label; opt-in\n * labels have built-in detection rules but must be requested explicitly.\n *\n * These labels are ephemeral: entities are regenerated on\n * every pipeline run and never persisted to the database.\n * Renaming a label here requires no migration.\n */\nexport const ENTITY_CAPABILITIES = [\n {\n label: \"person\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [\n DETECTION_SOURCES.TRIGGER,\n DETECTION_SOURCES.REGEX,\n DETECTION_SOURCES.DENY_LIST,\n DETECTION_SOURCES.COREFERENCE,\n ],\n },\n {\n label: \"organization\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [\n DETECTION_SOURCES.TRIGGER,\n DETECTION_SOURCES.DENY_LIST,\n DETECTION_SOURCES.LEGAL_FORM,\n DETECTION_SOURCES.GAZETTEER,\n DETECTION_SOURCES.COREFERENCE,\n ],\n },\n {\n label: \"phone number\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER],\n },\n {\n label: \"address\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [\n DETECTION_SOURCES.REGEX,\n DETECTION_SOURCES.TRIGGER,\n DETECTION_SOURCES.DENY_LIST,\n ],\n },\n {\n label: \"country\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [DETECTION_SOURCES.COUNTRY],\n },\n {\n label: \"email address\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [DETECTION_SOURCES.REGEX],\n },\n {\n label: \"date\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER],\n },\n {\n label: \"date of birth\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [DETECTION_SOURCES.TRIGGER],\n },\n {\n label: \"bank account number\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER],\n },\n {\n label: \"iban\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER],\n },\n {\n label: \"tax identification number\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER],\n },\n {\n label: \"identity card number\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER],\n },\n {\n label: \"birth number\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER],\n },\n {\n label: \"national identification number\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER],\n },\n {\n label: \"social security number\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER],\n },\n {\n label: \"registration number\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER],\n },\n {\n label: \"credit card number\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [DETECTION_SOURCES.REGEX],\n },\n {\n label: \"passport number\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [DETECTION_SOURCES.REGEX],\n },\n {\n label: \"crypto\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [DETECTION_SOURCES.REGEX],\n },\n {\n label: \"monetary amount\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.TRIGGER],\n },\n {\n label: \"land parcel\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [DETECTION_SOURCES.TRIGGER],\n },\n {\n label: \"misc\",\n selection: ENTITY_SELECTIONS.DEFAULT,\n detectionSources: [DETECTION_SOURCES.REGEX, DETECTION_SOURCES.DENY_LIST],\n },\n {\n label: \"ip address\",\n selection: ENTITY_SELECTIONS.OPT_IN,\n detectionSources: [DETECTION_SOURCES.REGEX],\n },\n {\n label: \"mac address\",\n selection: ENTITY_SELECTIONS.OPT_IN,\n detectionSources: [DETECTION_SOURCES.REGEX],\n },\n {\n label: \"url\",\n selection: ENTITY_SELECTIONS.OPT_IN,\n detectionSources: [DETECTION_SOURCES.REGEX],\n },\n] as const satisfies readonly EntityCapability[];\n\ntype KnownEntityCapability = (typeof ENTITY_CAPABILITIES)[number];\n\nexport type EntityLabel = KnownEntityCapability[\"label\"];\n\nexport const ENTITY_LABELS: readonly EntityLabel[] = ENTITY_CAPABILITIES.map(\n ({ label }) => label,\n);\n\nconst isDefaultEntityCapability = (\n capability: KnownEntityCapability,\n): capability is Extract<\n KnownEntityCapability,\n { selection: typeof ENTITY_SELECTIONS.DEFAULT }\n> => capability.selection === ENTITY_SELECTIONS.DEFAULT;\n\nexport type DefaultEntityLabel = Extract<\n KnownEntityCapability,\n { selection: typeof ENTITY_SELECTIONS.DEFAULT }\n>[\"label\"];\n\nexport const DEFAULT_ENTITY_LABELS: readonly DefaultEntityLabel[] =\n ENTITY_CAPABILITIES.filter(isDefaultEntityCapability).map(\n ({ label }) => label,\n );\n"],"mappings":";;;;;;;;;;;;;;;;;;;;;AAqBA,MAAa,oBAAoB;CAC/B,SAAS;CACT,OAAO;CACP,WAAW;CACX,YAAY;CACZ,WAAW;CACX,SAAS;CACT,KAAK;CACL,aAAa;AACf;;;;;;;AAWA,MAAa,oBAAoB;EAC9B,kBAAkB,YAAY;EAC9B,kBAAkB,UAAU;EAC5B,kBAAkB,aAAa;EAC/B,kBAAkB,QAAQ;EAC1B,kBAAkB,UAAU;EAC5B,kBAAkB,YAAY;EAC9B,kBAAkB,cAAc;EAChC,kBAAkB,MAAM;AAC3B;;;;;AAMA,MAAa,iBAAiB;CAAC;CAAW;CAAU;CAAQ;AAAM;AAIlE,MAAa,oBAAoB;CAC/B,SAAS;CACT,QAAQ;AACV;;;;;;;;;;AAoBA,MAAa,sBAAsB;CACjC;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB;GAChB,kBAAkB;GAClB,kBAAkB;GAClB,kBAAkB;GAClB,kBAAkB;EACpB;CACF;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB;GAChB,kBAAkB;GAClB,kBAAkB;GAClB,kBAAkB;GAClB,kBAAkB;GAClB,kBAAkB;EACpB;CACF;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,OAAO,kBAAkB,OAAO;CACvE;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB;GAChB,kBAAkB;GAClB,kBAAkB;GAClB,kBAAkB;EACpB;CACF;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,OAAO;CAC9C;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,KAAK;CAC5C;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,OAAO,kBAAkB,OAAO;CACvE;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,OAAO;CAC9C;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,OAAO,kBAAkB,OAAO;CACvE;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,OAAO,kBAAkB,OAAO;CACvE;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,OAAO,kBAAkB,OAAO;CACvE;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,OAAO,kBAAkB,OAAO;CACvE;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,OAAO,kBAAkB,OAAO;CACvE;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,OAAO,kBAAkB,OAAO;CACvE;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,OAAO,kBAAkB,OAAO;CACvE;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,OAAO,kBAAkB,OAAO;CACvE;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,KAAK;CAC5C;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,KAAK;CAC5C;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,KAAK;CAC5C;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,OAAO,kBAAkB,OAAO;CACvE;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,OAAO;CAC9C;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,OAAO,kBAAkB,SAAS;CACzE;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,KAAK;CAC5C;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,KAAK;CAC5C;CACA;EACE,OAAO;EACP,WAAW,kBAAkB;EAC7B,kBAAkB,CAAC,kBAAkB,KAAK;CAC5C;AACF;AAMA,MAAa,gBAAwC,oBAAoB,KACtE,EAAE,YAAY,KACjB;AAEA,MAAM,6BACJ,eAIG,WAAW,cAAc,kBAAkB;AAOhD,MAAa,wBACX,oBAAoB,OAAO,yBAAyB,CAAC,CAAC,KACnD,EAAE,YAAY,KACjB"}