@studio-foundation/anonymizer 0.3.0-beta.6 → 0.5.2

package/README.md

@@ -1,20 +1,25 @@
 # @studio-foundation/anonymizer
 
-PII detection and anonymization library. Replaces sensitive data with consistent tokens before sending to LLMs, with a keymap to restore the original values afterward.
+**Studio** is a declarative YAML runtime for AI agents. It orchestrates multi-stage agent workflows with structured output validation and automatic retry. This package is the **anonymizer**: a PII detection and anonymization library that replaces sensitive data with consistent tokens before sending to LLMs, with a keymap to restore the original values afterward.
 
-## Role
+anonymizer sits at the bottom of the stack, a pure utility with zero `@studio-foundation/*` dependencies. It's usable standalone in any TypeScript project, and it's what powers `anonymize: true` in Studio agent configs.
 
-anonymizer sits at the bottom of the stack — a pure utility with zero `@studio/*` dependencies. The runner wraps it in `AnonymizationMiddleware` and injects it transparently into the agent execution loop.
+- Homepage: https://github.com/studio-foundation/studio
+- Full docs: [README](https://github.com/studio-foundation/studio#readme) · [CONCEPTS](https://github.com/studio-foundation/studio/blob/main/CONCEPTS.md)
+- Use via the CLI: [`@studio-foundation/cli`](https://www.npmjs.com/package/@studio-foundation/cli)
 
-```
-user data → anonymize() → [PERSON_1], [EMAIL_1] → LLM → deanonymize() → original values
+## Install
+
+```bash
+npm install @studio-foundation/anonymizer
+# or
+pnpm add @studio-foundation/anonymizer
 ```
 
-## Key exports
+## Quick start
 
 ```typescript
-import { anonymize, deanonymize, Tokenizer } from '@studio-foundation/anonymizer';
-import type { PIICategory, PIIDetectionResult, AnonymizerOptions } from '@studio-foundation/anonymizer';
+import { anonymize, deanonymize } from '@studio-foundation/anonymizer';
 
 // Anonymize a string
 const { text, keymap } = anonymize('Hi Marie, call me at 555-867-5309');
@@ -30,26 +35,36 @@ const { text: text2, keymap: keymap2 } = anonymize(nextChunk, { seedKeymap: keym
 // PERSON_1 still maps to "Marie" across calls
 ```
 
+```
+user data → anonymize() → [PERSON_1], [EMAIL_1] → LLM → deanonymize() → original values
+```
+
 ## PII categories
 
 | Category | Token format | What it detects |
 |----------|-------------|-----------------|
-| `person` | `PERSON_N` | Names after salutations (Dear, Hi, Mr., Dr., etc.) — best effort |
+| `person` | `PERSON_N` | Names after salutations (Dear, Hi, Mr., Dr., etc.) (best effort) |
 | `email` | `EMAIL_N` | Email addresses |
 | `phone` | `PHONE_N` | US phone numbers (10 digits, various formats) |
 | `ssn` | `SSN_N` | Social security numbers (ddd-dd-dddd) |
 | `credit_card` | `CREDIT_CARD_N` | 16-digit card numbers |
-| `address` | `ADDRESS_N` | Reserved (detection not yet implemented) |
+| `address` | `ADDRESS_N` | Reserved — delegated to future NER detector (not regex-detected) |
 
 ## Detection strategy
 
 Two-phase detection on each call:
 
-1. **Regex (high precision)** — email, phone, SSN, credit card. Structural patterns anchored to avoid false positives (e.g. SSN regex uses strict hyphen format to avoid matching phone fragments).
+1. **Regex (high precision)**: email, phone, SSN, credit card. Structural patterns anchored to avoid false positives (e.g. SSN regex uses strict hyphen format to avoid matching phone fragments).
 
-2. **Person names (best effort)** — salutation-gated pattern (`Dear X`, `Hi X`, `Mr. X`, etc.). Only catches explicitly addressed names — not bare occurrences.
+2. **Person names (best effort)**: salutation-gated pattern (`Dear X`, `Hi X`, `Mr. X`, etc.). Only catches explicitly addressed names, not bare occurrences.
 
-Spans are non-overlapping. If two patterns match the same range, the first one wins and the position is marked occupied.
+Spans are non-overlapping. Overlaps are resolved by an explicit priority
+(credit_card > ssn > email > phone > person, most-specific wins): the
+highest-priority match takes the whole span and overlapping lower-priority
+matches are dropped. Detection is pluggable via the `DetectionProvider`
+interface (`detect(text) → Promise<Span[]>`); `RegexDetector` is the built-in
+provider. Person detection covers FR + EN salutations. `address` is not
+detected by regex — it is delegated to a future NER detector.
 
 ## Token consistency
 
@@ -81,10 +96,16 @@ The runner wraps this in `AnonymizationMiddleware` (in `runner/src/middleware/an
 2. Tool results are anonymized before being injected back into context
 3. The accumulated keymap is written to `.studio/runs/anonymization/<run-id>.keymap.json` for post-run inspection
 
-The middleware is wired by the engine and passed to `runAgent()` — user code doesn't call `anonymize()` directly.
+The middleware is wired by the engine and passed to `runAgent()`, user code doesn't call `anonymize()` directly when using the Studio CLI.
+
+## For contributors
+
+Internal rules that govern this package:
+
+- **Zero `@studio-foundation/*` dependencies.** This package must stay a pure utility.
+- `anonymize()` is stateless, the `Tokenizer` is created fresh each call (or seeded via `seedKeymap`).
+- Person detection is always best-effort and non-fatal, failures are silently skipped.
 
-## Rules
+## License
 
-- **Zero `@studio/*` dependencies.** This package must stay a pure utility.
-- `anonymize()` is stateless — the `Tokenizer` is created fresh each call (or seeded via `seedKeymap`).
-- Person detection is always best-effort and non-fatal — failures are silently skipped.
+AGPL-3.0-only

package/dist/detection/provider.d.ts

@@ -0,0 +1,18 @@
+/**
+ * A detected PII span — POSITIONS only. The kernel reconstructs the value via
+ * text.slice(start, end). `type` is a free string so a future network provider
+ * (Presidio/NER) can return its own vocabulary without changing this interface.
+ */
+export interface Span {
+    start: number;
+    end: number;
+    type: string;
+}
+/**
+ * Pluggable PII detection. Async by default so the same contract covers an
+ * in-process detector (regex) and a future network provider (Presidio HTTP).
+ */
+export interface DetectionProvider {
+    detect(text: string): Promise<Span[]>;
+}
+//# sourceMappingURL=provider.d.ts.map

package/dist/detection/provider.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"provider.d.ts","sourceRoot":"","sources":["../../src/detection/provider.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AACH,MAAM,WAAW,IAAI;IACnB,KAAK,EAAE,MAAM,CAAC;IACd,GAAG,EAAE,MAAM,CAAC;IACZ,IAAI,EAAE,MAAM,CAAC;CACd;AAED;;;GAGG;AACH,MAAM,WAAW,iBAAiB;IAChC,MAAM,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,EAAE,CAAC,CAAC;CACvC"}

package/dist/detection/provider.js

@@ -0,0 +1,3 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+//# sourceMappingURL=provider.js.map

package/dist/detection/provider.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"provider.js","sourceRoot":"","sources":["../../src/detection/provider.ts"],"names":[],"mappings":""}

package/dist/detection/regex-detector.d.ts

@@ -0,0 +1,13 @@
+import type { DetectionProvider, Span } from './provider.js';
+/**
+ * Base in-process detector. Covers the types where regex is reliable
+ * (formatted identifiers): email, phone, ssn, credit_card, and salutation-based
+ * person (FR + EN).
+ *
+ * address is intentionally delegated to a future NER detector — see PRIORITY in
+ * regex-matcher.ts. Do NOT add an address regex here.
+ */
+export declare class RegexDetector implements DetectionProvider {
+    detect(text: string): Promise<Span[]>;
+}
+//# sourceMappingURL=regex-detector.d.ts.map

package/dist/detection/regex-detector.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"regex-detector.d.ts","sourceRoot":"","sources":["../../src/detection/regex-detector.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,iBAAiB,EAAE,IAAI,EAAE,MAAM,eAAe,CAAC;AAG7D;;;;;;;GAOG;AACH,qBAAa,aAAc,YAAW,iBAAiB;IAC/C,MAAM,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,EAAE,CAAC;CAG5C"}

package/dist/detection/regex-detector.js

@@ -0,0 +1,19 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.RegexDetector = void 0;
+const regex_matcher_js_1 = require("./regex-matcher.js");
+/**
+ * Base in-process detector. Covers the types where regex is reliable
+ * (formatted identifiers): email, phone, ssn, credit_card, and salutation-based
+ * person (FR + EN).
+ *
+ * address is intentionally delegated to a future NER detector — see PRIORITY in
+ * regex-matcher.ts. Do NOT add an address regex here.
+ */
+class RegexDetector {
+    async detect(text) {
+        return (0, regex_matcher_js_1.matchPII)(text).map(({ start, end, type }) => ({ start, end, type }));
+    }
+}
+exports.RegexDetector = RegexDetector;
+//# sourceMappingURL=regex-detector.js.map

package/dist/detection/regex-detector.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"regex-detector.js","sourceRoot":"","sources":["../../src/detection/regex-detector.ts"],"names":[],"mappings":";;;AACA,yDAA8C;AAE9C;;;;;;;GAOG;AACH,MAAa,aAAa;IACxB,KAAK,CAAC,MAAM,CAAC,IAAY;QACvB,OAAO,IAAA,2BAAQ,EAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,KAAK,EAAE,GAAG,EAAE,IAAI,EAAE,EAAE,EAAE,CAAC,CAAC,EAAE,KAAK,EAAE,GAAG,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC;IAC9E,CAAC;CACF;AAJD,sCAIC"}

package/dist/detection/regex-matcher.d.ts

@@ -0,0 +1,9 @@
+export type PIIType = 'email' | 'phone' | 'ssn' | 'credit_card' | 'person';
+export interface RegexMatch {
+    start: number;
+    end: number;
+    type: PIIType;
+}
+export declare function resolveByPriority(candidates: RegexMatch[]): RegexMatch[];
+export declare function matchPII(text: string): RegexMatch[];
+//# sourceMappingURL=regex-matcher.d.ts.map

package/dist/detection/regex-matcher.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"regex-matcher.d.ts","sourceRoot":"","sources":["../../src/detection/regex-matcher.ts"],"names":[],"mappings":"AAKA,MAAM,MAAM,OAAO,GAAG,OAAO,GAAG,OAAO,GAAG,KAAK,GAAG,aAAa,GAAG,QAAQ,CAAC;AAE3E,MAAM,WAAW,UAAU;IACzB,KAAK,EAAE,MAAM,CAAC;IACd,GAAG,EAAE,MAAM,CAAC;IACZ,IAAI,EAAE,OAAO,CAAC;CACf;AAyFD,wBAAgB,iBAAiB,CAAC,UAAU,EAAE,UAAU,EAAE,GAAG,UAAU,EAAE,CAaxE;AAED,wBAAgB,QAAQ,CAAC,IAAI,EAAE,MAAM,GAAG,UAAU,EAAE,CAMnD"}

package/dist/detection/regex-matcher.js

@@ -0,0 +1,110 @@
+"use strict";
+// Shared PRIVATE regex internal for the anonymizer package.
+// Returns POSITIONS only — never the extracted substring. The kernel
+// reconstructs the value via text.slice(start, end); see detector.ts.
+// NOT exported from index.ts.
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.resolveByPriority = resolveByPriority;
+exports.matchPII = matchPII;
+// Most-specific format wins. credit_card beats ssn on the dangerous digit-run
+// collision; email is near-unambiguous (the @) so it stays high; phone is the
+// most permissive of the formatted types so it runs late; person is a
+// salutation heuristic, last, and only claims what the others leave.
+//
+// address: intentionally NOT here. Addresses are the least regex-able PII type;
+// a number+word+street-suffix pattern false-positives on non-addresses, and in
+// an email classifier a false positive (tokenizing legitimate text) is strictly
+// worse than a miss. Address detection is delegated to a future NER detector
+// (horizon 2). Do NOT add an address regex here.
+const PRIORITY = ['credit_card', 'ssn', 'email', 'phone', 'person'];
+const FORMAT_PATTERNS = {
+    credit_card: /\b(?:\d{4}[-\s]?){3}\d{4}\b/g,
+    ssn: /\b\d{3}-\d{2}-\d{4}\b/g,
+    email: /\b[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,}\b/g,
+    phone: /\b(?:\+?1[-.\s]?)?\(?\d{3}\)?[-.\s]\d{3}[-.\s]\d{4}\b/g,
+};
+// Person detection is salutation-anchored (not a pure format), so it inherits
+// the deployment language. The first deployment is French-language; an EN-only list
+// would silently miss French names and leak the most sensitive PII (parents and
+// minors). FR + EN salutations ship together.
+//
+// Case-sensitivity is SPLIT across two passes, because a single regex cannot
+// be case-insensitive in one part and case-sensitive in another (the `i` flag
+// is all-or-nothing, and `i` on the name capture is exactly the bug: `[A-Z]`
+// would then also match lowercase, tokenizing `bonjour marie` as a person):
+//
+//   1. SALUTATION_TRIGGER (`/gi`) — case-insensitive so `bonjour` / `Bonjour` /
+//      `BONJOUR` all fire. It consumes the trigger *and* the whitespace after
+//      it, leaving lastIndex at the first character of the name.
+//   2. NAME (`/y`, sticky, NO `i`) — case-SENSITIVE, anchored to start exactly
+//      where the trigger ended. It requires an initial uppercase on every word,
+//      so a lowercase word after a salutation produces no name. A false positive
+//      (tokenizing legitimate text) is strictly worse than a miss in an email
+//      classifier, so we fail closed.
+//
+// (ES2025 inline flags `(?i:…)` would express this in one regex, but V8 on the
+// supported runtime does not yet parse them.)
+//
+// `\b...` anchors the trigger to a word boundary. `m\.` REQUIRES the trailing
+// period so a word merely ending in "m" before a period (e.g. "forum.") does
+// not fire.
+const SALUTATION_TRIGGER = /\b(?:dear|hi|hello|greetings|hey(?:\s+there)?|mr\.?|mrs\.?|ms\.?|dr\.?|prof\.?|bonjour|bonsoir|all[oô]|salut|ch[eè]re?s?|madame|monsieur|mme\.?|m\.)\s+/gi;
+const NAME = /[A-Z][a-zA-ZÀ-ÿ'\-]+(?:\s+[A-Z][a-zA-ZÀ-ÿ'\-]+)*/y;
+function gatherPersonCandidates(text) {
+    const out = [];
+    SALUTATION_TRIGGER.lastIndex = 0;
+    let m;
+    while ((m = SALUTATION_TRIGGER.exec(text)) !== null) {
+        const start = m.index + m[0].length;
+        // Sticky: matches only if a Title-cased name begins exactly at `start`.
+        NAME.lastIndex = start;
+        const nameMatch = NAME.exec(text);
+        if (!nameMatch)
+            continue;
+        out.push({ start, end: start + nameMatch[0].length, type: 'person' });
+    }
+    return out;
+}
+function gatherFormatCandidates(text) {
+    const out = [];
+    for (const type of Object.keys(FORMAT_PATTERNS)) {
+        const re = FORMAT_PATTERNS[type];
+        re.lastIndex = 0;
+        let m;
+        while ((m = re.exec(text)) !== null) {
+            out.push({ start: m.index, end: m.index + m[0].length, type });
+        }
+    }
+    return out;
+}
+function rangeOccupied(occupied, start, end) {
+    // True interval intersection: ANY position in [start, end) being occupied
+    // means overlap. Never a start-only check.
+    for (let i = start; i < end; i++) {
+        if (occupied.has(i))
+            return true;
+    }
+    return false;
+}
+function resolveByPriority(candidates) {
+    const rank = (t) => PRIORITY.indexOf(t);
+    const ordered = [...candidates].sort((a, b) => rank(a.type) - rank(b.type) || a.start - b.start);
+    const occupied = new Set();
+    const accepted = [];
+    for (const c of ordered) {
+        if (rangeOccupied(occupied, c.start, c.end))
+            continue;
+        for (let i = c.start; i < c.end; i++)
+            occupied.add(i);
+        accepted.push(c);
+    }
+    return accepted.sort((a, b) => a.start - b.start);
+}
+function matchPII(text) {
+    const candidates = [
+        ...gatherFormatCandidates(text),
+        ...gatherPersonCandidates(text),
+    ];
+    return resolveByPriority(candidates);
+}
+//# sourceMappingURL=regex-matcher.js.map

package/dist/detection/regex-matcher.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"regex-matcher.js","sourceRoot":"","sources":["../../src/detection/regex-matcher.ts"],"names":[],"mappings":";AAAA,4DAA4D;AAC5D,qEAAqE;AACrE,sEAAsE;AACtE,8BAA8B;;AAiG9B,8CAaC;AAED,4BAMC;AA5GD,8EAA8E;AAC9E,8EAA8E;AAC9E,sEAAsE;AACtE,qEAAqE;AACrE,EAAE;AACF,gFAAgF;AAChF,+EAA+E;AAC/E,gFAAgF;AAChF,6EAA6E;AAC7E,iDAAiD;AACjD,MAAM,QAAQ,GAAuB,CAAC,aAAa,EAAE,KAAK,EAAE,OAAO,EAAE,OAAO,EAAE,QAAQ,CAAC,CAAC;AAIxF,MAAM,eAAe,GAA+B;IAClD,WAAW,EAAE,8BAA8B;IAC3C,GAAG,EAAE,wBAAwB;IAC7B,KAAK,EAAE,uDAAuD;IAC9D,KAAK,EAAE,wDAAwD;CAChE,CAAC;AAEF,8EAA8E;AAC9E,oFAAoF;AACpF,gFAAgF;AAChF,8CAA8C;AAC9C,EAAE;AACF,6EAA6E;AAC7E,8EAA8E;AAC9E,6EAA6E;AAC7E,4EAA4E;AAC5E,EAAE;AACF,gFAAgF;AAChF,8EAA8E;AAC9E,iEAAiE;AACjE,+EAA+E;AAC/E,gFAAgF;AAChF,iFAAiF;AACjF,8EAA8E;AAC9E,sCAAsC;AACtC,EAAE;AACF,+EAA+E;AAC/E,8CAA8C;AAC9C,EAAE;AACF,8EAA8E;AAC9E,6EAA6E;AAC7E,YAAY;AACZ,MAAM,kBAAkB,GACtB,2JAA2J,CAAC;AAC9J,MAAM,IAAI,GAAG,mDAAmD,CAAC;AAEjE,SAAS,sBAAsB,CAAC,IAAY;IAC1C,MAAM,GAAG,GAAiB,EAAE,CAAC;IAC7B,kBAAkB,CAAC,SAAS,GAAG,CAAC,CAAC;IACjC,IAAI,CAAyB,CAAC;IAC9B,OAAO,CAAC,CAAC,GAAG,kBAAkB,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,KAAK,IAAI,EAAE,CAAC;QACpD,MAAM,KAAK,GAAG,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC;QACpC,wEAAwE;QACxE,IAAI,CAAC,SAAS,GAAG,KAAK,CAAC;QACvB,MAAM,SAAS,GAAG,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QAClC,IAAI,CAAC,SAAS;YAAE,SAAS;QACzB,GAAG,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,GAAG,SAAS,CAAC,CAAC,CAAC,CAAC,MAAM,EAAE,IAAI,EAAE,QAAQ,EAAE,CAAC,CAAC;IACxE,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED,SAAS,sBAAsB,CAAC,IAAY;IAC1C,MAAM,GAAG,GAAiB,EAAE,CAAC;IAC7B,KAAK,MAAM,IAAI,IAAI,MAAM,CAAC,IAAI,CAAC,eAAe,CAAiB,EAAE,CAAC;QAChE,MAAM,EAAE,GAAG,eAAe,CAAC,IAAI,CAAC,CAAC;QACjC,EAAE,CAAC,SAAS,GAAG,CAAC,CAAC;QACjB,IAAI,CAAyB,CAAC;QAC9B,OAAO,CAAC,CAAC,GAAG,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,KAAK,IAAI,EAAE,CAAC;YACpC,GAAG,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,KAAK,EAAE,GAAG,EAAE,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC;QACjE,CAAC;IACH,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED,SAAS,aAAa,CAAC,QAAqB,EAAE,KAAa,EAAE,GAAW;IACtE,0EAA0E;IAC1E,2CAA2C;IAC3C,KAAK,IAAI,CAAC,GAAG,KAAK,EAAE,CAAC,GAAG,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC;QACjC,IAAI,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC;YAAE,OAAO,IAAI,CAAC;IACnC,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,SAAgB,iBAAiB,CAAC,UAAwB;IACxD,MAAM,IAAI,GAAG,CAAC,CAAU,EAAU,EAAE,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;IACzD,MAAM,OAAO,GAAG,CAAC,GAAG,UAAU,CAAC,CAAC,IAAI,CAClC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,CAC3D,CAAC;IACF,MAAM,QAAQ,GAAG,IAAI,GAAG,EAAU,CAAC;IACnC,MAAM,QAAQ,GAAiB,EAAE,CAAC;IAClC,KAAK,MAAM,CAAC,IAAI,OAAO,EAAE,CAAC;QACxB,IAAI,aAAa,CAAC,QAAQ,EAAE,CAAC,CAAC,KAAK,EAAE,CAAC,CAAC,GAAG,CAAC;YAAE,SAAS;QACtD,KAAK,IAAI,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE;YAAE,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC;QACtD,QAAQ,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;IACnB,CAAC;IACD,OAAO,QAAQ,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC;AACpD,CAAC;AAED,SAAgB,QAAQ,CAAC,IAAY;IACnC,MAAM,UAAU,GAAG;QACjB,GAAG,sBAAsB,CAAC,IAAI,CAAC;QAC/B,GAAG,sBAAsB,CAAC,IAAI,CAAC;KAChC,CAAC;IACF,OAAO,iBAAiB,CAAC,UAAU,CAAC,CAAC;AACvC,CAAC"}

package/dist/detector.d.ts

@@ -2,6 +2,11 @@ import type { PIISpan } from './types.js';
 /**
  * Detect PII spans in text.
  * Returns non-overlapping spans sorted by position.
+ *
+ * Thin adapter over the shared regex-matcher internal: the matcher owns the
+ * patterns, the canonical type vocabulary, and overlap resolution (positions
+ * only). The value is reconstructed HERE — never inside the internal. PIIType
+ * is a subset of PIICategory, so the type assignment is direct.
  */
 export declare function detectPII(text: string): PIISpan[];
 //# sourceMappingURL=detector.d.ts.map

package/dist/detector.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"detector.d.ts","sourceRoot":"","sources":["../src/detector.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAe,OAAO,EAAE,MAAM,YAAY,CAAC;AA0BvD;;;GAGG;AACH,wBAAgB,SAAS,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,EAAE,CAyBjD"}
+{"version":3,"file":"detector.d.ts","sourceRoot":"","sources":["../src/detector.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,YAAY,CAAC;AAG1C;;;;;;;;GAQG;AACH,wBAAgB,SAAS,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,EAAE,CAOjD"}

package/dist/detector.js

@@ -1,92 +1,22 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.detectPII = detectPII;
-// Regex patterns for structural PII (high precision)
-const PATTERNS = [
-    {
-        category: 'email',
-        regex: /\b[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,}\b/g,
-    },
-    {
-        category: 'phone',
-        // US phone: must be at least 10 digits. Require optional country code + area code.
-        // Anchored to avoid matching partial SSN-like strings.
-        regex: /\b(?:\+?1[-.\s]?)?\(?\d{3}\)?[-.\s]\d{3}[-.\s]\d{4}\b/g,
-    },
-    {
-        category: 'ssn',
-        // SSN: exactly ddd-dd-dddd with hyphens (strict format, not dots or spaces
-        // to avoid collision with phone numbers already consumed)
-        regex: /\b\d{3}-\d{2}-\d{4}\b/g,
-    },
-    {
-        category: 'credit_card',
-        regex: /\b(?:\d{4}[-\s]?){3}\d{4}\b/g,
-    },
-];
+const regex_matcher_js_1 = require("./detection/regex-matcher.js");
 /**
  * Detect PII spans in text.
  * Returns non-overlapping spans sorted by position.
+ *
+ * Thin adapter over the shared regex-matcher internal: the matcher owns the
+ * patterns, the canonical type vocabulary, and overlap resolution (positions
+ * only). The value is reconstructed HERE — never inside the internal. PIIType
+ * is a subset of PIICategory, so the type assignment is direct.
  */
 function detectPII(text) {
-    const spans = [];
-    const occupied = new Set();
-    // Phase 1: Structural PII via regex
-    for (const { category, regex } of PATTERNS) {
-        regex.lastIndex = 0;
-        let match;
-        while ((match = regex.exec(text)) !== null) {
-            const start = match.index;
-            const end = start + match[0].length;
-            if (isOccupied(occupied, start, end))
-                continue;
-            markOccupied(occupied, start, end);
-            spans.push({ start, end, category, value: match[0] });
-        }
-    }
-    // Phase 2: Person names — best-effort via @redactpii/node
-    try {
-        detectPersons(text, occupied, spans);
-    }
-    catch {
-        // Person detection is best-effort; silently skip on any error
-    }
-    return spans.sort((a, b) => a.start - b.start);
-}
-function isOccupied(occupied, start, end) {
-    for (let i = start; i < end; i++) {
-        if (occupied.has(i))
-            return true;
-    }
-    return false;
-}
-function markOccupied(occupied, start, end) {
-    for (let i = start; i < end; i++) {
-        occupied.add(i);
-    }
-}
-function detectPersons(text, occupied, spans) {
-    // The @redactpii/node NAME pattern only catches names after salutations
-    // (Dear, Hi, Hello, Hey, etc.) — not bare names. We replicate that pattern
-    // here and use it directly, avoiding the ESM/CJS import complexity.
-    // This gives best-effort detection for explicitly addressed names.
-    const salutationPattern = /(?:dear|hi|hello|greetings|hey(?:\s+there)?|mr\.?|mrs\.?|ms\.?|dr\.?|prof\.?)\s+([A-Z][a-zA-ZÀ-ÿ'\-]+(?:\s+[A-Z][a-zA-ZÀ-ÿ'\-]+)*)/gi;
-    let match;
-    salutationPattern.lastIndex = 0;
-    while ((match = salutationPattern.exec(text)) !== null) {
-        // match[1] is the captured name group
-        const nameValue = match[1];
-        if (!nameValue)
-            continue;
-        // Find the actual start position of the captured name within the full match
-        const fullMatchStart = match.index;
-        const nameOffset = match[0].indexOf(nameValue);
-        const start = fullMatchStart + nameOffset;
-        const end = start + nameValue.length;
-        if (isOccupied(occupied, start, end))
-            continue;
-        markOccupied(occupied, start, end);
-        spans.push({ start, end, category: 'person', value: nameValue });
-    }
+    return (0, regex_matcher_js_1.matchPII)(text).map(({ start, end, type }) => ({
+        start,
+        end,
+        category: type,
+        value: text.slice(start, end),
+    }));
 }
 //# sourceMappingURL=detector.js.map

package/dist/detector.js.map

@@ -1 +1 @@
-{"version":3,"file":"detector.js","sourceRoot":"","sources":["../src/detector.ts"],"names":[],"mappings":";;AA8BA,8BAyBC;AArDD,qDAAqD;AACrD,MAAM,QAAQ,GAAoD;IAChE;QACE,QAAQ,EAAE,OAAO;QACjB,KAAK,EAAE,uDAAuD;KAC/D;IACD;QACE,QAAQ,EAAE,OAAO;QACjB,mFAAmF;QACnF,uDAAuD;QACvD,KAAK,EAAE,wDAAwD;KAChE;IACD;QACE,QAAQ,EAAE,KAAK;QACf,2EAA2E;QAC3E,0DAA0D;QAC1D,KAAK,EAAE,wBAAwB;KAChC;IACD;QACE,QAAQ,EAAE,aAAa;QACvB,KAAK,EAAE,8BAA8B;KACtC;CACF,CAAC;AAEF;;;GAGG;AACH,SAAgB,SAAS,CAAC,IAAY;IACpC,MAAM,KAAK,GAAc,EAAE,CAAC;IAC5B,MAAM,QAAQ,GAAG,IAAI,GAAG,EAAU,CAAC;IAEnC,oCAAoC;IACpC,KAAK,MAAM,EAAE,QAAQ,EAAE,KAAK,EAAE,IAAI,QAAQ,EAAE,CAAC;QAC3C,KAAK,CAAC,SAAS,GAAG,CAAC,CAAC;QACpB,IAAI,KAA6B,CAAC;QAClC,OAAO,CAAC,KAAK,GAAG,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,KAAK,IAAI,EAAE,CAAC;YAC3C,MAAM,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC;YAC1B,MAAM,GAAG,GAAG,KAAK,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC;YACpC,IAAI,UAAU,CAAC,QAAQ,EAAE,KAAK,EAAE,GAAG,CAAC;gBAAE,SAAS;YAC/C,YAAY,CAAC,QAAQ,EAAE,KAAK,EAAE,GAAG,CAAC,CAAC;YACnC,KAAK,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,GAAG,EAAE,QAAQ,EAAE,KAAK,EAAE,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;QACxD,CAAC;IACH,CAAC;IAED,0DAA0D;IAC1D,IAAI,CAAC;QACH,aAAa,CAAC,IAAI,EAAE,QAAQ,EAAE,KAAK,CAAC,CAAC;IACvC,CAAC;IAAC,MAAM,CAAC;QACP,8DAA8D;IAChE,CAAC;IAED,OAAO,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC;AACjD,CAAC;AAED,SAAS,UAAU,CAAC,QAAqB,EAAE,KAAa,EAAE,GAAW;IACnE,KAAK,IAAI,CAAC,GAAG,KAAK,EAAE,CAAC,GAAG,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC;QACjC,IAAI,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC;YAAE,OAAO,IAAI,CAAC;IACnC,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,SAAS,YAAY,CAAC,QAAqB,EAAE,KAAa,EAAE,GAAW;IACrE,KAAK,IAAI,CAAC,GAAG,KAAK,EAAE,CAAC,GAAG,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC;QACjC,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC;IAClB,CAAC;AACH,CAAC;AAED,SAAS,aAAa,CAAC,IAAY,EAAE,QAAqB,EAAE,KAAgB;IAC1E,wEAAwE;IACxE,2EAA2E;IAC3E,oEAAoE;IACpE,mEAAmE;IACnE,MAAM,iBAAiB,GACrB,sIAAsI,CAAC;IAEzI,IAAI,KAA6B,CAAC;IAClC,iBAAiB,CAAC,SAAS,GAAG,CAAC,CAAC;IAEhC,OAAO,CAAC,KAAK,GAAG,iBAAiB,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,KAAK,IAAI,EAAE,CAAC;QACvD,sCAAsC;QACtC,MAAM,SAAS,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;QAC3B,IAAI,CAAC,SAAS;YAAE,SAAS;QAEzB,4EAA4E;QAC5E,MAAM,cAAc,GAAG,KAAK,CAAC,KAAK,CAAC;QACnC,MAAM,UAAU,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;QAC/C,MAAM,KAAK,GAAG,cAAc,GAAG,UAAU,CAAC;QAC1C,MAAM,GAAG,GAAG,KAAK,GAAG,SAAS,CAAC,MAAM,CAAC;QAErC,IAAI,UAAU,CAAC,QAAQ,EAAE,KAAK,EAAE,GAAG,CAAC;YAAE,SAAS;QAC/C,YAAY,CAAC,QAAQ,EAAE,KAAK,EAAE,GAAG,CAAC,CAAC;QACnC,KAAK,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,GAAG,EAAE,QAAQ,EAAE,QAAQ,EAAE,KAAK,EAAE,SAAS,EAAE,CAAC,CAAC;IACnE,CAAC;AACH,CAAC"}
+{"version":3,"file":"detector.js","sourceRoot":"","sources":["../src/detector.ts"],"names":[],"mappings":";;AAYA,8BAOC;AAlBD,mEAAwD;AAExD;;;;;;;;GAQG;AACH,SAAgB,SAAS,CAAC,IAAY;IACpC,OAAO,IAAA,2BAAQ,EAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,KAAK,EAAE,GAAG,EAAE,IAAI,EAAE,EAAE,EAAE,CAAC,CAAC;QACnD,KAAK;QACL,GAAG;QACH,QAAQ,EAAE,IAAI;QACd,KAAK,EAAE,IAAI,CAAC,KAAK,CAAC,KAAK,EAAE,GAAG,CAAC;KAC9B,CAAC,CAAC,CAAC;AACN,CAAC"}

package/dist/index.d.ts

@@ -1,12 +1,25 @@
 import type { PIIDetectionResult, AnonymizerOptions } from './types.js';
+import type { DetectionProvider } from './detection/provider.js';
 export type { PIICategory, PIIDetectionResult, AnonymizerOptions } from './types.js';
 export { Tokenizer } from './tokenizer.js';
+export { RegexDetector } from './detection/regex-detector.js';
+export type { DetectionProvider, Span } from './detection/provider.js';
 /**
  * Anonymize PII in text. Returns anonymized text + keymap (token → original).
  * Same PII value always gets the same token within a call.
  * Pass seedKeymap to maintain consistency across multiple calls.
  */
 export declare function anonymize(text: string, options?: AnonymizerOptions): PIIDetectionResult;
+/**
+ * Anonymize PII using a pluggable {@link DetectionProvider} instead of the
+ * built-in regex detector. The provider returns positional spans only; the
+ * value is reconstructed HERE via text.slice and tokenized with `span.type` as
+ * the category. Async to cover network providers (e.g. Presidio HTTP).
+ *
+ * Same contract as {@link anonymize}: pass seedKeymap to keep tokens consistent
+ * across calls (a value already mapped reuses its token).
+ */
+export declare function anonymizeWithProvider(text: string, provider: DetectionProvider, options?: AnonymizerOptions): Promise<PIIDetectionResult>;
 /**
  * Restore original PII values from keymap.
  * Tokens not in the keymap are left unchanged.

package/dist/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EAAE,kBAAkB,EAAE,iBAAiB,EAAE,MAAM,YAAY,CAAC;AAExE,YAAY,EAAE,WAAW,EAAE,kBAAkB,EAAE,iBAAiB,EAAE,MAAM,YAAY,CAAC;AACrF,OAAO,EAAE,SAAS,EAAE,MAAM,gBAAgB,CAAC;AAE3C;;;;GAIG;AACH,wBAAgB,SAAS,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,iBAAiB,GAAG,kBAAkB,CAyBvF;AAED;;;GAGG;AACH,wBAAgB,WAAW,CAAC,IAAI,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,GAAG,MAAM,CAQhF"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EAAE,kBAAkB,EAAE,iBAAiB,EAAe,MAAM,YAAY,CAAC;AACrF,OAAO,KAAK,EAAE,iBAAiB,EAAE,MAAM,yBAAyB,CAAC;AAEjE,YAAY,EAAE,WAAW,EAAE,kBAAkB,EAAE,iBAAiB,EAAE,MAAM,YAAY,CAAC;AACrF,OAAO,EAAE,SAAS,EAAE,MAAM,gBAAgB,CAAC;AAC3C,OAAO,EAAE,aAAa,EAAE,MAAM,+BAA+B,CAAC;AAC9D,YAAY,EAAE,iBAAiB,EAAE,IAAI,EAAE,MAAM,yBAAyB,CAAC;AAEvE;;;;GAIG;AACH,wBAAgB,SAAS,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,iBAAiB,GAAG,kBAAkB,CAyBvF;AAED;;;;;;;;GAQG;AACH,wBAAsB,qBAAqB,CACzC,IAAI,EAAE,MAAM,EACZ,QAAQ,EAAE,iBAAiB,EAC3B,OAAO,CAAC,EAAE,iBAAiB,GAC1B,OAAO,CAAC,kBAAkB,CAAC,CA2B7B;AAED;;;GAGG;AACH,wBAAgB,WAAW,CAAC,IAAI,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,GAAG,MAAM,CAQhF"}

package/dist/index.js

@@ -1,12 +1,15 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.Tokenizer = void 0;
+exports.RegexDetector = exports.Tokenizer = void 0;
 exports.anonymize = anonymize;
+exports.anonymizeWithProvider = anonymizeWithProvider;
 exports.deanonymize = deanonymize;
 const detector_js_1 = require("./detector.js");
 const tokenizer_js_1 = require("./tokenizer.js");
 var tokenizer_js_2 = require("./tokenizer.js");
 Object.defineProperty(exports, "Tokenizer", { enumerable: true, get: function () { return tokenizer_js_2.Tokenizer; } });
+var regex_detector_js_1 = require("./detection/regex-detector.js");
+Object.defineProperty(exports, "RegexDetector", { enumerable: true, get: function () { return regex_detector_js_1.RegexDetector; } });
 /**
  * Anonymize PII in text. Returns anonymized text + keymap (token → original).
  * Same PII value always gets the same token within a call.
@@ -34,6 +37,39 @@ function anonymize(text, options) {
     }
     return { text: result, keymap: tokenizer.getKeymap() };
 }
+/**
+ * Anonymize PII using a pluggable {@link DetectionProvider} instead of the
+ * built-in regex detector. The provider returns positional spans only; the
+ * value is reconstructed HERE via text.slice and tokenized with `span.type` as
+ * the category. Async to cover network providers (e.g. Presidio HTTP).
+ *
+ * Same contract as {@link anonymize}: pass seedKeymap to keep tokens consistent
+ * across calls (a value already mapped reuses its token).
+ */
+async function anonymizeWithProvider(text, provider, options) {
+    const spans = await provider.detect(text);
+    const filtered = options?.categories
+        ? spans.filter(s => options.categories.includes(s.type))
+        : spans;
+    const tokenizer = new tokenizer_js_1.Tokenizer();
+    if (options?.seedKeymap && Object.keys(options.seedKeymap).length > 0) {
+        tokenizer.loadKeymap(options.seedKeymap);
+    }
+    if (filtered.length === 0) {
+        return { text, keymap: tokenizer.getKeymap() };
+    }
+    // Replace spans right-to-left so earlier offsets stay valid as we splice.
+    const sortedDesc = [...filtered].sort((a, b) => b.start - a.start);
+    let result = text;
+    for (const span of sortedDesc) {
+        const value = text.slice(span.start, span.end);
+        // span.type is a free string (the provider's vocabulary); the Tokenizer
+        // derives a clean prefix from it — no cast, no undefined_N tokens.
+        const token = tokenizer.tokenize(value, span.type);
+        result = result.slice(0, span.start) + token + result.slice(span.end);
+    }
+    return { text: result, keymap: tokenizer.getKeymap() };
+}
 /**
  * Restore original PII values from keymap.
  * Tokens not in the keymap are left unchanged.

package/dist/index.js.map

@@ -1 +1 @@
-{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";;;AAYA,8BAyBC;AAMD,kCAQC;AAnDD,+CAA0C;AAC1C,iDAA2C;AAI3C,+CAA2C;AAAlC,yGAAA,SAAS,OAAA;AAElB;;;;GAIG;AACH,SAAgB,SAAS,CAAC,IAAY,EAAE,OAA2B;IACjE,MAAM,KAAK,GAAG,IAAA,uBAAS,EAAC,IAAI,CAAC,CAAC;IAC9B,MAAM,QAAQ,GAAG,OAAO,EAAE,UAAU;QAClC,CAAC,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,OAAO,CAAC,UAAW,CAAC,QAAQ,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC;QAC7D,CAAC,CAAC,KAAK,CAAC;IAEV,MAAM,SAAS,GAAG,IAAI,wBAAS,EAAE,CAAC;IAClC,uDAAuD;IACvD,IAAI,OAAO,EAAE,UAAU,IAAI,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,UAAU,CAAC,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACtE,SAAS,CAAC,UAAU,CAAC,OAAO,CAAC,UAAU,CAAC,CAAC;IAC3C,CAAC;IAED,IAAI,QAAQ,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC1B,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,SAAS,CAAC,SAAS,EAAE,EAAE,CAAC;IACjD,CAAC;IAED,mEAAmE;IACnE,MAAM,UAAU,GAAG,CAAC,GAAG,QAAQ,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC;IACnE,IAAI,MAAM,GAAG,IAAI,CAAC;IAClB,KAAK,MAAM,IAAI,IAAI,UAAU,EAAE,CAAC;QAC9B,MAAM,KAAK,GAAG,SAAS,CAAC,QAAQ,CAAC,IAAI,CAAC,KAAK,EAAE,IAAI,CAAC,QAAQ,CAAC,CAAC;QAC5D,MAAM,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC,EAAE,IAAI,CAAC,KAAK,CAAC,GAAG,KAAK,GAAG,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;IACxE,CAAC;IAED,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,MAAM,EAAE,SAAS,CAAC,SAAS,EAAE,EAAE,CAAC;AACzD,CAAC;AAED;;;GAGG;AACH,SAAgB,WAAW,CAAC,IAAY,EAAE,MAA8B;IACtE,IAAI,MAAM,GAAG,IAAI,CAAC;IAClB,yEAAyE;IACzE,MAAM,MAAM,GAAG,MAAM,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC;IAChF,KAAK,MAAM,CAAC,KAAK,EAAE,QAAQ,CAAC,IAAI,MAAM,EAAE,CAAC;QACvC,MAAM,GAAG,MAAM,CAAC,UAAU,CAAC,KAAK,EAAE,QAAQ,CAAC,CAAC;IAC9C,CAAC;IACD,OAAO,MAAM,CAAC;AAChB,CAAC"}
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";;;AAeA,8BAyBC;AAWD,sDA+BC;AAMD,kCAQC;AAhGD,+CAA0C;AAC1C,iDAA2C;AAK3C,+CAA2C;AAAlC,yGAAA,SAAS,OAAA;AAClB,mEAA8D;AAArD,kHAAA,aAAa,OAAA;AAGtB;;;;GAIG;AACH,SAAgB,SAAS,CAAC,IAAY,EAAE,OAA2B;IACjE,MAAM,KAAK,GAAG,IAAA,uBAAS,EAAC,IAAI,CAAC,CAAC;IAC9B,MAAM,QAAQ,GAAG,OAAO,EAAE,UAAU;QAClC,CAAC,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,OAAO,CAAC,UAAW,CAAC,QAAQ,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC;QAC7D,CAAC,CAAC,KAAK,CAAC;IAEV,MAAM,SAAS,GAAG,IAAI,wBAAS,EAAE,CAAC;IAClC,uDAAuD;IACvD,IAAI,OAAO,EAAE,UAAU,IAAI,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,UAAU,CAAC,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACtE,SAAS,CAAC,UAAU,CAAC,OAAO,CAAC,UAAU,CAAC,CAAC;IAC3C,CAAC;IAED,IAAI,QAAQ,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC1B,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,SAAS,CAAC,SAAS,EAAE,EAAE,CAAC;IACjD,CAAC;IAED,mEAAmE;IACnE,MAAM,UAAU,GAAG,CAAC,GAAG,QAAQ,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC;IACnE,IAAI,MAAM,GAAG,IAAI,CAAC;IAClB,KAAK,MAAM,IAAI,IAAI,UAAU,EAAE,CAAC;QAC9B,MAAM,KAAK,GAAG,SAAS,CAAC,QAAQ,CAAC,IAAI,CAAC,KAAK,EAAE,IAAI,CAAC,QAAQ,CAAC,CAAC;QAC5D,MAAM,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC,EAAE,IAAI,CAAC,KAAK,CAAC,GAAG,KAAK,GAAG,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;IACxE,CAAC;IAED,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,MAAM,EAAE,SAAS,CAAC,SAAS,EAAE,EAAE,CAAC;AACzD,CAAC;AAED;;;;;;;;GAQG;AACI,KAAK,UAAU,qBAAqB,CACzC,IAAY,EACZ,QAA2B,EAC3B,OAA2B;IAE3B,MAAM,KAAK,GAAG,MAAM,QAAQ,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;IAC1C,MAAM,QAAQ,GAAG,OAAO,EAAE,UAAU;QAClC,CAAC,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,OAAO,CAAC,UAAW,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAmB,CAAC,CAAC;QACxE,CAAC,CAAC,KAAK,CAAC;IAEV,MAAM,SAAS,GAAG,IAAI,wBAAS,EAAE,CAAC;IAClC,IAAI,OAAO,EAAE,UAAU,IAAI,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,UAAU,CAAC,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACtE,SAAS,CAAC,UAAU,CAAC,OAAO,CAAC,UAAU,CAAC,CAAC;IAC3C,CAAC;IAED,IAAI,QAAQ,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC1B,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,SAAS,CAAC,SAAS,EAAE,EAAE,CAAC;IACjD,CAAC;IAED,0EAA0E;IAC1E,MAAM,UAAU,GAAG,CAAC,GAAG,QAAQ,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC;IACnE,IAAI,MAAM,GAAG,IAAI,CAAC;IAClB,KAAK,MAAM,IAAI,IAAI,UAAU,EAAE,CAAC;QAC9B,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC;QAC/C,wEAAwE;QACxE,mEAAmE;QACnE,MAAM,KAAK,GAAG,SAAS,CAAC,QAAQ,CAAC,KAAK,EAAE,IAAI,CAAC,IAAI,CAAC,CAAC;QACnD,MAAM,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC,EAAE,IAAI,CAAC,KAAK,CAAC,GAAG,KAAK,GAAG,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;IACxE,CAAC;IAED,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,MAAM,EAAE,SAAS,CAAC,SAAS,EAAE,EAAE,CAAC;AACzD,CAAC;AAED;;;GAGG;AACH,SAAgB,WAAW,CAAC,IAAY,EAAE,MAA8B;IACtE,IAAI,MAAM,GAAG,IAAI,CAAC;IAClB,yEAAyE;IACzE,MAAM,MAAM,GAAG,MAAM,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC;IAChF,KAAK,MAAM,CAAC,KAAK,EAAE,QAAQ,CAAC,IAAI,MAAM,EAAE,CAAC;QACvC,MAAM,GAAG,MAAM,CAAC,UAAU,CAAC,KAAK,EAAE,QAAQ,CAAC,CAAC;IAC9C,CAAC;IACD,OAAO,MAAM,CAAC;AAChB,CAAC"}

package/dist/tokenizer.d.ts

@@ -1,12 +1,19 @@
-import type { PIICategory } from './types.js';
 export declare class Tokenizer {
     private inverse;
     private keymap;
     private counters;
     /**
      * Get or create a consistent sequential token for a PII value.
+     *
+     * `category` is a free string: the built-in detector emits the six PIICategory
+     * values, but a pluggable DetectionProvider (STU-397) may return its own
+     * vocabulary (e.g. "organization", "iban"). The token prefix is the category
+     * uppercased — for the built-in categories that is exactly the old fixed
+     * mapping (person → PERSON, credit_card → CREDIT_CARD), so existing tokens are
+     * unchanged — and the counter is keyed by the lowercased category so distinct
+     * types never collide on a single prefix.
      */
-    tokenize(value: string, category: PIICategory): string;
+    tokenize(value: string, category: string): string;
     getKeymap(): Record<string, string>;
     /**
      * Load an existing keymap (for cross-stage continuity).

package/dist/tokenizer.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"tokenizer.d.ts","sourceRoot":"","sources":["../src/tokenizer.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AAW9C,qBAAa,SAAS;IAEpB,OAAO,CAAC,OAAO,CAA6B;IAE5C,OAAO,CAAC,MAAM,CAA6B;IAE3C,OAAO,CAAC,QAAQ,CAAkC;IAElD;;OAEG;IACH,QAAQ,CAAC,KAAK,EAAE,MAAM,EAAE,QAAQ,EAAE,WAAW,GAAG,MAAM;IAetD,SAAS,IAAI,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC;IAInC;;;OAGG;IACH,UAAU,CAAC,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,GAAG,IAAI;CAiBjD"}
+{"version":3,"file":"tokenizer.d.ts","sourceRoot":"","sources":["../src/tokenizer.ts"],"names":[],"mappings":"AAAA,qBAAa,SAAS;IAEpB,OAAO,CAAC,OAAO,CAA6B;IAE5C,OAAO,CAAC,MAAM,CAA6B;IAE3C,OAAO,CAAC,QAAQ,CAA6B;IAE7C;;;;;;;;;;OAUG;IACH,QAAQ,CAAC,KAAK,EAAE,MAAM,EAAE,QAAQ,EAAE,MAAM,GAAG,MAAM;IAejD,SAAS,IAAI,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC;IAInC;;;OAGG;IACH,UAAU,CAAC,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,GAAG,IAAI;CAgBjD"}

package/dist/tokenizer.js

@@ -1,32 +1,32 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.Tokenizer = void 0;
-const CATEGORY_PREFIX = {
-    person: 'PERSON',
-    email: 'EMAIL',
-    phone: 'PHONE',
-    address: 'ADDRESS',
-    ssn: 'SSN',
-    credit_card: 'CREDIT_CARD',
-};
 class Tokenizer {
     // original value → token
     inverse = new Map();
     // token → original value
     keymap = new Map();
-    // category → counter
+    // category (lowercased) → counter
     counters = new Map();
     /**
      * Get or create a consistent sequential token for a PII value.
+     *
+     * `category` is a free string: the built-in detector emits the six PIICategory
+     * values, but a pluggable DetectionProvider (STU-397) may return its own
+     * vocabulary (e.g. "organization", "iban"). The token prefix is the category
+     * uppercased — for the built-in categories that is exactly the old fixed
+     * mapping (person → PERSON, credit_card → CREDIT_CARD), so existing tokens are
+     * unchanged — and the counter is keyed by the lowercased category so distinct
+     * types never collide on a single prefix.
      */
     tokenize(value, category) {
         const existing = this.inverse.get(value);
         if (existing)
             return existing;
-        const counter = (this.counters.get(category) ?? 0) + 1;
-        this.counters.set(category, counter);
-        const prefix = CATEGORY_PREFIX[category];
-        const token = `${prefix}_${counter}`;
+        const key = category.toLowerCase();
+        const counter = (this.counters.get(key) ?? 0) + 1;
+        this.counters.set(key, counter);
+        const token = `${category.toUpperCase()}_${counter}`;
         this.inverse.set(value, token);
         this.keymap.set(token, value);
         return token;
@@ -42,15 +42,14 @@ class Tokenizer {
         for (const [token, value] of Object.entries(keymap)) {
             this.keymap.set(token, value);
             this.inverse.set(value, token);
-            // Restore counter state from token name (e.g. PERSON_3 → counter = 3)
+            // Restore counter state from token name (e.g. PERSON_3 → counter = 3).
+            // The prefix lowercased is the counter key, mirroring tokenize().
             const match = token.match(/^([A-Z_]+)_(\d+)$/);
             if (match) {
-                const cat = Object.entries(CATEGORY_PREFIX).find(([, p]) => p === match[1])?.[0];
-                if (cat) {
-                    const n = parseInt(match[2], 10);
-                    if ((this.counters.get(cat) ?? 0) < n) {
-                        this.counters.set(cat, n);
-                    }
+                const key = match[1].toLowerCase();
+                const n = parseInt(match[2], 10);
+                if ((this.counters.get(key) ?? 0) < n) {
+                    this.counters.set(key, n);
                 }
             }
         }

package/dist/tokenizer.js.map

@@ -1 +1 @@
-{"version":3,"file":"tokenizer.js","sourceRoot":"","sources":["../src/tokenizer.ts"],"names":[],"mappings":";;;AAEA,MAAM,eAAe,GAAgC;IACnD,MAAM,EAAE,QAAQ;IAChB,KAAK,EAAE,OAAO;IACd,KAAK,EAAE,OAAO;IACd,OAAO,EAAE,SAAS;IAClB,GAAG,EAAE,KAAK;IACV,WAAW,EAAE,aAAa;CAC3B,CAAC;AAEF,MAAa,SAAS;IACpB,yBAAyB;IACjB,OAAO,GAAG,IAAI,GAAG,EAAkB,CAAC;IAC5C,yBAAyB;IACjB,MAAM,GAAG,IAAI,GAAG,EAAkB,CAAC;IAC3C,qBAAqB;IACb,QAAQ,GAAG,IAAI,GAAG,EAAuB,CAAC;IAElD;;OAEG;IACH,QAAQ,CAAC,KAAa,EAAE,QAAqB;QAC3C,MAAM,QAAQ,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC;QACzC,IAAI,QAAQ;YAAE,OAAO,QAAQ,CAAC;QAE9B,MAAM,OAAO,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC;QACvD,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,QAAQ,EAAE,OAAO,CAAC,CAAC;QAErC,MAAM,MAAM,GAAG,eAAe,CAAC,QAAQ,CAAC,CAAC;QACzC,MAAM,KAAK,GAAG,GAAG,MAAM,IAAI,OAAO,EAAE,CAAC;QAErC,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC;QAC/B,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC;QAC9B,OAAO,KAAK,CAAC;IACf,CAAC;IAED,SAAS;QACP,OAAO,MAAM,CAAC,WAAW,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;IACzC,CAAC;IAED;;;OAGG;IACH,UAAU,CAAC,MAA8B;QACvC,KAAK,MAAM,CAAC,KAAK,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,CAAC;YACpD,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC;YAC9B,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC;YAC/B,sEAAsE;YACtE,MAAM,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC,mBAAmB,CAAC,CAAC;YAC/C,IAAI,KAAK,EAAE,CAAC;gBACV,MAAM,GAAG,GAAG,MAAM,CAAC,OAAO,CAAC,eAAe,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,KAAK,KAAK,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAA4B,CAAC;gBAC5G,IAAI,GAAG,EAAE,CAAC;oBACR,MAAM,CAAC,GAAG,QAAQ,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;oBACjC,IAAI,CAAC,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,EAAE,CAAC;wBACtC,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,GAAG,EAAE,CAAC,CAAC,CAAC;oBAC5B,CAAC;gBACH,CAAC;YACH,CAAC;QACH,CAAC;IACH,CAAC;CACF;AAnDD,8BAmDC"}
+{"version":3,"file":"tokenizer.js","sourceRoot":"","sources":["../src/tokenizer.ts"],"names":[],"mappings":";;;AAAA,MAAa,SAAS;IACpB,yBAAyB;IACjB,OAAO,GAAG,IAAI,GAAG,EAAkB,CAAC;IAC5C,yBAAyB;IACjB,MAAM,GAAG,IAAI,GAAG,EAAkB,CAAC;IAC3C,kCAAkC;IAC1B,QAAQ,GAAG,IAAI,GAAG,EAAkB,CAAC;IAE7C;;;;;;;;;;OAUG;IACH,QAAQ,CAAC,KAAa,EAAE,QAAgB;QACtC,MAAM,QAAQ,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC;QACzC,IAAI,QAAQ;YAAE,OAAO,QAAQ,CAAC;QAE9B,MAAM,GAAG,GAAG,QAAQ,CAAC,WAAW,EAAE,CAAC;QACnC,MAAM,OAAO,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC;QAClD,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,GAAG,EAAE,OAAO,CAAC,CAAC;QAEhC,MAAM,KAAK,GAAG,GAAG,QAAQ,CAAC,WAAW,EAAE,IAAI,OAAO,EAAE,CAAC;QAErD,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC;QAC/B,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC;QAC9B,OAAO,KAAK,CAAC;IACf,CAAC;IAED,SAAS;QACP,OAAO,MAAM,CAAC,WAAW,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;IACzC,CAAC;IAED;;;OAGG;IACH,UAAU,CAAC,MAA8B;QACvC,KAAK,MAAM,CAAC,KAAK,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,CAAC;YACpD,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC;YAC9B,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC;YAC/B,uEAAuE;YACvE,kEAAkE;YAClE,MAAM,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC,mBAAmB,CAAC,CAAC;YAC/C,IAAI,KAAK,EAAE,CAAC;gBACV,MAAM,GAAG,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC,WAAW,EAAE,CAAC;gBACnC,MAAM,CAAC,GAAG,QAAQ,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;gBACjC,IAAI,CAAC,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,EAAE,CAAC;oBACtC,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,GAAG,EAAE,CAAC,CAAC,CAAC;gBAC5B,CAAC;YACH,CAAC;QACH,CAAC;IACH,CAAC;CACF;AA1DD,8BA0DC"}

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@studio-foundation/anonymizer",
-  "version": "0.3.0-beta.6",
-  "description": "PII detection and anonymization with consistent token mapping",
+  "version": "0.5.2",
+  "description": "PII detection and anonymization for LLM workflows, with consistent token mapping. Used by Studio.",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",
   "files": [