anonyma 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +457 -0
- package/LICENSE +21 -0
- package/README.md +605 -0
- package/dist/crypto.cjs +142 -0
- package/dist/crypto.cjs.map +1 -0
- package/dist/crypto.d.cts +51 -0
- package/dist/crypto.d.ts +51 -0
- package/dist/crypto.js +139 -0
- package/dist/crypto.js.map +1 -0
- package/dist/detectors/index.cjs +1487 -0
- package/dist/detectors/index.cjs.map +1 -0
- package/dist/detectors/index.d.cts +754 -0
- package/dist/detectors/index.d.ts +754 -0
- package/dist/detectors/index.js +1451 -0
- package/dist/detectors/index.js.map +1 -0
- package/dist/index.cjs +2877 -0
- package/dist/index.cjs.map +1 -0
- package/dist/index.d.cts +772 -0
- package/dist/index.d.ts +772 -0
- package/dist/index.js +2850 -0
- package/dist/index.js.map +1 -0
- package/dist/schemas.cjs +361 -0
- package/dist/schemas.cjs.map +1 -0
- package/dist/schemas.d.cts +1031 -0
- package/dist/schemas.d.ts +1031 -0
- package/dist/schemas.js +341 -0
- package/dist/schemas.js.map +1 -0
- package/dist/stream.cjs +2315 -0
- package/dist/stream.cjs.map +1 -0
- package/dist/stream.d.cts +76 -0
- package/dist/stream.d.ts +76 -0
- package/dist/stream.js +2311 -0
- package/dist/stream.js.map +1 -0
- package/dist/types-D-9HzVuy.d.cts +553 -0
- package/dist/types-D-9HzVuy.d.ts +553 -0
- package/dist/validators.cjs +186 -0
- package/dist/validators.cjs.map +1 -0
- package/dist/validators.d.cts +136 -0
- package/dist/validators.d.ts +136 -0
- package/dist/validators.js +176 -0
- package/dist/validators.js.map +1 -0
- package/package.json +158 -0
package/README.md
ADDED
|
@@ -0,0 +1,605 @@
|
|
|
1
|
+
# anonyma
|
|
2
|
+
|
|
3
|
+
> A modern, **zero-dependency** TypeScript library for PII detection and data anonymization — built for the AI era.
|
|
4
|
+
|
|
5
|
+
[](https://www.npmjs.com/package/anonyma)
|
|
6
|
+
[](https://github.com/izaccavalheiro/anonyma/actions/workflows/ci.yml)
|
|
7
|
+
[](#)
|
|
8
|
+
[](LICENSE)
|
|
9
|
+
[](https://www.typescriptlang.org/)
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## Features
|
|
14
|
+
|
|
15
|
+
- 🔍 **Detect 27 PII categories** — email, phone, SSN, credit card, IBAN, passport, address, VIN, API keys, crypto wallets, and more
|
|
16
|
+
- 🛡️ **8 anonymization strategies** — mask, redact, pseudonymize, hash (SHA-256), generalize, tokenize, encrypt (AES-GCM), synthesize
|
|
17
|
+
- 🔁 **Reversible tokenization** — `tokenize()` / `detokenize()` for round-trip fidelity
|
|
18
|
+
- 🤖 **LLM pipeline helpers** — `sanitizeForLLM()` / `restoreFromLLM()` for safe prompt injection with reversible tokens
|
|
19
|
+
- 📋 **Compliance presets** — built-in `gdpr`, `hipaa`, `ccpa`, `pci-dss`, `sox`, and `ferpa` presets
|
|
20
|
+
- 🌍 **Locale-aware detection** — locale flags for US, UK, EU, CA, AU, BR, IN, CN, JP, KR, ZA, and global
|
|
21
|
+
- ⚡ **Batch processing** — `anonymizeBatch()`, `anonymizeBatchAsync()`, `tokenizeBatch()`, `detectBatch()`
|
|
22
|
+
- 🌊 **Streaming support** — WHATWG `TransformStream` wrappers (`createAnonymizeStream()`, `createTokenizeStream()`)
|
|
23
|
+
- 🎯 **Field-level record anonymization** — dot-notation paths for nested objects
|
|
24
|
+
- 🌳 **Deep object anonymization** — `anonymizeObject()` recursively cleans entire JSON trees
|
|
25
|
+
- ✅ **PII presence check** — `hasPII()` with early-exit for fast gating
|
|
26
|
+
- 🔎 **Aggressive mode** — expanded, permissive patterns for obfuscated PII
|
|
27
|
+
- 🧩 **Custom patterns & detectors** — inject ad-hoc `RegExp` patterns or fully replace per-category detectors
|
|
28
|
+
- 🚫 **Allowlist support** — skip known-safe values by exact string or `RegExp` pattern
|
|
29
|
+
- 🔢 **Confidence threshold** — filter out low-confidence matches
|
|
30
|
+
- 🔌 **Plugin architecture** — extend detectors, strategies, and validators via `AnonymaPlugin`
|
|
31
|
+
- 🔒 **Encryption** — reversible AES-GCM `encrypt()` / `decrypt()` strategy (Web Crypto API)
|
|
32
|
+
- 🎭 **Synthesis** — format-preserving synthetic data replacement (deterministic, seeded)
|
|
33
|
+
- ✔️ **Checksum validators** — `luhn`, `verhoeff`, `nhsMod11`, `cpfChecksum`, `vinChecksum`, `deaChecksum`, `ibanMod97`, `ninoValid`, `aadhaarFormat` via `"anonyma/validators"`
|
|
34
|
+
- ⚡ **Zero runtime dependencies** — Zod is an optional peer dependency
|
|
35
|
+
- 🌲 **Tree-shakeable** — import only what you use
|
|
36
|
+
- 🤖 **AI-ready** — OpenAI/MCP tool definitions and Zod schemas included
|
|
37
|
+
- 📦 **Dual ESM + CJS** — works everywhere Node.js ≥ 18 runs
|
|
38
|
+
- 🔒 **Strict TypeScript** — no `any`, full declaration files
|
|
39
|
+
|
|
40
|
+
---
|
|
41
|
+
|
|
42
|
+
## Installation
|
|
43
|
+
|
|
44
|
+
```bash
|
|
45
|
+
npm install anonyma
|
|
46
|
+
# or
|
|
47
|
+
pnpm add anonyma
|
|
48
|
+
# or
|
|
49
|
+
yarn add anonyma
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
For runtime validation and AI schema features, also install the optional peer dependency:
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
npm install zod
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
---
|
|
59
|
+
|
|
60
|
+
## Quick Start
|
|
61
|
+
|
|
62
|
+
```ts
|
|
63
|
+
import {
|
|
64
|
+
anonymize, anonymizeAsync, detect, hasPII,
|
|
65
|
+
anonymizeObject, anonymizeRecord, createAnonymizer,
|
|
66
|
+
tokenize, tokenizeAsync, detokenize, sanitizeForLLM, restoreFromLLM,
|
|
67
|
+
anonymizeBatch, anonymizeBatchAsync,
|
|
68
|
+
} from "anonyma";
|
|
69
|
+
|
|
70
|
+
// ── Detect PII ─────────────────────────────────────────────────────────────
|
|
71
|
+
const matches = detect("Contact alice@example.com or call 555-867-5309.");
|
|
72
|
+
// [
|
|
73
|
+
// { category: "email", value: "alice@example.com", start: 8, end: 25, confidence: 0.99 },
|
|
74
|
+
// { category: "phone", value: "555-867-5309", start: 34, end: 46, confidence: 0.9 },
|
|
75
|
+
// ]
|
|
76
|
+
|
|
77
|
+
// ── Fast PII presence check (stops at first match) ─────────────────────────
|
|
78
|
+
if (hasPII(commentText)) {
|
|
79
|
+
return { error: "Comment contains personal information." };
|
|
80
|
+
}
|
|
81
|
+
|
|
82
|
+
// ── Anonymize free text (default: redact all PII) ──────────────────────────
|
|
83
|
+
const { text } = anonymize("Contact alice@example.com or call 555-867-5309.");
|
|
84
|
+
// "Contact [REDACTED] or call [REDACTED]."
|
|
85
|
+
|
|
86
|
+
// ── Async anonymization (required for hash/encrypt strategies) ─────────────
|
|
87
|
+
const { text: hashed } = await anonymizeAsync("alice@example.com", {
|
|
88
|
+
defaultStrategy: { strategy: "hash", pepper: "my-pepper" },
|
|
89
|
+
});
|
|
90
|
+
// "5f3e4b3a9c1d8f2a"
|
|
91
|
+
|
|
92
|
+
// ── Consistent token mapping ───────────────────────────────────────────────
|
|
93
|
+
anonymize("From alice@example.com (alice@example.com)", { consistentTokens: true }).text;
|
|
94
|
+
// "From EMAIL_1 (EMAIL_1)"
|
|
95
|
+
|
|
96
|
+
// ── Global replacement ─────────────────────────────────────────────────────
|
|
97
|
+
anonymize("alice@example.com — 555-867-5309", { globalReplacement: "***" }).text;
|
|
98
|
+
// "*** — ***"
|
|
99
|
+
|
|
100
|
+
// ── Aggressive mode — catch obfuscated PII ────────────────────────────────
|
|
101
|
+
anonymize("user [at] example [dot] com", { aggressive: true }).text;
|
|
102
|
+
// "[REDACTED]"
|
|
103
|
+
|
|
104
|
+
// ── Custom pattern ─────────────────────────────────────────────────────────
|
|
105
|
+
anonymize("Order ACME-001234 confirmed.", {
|
|
106
|
+
customPatterns: [{ pattern: /\bACME-\d{6}\b/g, label: "[ORDER_ID]" }],
|
|
107
|
+
rules: [],
|
|
108
|
+
}).text;
|
|
109
|
+
// "Order [ORDER_ID] confirmed."
|
|
110
|
+
|
|
111
|
+
// ── Deep object anonymization ──────────────────────────────────────────────
|
|
112
|
+
anonymizeObject({
|
|
113
|
+
user: { email: "alice@example.com", phone: "555-867-5309" },
|
|
114
|
+
notes: ["Call later", "IP: 192.168.1.1"],
|
|
115
|
+
});
|
|
116
|
+
// {
|
|
117
|
+
// user: { email: "[REDACTED]", phone: "[REDACTED]" },
|
|
118
|
+
// notes: ["Call later", "[REDACTED]"],
|
|
119
|
+
// }
|
|
120
|
+
|
|
121
|
+
// ── Custom strategy per category ───────────────────────────────────────────
|
|
122
|
+
const { text: masked } = anonymize("alice@example.com and 192.168.1.1", {
|
|
123
|
+
rules: [
|
|
124
|
+
{ category: "email", strategy: { strategy: "mask", keepLeading: 1, keepTrailing: 3 } },
|
|
125
|
+
{ category: "ipv4", strategy: { strategy: "redact", label: "[IP REMOVED]" } },
|
|
126
|
+
],
|
|
127
|
+
});
|
|
128
|
+
// "a***************com and [IP REMOVED]"
|
|
129
|
+
|
|
130
|
+
// ── Format-preserving mask ─────────────────────────────────────────────────
|
|
131
|
+
anonymize("123-45-6789", {
|
|
132
|
+
rules: [{ category: "ssn", strategy: { strategy: "mask", preserveFormat: true } }],
|
|
133
|
+
}).text;
|
|
134
|
+
// "000-00-0000"
|
|
135
|
+
|
|
136
|
+
// ── Enable only specific categories ───────────────────────────────────────
|
|
137
|
+
anonymize("alice@example.com — call 555-867-5309", {
|
|
138
|
+
enabledCategories: { email: true },
|
|
139
|
+
}).text;
|
|
140
|
+
// "[REDACTED] — call 555-867-5309"
|
|
141
|
+
|
|
142
|
+
// ── Allowlist — skip known-safe values ────────────────────────────────────
|
|
143
|
+
anonymize("noreply@company.com or alice@example.com", {
|
|
144
|
+
allowlist: ["noreply@company.com"],
|
|
145
|
+
}).text;
|
|
146
|
+
// "noreply@company.com or [REDACTED]"
|
|
147
|
+
|
|
148
|
+
// ── Confidence threshold ──────────────────────────────────────────────────
|
|
149
|
+
anonymize("maybe a name here", { confidenceThreshold: 0.8 }).text;
|
|
150
|
+
// Low-confidence matches are left untouched
|
|
151
|
+
|
|
152
|
+
// ── Compliance presets ─────────────────────────────────────────────────────
|
|
153
|
+
anonymize(medicalNote, { preset: "hipaa" }).text;
|
|
154
|
+
anonymize(userData, { preset: "gdpr" }).text;
|
|
155
|
+
|
|
156
|
+
// ── Locale-aware detection ─────────────────────────────────────────────────
|
|
157
|
+
anonymize("NHS: 943 476 5919", { locales: ["uk"] }).text;
|
|
158
|
+
// "[REDACTED]"
|
|
159
|
+
|
|
160
|
+
// ── Anonymize object fields ─────────────────────────────────────────────────
|
|
161
|
+
anonymizeRecord(
|
|
162
|
+
{ name: "Alice", email: "alice@example.com", age: "27" },
|
|
163
|
+
{
|
|
164
|
+
email: { strategy: { strategy: "redact" } },
|
|
165
|
+
age: { strategy: { strategy: "generalize" } }, // 27 → "20-29"
|
|
166
|
+
}
|
|
167
|
+
);
|
|
168
|
+
// { name: "Alice", email: "[REDACTED]", age: "20-29" }
|
|
169
|
+
|
|
170
|
+
// ── Reusable anonymizer ─────────────────────────────────────────────────────
|
|
171
|
+
const anonymizer = createAnonymizer({
|
|
172
|
+
categories: ["email", "phone"],
|
|
173
|
+
defaultStrategy: { strategy: "pseudonymize", seed: "my-secret" },
|
|
174
|
+
consistentTokens: true,
|
|
175
|
+
});
|
|
176
|
+
anonymizer.anonymize("alice@example.com").text;
|
|
177
|
+
// "id_3a7f1c2b9e4d0f1a"
|
|
178
|
+
anonymizer.hasPII("no pii here"); // false
|
|
179
|
+
|
|
180
|
+
// ── Reversible tokenization ────────────────────────────────────────────────
|
|
181
|
+
const { text: tokenized, mapping } = tokenize("alice@example.com called 555-867-5309");
|
|
182
|
+
// text: "[EMAIL_0001] called [PHONE_0001]"
|
|
183
|
+
const { text: restored } = detokenize(tokenized, mapping);
|
|
184
|
+
// "alice@example.com called 555-867-5309"
|
|
185
|
+
|
|
186
|
+
// ── LLM pipeline — sanitize then restore ──────────────────────────────────
|
|
187
|
+
const { text: sanitized, mapping: llmMapping } = sanitizeForLLM("Send invoice to alice@example.com");
|
|
188
|
+
// "Send invoice to [EMAIL_0001]"
|
|
189
|
+
const llmResponse = await callLLM(sanitized);
|
|
190
|
+
const final = restoreFromLLM(llmResponse, llmMapping);
|
|
191
|
+
// Tokens in the LLM's response are swapped back to the original values
|
|
192
|
+
|
|
193
|
+
// ── Batch processing ───────────────────────────────────────────────────────
|
|
194
|
+
const results = anonymizeBatch(["alice@example.com", "192.0.2.1", "123-45-6789"]);
|
|
195
|
+
for (const r of results) {
|
|
196
|
+
if (r.ok) console.log(r.value.text);
|
|
197
|
+
else console.error(`Item ${r.index} failed:`, r.error.message);
|
|
198
|
+
}
|
|
199
|
+
|
|
200
|
+
// ── AES-GCM encryption (reversible) ───────────────────────────────────────
|
|
201
|
+
import { encrypt, decrypt } from "anonyma";
|
|
202
|
+
const ciphertext = await encrypt("alice@example.com", { passphrase: "s3cr3t" });
|
|
203
|
+
// "base64:<iv>:<ciphertext>"
|
|
204
|
+
const original = await decrypt(ciphertext, { passphrase: "s3cr3t" });
|
|
205
|
+
// "alice@example.com"
|
|
206
|
+
|
|
207
|
+
// ── Synthetic data replacement ────────────────────────────────────────────
|
|
208
|
+
anonymize("alice@example.com", {
|
|
209
|
+
defaultStrategy: { strategy: "synthesize", seed: "project-x" },
|
|
210
|
+
}).text;
|
|
211
|
+
// "carol.smith42@example.com" (deterministic, structurally valid)
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
---
|
|
215
|
+
|
|
216
|
+
## Strategies
|
|
217
|
+
|
|
218
|
+
| Strategy | Description | Async | Deterministic | Reversible |
|
|
219
|
+
|----------------|---------------------------------------------------------------------|:-----:|:-------------:|:----------:|
|
|
220
|
+
| `redact` | Replace with `[REDACTED]` label (customizable) | ❌ | ✅ | ❌ |
|
|
221
|
+
| `mask` | Replace inner chars with `*`; optional format-preserving mode | ❌ | ✅ | ❌ |
|
|
222
|
+
| `pseudonymize` | Replace with a hex pseudonym (seeded or random) | ❌ | ✅† | ❌ |
|
|
223
|
+
| `hash` | SHA-256 one-way hash with optional pepper (Web Crypto) | ✅ | ✅ | ❌ |
|
|
224
|
+
| `generalize` | Replace numbers with a bucket range (e.g. `27` → `20-29`) | ❌ | ✅ | ❌ |
|
|
225
|
+
| `tokenize` | Replace with a reversible placeholder token | ❌ | ✅ | ✅ |
|
|
226
|
+
| `encrypt` | AES-256-GCM encryption via passphrase or raw key (Web Crypto) | ✅ | ❌‡ | ✅ |
|
|
227
|
+
| `synthesize` | Format-preserving synthetic replacement (seeded, no real PII) | ❌ | ✅† | ❌ |
|
|
228
|
+
|
|
229
|
+
† Deterministic when `seed` is provided. ‡ Random IV per encryption; decryptable with the same key.
|
|
230
|
+
|
|
231
|
+
---
|
|
232
|
+
|
|
233
|
+
## Detected PII Categories
|
|
234
|
+
|
|
235
|
+
### Personal Information
|
|
236
|
+
|
|
237
|
+
| Category | Examples | Validator / Notes |
|
|
238
|
+
|-------------------|---------------------------------------------|-------------------------------------|
|
|
239
|
+
| `email` | `alice@example.com` | RFC 5321 regex |
|
|
240
|
+
| `phone` | `+1 (555) 867-5309`, `415.555.2671` | Multi-format regex |
|
|
241
|
+
| `ssn` | `123-45-6789` | Regex + exclusions |
|
|
242
|
+
| `name` | `Dear Alice Smith`, `Patient: John Doe` | Heuristic (greeting/title context) |
|
|
243
|
+
| `date-of-birth` | `1990-04-15`, `April 15, 1990` | Multi-format regex |
|
|
244
|
+
| `address` | `123 Main St, Springfield, IL 62701` | Pattern + keyword heuristic |
|
|
245
|
+
| `passport` | `A12345678`, `P1234567` | Country-specific patterns |
|
|
246
|
+
| `drivers-license` | `D123-4567-8901`, `F123-456-78-910-1` | Multi-state/country patterns |
|
|
247
|
+
| `national-id` | Aadhaar, NHS number, NINO, CPF, etc. | Country-specific patterns |
|
|
248
|
+
|
|
249
|
+
### Financial
|
|
250
|
+
|
|
251
|
+
| Category | Examples | Validator / Notes |
|
|
252
|
+
|--------------------|-------------------------------------------|-------------------------------------|
|
|
253
|
+
| `credit-card` | `4111 1111 1111 1111` | Luhn algorithm |
|
|
254
|
+
| `iban` | `GB82 WEST 1234 5698 7654 32` | MOD-97 (ISO 13616) |
|
|
255
|
+
| `bank-account` | Routing + account number pairs | Pattern regex |
|
|
256
|
+
| `cryptocurrency` | BTC, ETH, XRP wallet addresses | Format regex per chain |
|
|
257
|
+
| `tax-id` | EIN `12-3456789`, VAT `GB123456789` | Multi-country patterns |
|
|
258
|
+
|
|
259
|
+
### Healthcare
|
|
260
|
+
|
|
261
|
+
| Category | Examples | Validator / Notes |
|
|
262
|
+
|-------------------|---------------------------------------------|-------------------------------------|
|
|
263
|
+
| `medical-record` | `MRN: 1234567`, `MR#00456789` | Keyword + pattern |
|
|
264
|
+
| `health-insurance`| `Subscriber ID`, `Group/Member #` | Pattern regex |
|
|
265
|
+
| `prescription` | `Rx# 1234567`, DEA numbers | Pattern + DEA checksum |
|
|
266
|
+
|
|
267
|
+
### Digital Identity
|
|
268
|
+
|
|
269
|
+
| Category | Examples | Validator / Notes |
|
|
270
|
+
|----------------|------------------------------------------------|------------------------------|
|
|
271
|
+
| `ipv4` | `192.168.1.1`, `10.0.0.0/8` | Octet-range regex |
|
|
272
|
+
| `ipv6` | `2001:0db8::8a2e:0370:7334` | Regex |
|
|
273
|
+
| `url` | `https://example.com/path?q=1` | http/https (scheme-required) |
|
|
274
|
+
| `api-key` | Bearer tokens, AWS keys, GitHub PATs, etc. | Pattern regex per provider |
|
|
275
|
+
| `social-media` | `@username`, profile URLs, handles | Pattern regex |
|
|
276
|
+
|
|
277
|
+
### Vehicles & Transportation
|
|
278
|
+
|
|
279
|
+
| Category | Examples | Validator / Notes |
|
|
280
|
+
|-------------------|---------------------------------------|------------------------------|
|
|
281
|
+
| `vin` | `1HGCM82633A004352` | VIN checksum (pos 9) |
|
|
282
|
+
| `license-plate` | `ABC-1234`, `AB12CDE` | Multi-country patterns |
|
|
283
|
+
| `tracking-number` | FedEx, UPS, USPS, DHL tracking codes | Pattern regex per carrier |
|
|
284
|
+
|
|
285
|
+
### Government & Legal
|
|
286
|
+
|
|
287
|
+
| Category | Examples | Validator / Notes |
|
|
288
|
+
|-----------------------|---------------------------------------|------------------------------|
|
|
289
|
+
| `case-number` | `2023-CV-001234`, `CR-2022-5678` | Pattern regex |
|
|
290
|
+
| `company-registration`| EIN, CRN, SIREN, ABN, etc. | Multi-country patterns |
|
|
291
|
+
|
|
292
|
+
---
|
|
293
|
+
|
|
294
|
+
## Compliance Presets
|
|
295
|
+
|
|
296
|
+
Built-in presets pre-configure which categories are detected and which default strategy is applied.
|
|
297
|
+
|
|
298
|
+
| Preset | Categories Covered | Default Strategy |
|
|
299
|
+
|------------|--------------------------------------------------------------------------|-------------------|
|
|
300
|
+
| `gdpr` | All personal, financial, healthcare, and digital identity categories | `pseudonymize` |
|
|
301
|
+
| `hipaa` | All 18 HIPAA Safe Harbor PHI identifiers | `redact` |
|
|
302
|
+
| `ccpa` | Consumer identifiers, financial data, online activity | `redact` |
|
|
303
|
+
| `pci-dss` | Credit card, cardholder name, bank account, address | `mask` (last 4) |
|
|
304
|
+
| `sox` | Financial identifiers for audit trails | `redact` |
|
|
305
|
+
| `ferpa` | Student PII — name, SSN, DOB, address | `redact` |
|
|
306
|
+
|
|
307
|
+
```ts
|
|
308
|
+
import { anonymize, getPreset, PRESET_REGISTRY } from "anonyma";
|
|
309
|
+
|
|
310
|
+
// Apply a preset
|
|
311
|
+
anonymize(text, { preset: "hipaa" });
|
|
312
|
+
|
|
313
|
+
// Extend a preset — add API key detection on top of GDPR
|
|
314
|
+
anonymize(text, { preset: "gdpr", enabledCategories: { "api-key": true } });
|
|
315
|
+
|
|
316
|
+
// Inspect a preset's configuration
|
|
317
|
+
const hipaa = getPreset("hipaa");
|
|
318
|
+
console.log(hipaa.categories); // [...18 categories...]
|
|
319
|
+
```
|
|
320
|
+
|
|
321
|
+
---
|
|
322
|
+
|
|
323
|
+
## Reversible Tokenization
|
|
324
|
+
|
|
325
|
+
`tokenize()` replaces PII with opaque placeholder tokens and returns a mapping for lossless restoration.
|
|
326
|
+
|
|
327
|
+
```ts
|
|
328
|
+
import { tokenize, detokenize } from "anonyma";
|
|
329
|
+
|
|
330
|
+
const { text, mapping, tokens } = tokenize("alice@example.com called 555-867-5309", {
|
|
331
|
+
format: "bracket", // "[EMAIL_0001]", "[PHONE_0001]" (default)
|
|
332
|
+
// format: "angle", // "<Email_1>", "<Phone_1>" (LLM-friendly)
|
|
333
|
+
// format: "custom", tokenTemplate: (cat, n) => `{{${cat}_${n}}}`,
|
|
334
|
+
deterministic: true, // same value → same token (default: true)
|
|
335
|
+
});
|
|
336
|
+
// text: "[EMAIL_0001] called [PHONE_0001]"
|
|
337
|
+
|
|
338
|
+
const { text: restored } = detokenize(text, mapping);
|
|
339
|
+
// "alice@example.com called 555-867-5309"
|
|
340
|
+
```
|
|
341
|
+
|
|
342
|
+
---
|
|
343
|
+
|
|
344
|
+
## LLM Pipeline Helpers
|
|
345
|
+
|
|
346
|
+
Sanitize user input before sending it to a language model, then restore PII from the model's response.
|
|
347
|
+
|
|
348
|
+
```ts
|
|
349
|
+
import { sanitizeForLLM, restoreFromLLM } from "anonyma";
|
|
350
|
+
|
|
351
|
+
// 1. Replace PII with bracket tokens before the LLM sees it
|
|
352
|
+
const { text: prompt, mapping } = sanitizeForLLM("Send invoice to alice@example.com");
|
|
353
|
+
// prompt: "Send invoice to [EMAIL_0001]"
|
|
354
|
+
|
|
355
|
+
// 2. Call your LLM
|
|
356
|
+
const response = await callLLM(prompt);
|
|
357
|
+
// response: "I have sent the invoice to [EMAIL_0001]."
|
|
358
|
+
|
|
359
|
+
// 3. Restore original values in the LLM's response
|
|
360
|
+
const { text: final, unresolved } = restoreFromLLM(response, mapping);
|
|
361
|
+
// final: "I have sent the invoice to alice@example.com."
|
|
362
|
+
// unresolved: [] (any tokens the LLM dropped are reported here)
|
|
363
|
+
```
|
|
364
|
+
|
|
365
|
+
---
|
|
366
|
+
|
|
367
|
+
## Batch Processing
|
|
368
|
+
|
|
369
|
+
Process arrays of strings efficiently. Each item is independent — failures don't abort the batch.
|
|
370
|
+
|
|
371
|
+
```ts
|
|
372
|
+
import { anonymizeBatch, anonymizeBatchAsync, tokenizeBatch, detectBatch } from "anonyma";
|
|
373
|
+
|
|
374
|
+
// Sync batch
|
|
375
|
+
const results = anonymizeBatch(["alice@example.com", "192.0.2.1", "bad\x00input"]);
|
|
376
|
+
for (const r of results) {
|
|
377
|
+
if (r.ok) console.log(r.value.text);
|
|
378
|
+
else console.error(`Item ${r.index}:`, r.error.message);
|
|
379
|
+
}
|
|
380
|
+
|
|
381
|
+
// Async batch (supports hash/encrypt strategies, runs in parallel)
|
|
382
|
+
const asyncResults = await anonymizeBatchAsync(texts, {
|
|
383
|
+
defaultStrategy: { strategy: "hash", pepper: "secret" },
|
|
384
|
+
concurrency: 10,
|
|
385
|
+
});
|
|
386
|
+
|
|
387
|
+
// Batch tokenization
|
|
388
|
+
const tokenResults = tokenizeBatch(texts, { format: "bracket" });
|
|
389
|
+
|
|
390
|
+
// Batch detection only
|
|
391
|
+
const detections = detectBatch(texts);
|
|
392
|
+
```
|
|
393
|
+
|
|
394
|
+
---
|
|
395
|
+
|
|
396
|
+
## Streaming
|
|
397
|
+
|
|
398
|
+
Process data through WHATWG `TransformStream` pipelines (Node ≥ 18 / browsers with Streams API).
|
|
399
|
+
|
|
400
|
+
```ts
|
|
401
|
+
import {
|
|
402
|
+
createAnonymizeStream,
|
|
403
|
+
createAnonymizeStreamAsync,
|
|
404
|
+
createTokenizeStream,
|
|
405
|
+
} from "anonyma/stream";
|
|
406
|
+
|
|
407
|
+
// Anonymize line by line
|
|
408
|
+
const readable = ReadableStream.from(lines);
|
|
409
|
+
const anonymized = readable.pipeThrough(
|
|
410
|
+
createAnonymizeStream({ defaultStrategy: { strategy: "mask" } })
|
|
411
|
+
);
|
|
412
|
+
for await (const { text } of anonymized) {
|
|
413
|
+
process.stdout.write(text + "\n");
|
|
414
|
+
}
|
|
415
|
+
|
|
416
|
+
// Async stream (for hash/encrypt strategies)
|
|
417
|
+
const asyncStream = createAnonymizeStreamAsync({ preset: "hipaa" });
|
|
418
|
+
|
|
419
|
+
// Tokenize stream
|
|
420
|
+
const tokenStream = createTokenizeStream({ format: "bracket" });
|
|
421
|
+
```
|
|
422
|
+
|
|
423
|
+
---
|
|
424
|
+
|
|
425
|
+
## Encryption Strategy
|
|
426
|
+
|
|
427
|
+
Reversible AES-256-GCM encryption using the Web Crypto API. Requires Node.js ≥ 18.
|
|
428
|
+
|
|
429
|
+
```ts
|
|
430
|
+
import { encrypt, decrypt } from "anonyma";
|
|
431
|
+
|
|
432
|
+
// Encrypt with a passphrase (key derived via PBKDF2)
|
|
433
|
+
const ciphertext = await encrypt("alice@example.com", { passphrase: "s3cr3t" });
|
|
434
|
+
// "base64:<iv>:<ciphertext>"
|
|
435
|
+
|
|
436
|
+
// Or supply raw key bytes (16 or 32 bytes for AES-128/256)
|
|
437
|
+
const ct = await encrypt("alice@example.com", { keyBytes: myKeyBytes, encoding: "hex" });
|
|
438
|
+
|
|
439
|
+
// Decrypt
|
|
440
|
+
const original = await decrypt(ciphertext, { passphrase: "s3cr3t" });
|
|
441
|
+
// "alice@example.com"
|
|
442
|
+
|
|
443
|
+
// Use as a strategy inside anonymize
|
|
444
|
+
const { text } = await anonymizeAsync("alice@example.com", {
|
|
445
|
+
rules: [{ category: "email", strategy: { strategy: "encrypt", passphrase: "s3cr3t" } }],
|
|
446
|
+
});
|
|
447
|
+
```
|
|
448
|
+
|
|
449
|
+
---
|
|
450
|
+
|
|
451
|
+
## Synthesis Strategy
|
|
452
|
+
|
|
453
|
+
Replace PII with structurally valid, format-preserving synthetic data. Deterministic when a seed is provided.
|
|
454
|
+
|
|
455
|
+
```ts
|
|
456
|
+
import { synthesize } from "anonyma";
|
|
457
|
+
import { anonymize } from "anonyma";
|
|
458
|
+
|
|
459
|
+
synthesize("alice@example.com", { category: "email", seed: "project-x" });
|
|
460
|
+
// "carol.smith42@example.com"
|
|
461
|
+
|
|
462
|
+
anonymize("Call 555-867-5309 or email alice@example.com", {
|
|
463
|
+
defaultStrategy: { strategy: "synthesize", seed: "my-seed" },
|
|
464
|
+
}).text;
|
|
465
|
+
// "Call +1-312-408-7291 or email bob.jones15@test.org"
|
|
466
|
+
```
|
|
467
|
+
|
|
468
|
+
---
|
|
469
|
+
|
|
470
|
+
## Validators
|
|
471
|
+
|
|
472
|
+
Standalone checksum and format validators are exported from `"anonyma/validators"`. They are used internally by detectors but can also be used directly.
|
|
473
|
+
|
|
474
|
+
```ts
|
|
475
|
+
import {
|
|
476
|
+
luhn, // Luhn (ISO/IEC 7812) — credit cards
|
|
477
|
+
verhoeff, // Verhoeff — Indian Aadhaar
|
|
478
|
+
nhsMod11, // NHS mod-11 — UK NHS numbers
|
|
479
|
+
cpfChecksum, // CPF — Brazilian tax IDs
|
|
480
|
+
vinChecksum, // VIN position-9 check digit
|
|
481
|
+
deaChecksum, // DEA number check — US prescriptions
|
|
482
|
+
ibanMod97, // MOD-97 — IBAN (ISO 13616)
|
|
483
|
+
ninoValid, // NINO format — UK National Insurance
|
|
484
|
+
aadhaarFormat,// Aadhaar format — Indian national ID
|
|
485
|
+
} from "anonyma/validators";
|
|
486
|
+
|
|
487
|
+
luhn("4111111111111111"); // true
|
|
488
|
+
ibanMod97("GB82WEST12345698765432"); // true
|
|
489
|
+
nhsMod11("943-476-5919"); // true
|
|
490
|
+
```
|
|
491
|
+
|
|
492
|
+
---
|
|
493
|
+
|
|
494
|
+
## Plugin Architecture
|
|
495
|
+
|
|
496
|
+
Extend anonyma with custom detectors, strategies, and validators using `AnonymaPlugin`.
|
|
497
|
+
|
|
498
|
+
```ts
|
|
499
|
+
import { createAnonymizer } from "anonyma";
|
|
500
|
+
import type { AnonymaPlugin } from "anonyma";
|
|
501
|
+
|
|
502
|
+
const myPlugin: AnonymaPlugin = {
|
|
503
|
+
name: "my-plugin",
|
|
504
|
+
detectors: {
|
|
505
|
+
// Override or add custom category detection
|
|
506
|
+
"employee-id": (text) => {
|
|
507
|
+
const matches = [];
|
|
508
|
+
for (const m of text.matchAll(/\bEMP-\d{6}\b/g)) {
|
|
509
|
+
matches.push({ category: "employee-id", value: m[0], start: m.index!, end: m.index! + m[0].length, confidence: 0.95 });
|
|
510
|
+
}
|
|
511
|
+
return matches;
|
|
512
|
+
},
|
|
513
|
+
},
|
|
514
|
+
};
|
|
515
|
+
|
|
516
|
+
const anonymizer = createAnonymizer({ plugins: [myPlugin] });
|
|
517
|
+
anonymizer.anonymize("Employee EMP-001234 called.").text;
|
|
518
|
+
// "Employee [REDACTED] called."
|
|
519
|
+
```
|
|
520
|
+
|
|
521
|
+
---
|
|
522
|
+
|
|
523
|
+
## AI Integration
|
|
524
|
+
|
|
525
|
+
### OpenAI / Anthropic Function Calling
|
|
526
|
+
|
|
527
|
+
```ts
|
|
528
|
+
import {
|
|
529
|
+
ANONYMIZE_TOOL_DEFINITION,
|
|
530
|
+
DETECT_TOOL_DEFINITION,
|
|
531
|
+
HAS_PII_TOOL_DEFINITION,
|
|
532
|
+
ANONYMIZE_OBJECT_TOOL_DEFINITION,
|
|
533
|
+
} from "anonyma/schemas";
|
|
534
|
+
|
|
535
|
+
// Pass directly to the tools array:
|
|
536
|
+
const response = await openai.chat.completions.create({
|
|
537
|
+
model: "gpt-4o",
|
|
538
|
+
tools: [
|
|
539
|
+
{ type: "function", function: ANONYMIZE_TOOL_DEFINITION },
|
|
540
|
+
{ type: "function", function: HAS_PII_TOOL_DEFINITION },
|
|
541
|
+
{ type: "function", function: ANONYMIZE_OBJECT_TOOL_DEFINITION },
|
|
542
|
+
],
|
|
543
|
+
messages: [{ role: "user", content: "Anonymize: alice@example.com" }],
|
|
544
|
+
});
|
|
545
|
+
```
|
|
546
|
+
|
|
547
|
+
### MCP Tool Definition
|
|
548
|
+
|
|
549
|
+
```ts
|
|
550
|
+
import { ANONYMA_MANIFEST } from "anonyma/schemas";
|
|
551
|
+
|
|
552
|
+
// Use the capability manifest to let an AI agent discover anonyma's tools:
|
|
553
|
+
const manifest = ANONYMA_MANIFEST;
|
|
554
|
+
console.log(manifest.capabilities.anonymize.strategies);
|
|
555
|
+
```
|
|
556
|
+
|
|
557
|
+
### Runtime Validation with Zod
|
|
558
|
+
|
|
559
|
+
```ts
|
|
560
|
+
import { AnonymizeOptionsSchema } from "anonyma/schemas";
|
|
561
|
+
import { anonymize } from "anonyma";
|
|
562
|
+
|
|
563
|
+
// Validate untrusted input (e.g. from an HTTP request):
|
|
564
|
+
const opts = AnonymizeOptionsSchema.parse(req.body.options);
|
|
565
|
+
const result = anonymize(req.body.text as string, opts);
|
|
566
|
+
```
|
|
567
|
+
|
|
568
|
+
---
|
|
569
|
+
|
|
570
|
+
## Subpath Imports
|
|
571
|
+
|
|
572
|
+
| Import path | Contents |
|
|
573
|
+
|--------------------------|---------------------------------------------------------------------------|
|
|
574
|
+
| `"anonyma"` | Core API — no Zod dependency |
|
|
575
|
+
| `"anonyma/detectors"` | All detectors + `DETECTOR_REGISTRY` + `AGGRESSIVE_DETECTOR_REGISTRY` |
|
|
576
|
+
| `"anonyma/schemas"` | Zod schemas, JSON schemas, AI/MCP tool definitions (requires `zod`) |
|
|
577
|
+
| `"anonyma/validators"` | Standalone checksum validators (Luhn, MOD-97, NHS, VIN, etc.) |
|
|
578
|
+
| `"anonyma/stream"` | WHATWG `TransformStream` wrappers (Node ≥ 18 / browsers) |
|
|
579
|
+
| `"anonyma/crypto"` | Low-level Web Crypto utilities |
|
|
580
|
+
|
|
581
|
+
---
|
|
582
|
+
|
|
583
|
+
## API Documentation
|
|
584
|
+
|
|
585
|
+
Full API reference: [docs/api.md](docs/api.md)
|
|
586
|
+
|
|
587
|
+
---
|
|
588
|
+
|
|
589
|
+
## Requirements
|
|
590
|
+
|
|
591
|
+
- **Node.js** ≥ 18 (for `hash`, `encrypt`, and streaming features — Web Crypto + `TransformStream`)
|
|
592
|
+
- TypeScript ≥ 5.0 (optional)
|
|
593
|
+
- Zod ≥ 3.23 (optional peer-dependency for `"anonyma/schemas"`)
|
|
594
|
+
|
|
595
|
+
---
|
|
596
|
+
|
|
597
|
+
## Contributing
|
|
598
|
+
|
|
599
|
+
See [CONTRIBUTING.md](CONTRIBUTING.md).
|
|
600
|
+
|
|
601
|
+
---
|
|
602
|
+
|
|
603
|
+
## License
|
|
604
|
+
|
|
605
|
+
MIT — see [LICENSE](LICENSE).
|