@link-foundation/sensitive-data-sanitizer 0.20.0 → 0.21.1

package/CHANGELOG.md

@@ -1,5 +1,17 @@
 # Changelog
 
+## 0.21.1
+
+### Patch Changes
+
+- e0e05a4: Cover 34 missing strict personal-attribute terms, preserve their typed placeholders over generic prose names, cache caller vocabulary and cue sets, avoid unnecessary placeholder scans and JSON parsing, and preserve exact public technology names in Cyrillic prose.
+
+## 0.21.0
+
+### Minor Changes
+
+- d1d7a22: Resolve the sanitizer requests of issues 130–141. Placeholder mode accepts `transformation.existing: 'keep'` for a second pass over sanitized text, types whole JSON values (`Sensitive-1`, `Age-1`), and keeps pronouns, cue words and sentence-initial greetings and verbs readable beside `Person-n`; compact organization and location spans become one typed token. `reservedPlaceholderTypes()` and `isPlaceholderToken()` expose the namespace. `strict` covers more nationality, religion, health, sexuality, conviction and trade-union wording, and `vocabulary.personalAttributes` adds caller stems. A credentials-only entry (`@link-foundation/sensitive-data-sanitizer/credentials`) starts about three times faster, and both entries support `require`. Credential context stays readable and masks stay stable; configuration settings are kept and an `exempt(finding, text)` hook can keep findings by policy. New detections cover CI credential markers, XML attributes, hash rockets, markup values and plural credential flags. Wrapped base64 and hex keep their layout; candidates are bounded per 64 KiB window; `knownSecrets` also match their hex and base64 spellings. Environment secret discovery selects `PGPASSWORD`-style names and cookies, skips more configuration names, and accepts `minLength`.
+
 ## 0.20.0
 
 ### Minor Changes

package/README.md

@@ -20,6 +20,12 @@ two-pass publication:
 sensitive-data-sanitizer redact session.jsonl --stream --jsonl --two-pass --output safe.jsonl
 ```
 
+For loggers that only mask credentials, import the
+[credentials-only entry](docs/API.md#credentials-only-entry)
+`@link-foundation/sensitive-data-sanitizer/credentials`: it skips the
+personal-data modules and starts about three times faster. Both entries also
+support `require`.
+
 For synchronous log hooks, `createSyncSanitizer` supports custom detectors and
 verified callback transformations. Rule/type controls, caller vocabulary,
 structured field selectors and `compileKnown` configure reusable policies; see
@@ -89,6 +95,13 @@ Known Russian names match in all six cases and in any word order, with `ё`/`е`
 spelling variants and initials (`с Петровым Данилом Сергеевичем`, `Петров Д. С.`).
 A part of a longer name is matched alone only when it is listed itself.
 
+With `properNames: 'capitalized'`, placeholder output keeps the sentence around a
+name (`Кандидат Person-1`, `Меня зовут Person-1`, `Благодарим Вас, Person-1`), and
+`transformation.existing: 'keep'` lets a second pass over sanitized text keep its
+`Type-n` tokens and number new values after them. `isPlaceholderToken` and
+`reservedPlaceholderTypes` expose the namespace; see the
+[second-pass example](examples/second-pass-placeholders.mjs).
+
 For irreversible output, choose `transformation: { mode: 'placeholder', map:
 false }`: no restore map is returned or written. Detection limits still apply
 in either mode. See the [placeholder API](docs/API.md#reversible-placeholders),
@@ -248,7 +261,7 @@ See [contributing](docs/CONTRIBUTING.md). `add`, `multiply` and `delay` live in
 
 The universal app's preview images regenerate on pushes to the default branch. Run `npm run example:web:preview-images` to regenerate them locally; see [the template guide](docs/TEMPLATE-PIPELINE.md) for setup.
 
-Automatic `--env-secrets` discovery skips authentication configuration names such as `GIT_AUTH_TRANSPORT`, `AUTH_MODE` and `AUTH_METHOD`. Explicit `knownSecretsFromEnv(env, names)` selections still include their exact values.
+Automatic `--env-secrets` discovery also selects `PGPASSWORD`-style suffixes and cookies, and skips configuration names such as `GIT_AUTH_TRANSPORT`, `AUTH_MODE`, `*_AUTH_GUARD`, `MAX_TOKEN_COUNT`, `SSH_KEY_PATH`, `SECRET_NAME`, `AWS_ACCESS_KEY_ID` and `GIT_CONFIG_KEY_<n>`; `knownSecretsFromEnv(env, undefined, { minLength: 8 })` also skips short values. Explicit `knownSecretsFromEnv(env, names)` selections still include their exact values.
 
 For shared tool logs, `--credentials-only --env-secrets --gh-config` preserves
 public account details while redacting credential patterns and exact local

package/bin/sensitive-data-sanitizer.js

@@ -1,4 +1,5 @@
 #!/usr/bin/env node
+import '../src/personal-features.js';
 import { readFileSync, realpathSync } from 'node:fs';
 import { fileURLToPath, URL } from 'node:url';
 import { resolve } from 'node:path';

package/docs/API.md

@@ -1,6 +1,6 @@
 # Library and detector contract
 
-Import the ESM root entry; complete declarations are in `src/index.d.ts`. Inputs are strings, offsets are UTF-16 code units, starts are zero-based, and ends are exclusive. Returned line and column numbers are one-based; columns also count UTF-16 units.
+Import the root entry with `import` or `require`; complete declarations are in `src/index.d.ts`. Inputs are strings, offsets are UTF-16 code units, starts are zero-based, and ends are exclusive. Returned line and column numbers are one-based; columns also count UTF-16 units.
 
 ## Methods
 
@@ -23,6 +23,31 @@ Import the ESM root entry; complete declarations are in `src/index.d.ts`. Inputs
 
 `redactions` counts merged replacement spans. `findings` can contain several detector hits for one replacement. Findings have `{start, end, type, category, rule, line, column}`; no text, quote, match, entropy score, provider details, or raw diagnostic is copied. The optional `strict-attributes` debug event reports only detected and suppressed match counts. Debugging is off by default; `debug(event)` receives value-free counts, rule timings, residual rounds and registry event metadata (see `DebugEvent` in the TypeScript declarations).
 
+### Credentials-only entry
+
+`@link-foundation/sensitive-data-sanitizer/credentials` exports `inspect`,
+`sanitize`, `redact`, `createSanitizer`, `createSyncSanitizer`, `REDACTED`,
+`entropy`, `builtInRules`, `tokenChecksum`, `tokenWithChecksum` and the
+`knownSecretsFrom*` helpers. It always runs with `credentialsOnly: true` and
+does not load the personal-data detectors, faker or libphonenumber, so it starts
+about three times faster than the root entry (about 0.25 s against 0.75 s on
+Node.js 26). Use it for loggers that must mask every line from the first write:
+
+```js
+const {
+  createSyncSanitizer,
+  knownSecretsFromEnv,
+} = require('@link-foundation/sensitive-data-sanitizer/credentials');
+const mask = createSyncSanitizer({ knownSecrets: knownSecretsFromEnv() });
+console.log(mask.sanitize('Authorization: Bearer abc123def456ghi789').text);
+// Authorization: Bearer [REDACTED]
+```
+
+Options that need the personal-data modules, `credentialsOnly: false`,
+`mode: 'fake'` and `properNames`, fail with `ERR_CONFIG`. Known personal values
+(`knownPersonal`) still work. Both entries offer `import`, `require` (Node.js
+22.12 or later) and `default` export conditions.
+
 ## Options
 
 ### Replace only known values
@@ -71,6 +96,31 @@ Unicode letters, marks, digits and underscores bound those tokens. Ordinary
 technical names such as GPT-4, ISO-9001 and Windows-10 are not reserved. The map's format cannot change
 on reuse. A malformed map or ambiguous exact restoration blocks output.
 
+`existing: 'keep'` on the placeholder transformation accepts input that already
+contains tokens of the chosen namespace, for example the output of an earlier
+pass or a model answer that quotes tokens. Those tokens stay as they are and are
+never replaced, faked or treated as names; new values of a type are numbered
+after the highest kept number of that type:
+
+```js
+const policy = { transformation: { mode: 'placeholder', existing: 'keep' } };
+sanitize('Person-1 встретил Анну Смирнову', policy).text;
+// Person-1 встретил Person-2
+```
+
+A second pass over sanitized text is then a fixed point. The default
+`existing: 'error'` keeps the `ERR_PLACEHOLDER_COLLISION` check; other values and
+a mix with per-type `transformations` are `ERR_CONFIG`. A kept token restores to
+itself, so the map still reconstructs every newly replaced value exactly.
+
+`reservedPlaceholderTypes()` returns the sorted, frozen list of every type a
+built-in rule can turn into a token. `isPlaceholderToken(text, {format?, types?})`
+checks whether `text` is one whole token of the namespace (`Person-1`,
+`UsBankNumber-12`, `<PERSON_3>` with `format: '<{TYPE}_{n}>'`); `types` adds
+custom types such as those of `createPatternDetector`. Both are also exported by
+the credentials-only entry. The namespace pattern is compiled once per format and
+cached.
+
 `createRestoreMap({format?})` creates an empty reusable map. Pass a map as
 `transformation.map` to continue a corpus; the operation updates that same
 object. A `createSanitizer()` engine also keeps its registry across calls.
@@ -151,6 +201,13 @@ JSON keys remain unchanged by automatic personal detection in all transformation
 modes. Explicit known values and credentials retain priority. Parseability and
 duplicate-key checks remain enforced. Plain redact/fake policies retain their
 existing metadata/key behavior. Use `structured` for valid structured input.
+A whole JSON value replaced by a structured rule (`sensitiveFields`, or a
+personal field such as `age` replaced with its JSON type) becomes one JSON
+string token typed by its field: `{"notes":"любой текст"}` reads
+`{"notes":"Sensitive-1"}` and `{"age":34}` reads `{"age":"Age-1"}`. Such findings
+carry `valueType` when the field has a personal type. `restore()` brings back
+the original JSON value, including numbers and objects. Other modes keep
+`"[REDACTED]"`.
 The normal encoding limits and credential redaction policy continue to apply.
 
 CLI: `redact FILE --placeholder --map private.json` creates or continues a map;
@@ -169,7 +226,7 @@ new document recognizers; `documentEntities: false` turns them off explicitly.
 | `scripts`                | `['Cyrillic']`; Unicode Script aliases that activate the prose rule; `[]` disables only script activation                                                        |
 | `profile`                | Off; `'publication'` sets a 0.3 threshold and identity masking; `'documents'` enables document entities                                                          |
 | `knownOnly`              | `false`; disable all automatic detection (including credentials, Secretlint and custom detectors); use explicit findings, known values and reused map identities |
-| `knownSecrets`           | `[]`; exact literal credential values, regardless of length or format                                                                                            |
+| `knownSecrets`           | `[]`; exact literal credential values, regardless of length or format, plus their standalone hex (either case) and base64/base64url spellings                    |
 | `knownPersonal`          | `[]`; `{type, value}` values with whole-word Unicode case-insensitive matching and bounded Russian forms; list parts to match them alone                         |
 | `publicEntities`         | `[]`; exact PERSON/ORGANIZATION/EDUCATION/LOCATION/EMAIL exception with HTTPS evidence and review date                                                           |
 | `findings`               | `[]`; validated detections for the original input, supplied by the caller                                                                                        |
@@ -196,7 +253,7 @@ new document recognizers; `documentEntities: false` turns them off explicitly.
 | `strict`                 | `false`; aggressive stored-log personal attributes and bare values                                                                                               |
 | `structuralFields`       | `[]`; declared property names suppress weak name/ID/entropy heuristics in their string values                                                                    |
 
-Unknown options and invalid limits are errors. `knownSecretsFromEnv(env, names?)` discovers credential-named variables (excluding auth configuration names) when names are omitted, or uses an explicit selection, deduplicating nonempty values. It never invokes shell commands or reads auth files. `personalVariants(entries)` generates numeric and ISO/day-first date spellings; name case variants are already handled by detection. Neither helper invents transliterations.
+Unknown options and invalid limits are errors. `knownSecretsFromEnv(env, names?, {minLength?})` discovers credential-named variables (excluding configuration names) when names are omitted, or uses an explicit selection, deduplicating nonempty values (or values of at least `minLength` characters). It never invokes shell commands or reads auth files. `personalVariants(entries)` generates numeric and ISO/day-first date spellings; name case variants are already handled by detection. Neither helper invents transliterations.
 
 ## Required detectors
 
@@ -260,7 +317,7 @@ Generated replacements carry `faked: true`. A private bounded registry recognize
 
 Faking is pseudonymisation: a private key and retained ages, relations and structure can permit linking. Keep the key out of published data. See [the realistic fake example](../examples/realistic-fakes.mjs) and [the complete requirement/research record](case-studies/issue-24/REQUIREMENTS.md).
 
-`preserveEncoding: true` sanitizes decoded payloads with the same required engine union, re-encodes them and verifies decoding reproduces the sanitized content. Supported formats include base64/base64url, hex, percent, numeric HTML and UTF-8 byte escapes. Escaped JSON string content is sanitized and re-encoded even under default redaction so nested session JSON stays parseable. Malformed or unsupported encodings block opted-in preservation. Known encoding candidates and nesting are bounded.
+`preserveEncoding: true` sanitizes decoded payloads with the same required engine union, re-encodes them and verifies decoding reproduces the sanitized content. Supported formats include base64/base64url, hex, percent, numeric HTML and UTF-8 byte escapes. Escaped JSON string content is sanitized and re-encoded even under default redaction so nested session JSON stays parseable. Wrapped base64 and hex keep their line width and line separator (`\n` or `\r\n`), and a wrapped run is reported as one finding. When the sanitized payload cannot be re-encoded in a form the decoder recognizes (for example hex shorter than 32 digits), the encoded text is masked as it stands rather than blocking the input. Known encoding candidates and nesting are bounded.
 
 ## Identity policy and structured sessions
 
@@ -308,9 +365,15 @@ status identifiers. Explicit `knownPersonal` values override these heuristics.
 
 Oversized encoded runs receive a conservative complete-run `ENCODED_LIMIT`
 redaction rather than aborting the record. Decoding probes are capped at 4,096
-characters; full decoding stays capped at 8,192 characters. Whole-input,
-candidate-count, record and worker limits still apply. This fallback can remove
-large non-sensitive encoded outputs; its finding makes that decision auditable.
+characters; full decoding stays capped at 8,192 characters. A multi-line run is
+treated as one wrapped payload only when its probe decodes as text or its lines
+share one base64 width (a multiple of four, standard alphabet, not plain hex);
+lists of identifiers or commit hashes are scanned line by line. Up to 4,096
+textually decodable candidates are scanned per 64 KiB window of the input;
+further candidates in a denser window receive `ENCODED_LIMIT` redactions rather
+than an `ERR_LIMIT` for the whole text. Whole-input, record and worker limits
+still apply. This fallback can remove large non-sensitive encoded outputs; its
+finding makes that decision auditable.
 
 ## Streaming and worker publication
 
@@ -451,12 +514,28 @@ findings retain the normal engine union, confidence rules and verification.
 This keeps EMAIL/account-ID values without per-value `publicEntities` entries.
 The default policy continues to protect personal data.
 
-`knownSecretsFromEnv(env = process.env, names?)` discovers nonempty exact values
-when `names` is omitted, using
-`/(?:^|_)(?:TOKEN|(?:API|ACCESS|PRIVATE)?KEY|SECRET|PASSWORD|PASSWD|CREDENTIALS?|AUTH|PAT)(?:_|$)/i`.
-This covers `MY_SERVICE_TOKEN`, `CLAUDE_CODE_OAUTH_TOKEN`, `CODEX_API_KEY`,
-`OPENCODE_API_KEY`, `DASHSCOPE_API_KEY`, `HF_TOKEN` and `TELEGRAM_BOT_TOKEN`.
-Automatic discovery excludes names ending in `AUTH_TRANSPORT`, `AUTH_MODE`, `AUTH_METHOD`, `AUTH_TYPE`, `AUTH_URL`, `AUTH_URI`, `AUTH_HOST`, `AUTH_HOSTNAME`, `AUTH_PATH`, `AUTH_ENABLED`, `AUTH_DISABLED`, `AUTH_PORT`, `AUTH_PROTOCOL` or `AUTH_SCHEME` (including prefixed forms such as `HIVE_MIND_GIT_AUTH_TRANSPORT`). Credential names such as `AUTH_TOKEN` still contribute their exact values, including short/low-entropy values; no value-shape filter can reliably distinguish passwords from configuration. An explicit array bypasses the name exclusions; `[]` selects nothing.
+`knownSecretsFromEnv(env = process.env, names?, { minLength = 1 }?)` discovers
+nonempty exact values when `names` is omitted. A name is selected when it has a
+credential keyword between `_` boundaries
+(`/(?:^|_)(?:TOKEN|(?:API|ACCESS|PRIVATE)?KEY|SECRET|PASSWORD|PASSWD|CREDENTIALS?|O?AUTH|PAT|COOKIE)(?:_|$)/i`)
+or ends in `PASSWORD`, `PASSWD`, `TOKEN` or `COOKIE` without one. This covers
+`MY_SERVICE_TOKEN`, `CLAUDE_CODE_OAUTH_TOKEN`, `CODEX_API_KEY`,
+`OPENCODE_API_KEY`, `DASHSCOPE_API_KEY`, `HF_TOKEN`, `TELEGRAM_BOT_TOKEN`,
+`PGPASSWORD`, `GITHUBTOKEN`, `OAUTH` and `SESSION_COOKIE`.
+Automatic discovery excludes names whose last segment describes configuration
+rather than a credential: `_TRANSPORT`, `_MODE`, `_METHOD`, `_TYPE`, `_URL`,
+`_URI`, `_HOST`, `_HOSTNAME`, `_PATH`, `_ENABLED`, `_DISABLED`, `_PORT`,
+`_PROTOCOL`, `_SCHEME`, `_GUARD`, `_COUNT`, `_LIMIT`, `_FILE`, `_FILENAME`,
+`_DIR`, `_DIRECTORY`, `_NAME` and `_ID` (for example `HIVE_MIND_GIT_AUTH_TRANSPORT`,
+`HIVE_MIND_ROUTER_AUTH_GUARD`, `MAX_TOKEN_COUNT`, `SSH_KEY_PATH`, `SECRET_NAME`
+and `AWS_ACCESS_KEY_ID`), plus `GOOGLE_APPLICATION_CREDENTIALS` (a file path) and
+`GIT_CONFIG_KEY_<n>` (the name of a git setting). `GIT_CONFIG_VALUE_<n>` is
+selected when its `GIT_CONFIG_KEY_<n>` names a header, password, token, secret,
+cookie file or auth setting, such as `http.extraheader`. Credential names such as
+`AUTH_TOKEN` and `AUTH_MODE_TOKEN` still contribute their exact values, including
+short/low-entropy values; no value-shape filter can reliably distinguish passwords
+from configuration. Pass `minLength` to skip shorter values. An explicit array
+bypasses the name exclusions; `[]` selects nothing.
 These values are loaded only when the helper or CLI `--env-secrets` is requested.
 
 `await knownSecretsFromGitHubConfig({path?})` reads `oauth_token` and `api_token`
@@ -467,6 +546,9 @@ symlinks, aliases, files above 1 MiB and read failures produce value-free
 `ERR_AUTH`. YAML quoting is parsed with the `yaml` package, with alias expansion
 disabled. CLI `--gh-config` adds these exact values to `knownSecrets`.
 `knownSecretsFromGitHubAuth` / `--gh-auth` remain available independently.
+A `hosts.yml` that uses YAML aliases or is a symlink stays a hard `ERR_AUTH`
+failure, so a caller relying on `--gh-config` never runs with a silently
+incomplete list.
 
 Plain log streams and CLI file/stdin/output/in-place paths preserve invalid
 UTF-8 bytes exactly. Valid Unicode keeps its normal meaning for detection.
@@ -559,9 +641,33 @@ also applies independently within serialized JSON strings, to the existing
 bounded decoding depth. Findings do not seed document-wide name propagation.
 Repeated redaction is stable. Reusing a placeholder map for the same original
 produces stable tokens and exact restoration; the existing placeholder-namespace
-collision checks still reject input that already contains typed placeholders.
+collision checks still reject input that already contains typed placeholders
+unless the transformation sets `existing: 'keep'`.
+
+Capitalized words that identify nobody stay readable, so placeholder output keeps
+its sentence: pronouns (`Вас`, `Я`), role and label cues (`Кандидат`, `Сотрудник`,
+the default and configured `roles` and `personalLabels`) anywhere, and greetings,
+function words and verbs (past tense `Звонила`, `Перенёс`, `Работал` before a
+lowercase word; present `Работаю`, `Рекомендует`) at the start of a sentence:
 
-This is a structural heuristic, so capitalized ordinary words also disappear.
+```js
+sanitize('Кандидат Петров Иван Сергеевич. Звонила Анна Смирнова', {
+  properNames: 'capitalized',
+  transformation: { mode: 'placeholder', map: false },
+}).text;
+// Кандидат Person-1. Звонила Person-2
+```
+
+A word that is also a given-name form (`Вера`, `Любовь`) never counts as such a
+word, and a bare masculine `-л` form before a capitalized word may be a surname
+(`Шагал Марк`), so both are still replaced. A compact organization, location,
+school or person span found by the document rules (`ООО «Ромашка»`, `в Казани`)
+replaces the proper-name fragments inside it as one typed value
+(`Organization-1`, `Location-1`); spans with tabs or line breaks, or longer than
+160 characters, keep the separate fragments. A bare oblique surname (`Иванову`)
+links to the one person whose name contains it.
+
+Other capitalized ordinary words still disappear; this is a structural heuristic.
 Lowercase organizations without employment context stay; the document rules
 replace a lowercase employer after `работал в` / `работала в`. Complete contact
 and identifier findings take priority over overlapping proper-name fragments.
@@ -644,7 +750,29 @@ Cues ignore case and support bounded Russian noun/adjective forms; personal fiel
 use normalized field names. Capitalized proper values are required after prose
 cues. Inputs allow at most 256 nonempty strings of 128 code units per list, with
 no line breaks/NUL. Unknown lists and malformed replacement records fail with
-`ERR_CONFIG`. Compilation uses a bounded 64-configuration cache.
+`ERR_CONFIG`. Compilation uses a bounded 64-configuration cache and a weak
+object-identity cache. Treat a vocabulary object as immutable after first use;
+pass a new object to change its lists. Repeated lookups reuse compiled patterns
+and a cue-word Set without normalizing or serializing the lists again.
+
+`personalAttributes: ['вепс', 'старовер']` adds special-category stems to the
+`strict` rule (it has no effect without `strict: true`). Each word matches as a
+stem followed by up to eight letters, so `вепска` and `староверами` match, and the
+words of a multi-word entry may be separated by spaces or tabs. Matches are
+reported as `PERSONAL_ATTRIBUTE` with the `strict-attribute` rule. The default
+list is empty.
+
+Typed special-category attributes take precedence over generic `ProperName`
+guesses, including capitalized caller stems in Cyrillic prose. Strict coverage
+includes the 34 additional nationality, religion, health, criminal-record,
+political and union terms reported in issue 144, even as standalone words.
+
+With default `publicKnowledge`, exact public technology spellings such as Python,
+SQL, PostgreSQL, Docker, Kubernetes, React, AWS and Azure remain readable when
+the capitalized prose detector flags them. The offline catalog includes source
+URLs in `src/public-technologies.js`. `publicKnowledge: false` disables this
+exception; known private values and credentials still redact, and an adjacent
+personal name never inherits a technology's exemption.
 
 ## Callback transformations and stages
 
@@ -787,6 +915,41 @@ A mixed value such as `alice:${DB_PASSWORD}` or `<x>abc123…` is still redacted
 Findings from custom detectors are never dropped this way. The value-free debug
 event `{event: 'template-value', rule}` reports each skipped match.
 
+### Exempting findings with a hook
+
+`exempt(finding, text) => boolean` keeps findings that the caller's own policy
+knows are not sensitive: a configuration flag, a token counter, a version
+range, a value another tool already masked. It runs after detection and before
+transformation, once per finding. The finding carries `value` and the offsets
+into `text`. Return `true` to keep the span unchanged and `false` to transform
+it. Any other result, including `undefined` from a forgotten `return`, is
+`ERR_CONFIG`.
+
+```js
+createSyncSanitizer({
+  credentialsOnly: true,
+  // The key before the value decides; offsets point into `text`.
+  exempt: (finding, text) =>
+    /FEATURE_TOKEN=$/.test(text.slice(0, finding.start)),
+}).sanitize(
+  'FEATURE_TOKEN=keep-this-value-123 other_token=keep-this-value-123'
+);
+// → 'FEATURE_TOKEN=keep-this-value-123 other_token=[REDACTED]'
+```
+
+Unlike a callback transformation that returns the original value, which fails
+with `ERR_RESIDUAL`, an exempted span is excluded from residual verification
+by span, not by value. Verification of the published text calls the hook again
+for every finding in the output, so the same value elsewhere is still caught
+unless the hook keeps that span too. The hook is explicit code, so it can keep
+credential findings without `allowCredentialDisable`. Declared
+`knownSecrets`/`knownPersonal` values, confirmed personal values and structured
+`JSON_VALUE` denials are never offered to it.
+
+`createSanitizer` awaits a hook that returns a `Promise`.
+`createSyncSanitizer` and `sanitize()` need a synchronous hook. A `Promise`
+there is `ERR_CONFIG`.
+
 ## Per-type thresholds
 
 `minConfidenceByType` maps finding types to a threshold that overrides

package/docs/COVERAGE.md

@@ -8,7 +8,7 @@ The native provider supplements in `src/rules.js` cover GitHub, GitLab, OpenAI,
 
 The default native engine bundles 221 text rules from Gitleaks 8.30.1 (222 upstream rules; the path-only PKCS12 rule is outside text detection), compiled unchanged with RE2/WASM or RE2JS for inputs above 1 MiB (re2-wasm has a fixed 16 MiB heap shared by every compiled rule and copied subject). Dense rule matches switch to RE2JS after at most eight WASM matches per view, bounding repeated whole-input copies without truncating secrets or changing anchors. The MIT license, commit and sync script are retained in `src/vendor/gitleaks`. Because Gitleaks is no longer updated, 203 further rules from its maintained successor Betterleaks 1.10.0 (ids absent from the Gitleaks set, such as Groq, Cerebras, xAI, Vercel, Supabase and Cloudflare v2 tokens) run the same way as `betterleaks/<id>` findings, with their MIT license, provenance and `scripts/sync-betterleaks.py` in `src/vendor/betterleaks`. Their entropy floors and secret-only `matchesAny`/`containsAny` filters are honoured; BPE token-ratio and nearby-line filters, live validation, validation-only components and the context-dependent `generic-*` rules are not, so these rules can only report more than Betterleaks, not less. Keyword prefilters and configured entropy thresholds run natively; upstream comment/path allowlists cannot suppress a sanitizer finding. Quoted hex/base64 entropy detection supplements those rules. The default asynchronous engine additionally uses the pinned Secretlint recommended scanners, excluding its comment-filter rule. Embedded disable directives in untrusted text cannot suppress findings. Optional Gitleaks and TruffleHog invoke their installed detector sets, and custom detectors can add formats. Service counts are not a measure of comparative recall.
 
-Context rules recognize credential labels with bounded key affixes, JSON and nested escaped JSON, YAML/env/INI assignments, shell flags, XML values, URL userinfo, authorization headers, cookies, webhook paths, and complete or truncated private-key PEM blocks. Low-entropy contextual values such as `password=1234` are protected. Generic token counters, booleans/null/language literals and ordinary Git hashes are excluded to retain useful logs. Supply literal known values for ambiguous credentials or unusual key names.
+Context rules recognize credential labels with bounded key affixes, JSON and nested escaped JSON, YAML/env/INI assignments, Ruby/Perl/PHP hash rockets before a quoted literal (`'password' => '…'`), shell flags including plural credential names (`--access-tokens`, but not `--max-tokens`), XML values (also with attributes and camelCase element names), values that start with markup (`token: <script>…`; a bare `<placeholder>` stays), all-caps CI/test credential markers that contain a digit (`SYNTHETIC_TOKEN_ALPHA_123456`, rule `marker-token`), URL userinfo, authorization headers, cookies, webhook paths, and complete or truncated private-key PEM blocks. Low-entropy contextual values such as `password=1234` are protected; a number or boolean under a configuration name (`AUTH_MODE=2`, `ROUTER_AUTH_GUARD=0`) is a setting. Generic token counters, booleans/null/language literals and ordinary Git hashes are excluded to retain useful logs. Supply literal known values for ambiguous credentials or unusual key names.
 
 Broader unquoted entropy scanning is opt-in (`paranoid: true`), uses Shannon entropy ≥4.2 on bounded candidates, and exempts ordinary hex hashes and common tool identifiers. Quoted hex/base64 and entropy checks belonging to bundled provider rules run by default. Entropy has both false positives and false negatives; contextual detection and known-secret dictionaries remain necessary.
 
@@ -480,17 +480,27 @@ remove ordinary uses of unambiguous category words. Bounded Russian case/stem
 rules cover disability, ethnicity, religion, diagnoses, intimate life and criminal
 status, including `инвалидом`, `армянин`, `еврейка`, `казах`, `христианин`, `иудей`,
 `эпилепсией`, `астма`, `онкологию`, `туберкулез`, `психиатра`, `депрессия`,
-`гепатит`, `гей` and `отбывал наказание`. English coverage includes `epilepsy`,
-`gay` and `Armenian`. Explicit category cues such as `по национальности X`,
+`гепатит`, `гей` and `отбывал наказание`, plus nationalities (`узбечка`,
+`таджик`, `чеченец`, `грузинка`, `азербайджанец`, `цыганка`, `кореянка`),
+religions (`протестант`), health (`инсульт`, `шизофрения`, `слабослышащий`,
+`колясочник`, `передвигается на коляске`, `аутизм`), sexuality (`бисексуал`,
+`трансгендер`), convictions (`сидел за кражу`) and trade-union membership
+(`член профсоюза`). English coverage includes `epilepsy`, `gay`, `Armenian`,
+`autism`, `transgender`, `wheelchair user`, `trade union member` and
+`served time for`. Country, region and language words (`Узбекистан`, `Чувашия`,
+`грузинский язык`) are not attributes. `vocabulary.personalAttributes` adds caller
+stems. Explicit category cues such as `по национальности X`,
 `исповедует X`, `болеет X`, `на учете у X`, `diagnosed with X` and `ethnicity: X`
 protect one following word (including hyphenated words) up to 64 characters,
 including unfamiliar values outside the lexicon. These cues stay on one line.
-Ambiguous English `disabled`, `single`, `man`, `men`, `russian`
-and `asian` require an adjacent personal cue (pronoun/person role, personal
+Ambiguous English `disabled`, `single`, `man`, `men`, `russian`,
+`asian`, `uzbek`, `korean`, `chinese`, `georgian`, `kazakh`, `roma` and
+`union member` require an adjacent personal cue (pronoun/person role, personal
 label, or following person role). Thus `he is disabled`, `candidate is single`
 and `gender: male` redact, while `the module is disabled`, `single run finished`,
 `single-flight lock acquired`, `man page missing`, `locale russian selected`
-and `asian region endpoint` remain intact. Bare operational JSON statuses
+`asian region endpoint`, `Korean locale enabled` and `union member list`
+remain intact. Bare operational JSON statuses
 `disabled`, `single` and `Russian` also remain intact; explicit personal labels
 and caller-declared known personal values still redact. A cue in another clause
 or string value does not make an ambiguous word personal.

package/docs/RULES.md

@@ -540,6 +540,7 @@ Regenerate with `node experiments/issue-118/generate-rule-catalog.mjs`.
 - `mac`
 - `mail-display-name`
 - `mailchimp`
+- `marker-token`
 - `messenger-handle`
 - `messenger-label`
 - `mrz`

package/examples/second-pass-placeholders.mjs

@@ -0,0 +1,28 @@
+// Sanitizes a CV note into readable placeholders, then sanitizes the result
+// again after a model added text, keeping the tokens already in it.
+import {
+  createSyncSanitizer,
+  isPlaceholderToken,
+} from '@link-foundation/sensitive-data-sanitizer';
+
+const policy = (existing) => ({
+  profile: 'documents',
+  properNames: 'capitalized',
+  strict: true,
+  transformation: { mode: 'placeholder', map: false, existing },
+});
+
+const first = createSyncSanitizer(policy()).sanitize(
+  'Кандидат Петров Иван Сергеевич, тел. +7 916 123-45-67. Работал в ООО «Ромашка», живу в Казани. Перенёс инсульт.'
+).text;
+console.log(first);
+// Кандидат Person-1, тел. Phone-1. Работал в Organization-1, живу в Location-1. Перенёс PersonalAttribute-1.
+
+const answer = `${first} Рекомендует Анна Смирнова.`;
+const second = createSyncSanitizer(policy('keep')).sanitize(answer).text;
+console.log(second);
+// ... Перенёс PersonalAttribute-1. Рекомендует Person-2.
+
+console.log(
+  second.split(/[\s,.]+/u).filter((word) => isPlaceholderToken(word))
+);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@link-foundation/sensitive-data-sanitizer",
-  "version": "0.20.0",
+  "version": "0.21.1",
   "description": "Offline sensitive-text sanitization with credential, multilingual personal-data and scanner union detection",
   "type": "module",
   "main": "src/index.js",
@@ -27,12 +27,21 @@
     "examples/reversible-placeholders.mjs",
     "examples/prose-logs.mjs",
     "docs/RULES.md",
-    "examples/custom-log-policy.mjs"
+    "examples/custom-log-policy.mjs",
+    "examples/second-pass-placeholders.mjs"
   ],
   "exports": {
     ".": {
       "types": "./src/index.d.ts",
-      "import": "./src/index.js"
+      "import": "./src/index.js",
+      "require": "./src/index.js",
+      "default": "./src/index.js"
+    },
+    "./credentials": {
+      "types": "./src/credentials-entry.d.ts",
+      "import": "./src/credentials-entry.js",
+      "require": "./src/credentials-entry.js",
+      "default": "./src/credentials-entry.js"
     },
     "./eslint-plugin": {
       "types": "./src/eslint-plugin.d.ts",