@link-foundation/sensitive-data-sanitizer 0.15.0 → 0.16.1

package/CHANGELOG.md

@@ -1,5 +1,17 @@
 # Changelog
 
+## 0.16.1
+
+### Patch Changes
+
+- 4dfe2b7: Scan standalone and embedded JSON string values with their own line boundaries. Extend the documents profile's contact, identity, name, attribute, employer, school and residence rules, and preserve ordinary relative URLs, operational timestamps and context-free SHA-1-shaped values consistently.
+
+## 0.16.0
+
+### Minor Changes
+
+- 75fd2d9: Match a reused restore map PERSON entry like an explicit `knownPersonal` value, so a bare or lower-case part no longer merges people or fails with `ERR_RESIDUAL`, keep parts listed as their own values in an optional `standalone` entry field, cover the adjacent given name or patronymic when a bare part matches, and read labelled CV experience lines (`Опыт: Менеджер, Ромашка (2018 – 2020)`), titles of up to nine words and the `по н.в.` open range in the `documents` profile.
+
 ## 0.15.0
 
 ### Minor Changes

package/docs/API.md

@@ -105,6 +105,24 @@ forms consume whole Unicode words; minimum stem lengths limit overmatching.
 This is conservative heuristic identity linking, not general coreference:
 ambiguous surnames/initials and unrelated people are not intentionally merged.
 
+A PERSON entry of a reused map matches the same forms as an explicit
+`knownPersonal` value: the full value and its contiguous parts together, all
+cases, both orders and initials. A bare part does not match in the next document
+unless it is stored as its own value, so `Петров Данил` from one CV leaves
+`петров день` and `Петров пришёл.` in the next one untouched. A single stored word
+(`Мороз`) matches its capitalized and upper-case forms but not lower-case text
+(`сильный мороз`); a multi-word value also matches in lower case (`петров данил`).
+A part listed in `knownPersonal` is kept in the entry's `standalone` list once it
+has been replaced, and bare surfaces recorded by earlier versions are ignored on
+reuse. The evidence of the current document is separate: when its text names
+`Петров Данил`, a bare `Петров` or `петрову` in that document, including its stream
+batches and decoded values, links to that person exactly as with a fresh map.
+
+When a bare part matches, the span in placeholder mode takes the capitalized
+given name or patronymic next to it on the same line, so `Алина Мороз` becomes one
+new token instead of `Алина Person-1`. Given names come from a bounded list with
+Russian case endings, so an unlisted name can still stay beside the placeholder.
+
 A listed Russian PERSON value also matches all six grammatical cases, including
 the instrumental (`Петровым Данилом Сергеевичем`), every word order, `ё`/`е`
 spelling differences in either direction, upper case, and initials with or

package/docs/COVERAGE.md

@@ -59,9 +59,15 @@ missed. Legal-form words and uncased scripts can also yield false positives.
 The `documents` profile also reads employers from CV experience lines anchored on
 a date range: `Title, Employer (2019-05 – 2021-03)`, `Title — Employer, 2019–2021`
 and `Title (Employer), 2019–2021`. Ranges use years, `YYYY-MM`, `MM.YYYY`, full
-dates or `month YYYY`, with `по настоящее время`, `н.в.`, `present` and similar
-open ends. Confidence is 0.60, and the employer then propagates through the
-document like other organizations. The default profile ignores these lines. Lines
+dates or `month YYYY`, with `по настоящее время`, `н.в.`, `по н.в.`, `present` and
+similar open ends. Titles have one to nine words. A label of one or two words
+before the title (`Опыт: `, `Опыт работы - `, `Опыт — `) is read like a list marker,
+and a segment after `|` is read like a line start, so
+`Должность: Менеджер | Опыт: Аналитик, Ромашка (2018 – 2020)` also yields the
+employer. The date range stays the anchor and the employer must still start with a
+capital letter, a digit or a quote. Confidence is 0.60, and the employer then
+propagates through the document like other organizations. The default profile
+ignores these lines. Lines
 with more fields (`Title, Employer, City (dates)`) and titles containing a comma
 are not targeted and can leave the employer in place; date-anchored prose with
 the same shape can be flagged.
@@ -196,3 +202,71 @@ Structured JSONL streaming supports 8 MiB records and ordered record workers;
 and reproducible throughput measurement. Realistic fake mode preserves supported
 scripts/formats/checksums and uses reserved contact ranges where available.
 Generated identities are pseudonyms; valid arithmetic does not prove non-issuance.
+
+## Document forms reviewed in issues 77–83
+
+Structured JSON/JSONL scans each decoded string at its own line start as well as
+with its field label. This includes array elements, multiline strings and up to
+two embedded JSON container layers. Both the synchronous API and asynchronous
+engine union use these views. Source offsets include original JSON escapes;
+unchanged values and whitespace remain byte-identical. Numeric primitives still
+require a caller schema policy. `debug` can observe `structured-value` events
+with finding counts, without logging the string contents.
+
+The `documents` profile additionally recognizes:
+
+- Russian mobile numbers with dashed, dotted, spaced or parenthesized forms,
+  including ten-digit `9xx` forms and domestic `8` prefixes; `тел.`, `моб.` and
+  WhatsApp labels; `wa.me` contact links.
+- Obfuscated email separators `(at)`, `[at]`, `собака`, and English `dot`/`at`.
+  This is a finite spelling grammar, not arbitrary email deobfuscation.
+- Personal paths on LinkedIn (`/in/`), GitHub, VK, Instagram and Telegram,
+  including scheme-less links; Telegram/телеграм/tg/skype/ник labels without `@`.
+  A provider root such as `https://github.com/` remains unchanged. These rules
+  cannot distinguish a person from an organization account on the same service.
+- Russian passport series/number phrases, ОГРНИП, subdivision codes, dated
+  ОВД/УВД/МВД issuer phrases, Russian/English driving licence labels, ОМС/полис,
+  Russian account numbers and numbered diplomas. These contextual patterns do
+  not validate issuance; digits are protected even with invalid checksums.
+- Numbered schools, gymnasiums, lyceums and colleges; school acronyms after
+  Образование/ВУЗ/Университет/окончил/окончила, with optional `им. Баумана`-style
+  suffixes. Acronym detection is contextual rather than a complete school list.
+- Surname/initial shapes in Cyrillic and selected Latin Slavic forms, known
+  given-name plus initial, Russian surname/given-name pairs using finite case
+  forms and lower case, and surnames after г-жа/господин/госпожа.
+- Birth years with д.р./г.р./год рождения/родился/родилась/Born in context;
+  Russian age expressions and Пол labels, English Sex, and gender beside age.
+  Age-in-years expressions can also describe nonpersonal durations.
+- Labelled citizenship, marital status, children, nationality, religion,
+  disability and criminal record, including the reported English labels.
+- Company/employer/workplace labels, capitalized company/holding/bank names,
+  employment prose and year-range employer lines; residence/metro/district
+  labels and contextual six-digit postal codes.
+- Russian street-plus-house/flat structures at line start or after
+  `проживает по адресу`. Public-place prose such as
+  `Музей находится на ул. Ленина, д. 5, кв. 12` stays unchanged. Public addresses
+  presented alone have the same shape as personal addresses and may be flagged;
+  inspect and review standalone public addresses before publication.
+
+Explicit limits accepted for the ambiguous examples in #82/#83:
+`Спасибо, Анна!`, `Анна, добрый день!`, `Менеджер Анна` and `Лютик, менеджер`
+remain unchanged without known values or additional context. Bare given names
+and unmarked employer/title fragments cannot reliably establish identity.
+`зарплата 150 000 руб.` and `Salary: 150000` also remain unchanged: salary is
+outside the native document rules. Supply `knownPersonal` or an additional
+recognizer when these fields must be removed. Initial/name rules do not cover
+all surnames, languages, cases or arbitrary lower-case names.
+
+Operational value policy (#79): a single-slash relative URL with no query,
+fragment, userinfo or explicit user/profile/member/home path stays unchanged
+under `url`; other detectors and known values still apply. An ISO-style
+`YYYY-MM-DDTHH:mm:ss[.fraction](Z|±HH:mm)` timestamp under `time` stays unchanged.
+This exemption is syntactic, not date validation. Userinfo, token-bearing URLs
+and personal paths still redact. Context-free 40-character hexadecimal values
+are treated as SHA-1-shaped identifiers in both text and structured modes,
+including quoted values. This does not prove they are harmless: actual bare
+40-hex secrets require `knownSecrets` or credential context. Credential labels,
+provider-specific rules and required external scanners remain authoritative.
+
+The [complete requirement inventory and alternatives](case-studies/issue-84/README.md)
+links every reported text row to its regression case.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@link-foundation/sensitive-data-sanitizer",
-  "version": "0.15.0",
+  "version": "0.16.1",
   "description": "Offline sensitive-text sanitization with credential, multilingual personal-data and scanner union detection",
   "type": "module",
   "main": "src/index.js",

package/src/context.js

@@ -228,6 +228,9 @@ function quotedRange(text, start, opening, maxLength) {
 }
 
 function plausibleLabel(value, type) {
+  if (operationalValue(value, type)) {
+    return false;
+  }
   if (type === 'PERSON') {
     return plausiblePerson(value.replace(/[,;\d].*$/s, '').trim());
   }
@@ -240,6 +243,20 @@ function plausibleLabel(value, type) {
   return true;
 }
 
+function operationalValue(value, type) {
+  return (
+    (type === 'URL' &&
+      /^\/(?!\/)[^\s?@#]*$/.test(value) &&
+      !/(?:^|\/)(?:users?|profiles?|people|members?|home)\/[^/]+/i.test(
+        value
+      )) ||
+    (type === 'TIME' &&
+      /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?(?:Z|[+-]\d{2}:\d{2})$/.test(
+        value
+      ))
+  );
+}
+
 // Shared by native labels and the vendor catalogs. Code expressions and
 // line escapes are not personal field values, even when a key is familiar.
 export function personalValueAllowed(text, range, type, labelStart) {

package/src/detection.js

@@ -185,7 +185,9 @@ export function nativeDetect(text, options, emit) {
     emit,
     { ...credential, rule: 'quoted-entropy' },
     1,
-    (value) => entropy(value) >= (/^[a-f0-9]+$/i.test(value) ? 3 : 4.5)
+    (value) =>
+      !/^[a-f0-9]{40}$/i.test(value) &&
+      entropy(value) >= (/^[a-f0-9]+$/i.test(value) ? 3 : 4.5)
   );
   collectMatches(
     text,

package/src/document-entities.js

@@ -3,6 +3,7 @@ import { firstNames } from './names.js';
 import { organizationCore, personalRegex } from './personal-variants.js';
 import { documentVocabulary } from './document-vocabulary.js';
 import { months } from './personal-context.js';
+import { detectDocumentPatterns } from './document-patterns.js';
 
 const words = (column) =>
   [...new Set(documentVocabulary.flatMap((row) => row[column].split('|')))]
@@ -61,14 +62,17 @@ const month = [
   ...new Set(months.split('|').flatMap((word) => [word, capitalized(word)])),
 ].join('|');
 const rangeStart = `(?:(?:19|20)\\d{2}(?:-\\d{2}){0,2}|\\d{2}[./](?:\\d{2}\\.)?\\d{4}|(?:${month})\\.?[ \\t]+(?:19|20)\\d{2})`;
-const rangeOpen = ['по настоящее время', 'настоящее время', 'сейчас', 'н.в.']
-  .concat(['н/в', 'present', 'now', 'current'])
+const rangeOpen = ['по настоящее время', 'настоящее время', 'сейчас', 'по н.в.']
+  .concat(['н.в.', 'по н/в', 'н/в', 'present', 'now', 'current'])
   .flatMap((word) => [word, capitalized(word)])
   .map(escapePattern)
   .join('|');
 const range = `${rangeStart}(?:[ \\t]*[-–—][ \\t]*|[ \\t]+(?:по|to)[ \\t]+)(?:${rangeStart}|${rangeOpen})(?!${boundary})`;
 const titleWord = '[\\p{L}\\p{N}][\\p{L}\\p{N}/+#.&-]{0,39}';
-const lineTitle = `^[ \\t]*(?:[-–—•*·][ \\t]+|\\d{1,2}[.)][ \\t]+)?${titleWord}(?:[ \\t]+${titleWord}){0,5}`;
+// A segment starts a line or follows `|`. `Опыт:` or `Опыт -` before the title
+// reads like a list marker: the date range remains the anchor of the rule.
+const lineLabel = `\\p{L}[\\p{L}\\p{N}/+#.&-]{0,39}(?:[ \\t]+${titleWord})?(?:[ \\t]*:[ \\t]*|[ \\t]+[-–—][ \\t]+)`;
+const lineTitle = `(?:^|(?<=\\|))[ \\t]*(?:[-–—•*·][ \\t]+|\\d{1,2}[.)][ \\t]+)?(?:${lineLabel}){0,2}${titleWord}(?:[ \\t]+${titleWord}){0,8}`;
 const employerWord = "[\\p{L}\\p{M}\\p{N}&'’.+-]{1,40}";
 const employer = `«[^«»\\r\\n]{1,160}»|"[^"\\r\\n]{1,160}"|“[^“”\\r\\n]{1,160}”|(?:\\p{Lu}|\\d{1,4}-?\\p{L})${employerWord}?(?:[ \\t]+${employerWord}){0,5}`;
 const experienceLine = new RegExp(
@@ -84,6 +88,7 @@ export function detectDocumentEntities(text, emit, options) {
   if (!options.documentEntities) {
     return;
   }
+  detectDocumentPatterns(text, emit);
   const found = [];
   const capture = (finding) => {
     found.push(finding);

package/src/document-patterns.js

@@ -0,0 +1,224 @@
+import { firstNames } from './names.js';
+import { isNameForm } from './russian-names.js';
+
+const boundary = '[\\p{L}\\p{N}_]';
+const word = '[А-ЯЁ][а-яё-]{1,47}';
+const surname =
+  '[А-ЯЁа-яё][А-ЯЁа-яё-]{1,28}(?:ов|ев|ин|ын|ск|цк)(?:а|ой|ою|у|ым|ом|е|ы|и|их|ий|ая|ого|ому|ую|ие)?';
+const initials = '(?:[А-ЯЁA-Z]\\.[ \\t]*){1,2}';
+const latinSurname = '[A-Z][a-z]{1,28}(?:ova|eva|ov|ev|in|ina|sky|ski|skaya)';
+const given = [
+  ...new Set(Object.values(firstNames).flatMap((s) => s.split(' '))),
+].join('|');
+const nameInitials = new RegExp(
+  `(?<!${boundary})(?:${surname}[ \\t]+${initials}|${initials}(?:${surname}|${latinSurname})|${latinSurname}[ \\t]+${initials}|(?:${given})[ \\t]+[А-ЯЁA-Z]\\.)(?!${boundary})`,
+  'gu'
+);
+const namePairs = new RegExp(
+  `(?<!${boundary})(${surname})[ \\t]+([А-ЯЁа-яё]{2,32})|(?<!${boundary})([А-ЯЁа-яё]{2,32})[ \\t]+(${surname})(?!${boundary})`,
+  'gu'
+);
+// Every rule is finite and requires either an identifying structure or a label.
+// Public locations/ambiguous bare words are not inferred from arbitrary prose.
+const rules = [
+  [
+    'PHONE',
+    'document-phone',
+    /(?<![\p{L}\p{N}_])(?:[78][ .-]?)?9\d{2}(?:[ .-]\d{3}[ .-]\d{2}[ .-]\d{2})(?!\d)|(?<![\p{L}\p{N}_])8[ .-]?\(9\d{2}\)[ .-]?\d{3}[ .-]?\d{2}[ .-]?\d{2}(?!\d)/gu,
+    0,
+  ],
+  [
+    'PHONE',
+    'document-phone-label',
+    /(?<![\p{L}\p{N}_])(?:тел\.?|моб\.?|WhatsApp)(?:[ \t]*:[ \t]*|[ \t]+)((?:[78][ () .-]*)?9\d{2}[ () .-]*\d{3}[ .-]*\d{2}[ .-]*\d{2})(?!\d)/giu,
+    1,
+  ],
+  [
+    'EMAIL',
+    'obfuscated-email',
+    /(?<![\p{L}\p{N}_.+-])[\p{L}\p{N}_+-]{1,64}(?:(?:\.|[ \t]{1,8}dot[ \t]{1,8})[\p{L}\p{N}_+-]{1,64}){0,4}(?:[ \t]{0,8}\(at\)[ \t]{0,8}|[ \t]{0,8}\[at\][ \t]{0,8}|[ \t]{1,8}(?:at|собака)[ \t]{1,8})[\p{L}\p{N}-]{1,63}(?:(?:\.|[ \t]{1,8}dot[ \t]{1,8})[\p{L}\p{N}-]{1,63}){1,8}(?![\p{L}\p{N}_])/giu,
+    0,
+  ],
+  [
+    'USERNAME',
+    'personal-profile-url',
+    /(?<![\p{L}\p{N}_./-])(?:https?:\/\/)?(?:www\.)?(?:linkedin\.com\/in|github\.com|vk\.com|instagram\.com|t\.me|wa\.me)\/[\p{L}\p{N}_][\p{L}\p{N}_.-]{1,100}(?:\/[\p{L}\p{N}_.-]{1,100}){0,4}(?![\p{L}\p{N}_])/giu,
+    0,
+  ],
+  [
+    'USERNAME',
+    'messenger-label',
+    /(?<![\p{L}\p{N}_])(?:Telegram|телеграм|tg|skype|ник)(?:[ \t]*:[ \t]*|[ \t]+)(@?[a-z][a-z\d_.-]{2,63})(?![\p{L}\p{N}_])/giu,
+    1,
+  ],
+  [
+    'PASSPORT_NUMBER',
+    'passport-series',
+    /(?<![\p{L}\p{N}_])(?:серия|series)[ \t]+(\d{2}[ \t]?\d{2}[ \t]+(?:№|No|номер|number|#)[ \t]*\d{6})(?!\d)/giu,
+    1,
+  ],
+  [
+    'ID',
+    'identity-number',
+    /(?<![\p{L}\p{N}_])(?:ОГРНИП[ \t]+(\d{15})|код подразделения[ \t]+(\d{3}-\d{3})|(?:водительское удостоверение|в\/у|Driver's licen[cs]e)[ \t]+(\d{2}[ \t]?\d{2}[ \t]?\d{6})|(?:ОМС|полис)[ \t]+(\d{16})|сч[её]т[ \t]+(\d{20})|диплом[ \t]+(?:№|No|#)[ \t]*(\d{6}))(?!\d)/giu,
+    -1,
+  ],
+  [
+    'ID',
+    'passport-issuer',
+    /(?<![\p{L}\p{N}_])выдан[ \t]+((?:ОВД|УВД|МВД)[ \t]+[^\r\n";|]{1,120}?\d{2}\.\d{2}\.\d{4})(?!\d)/giu,
+    1,
+  ],
+  [
+    'EDUCATION',
+    'numbered-school',
+    /(?<![\p{L}\p{N}_])(?:школа|гимназия|лицей|колледж)[ \t]*(?:№|No|#|:)[ \t]*\d{1,4}(?!\d)/giu,
+    0,
+  ],
+  [
+    'EDUCATION',
+    'school-acronym-context',
+    /(?<![\p{L}\p{N}_])(?:образование|ВУЗ|университет|окончил[аи]?)(?:[ \t]*:[ \t]*|[ \t]+)([А-ЯЁ]{2,12}(?:[ \t]+им\.[ \t]+[А-ЯЁ][а-яё-]{2,40})?)(?![\p{L}\p{N}_])/giu,
+    1,
+  ],
+  ['PERSON', 'name-initials', nameInitials, 0],
+  [
+    'PERSON',
+    'name-title',
+    new RegExp(
+      `(?<!${boundary})(?:г-жа|господин|госпожа)[ \\t]+(${word})(?!${boundary})`,
+      'gu'
+    ),
+    1,
+  ],
+  [
+    'DATE_OF_BIRTH',
+    'birth-year',
+    /(?<![\p{L}\p{N}_])(?:д\.р\.|год рождения|родил(?:ся|ась)(?:[ \t]+в)?|Born(?:[ \t]+in)?)[ \t]*:?[ \t]*((?:19|20)\d{2})(?!\d)|(?<!\d)((?:19|20)\d{2})(?=[ \t]+г\.р\.)/giu,
+    -1,
+  ],
+  [
+    'AGE',
+    'age-context',
+    /(?<![\p{L}\p{N}_])(?:возраст[ \t]*:|мне|исполнилось)[ \t]*(\d{1,3})(?!\d)|(?<![\p{L}\p{N}_])(\d{1,3})(?=[ \t]+(?:года|год|лет)(?![\p{L}\p{N}_]))/giu,
+    -1,
+  ],
+  [
+    'GENDER',
+    'gender-context',
+    /(?<![\p{L}\p{N}_])(?:Пол|Sex)[ \t]*:[ \t]*(женский|мужской|male|female)(?![\p{L}\p{N}_])|(?<![\p{L}\p{N}_])(Женщина|Мужчина)(?=,[ \t]*\d{1,3}[ \t]+(?:года|год|лет))/giu,
+    -1,
+  ],
+  [
+    'PERSONAL_ATTRIBUTE',
+    'profile-attribute',
+    /(?<![\p{L}\p{N}_])(?:Гражданство|Citizenship|Семейное положение|Marital status|Дети|национальность|Вероисповедание|инвалидность|судимость)[ \t]*:[ \t]*([^\r\n";|]{1,160})/giu,
+    1,
+  ],
+  [
+    'ORGANIZATION',
+    'employer-label',
+    new RegExp(
+      `(?<!${boundary})(?:[Кк]омпания|[Рр]аботодатель|[Мм]есто работы)[ \\t]*:[ \\t]*(${word})(?!${boundary})`,
+      'gu'
+    ),
+    1,
+  ],
+  [
+    'ORGANIZATION',
+    'employer-prose',
+    new RegExp(
+      `(?<!${boundary})(?:компания|работал[аи]?[ \\t]+в|холдинг|[Бб]анк)[ \\t]+(«[^«»\\r\\n]{1,120}»|${word})(?!${boundary})`,
+      'gu'
+    ),
+    1,
+  ],
+  [
+    'ORGANIZATION',
+    'employer-years',
+    new RegExp(
+      `(?:^|\\n)[ \\t]*(?:19|20)\\d{2}[–—-](?:19|20)\\d{2}[ \\t]+(${word})(?!${boundary})`,
+      'gu'
+    ),
+    1,
+  ],
+  [
+    'LOCATION',
+    'residence-prose',
+    new RegExp(
+      `(?<!${boundary})(?:живу[ \\t]+в|из|м\\.|метро|район)[ \\t]+(${word})(?!${boundary})`,
+      'gu'
+    ),
+    1,
+  ],
+  [
+    'LOCATION',
+    'postal-code',
+    /(?<![\p{L}\p{N}_])индекс[ \t]+(\d{6})(?!\d)|(?<!\d)(\d{6})(?=,[ \t]*[А-ЯЁ][а-яё-]{2,47})/gu,
+    -1,
+  ],
+];
+
+export function detectDocumentPatterns(text, emit) {
+  for (const [type, rule, pattern, group] of rules) {
+    for (const match of text.matchAll(pattern)) {
+      const value =
+        group === -1
+          ? match.slice(1).find((part) => part !== undefined)
+          : match[group].trim();
+      if (!documentValueAllowed(rule, value)) {
+        continue;
+      }
+      const start =
+        match.index + (group === 0 ? 0 : match[0].lastIndexOf(value));
+      emit({
+        start,
+        end: start + value.length,
+        type,
+        rule,
+        category: 'personal',
+        confidence: group === -1 ? 0.85 : 0.75,
+      });
+    }
+  }
+  for (const match of text.matchAll(namePairs)) {
+    if (isNameForm(match[2] ?? match[3])) {
+      emit({
+        start: match.index,
+        end: match.index + match[0].length,
+        type: 'PERSON',
+        rule: 'name-case',
+        category: 'personal',
+        confidence: 0.8,
+      });
+    }
+  }
+  // An address at line start or in explicit residence context is personal.
+  // Public-place prose with a street address does not supply that context.
+  const street =
+    /(?:^|\n)[ \t]*((?:ул\.|улица)[ \t]+[А-ЯЁ][а-яё-]{1,40},[ \t]*(?:(?:д\.|дом)[ \t]*)?\d{1,5}(?:[ \t]*,[ \t]*кв\.[ \t]*\d{1,5})?)|проживает по адресу[ \t]+((?:ул\.|улица)[ \t]+[А-ЯЁ][а-яё-]{1,40},[ \t]*(?:(?:д\.|дом)[ \t]*)?\d{1,5}(?:[ \t]*,[ \t]*кв\.[ \t]*\d{1,5})?)/gu;
+  for (const match of text.matchAll(street)) {
+    const value = match[1] ?? match[2];
+    const start = match.index + match[0].lastIndexOf(value);
+    emit({
+      start,
+      end: start + value.length,
+      type: 'ADDRESS',
+      rule: 'document-street',
+      category: 'personal',
+      confidence: 0.85,
+    });
+  }
+}
+
+function documentValueAllowed(rule, value) {
+  return (
+    Boolean(value) &&
+    value !== '[REDACTED]' &&
+    !(
+      rule === 'school-acronym-context' &&
+      !/^[А-ЯЁ]{2,12}(?![А-ЯЁа-яё])/.test(value)
+    ) &&
+    !(rule === 'messenger-label' && /^(?:недоступен|unavailable)$/i.test(value))
+  );
+}

package/src/engines.js

@@ -24,6 +24,7 @@ import {
 } from './placeholders.js';
 import {
   structuredBatches,
+  structuredValueViews,
   mapStructuredFindings,
   verifyStructured,
 } from './structured.js';
@@ -143,6 +144,21 @@ async function analyzeStructured(
     )) {
       findings.push(finding);
     }
+    for (const view of structuredValueViews(batch, options)) {
+      const detected = await analyze(
+        view.text,
+        [],
+        true,
+        { ...options, structured: undefined },
+        nativeFindings === undefined ? undefined : []
+      );
+      for (const finding of detected) {
+        const mapped = view.map(finding);
+        if (mapped) {
+          findings.push(mapped);
+        }
+      }
+    }
   }
   // Native structured inspection validates, locates and reapplies the
   // same fake-value/public policy to source offsets from all engines.

package/src/index.d.ts

@@ -269,6 +269,8 @@ export interface RestoreMap {
     number: number;
     values: string[];
     aliases: string[];
+    /** PERSON parts listed by the caller, which also match alone on reuse. */
+    standalone?: string[];
   }>;
   documents: Array<{
     digest: string;

package/src/personal-variants.js

@@ -167,6 +167,10 @@ export function detectKnownPersonal(text, entries, rule, emit, options) {
       ...(family ? [family.pattern] : []),
     ]) {
       for (const match of text.matchAll(pattern)) {
+        // A stored capitalized word in lower case is ordinary text: `Мороз`, `мороз`.
+        if (entry.properCase && /^\p{Ll}/u.test(match[0])) {
+          continue;
+        }
         if (++count > (options.maxFindings ?? 100000)) {
           throw failure('ERR_LIMIT');
         }

package/src/placeholders.js

@@ -1,6 +1,7 @@
 import { failure, REDACTED, escapePattern } from './detection.js';
 import { reservedPlaceholderTypes } from './placeholder-types.js';
 import { samePersonal } from './personal-variants.js';
+import { isNameForm } from './russian-names.js';
 import { nameAliases } from './transliteration.js';
 import { createHash } from 'node:crypto';
 import { jsonStringLiterals } from './json-string-runs.js';
@@ -85,13 +86,50 @@ export function placeholderKnown(options) {
       ...map.entries
         .filter((entry) => entry.type !== 'CREDENTIAL')
         .flatMap((entry) =>
-          entry.values
+          (entry.type === 'PERSON' ? reusablePersonValues(entry) : entry.values)
             .filter((value) => value.length <= 256)
-            .map((value) => ({ type: entry.type, value }))
+            .map((value) => ({
+              type: entry.type,
+              value,
+              ...(entry.type === 'PERSON' && properCaseWord(value)
+                ? { properCase: true }
+                : {}),
+            }))
         ),
     ],
   };
 }
+const singleWord = (value) => !/\s/u.test(value.trim());
+const properCaseWord = (value) =>
+  singleWord(value) && /^\p{Lu}/u.test(value.trim());
+// A bare part recorded while replacing a fuller name in its own document is not
+// evidence on its own in the next one, so a reused map offers only the full
+// values, parts listed by the caller and values that were stored alone.
+function reusablePersonValues(entry) {
+  const longer = entry.values.filter((value) => !singleWord(value));
+  return [
+    ...new Set([
+      ...entry.values.filter(
+        (value) =>
+          !singleWord(value) ||
+          !longer.some((full) => partOfPerson(value, full))
+      ),
+      ...(entry.standalone ?? []),
+    ]),
+  ];
+}
+function partOfPerson(value, full) {
+  return (
+    related(
+      value,
+      { type: 'PERSON', aliases: aliases(full, 'PERSON') },
+      true
+    ) ||
+    full
+      .split(/\s+/u)
+      .some((word) => word && samePersonal(value, word, 'PERSON'))
+  );
+}
 const stem = (word) =>
   word.replace(
     /(?:овым|евым|овой|евой|ова|ева|ову|еву|ове|еве|ов|ев|ovym|evym|ovoi|evoi|ova|eva|ovu|evu|ove|eve|ov|ev)$/u,
@@ -115,7 +153,22 @@ function numberedOrganization(value, entry) {
     return base !== original && samePersonal(value, base, entry.type);
   });
 }
-function related(value, entry) {
+function wordsMatchAlias(words, full, parts) {
+  if (words.length === 1) {
+    return parts && words[0].length > 1 && full.includes(words[0]);
+  }
+  return (
+    words.length > 1 &&
+    words.length === full.length &&
+    words.every(
+      (part, i) =>
+        part === full[i] || (part.length === 1 && full[i].startsWith(part))
+    )
+  );
+}
+// `parts` links a bare word to a fuller alias. Only callers that hold evidence
+// from the current document may ask for it; a stored entry alone is not enough.
+function related(value, entry, parts = false) {
   if (
     (entry.values ?? []).some((original) =>
       samePersonal(value, original, entry.type)
@@ -133,46 +186,35 @@ function related(value, entry) {
   if (entry.type !== 'PERSON') {
     return false;
   }
-  for (const candidate of candidates) {
-    const parts = candidate.match(/[\p{L}\p{M}]+/gu) ?? [];
-    for (const alias of entry.aliases) {
-      const full = alias.match(/[\p{L}\p{M}]+/gu) ?? [];
-      if (
-        parts.length === 1 &&
-        parts[0].length > 1 &&
-        full.includes(parts[0])
-      ) {
-        return true;
-      }
-      if (
-        parts.length === full.length &&
-        parts.length > 1 &&
-        parts.every(
-          (part, i) =>
-            part === full[i] || (part.length === 1 && full[i].startsWith(part))
-        )
-      ) {
-        return true;
-      }
-    }
-  }
-  return false;
+  return candidates.some((candidate) => {
+    const words = candidate.match(/[\p{L}\p{M}]+/gu) ?? [];
+    return entry.aliases.some((alias) =>
+      wordsMatchAlias(words, alias.match(/[\p{L}\p{M}]+/gu) ?? [], parts)
+    );
+  });
 }
 export function placeholderNameFindings(text, findings, options) {
   if (!hasPlaceholders(options) || options.knownOnly) {
     return findings;
   }
-  const people = [
+  const state = options[placeholderState];
+  const patterns = new Set();
+  // Bare surname forms need the full name in this document. A stored person
+  // keeps only the initial form, which is itself a full mention.
+  for (const person of [
     ...findings
       .filter((f) => f.type === 'PERSON')
       .map((f) => text.slice(f.start, f.end)),
-    ...(options[placeholderState]?.map.entries ?? [])
-      .filter((e) => e.type === 'PERSON')
-      .flatMap((e) => e.values),
-  ];
-  const patterns = new Set();
-  for (const person of people) {
-    addNamePatterns(person, patterns);
+    ...earlierPeople(options),
+  ]) {
+    addNamePatterns(person, patterns, true);
+  }
+  for (const entry of state?.map.entries ?? []) {
+    if (entry.type === 'PERSON') {
+      for (const person of entry.values) {
+        addNamePatterns(person, patterns, false);
+      }
+    }
   }
   const result = [...findings];
   for (const pattern of patterns) {
@@ -194,7 +236,56 @@ export function placeholderNameFindings(text, findings, options) {
   }
   return result;
 }
-function addNamePatterns(person, patterns) {
+// Stream batches and decoded layers are pieces of the document being replaced.
+function earlierPeople(options) {
+  const state = options[placeholderState];
+  return state?.stream || options[placeholderNested]
+    ? (state?.people ?? [])
+    : [];
+}
+const nameNeighbour = "\\p{Lu}[\\p{L}\\p{M}'’-]{1,31}";
+const nameBefore = new RegExp(
+  `(?<![\\p{L}\\p{M}\\p{N}_-])(${nameNeighbour})[ \\t]+$`,
+  'u'
+);
+const nameAfter = new RegExp(
+  `^[ \\t]+(${nameNeighbour})(?![\\p{L}\\p{M}\\p{N}_])`,
+  'u'
+);
+const neighbourWindow = 64;
+// `Алина` beside a bare `петрову` is the rest of that mention: replacing only the
+// surname would leave that given name beside the token of the surname's owner.
+export function coverAdjacentNames(text, findings, options) {
+  if (!hasPlaceholders(options)) {
+    return findings;
+  }
+  return findings.map((finding) => {
+    if (
+      finding.type !== 'PERSON' ||
+      !singleWord(text.slice(finding.start, finding.end))
+    ) {
+      return finding;
+    }
+    let { start, end } = finding;
+    // A given name and a patronymic can both stand on one side of a surname.
+    for (let step = 0; step < 2; step++) {
+      const before = nameBefore.exec(
+        text.slice(Math.max(0, start - neighbourWindow), start)
+      );
+      if (before && isNameForm(before[1])) {
+        start -= before[0].length;
+      }
+      const after = nameAfter.exec(text.slice(end, end + neighbourWindow));
+      if (after && isNameForm(after[1])) {
+        end += after[0].length;
+      }
+    }
+    return start === finding.start && end === finding.end
+      ? finding
+      : { ...finding, start, end, rule: 'name-variant' };
+  });
+}
+function addNamePatterns(person, patterns, bareForms) {
   const words = person.match(/[\p{L}\p{M}]+/gu) ?? [];
   if (words.length < 2 || words.some((word) => word.length < 2)) {
     return;
@@ -207,7 +298,7 @@ function addNamePatterns(person, patterns) {
     patterns.add(
       `${escapePattern(parts[0][0])}\\.[ \\t]+${parts.slice(1).map(escapePattern).join('[ \\t]+')}`
     );
-    for (const word of parts) {
+    for (const word of bareForms ? parts : []) {
       if (/^[а-яё]+(?:ов|ев)$/u.test(word)) {
         patterns.add(`${escapePattern(word)}(?:ым|ой|а|у|е|ы|ых|ами)?`);
       }
@@ -241,14 +332,16 @@ export function preparePlaceholderText(text, findings, options) {
   if (namespaceCollision(text, pattern)) {
     throw failure('ERR_PLACEHOLDER_COLLISION');
   }
-  const people = findings
-    .filter((f) => f.type === 'PERSON')
-    .map((f) => text.slice(f.start, f.end));
   state.people = [
-    ...people,
-    ...state.map.entries
-      .filter((e) => e.type === 'PERSON')
-      .flatMap((e) => e.values),
+    ...new Set([
+      ...earlierPeople(options),
+      ...findings
+        .filter((f) => f.type === 'PERSON')
+        .map((f) => text.slice(f.start, f.end)),
+      ...(options.knownPersonal ?? [])
+        .filter((e) => e.type === 'PERSON')
+        .map((e) => e.value),
+    ]),
   ];
 }
 function namespaceCollision(text, pattern) {
@@ -275,7 +368,11 @@ export function canonicalPerson(value, people) {
     return (
       words.length > 1 &&
       words.every((word) => word.length > 1) &&
-      related(value, { type: 'PERSON', aliases: aliases(person, 'PERSON') })
+      related(
+        value,
+        { type: 'PERSON', aliases: aliases(person, 'PERSON') },
+        true
+      )
     );
   });
   const distinct = new Map(
@@ -293,10 +390,33 @@ function canonicalKnown(value, type, options, state) {
   // `Петров Данил` and `Петров Данил Сергеевич` are one person: the fullest wins.
   const longest = Math.max(0, ...known.map((e) => e.value.length));
   const fullest = known.filter((e) => e.value.length === longest);
-  if (fullest.length === 1) {
-    return fullest[0].value;
+  const listed = fullest.length === 1 ? fullest[0].value : value;
+  if (type !== 'PERSON') {
+    return listed;
+  }
+  // The caller's list names each person once, while the document may hold the
+  // same full name in several cases, so the list settles a listed part first.
+  const named = canonicalPerson(
+    listed,
+    (options.knownPersonal ?? [])
+      .filter((e) => e.type === 'PERSON')
+      .map((e) => e.value)
+  );
+  return named === listed ? canonicalPerson(listed, state.people ?? []) : named;
+}
+// A part the caller listed is stored as its own value, unlike a part that was
+// only recorded as a surface of a longer name.
+function rememberListedParts(entry, value, options) {
+  for (const known of options.knownPersonal ?? []) {
+    if (
+      known.type === 'PERSON' &&
+      singleWord(known.value) &&
+      samePersonal(value, known.value, 'PERSON') &&
+      !entry.standalone?.includes(known.value)
+    ) {
+      (entry.standalone ??= []).push(known.value);
+    }
   }
-  return type === 'PERSON' ? canonicalPerson(value, state.people ?? []) : value;
 }
 export function placeholderReplacement(value, finding, transform, options) {
   if (finding.category === 'credential' && !transform.credentials) {
@@ -352,6 +472,9 @@ export function placeholderReplacement(value, finding, transform, options) {
   if (!entry.values.includes(value)) {
     entry.values.push(value);
   }
+  if (type === 'PERSON') {
+    rememberListedParts(entry, value, options);
+  }
   return entry.token;
 }
 export function finishPlaceholders(result, occurrences, options) {

package/src/restore-map.js

@@ -85,7 +85,8 @@ function validateEntries(map) {
       tokens.has(entry.token) ||
       !nonemptyStrings(entry.values) ||
       !Array.isArray(entry.aliases) ||
-      entry.aliases.some((v) => typeof v !== 'string' || !v)
+      entry.aliases.some((v) => typeof v !== 'string' || !v) ||
+      (entry.standalone !== undefined && !nonemptyStrings(entry.standalone))
     ) {
       throw failure('ERR_MAP');
     }

package/src/russian-names.js

@@ -9,6 +9,34 @@ const givenNames = new Set(firstNames.ru.split(' ').map(fold));
 export const isPatronymic = (word) => patronymic.test(word);
 export const isGivenName = (word) => givenNames.has(fold(word));
 
+// Only decides whether a word beside a known person belongs to that mention, so
+// it stays apart from the detector's list, which would match these names anywhere.
+const mentionGivenNames =
+  'Роман Антон Илья Артём Никита Егор Глеб Денис Игорь Константин Кирилл Леонид Матвей Тимофей Фёдор Руслан Станислав Вячеслав Вадим Валерий Геннадий Григорий Степан Тимур Ярослав Юлия Дарья Полина Ксения Виктория Анастасия Вероника Валерия Алёна Оксана Людмила Галина Лариса Надежда Вера Любовь Кристина Софья Диана Яна Лилия Альбина Жанна Регина Инна Карина Милана Арина Варвара Елизавета Маргарита';
+const anyGivenName = new Set(
+  [...Object.values(firstNames), mentionGivenNames]
+    .flatMap((list) => list.split(' '))
+    .map(fold)
+);
+const inflectedPatronymic =
+  /(?:[оеь]вич|[оеь]ич|ич)(?:а|у|ем|е)?$|(?:овн|евн|ичн)(?:а|ы|е|у|ой|ою)$/iu;
+const nameCase = /(?:ом|ем|ой|ей|ою|ею|а|я|у|ю|е|ы|и)$/u;
+// A capitalized given name or patronymic in any case, for deciding whether a
+// neighbouring word belongs to the same mention. Lemmas are recovered by a
+// finite ending list, so this stays a lookup and never parses the sentence.
+export function isNameForm(word) {
+  const lower = fold(word);
+  if (anyGivenName.has(lower) || inflectedPatronymic.test(lower)) {
+    return true;
+  }
+  const stem = lower.replace(nameCase, '');
+  return (
+    stem !== lower &&
+    stem.length >= 2 &&
+    ['', 'а', 'я', 'й', 'ь'].some((ending) => anyGivenName.has(stem + ending))
+  );
+}
+
 // Returns word indexes by role, or undefined when the roles are not evident.
 // `Имя Отчество Фамилия` is checked first so that surnames in -ич, as in
 // `Шостакович Дмитрий Дмитриевич`, are not mistaken for patronymics.

package/src/sanitizer.js

@@ -26,10 +26,12 @@ import {
   hasPlaceholders,
   placeholderValues,
   placeholderNameFindings,
+  coverAdjacentNames,
   nestedPlaceholderOptions,
 } from './placeholders.js';
 import {
   structuredBatches,
+  inspectStructuredValues,
   mapStructuredFindings,
   preserveIdentifierKeys,
   structuredValueRanges,
@@ -391,8 +393,8 @@ function inspectText(text, options = {}) {
   validateOptions(options);
   validateInput(text, options);
   if (options.structured) {
-    const findings = structuredBatches(text, options).flatMap((batch) =>
-      mapStructuredFindings(
+    const findings = structuredBatches(text, options).flatMap((batch) => [
+      ...mapStructuredFindings(
         batch,
         inspect(batch.text, {
           ...options,
@@ -400,8 +402,11 @@ function inspectText(text, options = {}) {
           findings: [],
         }),
         options
-      )
-    );
+      ),
+      ...inspectStructuredValues(batch, options, (value) =>
+        inspect(value, { ...options, structured: undefined, findings: [] })
+      ),
+    ]);
     return inspectFindingsOnly(text, findings, options);
   }
   const findings = [];
@@ -466,7 +471,11 @@ export function reviewFindings(text, findings, options) {
     text,
     markFake(
       text,
-      propagate(text, markFake(text, [...unique.values()], options), options),
+      coverAdjacentNames(
+        text,
+        propagate(text, markFake(text, [...unique.values()], options), options),
+        options
+      ),
       options
     ).sort(
       (a, b) =>

package/src/structured.js

@@ -280,3 +280,87 @@ export function verifyStructured(text, options) {
     }
   }
 }
+
+// Scan decoded strings at their own line start, retaining a map through every
+// JSON escape. Both native and asynchronous engines consume these same views.
+export function* structuredValueViews(
+  batch,
+  options,
+  depth = 0,
+  outerMap = (f) => f
+) {
+  for (const entry of batch.entries) {
+    const value = JSON.parse(entry.raw);
+    const offsets = stringOffsets(entry.raw);
+    const map = (finding) => {
+      const raw = {
+        ...finding,
+        start: offsets ? offsets.starts[finding.start] : finding.start + 1,
+        end: offsets ? offsets.ends[finding.end - 1] : finding.end + 1,
+      };
+      if (
+        preserveStructuredEntry(
+          entry,
+          {
+            ...raw,
+            start: entry.batchStart + raw.start,
+            end: entry.batchStart + raw.end,
+          },
+          batch.text,
+          options
+        )
+      ) {
+        return undefined;
+      }
+      return outerMap({
+        ...raw,
+        start: entry.start + raw.start,
+        end: entry.start + raw.end,
+      });
+    };
+    yield { text: value, map };
+    // Match the encoded detector's finite depth; arbitrary nesting is a limit.
+    if (depth < 2 && /^[ \t\r\n]*[[{]/.test(value)) {
+      let batches;
+      try {
+        batches = structuredBatches(value, { ...options, structured: 'json' });
+      } catch (error) {
+        if (error.code !== 'ERR_JSON') {
+          throw error;
+        }
+        batches = [];
+      }
+      for (const nested of batches) {
+        yield* structuredValueViews(nested, options, depth + 1, map);
+      }
+    }
+  }
+}
+export function inspectStructuredValues(batch, options, inspectValue) {
+  const result = [];
+  for (const view of structuredValueViews(batch, options)) {
+    const findings = inspectValue(view.text);
+    options.debug?.({ event: 'structured-value', count: findings.length });
+    for (const finding of findings) {
+      const mapped = view.map(finding);
+      if (mapped) {
+        result.push(mapped);
+      }
+    }
+  }
+  return result;
+}
+function stringOffsets(raw) {
+  if (!raw.includes('\\')) {
+    return undefined;
+  }
+  const starts = [],
+    ends = [];
+  for (let i = 1; i < raw.length - 1; ) {
+    const start = i;
+    i += raw[i] === '\\' ? (raw[i + 1] === 'u' ? 6 : 2) : 1;
+    starts.push(start);
+    ends.push(i);
+  }
+  return { starts, ends };
+}