euredact 0.3.7 → 0.3.9

package/README.md

@@ -167,7 +167,7 @@ interface Detection {
   text: string;                    // The matched substring
   source: DetectionSource;         // "rules" or "cloud"
   country: string | null;          // ISO code or null for shared/custom patterns
-  confidence: string;              // Confidence level
+  confidence: string;              // "high" | "medium" | "low" — see below
   countryConfidence?: number;      // How strongly the document supports `country`,
                                    // in [0, 1]. 0 means the attribution rests on
                                    // a checksum alone.
@@ -176,6 +176,26 @@ interface Detection {
 }
 ```
 
+**`confidence`** describes how the *type* was arrived at. It never says anything
+about whether the span is masked — every detection is, at every level.
+
+| value | meaning |
+|---|---|
+| `"high"` | a pattern matched and, where one exists, its checksum passed |
+| `"medium"` | the type comes from a label touching the span, because no pattern of that type claimed it — `Αρ. Ταυτότητας: 00892341` is a `NATIONAL_ID` although nothing can checksum it |
+| `"low"` | a pattern matched, its checksum *failed*, and the document labels the span as that very type — `Rijksregisternummer: 85.03.19-284.73` is a national number with a bad check digit |
+
+Filter on it when you need only checksum-backed detections:
+
+```ts
+const strict = result.detections.filter(d => d.confidence === "high");
+```
+
+A `"low"` detection is the honest description of a mistyped, OCR'd or invented
+identifier: the shape and the label agree, the check digit does not. Earlier
+versions dropped these, which meant a redaction library printed in full an
+identifier it had recognised and rejected.
+
 #### `RedactResult`
 
 ```ts
@@ -205,9 +225,43 @@ PASSPORT          DRIVERS_LICENSE   RESIDENCE_PERMIT  LICENSE_PLATE
 VIN               VAT               POSTAL_CODE       IP_ADDRESS
 IPV6_ADDRESS      MAC_ADDRESS       HEALTH_INSURANCE  HEALTHCARE_PROVIDER
 CHAMBER_OF_COMMERCE  IMEI          GPS_COORDINATES   UUID
-SOCIAL_HANDLE     SECRET            OTHER
+SOCIAL_HANDLE     SECRET            INTERNAL_ID       OTHER
 ```
 
+`INTERNAL_ID` — an employee, badge or customer number tied to a person — is
+emitted **only** when an explicit label names it (`medarbejdernummer:`,
+`Personalnummer:`, `Employee No:`, `Betriebsstättennr.`, `Badge`). There is no
+pattern for one, because there is no shape for one: without the label, a digit
+run is not distinguishable from any other. The type exists so that a labelled
+employee number is filed correctly instead of being claimed by the phone
+pattern.
+
+### What a label in front of a value decides
+
+A label touching the left edge of a value decides what that value is called.
+The label may be the abbreviation or the word it abbreviates — `BSN:` and
+`Burgerservicenummer:` both reach `NATIONAL_ID` — and it may be the official
+name in the document's own language:
+
+| label | type |
+|---|---|
+| `Companies House Registration:`, `Company Registration Number:` | `CHAMBER_OF_COMMERCE` |
+| `Sozialversicherungsnummer:` | `SSN` |
+| `Passport No.:`, `Paspoortnummer:` | `PASSPORT` |
+| `AGB-code:`, `LANR`, `GMC Number:` | `HEALTHCARE_PROVIDER` |
+| `Medical Card No.:` | `HEALTH_INSURANCE` |
+| `sort code`, `account number` | `BANK_ACCOUNT` |
+| `TAN-activatiecode`, `activation code` | `SECRET` |
+
+A label can also *rule a type out*. A four-digit run is not a postal code when
+a founding or payment participle introduces it (`Opgericht in 2016`,
+`Fondée en 2017`) or when it sits in a telephone parenthetical (`(toest. 3841)`,
+`(ext. 2219)`). Postal codes that merely look like years are unaffected —
+Antwerp's `2018` in `rustige ligging in 2018` is still a `POSTAL_CODE`.
+
+A label never moves a span; it only decides the label. Which characters are
+masked is unchanged either way.
+
 For custom patterns, `entityType` is a plain string (e.g. `"EMPLOYEE_ID"`).
 
 ## Country codes

package/dist/cjs/rules/cues.js

@@ -0,0 +1,240 @@
+"use strict";
+/**
+ * Local cues: what the document calls the value sitting next to it.
+ *
+ * A label touching the left edge of a span is the strongest statement a
+ * document makes about what that value *is*. "Phone: 0705237535" is a phone
+ * number even though those digits pass the Swedish personnummer checksum, and
+ * "Αρ. Ταυτότητας: 00892341" is an identity-card number even though nothing in
+ * the engine can checksum it.
+ *
+ * This table is the single source of that knowledge. Three callers read it, and
+ * none of them may move a span:
+ *
+ * 1. `localCueBonus` — ranks a candidate above its rivals on the *same span*
+ *    when the cue names its type. The oldest use, and why the table exists.
+ * 2. `detectWithEvidence` — re-admits a candidate whose checksum failed when the
+ *    cue names that same type. A declined identifier with its own label in front
+ *    of it is still that identifier, mistyped or truncated.
+ * 3. `retypedBy` — relabels a winning generic candidate whose type the cue
+ *    contradicts, when nothing of the cued type claimed the span.
+ *
+ * Consolidated here from two places that had drifted apart: `LOCAL_CUES` in
+ * `engine.ts` (typed, but missing most of Europe) and `ID_LABEL_BEFORE` in
+ * `suppressors.ts` (broader, but untyped — it could only *delete* a phone
+ * candidate, which left the value unmasked).
+ *
+ * Kept character-for-character in step with `euredact/rules/cues.py`. Two
+ * constructs are written the way they are so that both engines agree:
+ *
+ * * Boundaries are `(?<![A-Za-z0-9_])`, not `\b`. JavaScript's `\b` is
+ *   ASCII-only, so `\bΑΦΜ` never matches — a Greek letter is not a word
+ *   character there, and a space before it is not a transition. A boundary of
+ *   some kind is required regardless: without one the short alternatives match
+ *   the *tail* of an ordinary word. "NIS" matched "tālru**nis**", the Latvian
+ *   for telephone, and suppressed 653 real phone numbers; "iva" matched
+ *   "Pr**iva**t:" and read a personal e-mail address as a VAT number.
+ * * Neither the qualifier nor the run-on may use `\w`. The qualifier is a
+ *   negated ASCII class, fixed for this in 0.3.8; `[^\W\d_]` would reject
+ *   "εταιρειας" here for the same ASCII reason. The run-on was *not* fixed then
+ *   and went on diverging for another release: Slovak "telefón 0956550012"
+ *   needs it to absorb "efón" after `tel`, and Bulgarian
+ *   "пощенски кодове 4000-4999" needs "ове" after `пощенски код`. Python's
+ *   Unicode `\w` did; this SDK's ASCII `\w` did not, so one engine said `PHONE`
+ *   and the other `NATIONAL_ID` on the same document. They were the last two
+ *   type divergences on the corpus, and only the type-aware half of
+ *   `make parity` could see them.
+ *
+ *   The run-on is spelled `(?:[A-Za-z0-9_]|[^\x00-\x7F])` — an ASCII word
+ *   character, or any non-ASCII character — rather than as a negated class of
+ *   separators. A negated class also admits `'`, `%`, `&`, `@` and the rest of
+ *   ASCII punctuation, which `\w` never did, and widening it that far was
+ *   measured at 46 extra false positives with hints and 57 blind over the
+ *   152,300-document corpus. This is `\w` plus the non-ASCII letters `\w`
+ *   should have matched all along, and nothing else.
+ *
+ * Every label ends in `(?:<run-on>|\s+<one word>)`: the label either runs on
+ * into a longer word ("telefoon" → "telefoonnummer") **or** takes one qualifier
+ * word naming whose identifier it is ("ΑΦΜ εταιρειας:", "Tel Nr:") — never both.
+ * Allowing both let "nie" grow into the surname "Nieminen" and then swallow the
+ * real label in "Matti Nieminen\nDOB: 08.05.1994", reading a date of birth as a
+ * Spanish NIE: 52 false positives across the corpus.
+ */
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.RESCUE_TARGETS = exports.CUE_TARGETS = exports.CUES = exports.CUE_WINDOW = void 0;
+exports.cuedType = cuedType;
+exports.retypedBy = retypedBy;
+const types_js_1 = require("../types.js");
+/**
+ * How far back a cue may sit. Short on purpose: a cue is only evidence about
+ * the value it introduces, which is the lesson of the postal-code year defect,
+ * where a cue 150 characters away licensed every number in the window.
+ */
+exports.CUE_WINDOW = 32;
+/**
+ * (entity type, label pattern). Ordered: the first match wins, so the narrower
+ * label comes first where two could read the same text.
+ */
+exports.CUES = [
+    // Phone. Ranking only — nothing is ever relabelled *to* PHONE: a phone cue in
+    // front of a span that no phone pattern matched says the document is laid out
+    // unusually, not that the value is a phone number.
+    [types_js_1.EntityType.PHONE,
+        /(?<![A-Za-z0-9_])(?:phone|tel|telephone|téléphone|telefoon|telefon|telefono|teléfono|tlf|mobil|mobile|gsm|handy|sími|puh|móvil|cell|tālrunis|τηλέφωνο|τηλ)(?:(?:[A-Za-z0-9_]|[^\x00-\x7F])*|\s+[^\s:.\-\d,;()\/]{2,20})\s*\)?\s*[:.\-]*\s*$/i],
+    // National identity number.
+    [types_js_1.EntityType.NATIONAL_ID,
+        /(?<![A-Za-z0-9_])(?:bsn|burgerservicenummer(?:\s*\(bsn\))?|personnummer|rijksregisternummer|nationaal\s*nummer|numéro\s*national|national\s*(?:id|number)|identiteitsnummer|henkilötunnus|kennitala|cpr(?:-?nummer)?|nif|nie|dni|rr|nn|nir|insz|niss|nis|matricule|ausweisnummer|personalausweis|osobní\s*číslo|ЕГН|EGN)(?:(?:[A-Za-z0-9_]|[^\x00-\x7F])*|\s+[^\s:.\-\d,;()\/]{2,20})\s*\)?\s*[:.\-]*\s*$/i],
+    // Cyprus: "ID number" in both official languages, plus the Greek abbreviation
+    // ΑΔΤ (Αστυνομική Ταυτότητα). Neither has a phone reading.
+    [types_js_1.EntityType.NATIONAL_ID,
+        /(?<![A-Za-z0-9_])(?:αρ\.?\s*ταυτότητας|ταυτότητας|αδτ|kimlik\s*numaras[ıi])(?:(?:[A-Za-z0-9_]|[^\x00-\x7F])*|\s+[^\s:.\-\d,;()\/]{2,20})\s*\)?\s*[:.\-]*\s*$/i],
+    // Social security.
+    [types_js_1.EntityType.SSN,
+        /(?<![A-Za-z0-9_])(?:ssn|social\s*security|sozialversicherungsnummer|svnr|sofi)(?:(?:[A-Za-z0-9_]|[^\x00-\x7F])*|\s+[^\s:.\-\d,;()\/]{2,20})\s*\)?\s*[:.\-]*\s*$/i],
+    // Tax number. Greece's ΑΦΜ is issued by the tax authority; the identity-card
+    // number is the ΑΔΤ, above.
+    [types_js_1.EntityType.TAX_ID,
+        /(?<![A-Za-z0-9_])(?:αφμ|α\.φ\.μ\.|steuer-?id|steuernummer|st\.?-?nr|stnr|tin|finanzamt\s+ist|tax\s*(?:id|no|number)|daňové\s*číslo|numer\s*podatkowy)(?:(?:[A-Za-z0-9_]|[^\x00-\x7F])*|\s+[^\s:.\-\d,;()\/]{2,20})\s*\)?\s*[:.\-]*\s*$/i],
+    // Health insurance: the insured person.
+    [types_js_1.EntityType.HEALTH_INSURANCE,
+        /(?<![A-Za-z0-9_])(?:versichertennummer|krankenversicherung|kvnr|kv-?nr|mutualité|ziekenfonds|aansluitingsnummer|medical\s*card(?:\s*(?:no|number))?)(?:(?:[A-Za-z0-9_]|[^\x00-\x7F])*|\s+[^\s:.\-\d,;()\/]{2,20})\s*\)?\s*[:.\-]*\s*$/i],
+    // Healthcare provider: the clinician or practice, not the patient — the Dutch
+    // AGB-code, the German LANR, the UK GMC number.
+    [types_js_1.EntityType.HEALTHCARE_PROVIDER,
+        /(?<![A-Za-z0-9_])(?:agb-?code|lanr|bsnr|gmc(?:\s*(?:no|number|reg))?|big-?nummer|zorgverlener(?:snummer)?)(?:(?:[A-Za-z0-9_]|[^\x00-\x7F])*|\s+[^\s:.\-\d,;()\/]{2,20})\s*\)?\s*[:.\-]*\s*$/i],
+    // Company / trade register. "Companies House" and "Company Registration
+    // Number" are written out: the tail allows a run-on *or* one qualifier word,
+    // never a second word, so a three-word label needs to be spelled.
+    [types_js_1.EntityType.CHAMBER_OF_COMMERCE,
+        /(?<![A-Za-z0-9_])(?:siren|siret|kbo|bce|kvk|ondernemingsnummer|ondernemingen\s+onder\s+nummer|kruispuntbank|numéro\s*d'entreprise|enterprise\s*number|handelsregister|companies\s*house(?:\s*(?:registration|reg|no|number))?|company\s*registration(?:\s*(?:number|no))?|company\s*(?:no|number|reg)|organisationsnummer|orgnr|i[čc]o|identifikační\s*číslo|ЕИК|eik|bulstat|cvr(?:-?nummer)?|virksomhedsnummer)(?:(?:[A-Za-z0-9_]|[^\x00-\x7F])*|\s+[^\s:.\-\d,;()\/]{2,20})\s*\)?\s*[:.\-]*\s*$/i],
+    // VAT.
+    [types_js_1.EntityType.VAT,
+        /(?<![A-Za-z0-9_])(?:vat|btw|tva|ust|iva|moms|alv|mwst|dič|vsk)(?:(?:[A-Za-z0-9_.\-]|[^\x00-\x7F])*|\s+[^\s:.\-\d,;()\/]{2,20})\s*\)?\s*[:.\-]*\s*$/i],
+    // Bank account. A UK sort code is NN-NN-NN, also a licence-plate shape, so
+    // "sort code 20-45-91" was typed LICENSE_PLATE with the label in front of it.
+    // Retype target only — a failed mod-97 is never rescued by a label.
+    [types_js_1.EntityType.BANK_ACCOUNT,
+        /(?<![A-Za-z0-9_])(?:sort\s*code|account\s*(?:no|number)|rekeningnummer|kontonummer|numéro\s*de\s*compte|sort-?code)(?:(?:[A-Za-z0-9_]|[^\x00-\x7F])*|\s+[^\s:.\-\d,;()\/]{2,20})\s*\)?\s*[:.\-]*\s*$/i],
+    // Passport.
+    [types_js_1.EntityType.PASSPORT,
+        /(?<![A-Za-z0-9_])(?:passport(?:\s*(?:no|number))?|paspoort(?:nummer)?|reisepass(?:nummer)?|passeport|passnummer)(?:(?:[A-Za-z0-9_]|[^\x00-\x7F])*|\s+[^\s:.\-\d,;()\/]{2,20})\s*\)?\s*[:.\-]*\s*$/i],
+    // BIC / SWIFT.
+    [types_js_1.EntityType.BIC,
+        /(?<![A-Za-z0-9_])(?:bic|swift)(?:[A-Za-z0-9_\/]|[^\x00-\x7F])*\s*[:.\-]?\s*\(?\s*$/i],
+    // Postal code. A postal code loses every contested span on priority (it is
+    // the weakest evidence the engine has), so without a cue entry
+    // "Cod postal: 040171" was handed to a German phone pattern.
+    [types_js_1.EntityType.POSTAL_CODE,
+        /(?<![A-Za-z0-9_])(?:cod\s*po[sș]tal|postnummer|postnr|postleitzahl|plz(?:-?bereiche?)?|postcode|code\s*postal|codice\s*postale|kod\s*pocztowy|ps[čc]|τ\.?κ\.?|пощенски\s*код)(?:(?:[A-Za-z0-9_]|[^\x00-\x7F])*|\s+[^\s:.\-\d,;()\/]{2,20})\s*\)?\s*[:.\-]*\s*$/i],
+    // Internal / employee identifier. The canon calls these INTERNAL_ID and
+    // annotates them with an LLM, so the engine has no pattern for them — but it
+    // does have a phone pattern that was happily claiming them. The cue is the
+    // only thing that identifies one, which is why the engine emits this type
+    // from here and nowhere else.
+    [types_js_1.EntityType.INTERNAL_ID,
+        /(?<![A-Za-z0-9_])(?:medarbejdernummer|ansattnummer|anställningsnummer|personalnummer|personeelsnummer|matricule\s*salarié|employee\s*(?:no|number|id)|badge(?:\s*(?:no|number))?|betriebsstätten(?:nr|nummer)|grundstücks(?:nr|nummer)|dossier-?(?:nr|nummer)|aktenzeichen|osobní\s*(?:číslo\s*)?zaměstnance)(?:(?:[A-Za-z0-9_]|[^\x00-\x7F])*|\s+[^\s:.\-\d,;()\/]{2,20})\s*\)?\s*[:.\-]*\s*$/i],
+    // Credential. Typed SECRET because that is the type this SDK has; the
+    // training pipeline's report asks for CREDENTIAL, which does not exist here.
+    [types_js_1.EntityType.SECRET,
+        /(?<![A-Za-z0-9_])(?:tan-?activatiecode|activatiecode|activation\s*code|verificatiecode|verification\s*code|otp|pincode)(?:(?:[A-Za-z0-9_]|[^\x00-\x7F])*|\s+[^\s:.\-\d,;()\/]{2,20})\s*\)?\s*[:.\-]*\s*$/i],
+];
+/**
+ * The entity type a label immediately before `start` names, or null.
+ *
+ * Reads at most `CUE_WINDOW` characters, and every pattern is anchored to the
+ * end of that window, so the cue must *touch* the span. A label further away is
+ * about some other value.
+ */
+function cuedType(text, start) {
+    const before = text.slice(Math.max(0, start - exports.CUE_WINDOW), start);
+    for (const [entityType, pattern] of exports.CUES) {
+        if (pattern.test(before))
+            return entityType;
+    }
+    return null;
+}
+/**
+ * Shapes so generic that a label in front is better evidence than the pattern
+ * that matched. A bare digit run is not a phone number because it *could* be
+ * dialled.
+ */
+const RETYPABLE = new Set([
+    types_js_1.EntityType.PHONE, types_js_1.EntityType.POSTAL_CODE, types_js_1.EntityType.LICENSE_PLATE,
+]);
+/**
+ * Retypable, but only when the document gives its country no support at all.
+ *
+ * These carry a checksum, so normally the pattern outranks any label. But a
+ * checksum only says the digits fit *some* national scheme, and a weak one fits
+ * by luck — the same reasoning that demotes an uncorroborated validated
+ * candidate to priority 1 in `engine.ts`. At a country score of 0.0 the engine
+ * is already saying "this passed a Czech checksum in a document with nothing
+ * Czech about it", and an explicit "Passport No.:" is the better evidence.
+ *
+ * Restricted to zero country support on purpose: a Dutch BSN in a Dutch
+ * document scores above zero and stays a NATIONAL_ID whatever label sits near
+ * it, so this cannot relabel a domestic identifier — only the coincidental
+ * foreign ones, which is where "Passport No.: 512847603" and
+ * "Betriebsstättennr. 260326822" both landed, each reported as Czech.
+ */
+const RETYPABLE_UNCORROBORATED = new Set([types_js_1.EntityType.NATIONAL_ID]);
+/**
+ * Types a label can assert on its own.
+ *
+ * PHONE is absent on purpose: "Tel:" in front of something no phone pattern
+ * matched means the document is laid out oddly, not that the value is a phone
+ * number. So are EMAIL, DOB, VIN and the rest — those carry their own
+ * structure, and a nearby word is not entitled to overrule it.
+ */
+exports.CUE_TARGETS = new Set([
+    types_js_1.EntityType.NATIONAL_ID, types_js_1.EntityType.SSN, types_js_1.EntityType.TAX_ID,
+    types_js_1.EntityType.HEALTH_INSURANCE, types_js_1.EntityType.HEALTHCARE_PROVIDER,
+    types_js_1.EntityType.CHAMBER_OF_COMMERCE, types_js_1.EntityType.VAT, types_js_1.EntityType.POSTAL_CODE,
+    types_js_1.EntityType.PASSPORT, types_js_1.EntityType.INTERNAL_ID, types_js_1.EntityType.BANK_ACCOUNT,
+    types_js_1.EntityType.SECRET,
+]);
+/**
+ * Which of those a label may also *rescue* — re-admit after its checksum
+ * failed. Until 0.3.9 this was the same set as CUE_TARGETS and it cannot stay
+ * that way: re-typing a span the engine already decided to mask is cheap to be
+ * wrong about, while rescuing one asserts that a value which failed its check
+ * digit is a real identifier anyway.
+ *
+ * BIC's absence is what keeps the rescue honest. Its validator is a *registry
+ * lookup*, not a checksum, so a failure means "no such bank", which no label can
+ * talk you out of. Rescuing it read 34 unlisted codes behind a "BIC:" label as
+ * real ones. The same reasoning keeps CREDIT_CARD and BANK_ACCOUNT out: Luhn and
+ * mod-97 are strong enough that a failure really does mean "not one of these".
+ * BANK_ACCOUNT joined CUE_TARGETS in 0.3.9 so "sort code 20-45-91" stops being a
+ * LICENSE_PLATE, and is kept out here for exactly the 0.3.8 reason.
+ *
+ * HEALTHCARE_PROVIDER and SECRET are absent because neither is checksummed;
+ * there is no failed validation for a label to overrule in the first place.
+ */
+exports.RESCUE_TARGETS = new Set([
+    types_js_1.EntityType.NATIONAL_ID, types_js_1.EntityType.SSN, types_js_1.EntityType.TAX_ID,
+    types_js_1.EntityType.HEALTH_INSURANCE, types_js_1.EntityType.CHAMBER_OF_COMMERCE,
+    types_js_1.EntityType.VAT, types_js_1.EntityType.POSTAL_CODE, types_js_1.EntityType.PASSPORT,
+    types_js_1.EntityType.INTERNAL_ID,
+]);
+/**
+ * The type a cue overrules `entityType` with, or null to leave it alone.
+ *
+ * Both gates are load-bearing. Measured over 2,000 corpus documents, re-typing
+ * without them fired 52 times and got roughly 30 of those wrong — EMAIL read as
+ * VAT, a date of birth as a phone number, a VIN as VAT — against 20 real fixes.
+ * With them: 14 re-typings, every one of them PHONE to a structured identifier.
+ *
+ * `countryScore` is the document's support for the country the winning pattern
+ * came from; see RETYPABLE_UNCORROBORATED.
+ */
+function retypedBy(text, start, entityType, countryScore) {
+    const type = String(entityType);
+    if (!RETYPABLE.has(type)
+        && !(RETYPABLE_UNCORROBORATED.has(type) && countryScore === 0))
+        return null;
+    const cued = cuedType(text, start);
+    if (cued === null || cued === entityType || !exports.CUE_TARGETS.has(String(cued)))
+        return null;
+    return cued;
+}

package/dist/cjs/rules/engine.js

@@ -35,9 +35,11 @@ var __importStar = (this && this.__importStar) || (function () {
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.RuleEngine = exports.COUNTRY_CODE_ALIASES = void 0;
 exports.resolveCountryCode = resolveCountryCode;
+exports.tieKey = tieKey;
 const types_js_1 = require("../types.js");
 const matchers_js_1 = require("./matchers.js");
 const suppressors_js_1 = require("./suppressors.js");
+const cues_js_1 = require("./cues.js");
 const structural_js_1 = require("./structural.js");
 const index_js_1 = require("./countries/index.js");
 const evidenceMod = __importStar(require("./evidence.js"));
@@ -80,6 +82,48 @@ function resolveCountryCode(countryCode) {
     code = exports.COUNTRY_CODE_ALIASES[code] ?? code;
     return code in index_js_1.COUNTRY_CONFIGS && code !== "SHARED" ? code : null;
 }
+/**
+ * Last-resort ordering for candidates every other field has tied on.
+ *
+ * Reaching this means the engine has no evidence left: same span, no cue, same
+ * priority, the same (usually zero) country score, and the same declared scope.
+ * Something still has to win, and before this the winner was whichever candidate
+ * happened to be generated first — which is country *registration* order, and
+ * the two SDKs do not share one. Python registers alphabetically
+ * (`AT, BE, BG, CH, CY, CZ, ...`); this SDK uses a curated order
+ * (`NL, BE, DE, AT, CH, FR, ..., PT, LU, PL, ..., CZ`). So a bare digit run that
+ * several national schemes accept was filed as a Czech `NATIONAL_ID` by Python
+ * and a Portuguese `PHONE` here — identical characters masked, contradictory
+ * labels, and `make parity` compared only characters, so it stayed green through
+ * the whole of 0.3.8.
+ *
+ * The order reproduces what Python already did rather than inventing a new one,
+ * because every accuracy figure this engine has published was measured with
+ * Python resolving these ties by its registration order. Ordering on the entity
+ * type instead was tried first and re-tuned the corpus twice over: it turned the
+ * German `Steuerß12345678911` from `TAX_ID`/DE into `NATIONAL_ID`/EL, and moved
+ * 820 further spans from `DOB` to `DATE_OF_DEATH`. So the key is deliberately
+ * not total — candidates from the same country are left to the stable sort,
+ * which preserves the order that country's pattern list was written in, and both
+ * SDKs write theirs in the same order.
+ *
+ * Taken from the **pattern**, never the finished detection: this SDK re-types a
+ * cued candidate before the sort and Python re-types after it, so keying on the
+ * final type would reintroduce the divergence in a subtler form.
+ *
+ * Kept character-for-character in step with `tie_key` in `rules/engine.py`.
+ */
+function tieKey(countryCode) {
+    // "SHARED" and "CUSTOM" are not countries and sort as themselves under a
+    // plain alphabetical key — landing between PT and SK, rather than ahead of
+    // every country the way the registry loads them. The bucket restores that.
+    const bucket = countryCode === "SHARED" ? "0" : countryCode === "CUSTOM" ? "1" : "2";
+    // NUL separator, written as an escape so this file stays plain ASCII, and
+    // matching Python byte for byte. It must sort below every character a code
+    // or type name can contain, or a code that is a prefix of another would
+    // order differently in the two SDKs.
+    return `${bucket}\u0000${countryCode}`;
+}
 const warnedCountries = new Set();
 function warnUnknownCountry(code) {
     if (warnedCountries.has(code))
@@ -105,30 +149,13 @@ function warnUnknownCountry(code) {
 //
 // Measured: 263 phone numbers and 18 French SIREN were reported as national IDs,
 // each with an explicit cue in front of it.
-const LOCAL_CUES = [
-    [types_js_1.EntityType.PHONE,
-        /(?:phone|tel|telephone|téléphone|telefoon|telefon|telefono|teléfono|tlf|mobil|mobile|gsm|handy|sími|puh|móvil|cell)\w*\s*[:.\-]?\s*$/i],
-    [types_js_1.EntityType.CHAMBER_OF_COMMERCE,
-        /(?:siren|siret|kbo|bce|kvk|ondernemingsnummer|handelsregister|company\s*(?:no|number|reg)|organisationsnummer|orgnr)\w*\s*[:.\-]?\s*$/i],
-    [types_js_1.EntityType.VAT,
-        /(?:vat|btw|tva|ust|iva|moms|alv|mwst)[\w.\-]*\s*[:.\-]?\s*$/i],
-    [types_js_1.EntityType.BIC,
-        /(?:bic|swift)[\w/]*\s*[:.\-]?\s*\(?\s*$/i],
-    [types_js_1.EntityType.NATIONAL_ID,
-        /(?:bsn|personnummer|rijksregisternummer|national\s*id|identiteitsnummer|henkilötunnus|kennitala|cpr|nif|dni)\w*\s*[:.\-]?\s*$/i],
-];
-// How far back a cue may sit. Short on purpose: a cue is only evidence about the
-// value it introduces, which is the lesson of the postal-code year defect, where
-// a cue 150 characters away licensed every number in the window.
-const CUE_WINDOW = 22;
+//
+// The table itself lives in `cues.ts`, because two other steps read it: the
+// rescue of a declined identifier below, and the re-typing in deduplicate.
+// `CUE_WINDOW` is re-exported from there.
 /** 1 when a cue naming `entityType` sits immediately before the span. */
 function localCueBonus(text, start, entityType) {
-    const before = text.slice(Math.max(0, start - CUE_WINDOW), start);
-    for (const [cuedType, pattern] of LOCAL_CUES) {
-        if (cuedType === entityType && pattern.test(before))
-            return 1;
-    }
-    return 0;
+    return (0, cues_js_1.cuedType)(text, start) === entityType ? 1 : 0;
 }
 class RuleEngine {
     matcher = new matchers_js_1.MultiPatternMatcher();
@@ -235,7 +262,19 @@ class RuleEngine {
         // but failed the checksum is recorded with its country and entity type;
         // what it is allowed to demote is decided below, once the document's
         // countries are known.
+        //
+        // One kind of failure is rescued rather than recorded: a span the document
+        // itself labels as the very type whose checksum just failed.
+        // "Rijksregisternummer: 85.03.19-284.73" is a Belgian national number with a
+        // bad check digit — mistyped, OCR'd, or invented — and it is still a
+        // national number. Before this, the pattern declined, the generic phone rule
+        // was denied the span as a fragment, and the value was left in the clear: a
+        // redaction library printing an identifier it had recognised and rejected.
+        //
+        // Rescued candidates are still recorded as failed spans, so the demotion
+        // zones and the fragment rule below are unchanged.
         const validated = [];
+        const rescued = [];
         const failedSpans = [];
         for (const m of rawMatches) {
             if (this.matcher.validate(m)) {
@@ -244,6 +283,9 @@ class RuleEngine {
             else if (m.patternDef.validator !== null && !m.patternDef.requiresContext) {
                 failedSpans.push({ start: m.start, end: m.end, code: m.countryCode,
                     etype: String(m.patternDef.entityType) });
+                if (cues_js_1.RESCUE_TARGETS.has(String(m.patternDef.entityType))
+                    && (0, cues_js_1.cuedType)(text, m.start) === m.patternDef.entityType)
+                    rescued.push(m);
             }
         }
         // Evidence pass: work out which countries this document belongs to, from
@@ -438,8 +480,28 @@ class RuleEngine {
                 priority = 1;
                 blindPriority = 1;
             }
-            const country = (match.countryCode !== "SHARED" && match.countryCode !== "CUSTOM")
+            let entityType = match.patternDef.entityType;
+            let country = (match.countryCode !== "SHARED" && match.countryCode !== "CUSTOM")
                 ? match.countryCode : null;
+            let outOfScope = declared !== null && country !== null && !declared.has(match.countryCode);
+            let confidence = "high";
+            // The document labels this span as something else. Relabelling, not
+            // suppression: the span is found either way, so removing the claim would
+            // decide only whether the value is masked — and the answer was "no".
+            // "Rijksregisternummer: 85.03.19-284.73" produced no detection at all.
+            //
+            // The country came from the pattern that matched, and that pattern was
+            // just overruled: a Finnish phone rule saying "Finland" is worthless once
+            // the value is filed as a Cypriot identity number. Dropping it also clears
+            // outOfScope, which would otherwise claim the detection belongs to a
+            // country the caller did not declare while naming no country at all.
+            const retyped = (0, cues_js_1.retypedBy)(text, match.start, entityType, countryScores.get(match.countryCode) ?? 0);
+            if (retyped !== null) {
+                entityType = retyped;
+                country = null;
+                outOfScope = false;
+                confidence = "medium";
+            }
             candidates.push({
                 blindPriority,
                 cue: localCueBonus(text, match.start, match.patternDef.entityType),
@@ -447,6 +509,46 @@ class RuleEngine {
                 length: match.end - match.start,
                 countryScore: countryScores.get(match.countryCode) ?? 0,
                 inScope: declared === null || declared.has(match.countryCode) ? 1 : 0,
+                tie: tieKey(match.countryCode),
+                tier: 0,
+                det: {
+                    entityType,
+                    start: match.start,
+                    end: match.end,
+                    text: match.text,
+                    source: types_js_1.DetectionSource.RULES,
+                    country,
+                    confidence,
+                    outOfScope,
+                },
+            });
+        }
+        // A declined identifier the document itself labels. Cue-backed by
+        // construction, so `cue` is 1 — that is what lets it take a span from a
+        // foreign pattern whose checksum happened to pass, which is how
+        // "ΑΦΜ: 147382965" was reported as a Czech national ID.
+        //
+        // `priority` is 0: below every regex-only claim, so it wins only what
+        // nothing better wants. `blindPriority` is 1 so that span selection — the
+        // country-blind half of the sort — treats it like any other regex-strength
+        // claim and invariant I1 is untouched.
+        //
+        // Emitted as "low", which is the honest description: the shape and the label
+        // agree, the check digit does not. A caller wanting only checksum-backed
+        // detections filters on it.
+        for (const match of rescued) {
+            if ((0, suppressors_js_1.shouldSuppress)(text, match, scratch))
+                continue;
+            const country = (match.countryCode !== "SHARED" && match.countryCode !== "CUSTOM")
+                ? match.countryCode : null;
+            candidates.push({
+                blindPriority: 1,
+                cue: 1,
+                priority: 0,
+                length: match.end - match.start,
+                countryScore: countryScores.get(match.countryCode) ?? 0,
+                inScope: declared === null || declared.has(match.countryCode) ? 1 : 0,
+                tie: tieKey(match.countryCode),
                 tier: 0,
                 det: {
                     entityType: match.patternDef.entityType,
@@ -455,14 +557,15 @@ class RuleEngine {
                     text: match.text,
                     source: types_js_1.DetectionSource.RULES,
                     country,
-                    confidence: "high",
+                    confidence: "low",
                     outOfScope: declared !== null && country !== null && !declared.has(match.countryCode),
                 },
             });
         }
         for (const d of (0, structural_js_1.detectStructuralDob)(text)) {
             candidates.push({ blindPriority: 1, cue: 0, priority: 1, length: d.end - d.start,
-                countryScore: 0, inScope: 1, tier: 0, det: d });
+                countryScore: 0, inScope: 1, tier: 0, det: d,
+                tie: tieKey(d.country ?? "") });
         }
         const detections = this.deduplicate(candidates);
         detections.sort((a, b) => a.start - b.start || b.end - a.end);
@@ -530,7 +633,8 @@ class RuleEngine {
             b.cue - a.cue || // local cue         } country-aware
             b.priority - a.priority || // country priority  } — decides the
             b.countryScore - a.countryScore || // country evidence  } LABEL only
-            b.inScope - a.inScope); // declared scope    }
+            b.inScope - a.inScope || // declared scope    }
+            (a.tie < b.tie ? -1 : a.tie > b.tie ? 1 : 0)); // total   } — see tieKey
         const result = [];
         const occupied = new Set();
         for (const { det } of sorted) {

package/dist/cjs/rules/suppressors.js

@@ -44,11 +44,16 @@ const MATH_AFTER = /^\s*[=+\-×÷*/]/;
 const COUNTRY_PREFIX_HYPHEN = /(?:^|[^A-Za-z0-9])[A-Z]{1,2}-\s*$/;
 const SEQUENTIAL_PATTERNS = /^(?:0{6,}|1234567890?|0123456789|9876543210?|1111111111?|000000000|123456789)$/;
 // YEAR_PATTERN kept for reference but replaced by RECENT_YEAR in suppressYearAsPostal
-// The leading \b is load-bearing. Without it the short alternatives match the
-// *tail* of an ordinary word: "NIS" matched "tālru**nis** ", the Latvian for
-// "telephone", so every Latvian phone number was suppressed as though it sat
-// behind a Belgian national-ID label. 653 misses from a missing word boundary.
-const ID_LABEL_BEFORE = /\b(?:BSN|RR|NN|NIR|INSZ|NISS|NIS|Steuer-?ID|TIN|NIF|NIE|SSN|rijksregisternummer|numéro\s*national|national\s*number|matricule|Ausweisnummer|Personalausweis|Versichertennummer|KVNR|KV-Nr|Steuernummer|St\.\-Nr|StNr|Finanzamt\s+ist|Ondernemingen\s+onder\s+nummer|ondernemingsnummer|numéro\s*d'entreprise|enterprise\s*number|Kruispuntbank)\s*[:.]?\s*$/i;
+// `ID_LABEL_BEFORE` and `suppressPhoneAfterIdLabel` lived here. Their labels are
+// now typed entries in `cues.ts`, and a phone-shaped span behind one is
+// *relabelled* rather than dropped.
+//
+// Dropping was the wrong verb. The span is found either way, so removing the
+// claim decided only whether the value was masked — and the answer was "no":
+// "Rijksregisternummer: 85.03.19-284.73" produced no detection at all, a
+// redaction library printing in full an identifier it had recognised and
+// rejected. The 653-miss word-boundary lesson recorded here moved with the
+// labels; see the module comment in `cues.ts`.
 const SERVICE_NUMBER = /^0800[\-\s]/;
 const DATE_PATTERN_FULL = /^\d{2}[-/.]\d{2}[-/.]\d{4}$|^\d{4}[-/.]\d{2}[-/.]\d{2}$/;
 const PASSPORT_CONTEXT_BEFORE = /(?:Reisepass|passport|passeport|paspoort|Bisheriger\s+Reisepass)\s*(?:Nr\.?|Nummer|nummer|number|n[°o])?\s*[:.]?\s*$/i;
@@ -342,11 +347,52 @@ function suppressYearAsPostal(text, match) {
     // Suppress: recent years without postal context are almost never postal codes
     return true;
 }
-function suppressPhoneAfterIdLabel(text, match) {
-    if (match.patternDef.entityType !== types_js_1.EntityType.PHONE)
+// ── Postal code: disqualified by the word in front of it ───────────────
+//
+// The mirror image of `rules/cues.ts`. There a label promotes a type; here a
+// word refuses one. Both were reported together: a bare four-digit run becomes
+// a POSTAL_CODE as soon as *any* real postal code establishes the country,
+// which is to say in almost every real document, because POSTAL_CONTEXT_NEAR
+// above then rescues it from suppressYearAsPostal. Measured on the training
+// corpus: 55 bare years in prose ("Opgericht in 2016", "Fondée en 2017") plus
+// telephone extensions ("(toest. 3841)", "(poste interne 3318)", "(ext. 2219)").
+/**
+ * Prepositions that can only be temporal. Nothing is ever located "since 2018",
+ * so these disqualify a postal code on their own.
+ */
+const YEAR_WORD_BEFORE = /(?<![A-Za-z0-9_])(?:sinds|since|depuis|seit|siden|sedan|desde|vuonna|anno|dal)\s+$/i;
+/**
+ * "in", "en" and "im" are *both* temporal and locative, and in exactly the
+ * countries whose postal codes are year-shaped. Belgian 2000 is Antwerp and
+ * 2018 is one of its districts, so "Rustige ligging in 2018, vlakbij openbaar
+ * vervoer" is an address, not a date — an earlier version of this rule
+ * suppressed it and lost a real postal code, which is the worse error for a
+ * redaction tool. So an ambiguous preposition needs a founding or payment
+ * participle in front of it, which is what every case in the report had:
+ * "Opgericht in 2016", "Fondée en 2017", "versé en 2025".
+ */
+const YEAR_VERB_BEFORE = /(?<![A-Za-z0-9_])(?:opgericht|gesticht|opgestart|founded|established|created|fond[ée]e?|cr[ée][ée]e?|gegründet|gegruendet|errichtet|grundlagt|grundad|perustettu|fundada|fundado|fondata|costituita|vers[ée]|betaald|uitbetaald|ausgezahlt|paid|geboren|born)\s+(?:in|en|im|op|the)\s+$/i;
+/**
+ * A capitalised word immediately after keeps the candidate: "in 2000 Antwerpen"
+ * really is a postal code, and so is the "AZ" of a Dutch "1105 AZ". Without
+ * this the rule would suppress the very addresses it is meant to leave alone.
+ */
+const TOWN_AFTER = /^\s*[A-ZÀ-ÞĀ-Ž]/;
+/**
+ * A telephone extension marker. "poste interne 3318" puts one qualifier word
+ * between the marker and the number, so allow exactly one.
+ */
+const EXTENSION_BEFORE = /(?<![A-Za-z0-9_])(?:toest(?:el)?|ext|extension|poste|durchwahl|doorkiesnummer|tst|nebenstelle)\.?\s*(?:[a-zà-ÿ]{2,10}\s*)?[:.]?\s*$/i;
+/** Suppress a postal code the word in front of it rules out. */
+function suppressPostalAfterDisqualifyingWord(text, match) {
+    if (match.patternDef.entityType !== types_js_1.EntityType.POSTAL_CODE)
         return false;
-    const [before] = getContext(text, match.start, match.end);
-    return ID_LABEL_BEFORE.test(before);
+    const before = text.slice(Math.max(0, match.start - 24), match.start);
+    if (EXTENSION_BEFORE.test(before))
+        return true;
+    return RECENT_YEAR.test(match.text.trim())
+        && (YEAR_WORD_BEFORE.test(before) || YEAR_VERB_BEFORE.test(before))
+        && !TOWN_AFTER.test(text.slice(match.end, match.end + 3));
 }
 function suppressPhoneServiceNumber(_text, match) {
     if (match.patternDef.entityType !== types_js_1.EntityType.PHONE)
@@ -793,11 +839,11 @@ function suppressRequiresContext(text, match) {
     return !match.patternDef.contextKeywords.some(kw => context.includes(kw.toLowerCase()));
 }
 const TYPE_SUPPRESSORS = {
-    [types_js_1.EntityType.PHONE]: [suppressCurrency, suppressUnits, suppressReference, suppressMath, suppressPhoneAfterIdLabel, suppressPhoneServiceNumber, suppressPhoneDateOverlap, suppressPhoneAsNumberRange],
+    [types_js_1.EntityType.PHONE]: [suppressCurrency, suppressUnits, suppressReference, suppressMath, suppressPhoneServiceNumber, suppressPhoneDateOverlap, suppressPhoneAsNumberRange],
     [types_js_1.EntityType.NATIONAL_ID]: [suppressCurrency, suppressUnits, suppressReference, suppressLegal, suppressMath, suppressNatidAsPassport, suppressSeNatidAsOrg],
     [types_js_1.EntityType.SSN]: [suppressCurrency, suppressUnits, suppressReference, suppressMath],
     [types_js_1.EntityType.TAX_ID]: [suppressCurrency, suppressUnits, suppressReference, suppressMath, suppressTaxidAsIpAddress],
-    [types_js_1.EntityType.POSTAL_CODE]: [suppressCurrency, suppressUnits, suppressMath, suppressLegal, suppressYearAsPostal, suppressPostalInsideIban, suppressPostalAsHouseNumber, suppressPostalInLongerIdentifier, suppressReference],
+    [types_js_1.EntityType.POSTAL_CODE]: [suppressCurrency, suppressUnits, suppressMath, suppressLegal, suppressYearAsPostal, suppressPostalAfterDisqualifyingWord, suppressPostalInsideIban, suppressPostalAsHouseNumber, suppressPostalInLongerIdentifier, suppressReference],
     [types_js_1.EntityType.BIC]: [suppressBicWithoutEvidence],
     [types_js_1.EntityType.BANK_ACCOUNT]: [suppressReference],
     [types_js_1.EntityType.LICENSE_PLATE]: [suppressPlateInCompound, suppressDePlateUnknownDistrict, suppressPlateAsCurrencyAmount],