euredact 0.3.2__tar.gz → 0.3.4__tar.gz

{euredact-0.3.2 → euredact-0.3.4}/CHANGELOG.md

@@ -1,5 +1,190 @@
 # Changelog
 
+## 0.3.4 (2026-07-30)
+
+### Fixed
+
+- **Recovered the latency 0.3.3 gave away, without giving back its recall.**
+  0.3.3 made `\b` catch identifiers glued to a non-ASCII letter by rewriting it
+  to a three-branch lookaround union on all 303 patterns. Correct, but it cost
+  **1.8× on short records and 2.8× on real documents**, and its ASCII branch
+  manufactured boundaries *inside* words at non-ASCII letters — truncating
+  `@Ciarán` to `@Ciar` and typing `FICIAIRES`, a fragment of `BÉNÉFICIAIRES`,
+  as a national ID.
+
+  The boundary is now chosen per occurrence. Next to a digit, the ASCII reading
+  alone is *exactly* the union — a Unicode letter is never an ASCII word
+  character, so the ASCII branch already succeeds everywhere the Unicode branch
+  does — and one lookaround replaces the alternation. Everywhere else plain `\b`
+  stays, which is what a Greek e-mail local part needs.
+
+  | | 186 chars | 3,424 chars |
+  |---|---:|---:|
+  | pure Python | 3.0 ms → **907 µs** | 58 ms → **13.6 ms** |
+  | with `pyahocorasick` | 2.3 ms → **775 µs** | 42 ms → **10.9 ms** |
+  | with `[fast]` | 878 µs → **516 µs** | 14.4 ms → **5.3 ms** |
+
+  Within 4% of the pre-0.3.3 baseline, with all five glued-identifier cases and
+  both Unicode e-mail cases still detected. Per-type precision, recall and F1
+  are **byte-identical** across all 152,300 corpus documents.
+
+- **Social handles containing a non-ASCII letter were masked only up to it.**
+  `@Ciarán` came back as `@Ciar`, leaving `án` in the clear, because the pattern's
+  character class was ASCII-only and the old boundary let the match stop mid-word.
+  The class is now Unicode-aware, so the whole handle is masked. The TypeScript
+  SDK already had this right; the two now agree.
+
+- **German social-security numbers were unredacted whenever the document used
+  the abbreviation `SVNR`.** The pattern and its context gate were both correct;
+  the cue list simply had `SV-Nummer` and `Sozialversicherung` and not the short
+  form people actually type.
+
+  Measured across all 152,300 corpus documents, this was **every** German
+  social-security number the rule missed — 204 of them, in the clear:
+
+  | | recall before | after |
+  |---|---:|---:|
+  | `SOCIAL_SECURITY`, country hints | 84.16% | **100.00%** |
+  | `SOCIAL_SECURITY`, blind | 83.77% | **99.61%** |
+
+  No false-positive cost: 0 before, 0 after, and no other entity type moved.
+  `SV-Nr` and `RVNR` are added alongside it — the same gap for the other two
+  spellings, and the health-insurance rule already carries `KVNR` next to
+  `KV-Nummer` for exactly this reason.
+
+  Found by per-type measurement rather than a report. `SOCIAL_SECURITY` was the
+  only type below 90% recall that was not a known cloud-tier case, which is
+  what made it worth chasing.
+
+### Added
+
+- **`tests/metrics.py`** — per-entity-type precision, recall, F1 and
+  false-positive counts over the corpus, both detection modes, with `--csv`.
+  `eval_full.py` renders an HTML report; this is the plain-text counterpart with
+  its matching rules stated in the module docstring, so a figure quoted from it
+  can be reproduced and argued with.
+
+  Writing it surfaced two flaws in the shared evaluation config, both of which
+  had been quietly distorting published numbers:
+
+  - `HEALTH_ID` had no entry in `CATEGORY_MAP`, so its 252 labels could never be
+    matched — counted as 252 misses *and* charging the engine's correct
+    `HEALTH_INSURANCE` detections as 252 false positives. `SECRET` was also
+    unmapped and worked only by accident, its fallback happening to be a real
+    entity type. Both are mapped now, which slightly raises measured recall.
+  - False positives were attributed to a category that may carry no labels at
+    all: every `BIC` detection was charged to a zero-support `BIC` row while the
+    corpus labels them `SWIFT_BIC`, splitting one type's precision across two
+    rows and reporting it as 0.00%.
+
+## 0.3.3 (2026-07-29)
+
+### Fixed
+
+- **A shorter validated match could re-cut a longer one and leak the
+  remainder.** Under `countries` `["NL"]` the Dutch national-ID pattern
+  validates on `194.232.104`, outranked the IP address `194.232.104.77`, and
+  left `.77` in the output. The TypeScript SDK did the same with no country
+  declared at all.
+
+  Span length now outranks every other signal, including validation. Priority
+  still decides between candidates of *equal* length, which is where it was
+  always meant to apply — a valid IBAN beats a coincidental match on the same
+  characters. Masking more than necessary is recoverable; masking less is not.
+
+- **`countries` could change which spans were found, not just how they were
+  labelled.** Promotion depends on whether the document corroborates a
+  country, so the declared country decided which of two overlapping candidates
+  won. Two cases were found, the second only after fixing the first.
+
+  Ordering between different spans is now country-blind by construction —
+  length, then the span's structural tier, then leftmost — and `(length, start)`
+  identifies a span uniquely, so the country-aware signals can only choose the
+  **label within a span**, never its extent. The invariant holds by design
+  rather than by test.
+
+- **A bare string for `countries` silently discarded every detection.**
+  `countries="NL"` is iterable, so it was walked into the codes `"N"` and `"L"`.
+  Neither resolves, so nothing was declared — and every detection carrying a
+  country came back flagged out of scope, while the redacted text still looked
+  correct. The README tells callers to filter on exactly that field, so a
+  documented pipeline kept none of them. It failed toward "no PII here", from a
+  one-character typo.
+
+  Both SDKs now raise `TypeError` naming the argument and showing the fix, on
+  every entry point. A wrong *code* still only warns: raising there would invite
+  callers to wrap redaction in try/except and skip it. A wrong *type* is a
+  programming error with no correct interpretation to fall back on.
+
+- **The generic phone pattern claimed fragments of rejected identifiers.**
+  `Rijksregisternummer: 85.03.19-284.73` was reported as a `PHONE`. The Belgian
+  national-number pattern matched the whole value and failed its checksum; the
+  separator-tolerant phone pattern then took characters 3-14 of it.
+
+  A candidate covering only *part* of a rejected identifier is a fragment of
+  that identifier, not a separate entity, and is now dropped. An equal span is
+  left alone: that is two schemes competing for the whole value, and demoting
+  those is what previously handed a Swedish phone number to a Danish CPR.
+
+  Fragment detection ignores the declared country by design. It removes
+  candidates, so making it country-aware would let `countries` change which
+  spans are found — the invariant in `test_invariant_generation.py`.
+
+  No measurable cost: recall, precision and type-correct rate are unchanged on
+  all 152,300 corpus documents.
+
+- **Identifiers glued to a non-ASCII letter were missed.** Python's `\b` is
+  Unicode-aware, so it saw no boundary between a Cyrillic or accented letter
+  and a digit: `ЕГН7523169263`, `PESELŁ44051401359`, `čísloř7401011233`,
+  `Steuerß12345678911` and `kodasž38605181232` were all detected by the
+  TypeScript SDK and silently missed by this one.
+
+  `\b` is now compiled to the *union* of the Unicode and ASCII readings.
+  Swapping to ASCII-only — the obvious fix — trades one silent miss for
+  another: it drops Greek and Cyrillic e-mail local parts, which are
+  deliberately supported. `re.ASCII` is unusable for the same reason, since it
+  would also narrow `\w`.
+
+  **This is expensive.** The rewritten boundary is a lookaround union rather
+  than a single opcode, and every pattern carries it: **1.8× on short records
+  and 2.8× on real documents** (497 µs → 878 µs, and 5.2 ms → 14.4 ms, with
+  `[fast]` installed). An earlier revision of this entry said "about 20%" —
+  that was measured on 186-character synthetic records only and extrapolated,
+  wrongly, to real documents.
+
+  `[fast]` is close to required as a result: the RE2 prefilter absorbs most of
+  the cost by not running patterns that cannot match. Accuracy on the corpus is
+  unchanged.
+
+  Recovering the speed without giving back the five silent misses is open — the
+  likely route is to ASCII-ise only those boundaries that abut a digit or an
+  ASCII-only class, and leave plain `\b` where the neighbouring element is
+  Unicode-capable, rather than applying the union to all 303 patterns.
+
+### Added
+
+- **`make check` and `make verify`.** One entry point for every check.
+  `make check` is what CI runs; `make verify` adds the corpus checks CI cannot
+  run, because the corpus lives outside the repository.
+
+  - `tests/sweep.py` — structural properties over ~187,000 documents: offsets
+    index the text, spans do not overlap, detection is deterministic, the cache
+    is transparent, and no country argument changes which spans are found. Both
+    ranking bugs above were found here; the twenty-document version that runs in
+    CI showed nothing.
+  - `scripts/parity.py` — do both SDKs mask the same *characters*, over whole
+    corpora. Conformance vectors pin named cases; this is the broad counterpart,
+    and it is how a 19,014-character gap was found that no vector showed.
+
+
+### Known issues
+
+- A checksum-invalid identifier occupying a span no other detector claims can
+  still be reported under the wrong type — `7401011234` with `countries` `["CZ"]`
+  is masked as a Romanian `PHONE` at `countryConfidence` 0. The span *is*
+  redacted; only the label is wrong. The fragment rule above does not apply
+  because the spans are equal rather than nested.
+
 ## 0.3.2 (2026-07-29)
 
 ### Changed
@@ -76,9 +261,14 @@
   Identical digits: `0612345678` is both a valid Dutch mobile number and a
   valid Danish CPR. Only the document distinguishes them.
 
-  On 152,300 documents, blind detection (no `countries=`) improved from
-  **95.6% to 98.3% precision** and from **94.90% to 98.38% type-correct**,
-  closing the gap to hinted detection from 3.4 points to 0.3.
+  On all 152,300 documents, blind detection (no `countries=`) improved from
+  **95.6% to 98.3% precision** and from **96.10% to 98.84% type-correct**,
+  closing the precision gap to hinted detection from 3.4 points to 0.3.
+
+  (An earlier revision of this entry quoted 94.90% → 98.38% for type-correct.
+  Those came from a 30,000-document prefix of the corpus, which is drawn from
+  one dataset and is not representative; they were labelled as full-corpus in
+  error. The figures above are measured over all 152,300.)
 
   Weights are derived from the corpus, not hand-tuned. Inference influences
   scoring only; it can never cause a miss.

{euredact-0.3.2 → euredact-0.3.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: euredact
-Version: 0.3.2
+Version: 0.3.4
 Summary: European PII redaction SDK — rule engine for 31 countries
 Project-URL: Documentation, https://github.com/euredact/euredact#readme
 Project-URL: Repository, https://github.com/euredact/euredact
@@ -792,11 +792,17 @@ Each `PatternDef` can specify:
 Measured on one core, all 31 countries loaded, `detect_dates=True`. Cost scales
 with document length, so the input size is stated rather than averaged away.
 
-| Input | Base | With `[fast]` |
-|---|---:|---:|
-| Short record (186 chars) | 723 µs — 1,383/s | **496 µs — 2,017/s** |
-| Real document (3,424 chars) | 10.4 ms — 96/s | **5.4 ms — 187/s** |
-| Memory per country | ~50 KB compiled patterns | |
+| Input | Pure Python | Aho-Corasick | With `[fast]` |
+|---|---:|---:|---:|
+| Short record (186 chars) | 907 µs — 1,103/s | 775 µs — 1,290/s | **516 µs — 1,938/s** |
+| Real document (3,424 chars) | 13.6 ms — 73/s | 10.9 ms — 92/s | **5.3 ms — 190/s** |
+| Memory per country | ~50 KB | ~50 KB | ~50 KB |
+
+Making `\b` catch identifiers glued to a non-ASCII letter costs something, but
+only where it must: next to a digit the ASCII reading alone is exactly
+equivalent, so one lookaround replaces a three-way alternation, and plain `\b`
+stays everywhere else. 0.3.3 applied the union to all 303 patterns and was
+1.8×/2.8× slower for it.
 
 ```bash
 pip install euredact[fast]

{euredact-0.3.2 → euredact-0.3.4}/README.md

@@ -751,11 +751,17 @@ Each `PatternDef` can specify:
 Measured on one core, all 31 countries loaded, `detect_dates=True`. Cost scales
 with document length, so the input size is stated rather than averaged away.
 
-| Input | Base | With `[fast]` |
-|---|---:|---:|
-| Short record (186 chars) | 723 µs — 1,383/s | **496 µs — 2,017/s** |
-| Real document (3,424 chars) | 10.4 ms — 96/s | **5.4 ms — 187/s** |
-| Memory per country | ~50 KB compiled patterns | |
+| Input | Pure Python | Aho-Corasick | With `[fast]` |
+|---|---:|---:|---:|
+| Short record (186 chars) | 907 µs — 1,103/s | 775 µs — 1,290/s | **516 µs — 1,938/s** |
+| Real document (3,424 chars) | 13.6 ms — 73/s | 10.9 ms — 92/s | **5.3 ms — 190/s** |
+| Memory per country | ~50 KB | ~50 KB | ~50 KB |
+
+Making `\b` catch identifiers glued to a non-ASCII letter costs something, but
+only where it must: next to a digit the ASCII reading alone is exactly
+equivalent, so one lookaround replaces a three-way alternation, and plain `\b`
+stays everywhere else. 0.3.3 applied the union to all 303 patterns and was
+1.8×/2.8× slower for it.
 
 ```bash
 pip install euredact[fast]

{euredact-0.3.2 → euredact-0.3.4}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "euredact"
-version = "0.3.2"
+version = "0.3.4"
 description = "European PII redaction SDK — rule engine for 31 countries"
 requires-python = ">=3.10"
 license = "Apache-2.0"

{euredact-0.3.2 → euredact-0.3.4}/src/euredact/__init__.py

@@ -22,7 +22,7 @@ from __future__ import annotations
 
 from typing import Iterator
 
-__version__ = "0.3.2"
+__version__ = "0.3.4"
 
 from euredact.rules.bic_registry import set_bic_registry
 from euredact.rules.context import DocumentContext

{euredact-0.3.2 → euredact-0.3.4}/src/euredact/rules/countries/_shared.py

@@ -218,7 +218,11 @@ class SharedConfig(CountryConfig):
             # Must be preceded by whitespace or start-of-string, 1-30 alphanumeric/underscore/dot chars
             PatternDef(
                 entity_type=EntityType.SOCIAL_HANDLE,
-                pattern=r"(?<!\w)@[a-zA-Z][a-zA-Z0-9_.]{1,29}\b",
+                # Unicode-aware: an ASCII-only class cannot match "@Ciarán"
+                # at all, and the boundary rewrite used to paper over that by
+                # stopping at "@Ciar" and masking a fragment while leaving "án"
+                # in the clear. [^\W\d_] is "a letter in any script".
+                pattern=r"(?<!\w)@[^\W\d_][\w.]{1,29}\b",
                 validator=None,
                 description="Social media handle (@username)",
             ),

{euredact-0.3.2 → euredact-0.3.4}/src/euredact/rules/countries/de.py

@@ -132,6 +132,14 @@ class DEConfig(CountryConfig):
                     "Rentenversicherung", "RV-Nummer", "Sozialversicherung",
                     "SV-Nummer", "Versicherungsnummer", "Rentenversicherungsnummer",
                     "Sozialversicherungsnummer",
+                    # The punctuation-free abbreviations. "SVNR" alone accounted
+                    # for every German social-security number this rule missed
+                    # across the corpus — 204 of them, unredacted, because the
+                    # list had "SV-Nummer" and "Sozialversicherung" but not the
+                    # form people actually type. "SV-Nr" and "RVNR" are the same
+                    # gap for the other two spellings; cf. KVNR alongside
+                    # KV-Nummer on the health-insurance rule.
+                    "SVNR", "SV-Nr", "RVNR",
                 ],
                 requires_context=True,
             ),

{euredact-0.3.2 → euredact-0.3.4}/src/euredact/rules/engine.py

@@ -39,6 +39,31 @@ def _validate_custom_pattern(pattern: str) -> None:
         )
 
 
+def check_country_arg(value: object, param: str) -> None:
+    """Reject a bare string where a list of country codes is expected.
+
+    ``countries="NL"`` is iterable, so Python walks it into the codes ``"N"``
+    and ``"L"``. Neither resolves, so nothing is declared — and every detection
+    that *does* carry a country is then flagged ``out_of_scope``. A caller
+    following the documented pattern of filtering on that field silently keeps
+    none of them, while ``redacted_text`` still looks perfectly correct. The
+    failure direction is "no PII here", from a one-character typo.
+
+    A wrong *code* is data and only warns (see ``CountryRegistry.warn_unknown``)
+    — raising there would invite callers to wrap redaction in try/except and
+    skip it. A wrong *type* is a programming error, surfaces immediately, and
+    has no correct interpretation to fall back on.
+    """
+    if isinstance(value, (str, bytes)):
+        shown = value.decode(errors="replace") if isinstance(value, bytes) else value
+        raise TypeError(
+            f"{param} must be a list of country codes, not a bare string. "
+            f"Pass {param}=[{shown!r}] rather than {param}={shown!r} — a string "
+            f"is iterated character by character, which silently declares "
+            f"nothing and flags every detection out_of_scope."
+        )
+
+
 class RuleEngine:
     """The core rule-based PII detection engine.
 
@@ -135,6 +160,9 @@ class RuleEngine:
         It is scored alongside this text's own evidence but is **not** returned,
         so a caller accumulating evidence never counts it twice.
         """
+        check_country_arg(country_codes, "countries")
+        check_country_arg(country_hint, "country_hint")
+
         raw_matches = self.generate_candidates(text)
 
         # The declared countries are a scoring signal from here on: they rank
@@ -226,16 +254,50 @@ class RuleEngine:
         # phone number — and demoting on it cost every PHONE candidate in the
         # document its rank, which is how a Dutch mobile came back as a Danish
         # NATIONAL_ID even after the digits were correctly ranked.
-        # Zones are per entity type. A failed checksum means "these digits are
-        # not a valid X" — that is evidence against X, not against the span.
-        # Demoting every type at once meant Sweden's own personnummer checksum
-        # failing on 0708787668 demoted the Swedish *phone* candidate for the
-        # same digits, handing the span to a Danish CPR that happened to
-        # validate. Worth 878 mistyped phone numbers on 30,000 documents.
+        # Two rules, because a failed checksum carries two different messages
+        # depending on how the spans line up.
+        #
+        # Same type, any containment: "these digits are not a valid X" is
+        # evidence against X, not against the span. Demoting *every* type on an
+        # equal span meant Sweden's own personnummer checksum failing on
+        # 0708787668 demoted the Swedish *phone* candidate for the same digits,
+        # handing the span to a Danish CPR that happened to validate — 878
+        # mistyped phone numbers per 30,000 documents.
+        #
+        # Any type, strict subset: a candidate covering only *part* of a
+        # rejected identifier is a fragment of it, not a different entity. The
+        # generic separator-tolerant phone pattern claimed characters 3-14 of
+        # the rejected Belgian national number in
+        # "Rijksregisternummer: 85.03.19-284.73" and reported it as a PHONE.
+        # This is the case the original suppression zones were written for —
+        # licence-plate fragments inside an invalid IBAN — which the per-type
+        # split above had given up along with the over-reach.
+        #
+        # An equal span is two schemes competing for the whole value; a strict
+        # subset is one scheme picking over another's wreckage.
+        corroborated: list[tuple[int, int, object]] = [
+            (start, end, etype)
+            for start, end, cc, etype in failed_spans
+            if not country_scores or cc in ("SHARED", "CUSTOM") or cc in country_scores
+        ]
+        corroborated.sort()
+
+        # Fragment detection deliberately ignores corroboration and uses every
+        # failed span, because it *removes* candidates and must therefore be
+        # country-blind. Gating it on the document's countries made the set of
+        # spans found depend on `countries=`, which is precisely invariant I1 —
+        # caught by tests/test_invariant_generation.py within seconds of being
+        # written. Demotion may be country-aware because it only reorders; a
+        # demoted candidate still claims a span nothing else wants.
+        #
+        # Kept unmerged: merging adjacent spans would invent strict-subset
+        # relationships that neither original span had.
+        all_failed = sorted((start, end) for start, end, _cc, _et in failed_spans)
+        strict_starts = [f[0] for f in all_failed]
+
         by_type: dict[object, list[tuple[int, int]]] = {}
-        for start, end, cc, etype in failed_spans:
-            if not country_scores or cc in ("SHARED", "CUSTOM") or cc in country_scores:
-                by_type.setdefault(etype, []).append((start, end))
+        for start, end, etype in corroborated:
+            by_type.setdefault(etype, []).append((start, end))
 
         # Merge overlapping zones per type and index them for O(log n)
         # containment checks instead of O(n) per match.
@@ -274,7 +336,7 @@ class RuleEngine:
         # (priority, in_scope, start, end, match, prebuilt). in_scope ranks
         # candidates within a priority tier; it never removes any.
         candidates: list[
-            tuple[int, float, int, int, int, RawMatch | None, Detection | None]
+            tuple[int, int, float, int, int, int, RawMatch | None, Detection | None]
         ] = []
         for match, has_valid_validator in validated:
             # A validator-less match inside a failed-validation span used to be
@@ -295,9 +357,31 @@ class RuleEngine:
                     zone_starts, zone_ends = zone
                     idx = bisect.bisect_right(zone_starts, match.start) - 1
                     demoted = idx >= 0 and zone_ends[idx] >= match.end
+            # Strict subset of a rejected identifier, whatever its type: the
+            # candidate covers only part of something already recognised and
+            # rejected, so it is a fragment of that, not a separate entity.
+            #
+            # Dropped rather than demoted. Demotion cannot help here: nothing
+            # else wants a fragment, so a demoted candidate still wins its span
+            # by default — the generic phone pattern kept claiming characters
+            # 3-14 of a rejected Belgian national number. Dropping is safe in a
+            # way that whole-span deletion was not, because the characters stay
+            # covered by whatever claims the enclosing span, or by nothing —
+            # and "nothing" is the right answer for the tail of a rejected
+            # identifier.
+            fragment = False
+            if match.pattern_def.validator is None and not demoted:
+                hi = bisect.bisect_right(strict_starts, match.start)
+                for zs, ze in all_failed[:hi]:
+                    if match.end <= ze and (match.start > zs or match.end < ze):
+                        fragment = True
+                        break
+            if fragment:
+                continue
 
             if demoted:
                 priority = -1
+                blind_priority = -1
             elif has_valid_validator:
                 # A passing checksum normally outranks everything. But a
                 # checksum only says the digits fit *some* national scheme, and
@@ -321,28 +405,32 @@ class RuleEngine:
                     or match.country_code in country_scores
                 )
                 priority = 3 if vouched else 1
+                blind_priority = 3
             elif match.country_code == "CUSTOM":
                 priority = 2
+                blind_priority = 2
             elif match.pattern_def.entity_type == EntityType.POSTAL_CODE:
                 # A bare digit run is the weakest evidence in the engine and
                 # must never re-cut a span a structured detector claims
                 # (PHONE, SSN, NATIONAL_ID, BANK_ACCOUNT, VAT...). Postal
                 # codes take unclaimed spans only, whatever the span lengths.
                 priority = 0
+                blind_priority = 0
             else:
                 priority = 1
+                blind_priority = 1
 
             in_scope = 1 if (declared is None or match.country_code in declared) else 0
             country_score = country_scores.get(match.country_code, 0.0)
             candidates.append(
-                (priority, country_score, in_scope,
+                (blind_priority, priority, country_score, in_scope,
                  match.start, match.end, match, None)
             )
 
         # Structural detectors (JSON field names, CSV headers). These arrive as
         # finished Detections, carry no RawMatch, and are not suppressed.
         for d in detect_structural_dob(text):
-            candidates.append((1, 0.0, 1, d.start, d.end, None, d))
+            candidates.append((1, 1, 0.0, 1, d.start, d.end, None, d))
 
         # Deduplicate: validated > custom > regex-only, then longer wins
         detections = self._deduplicate(text, candidates, declared)
@@ -366,7 +454,7 @@ class RuleEngine:
     def _deduplicate(
         text: str,
         candidates: list[
-            tuple[int, float, int, int, int, RawMatch | None, Detection | None]
+            tuple[int, int, float, int, int, int, RawMatch | None, Detection | None]
         ],
         declared: set[str] | None = None,
     ) -> list[Detection]:
@@ -388,24 +476,51 @@ class RuleEngine:
         if not candidates:
             return []
 
-        # Sort by (priority, span_length) descending. Stable, so ties fall back
-        # to pattern registration order, as before.
+        # Ordering between *different* spans is country-blind; the country
+        # only ever picks the winner among candidates covering the *same* span.
+        #
+        # That split is what makes invariant I1 structural rather than merely
+        # tested. Country-aware promotion had twice decided which span won:
+        # under countries=["NL"] a validated national ID outranked a longer IP
+        # address and exposed ".77", and two equal-length candidates at
+        # different offsets swapped places. Both were found by sweeping the
+        # corpus, not by reasoning, and both are impossible now — the key below
+        # is country-blind up to and including `start`, and (length, start)
+        # already identifies a span uniquely.
+        #
+        # Ordering, in words: longest first (a shorter match winning re-cuts a
+        # longer one and leaks the remainder), then the structural tier, then
+        # leftmost. Only after a span is fixed do the country-aware fields
+        # break the tie, deciding the label rather than the extent.
+        #
+        # The structural tier is a property of the *span*, not of the candidate:
+        # the best tier anything claiming those exact characters can offer. Used
+        # per candidate it would also discriminate *within* a span, where the
+        # country-aware tier is supposed to have the final word — that silently
+        # disabled country inference entirely, handing every ambiguous value to
+        # whichever foreign scheme happened to validate.
+        span_tier: dict[tuple[int, int], int] = {}
+        for c in candidates:
+            key = (c[4], c[5])
+            if c[0] > span_tier.get(key, -2):
+                span_tier[key] = c[0]
+
         sorted_cands = sorted(
             candidates,
-            # Span length outranks scope: preferring the declared country over
-            # the longest match can truncate an entity. Measured: with
-            # countries=["BE"], the Belgian phone pattern claims 11 of the 14
-            # characters of "06 12 34 56 78" and leaves three digits exposed.
-            # Scope only breaks ties between candidates of equal length, so it
-            # can change WHICH country is attributed, never WHAT is masked.
-            key=lambda c: (c[0], c[4] - c[3], c[1], c[2]),
-            reverse=True,
+            key=lambda c: (
+                -(c[5] - c[4]),                # longest first    } country-blind
+                -span_tier[(c[4], c[5])],      # best tier here   } — decides
+                c[4],                          # leftmost         } WHICH span
+                -c[1],                         # country priority } country-aware
+                -c[2],                         # country evidence } — decides the
+                -c[3],                         # declared scope   } LABEL only
+            ),
         )
         result: list[Detection] = []
         occupied: set[int] = set()
         span_verdicts: dict[tuple[object, int, int], bool] = {}
 
-        for _priority, _score, in_scope, start, end, match, prebuilt in sorted_cands:
+        for _bp, _prio, _score, in_scope, start, end, match, prebuilt in sorted_cands:
             if any(p in occupied for p in range(start, end)):
                 continue