OpenScrub 1.0.76__tar.gz → 1.0.78__tar.gz

{openscrub-1.0.76 → openscrub-1.0.78}/.gitignore

@@ -126,3 +126,7 @@ CLAUDE.local.md
 # Windows installer build output (windows/build_installer.bat)
 windows/output/
 model_select.json
+
+# benchmark working dirs (regenerate with benchmark/corpus.py)
+benchmark/_corpus/
+benchmark/_results/

{openscrub-1.0.76 → openscrub-1.0.78}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: OpenScrub
-Version: 1.0.76
+Version: 1.0.78
 Summary: Local, GPU-accelerated redaction for video and screen recordings — blur, black-box, or mosaic faces, names, and any regex-matchable PII, with human review.
 Project-URL: Homepage, https://github.com/austinmabry/OpenScrub
 Project-URL: Repository, https://github.com/austinmabry/OpenScrub
@@ -261,6 +261,17 @@ rather than a half-second late. Defaults are tuned for the hardest case —
 dense, scrolling records screens (medical, financial, CRM, support
 consoles) — but the engine is general-purpose.
 
+<p align="center">
+  <picture>
+    <source srcset="https://raw.githubusercontent.com/austinmabry/OpenScrub/main/assets/screenshots/redaction-demo.webp" type="image/webp">
+    <img src="https://raw.githubusercontent.com/austinmabry/OpenScrub/main/assets/screenshots/redaction-demo.gif"
+         alt="Split-screen wipe: a skier carving down a slope, one side redacted by OpenScrub with a body-shaped silhouette blur that tracks them through the turn while the snow, skis and poles stay sharp"
+         width="900">
+  </picture>
+</p>
+<p align="center"><em>Real output. The wipe reveals the same frames redacted and untouched: the silhouette mask tracks the skier through a fast carve, and only the body is covered — the mountain stays sharp.<br>
+<sub>Compressed for GitHub — OpenScrub itself renders at full source quality and resolution (see <a href="https://openscrub.app">openscrub.app</a>). Footage: <a href="https://www.pexels.com/video/15155922/">Pexels</a>.</sub></em></p>
+
 <p align="center">
   <img src="https://raw.githubusercontent.com/austinmabry/OpenScrub/main/assets/screenshots/editor-demo.gif"
        alt="OpenScrub editor in action: a video loaded on the timeline with stackable detection windows, per-window categories and zones, and audio-track lanes"
@@ -766,17 +777,31 @@ This software is provided "AS IS" without warranty of any kind — see the
 
 ## Validation
 
-During development the pipeline is scored against a **synthetic corpus**:
-a generator plants fake PII at known locations across the hard cases —
-static charts, schedule grids, scrolling notes, OCR-disrupting
-highlights, embedded face photos — and a scorer checks the rendered
-output against the ground truth:
-
-    PII recall:           100.0%   (102/102 planted samples blurred)
-    Benign preservation:  100.0%   (39/39 benign samples left readable)
-
-(measured with the Tesseract fallback engine; PaddleOCR + spaCy NER, the
-recommended stack, is stronger). The shipped regression suite
+OpenScrub ships a **reproducible redaction benchmark**
+([`benchmark/`](benchmark/)) — you run the same commands and get your
+own numbers instead of trusting ours. It measures what a reader of the
+output actually cares about: is the planted PII still *readable*, and
+can a face recognizer still *identify* the person?
+
+Text PII (v1.0.78, CPU, corpus v2 — 8 scenarios including dark mode,
+sensor noise, 2.5° rotation and 120 kbps re-compression; 64 planted
+values × 5 sampled frames):
+
+    PII recall:           100.0%   (64/64 planted values never readable in the output)
+    Frame leak rate:        0.00%  (0/320 sampled region-frames readable)
+    Benign preservation:  100.0%   (43/43 non-PII samples left readable)
+
+Face re-identification (v1.0.78, default blur, SFace matcher, valid
+same-person controls): **0 of 511 redacted faces re-identified** across
+a handheld party clip at 1080p and native 4K, a public-domain close-up
+interview, and a mosaic-mode run — with post-redaction similarity below the
+*cross-person chance floor*, i.e. a redacted face matches its own
+original no better than a stranger's face does.
+
+Numbers on a fixed corpus mean the corpus no longer finds leaks — not
+that leaks are impossible. The benchmark README lists every defect the
+harness caught (all fixed and regression-pinned), and the rules for
+quoting these numbers honestly. The shipped test suite
 (`pytest test_openscrub.py`) exercises the same end-to-end pipeline on
 synthetic videos — for this tool a regression is not a bug, it's a leak,
 so the suite must stay green on every change.

{openscrub-1.0.76 → openscrub-1.0.78}/README.md

@@ -27,6 +27,17 @@ rather than a half-second late. Defaults are tuned for the hardest case —
 dense, scrolling records screens (medical, financial, CRM, support
 consoles) — but the engine is general-purpose.
 
+<p align="center">
+  <picture>
+    <source srcset="https://raw.githubusercontent.com/austinmabry/OpenScrub/main/assets/screenshots/redaction-demo.webp" type="image/webp">
+    <img src="https://raw.githubusercontent.com/austinmabry/OpenScrub/main/assets/screenshots/redaction-demo.gif"
+         alt="Split-screen wipe: a skier carving down a slope, one side redacted by OpenScrub with a body-shaped silhouette blur that tracks them through the turn while the snow, skis and poles stay sharp"
+         width="900">
+  </picture>
+</p>
+<p align="center"><em>Real output. The wipe reveals the same frames redacted and untouched: the silhouette mask tracks the skier through a fast carve, and only the body is covered — the mountain stays sharp.<br>
+<sub>Compressed for GitHub — OpenScrub itself renders at full source quality and resolution (see <a href="https://openscrub.app">openscrub.app</a>). Footage: <a href="https://www.pexels.com/video/15155922/">Pexels</a>.</sub></em></p>
+
 <p align="center">
   <img src="https://raw.githubusercontent.com/austinmabry/OpenScrub/main/assets/screenshots/editor-demo.gif"
        alt="OpenScrub editor in action: a video loaded on the timeline with stackable detection windows, per-window categories and zones, and audio-track lanes"
@@ -532,17 +543,31 @@ This software is provided "AS IS" without warranty of any kind — see the
 
 ## Validation
 
-During development the pipeline is scored against a **synthetic corpus**:
-a generator plants fake PII at known locations across the hard cases —
-static charts, schedule grids, scrolling notes, OCR-disrupting
-highlights, embedded face photos — and a scorer checks the rendered
-output against the ground truth:
-
-    PII recall:           100.0%   (102/102 planted samples blurred)
-    Benign preservation:  100.0%   (39/39 benign samples left readable)
-
-(measured with the Tesseract fallback engine; PaddleOCR + spaCy NER, the
-recommended stack, is stronger). The shipped regression suite
+OpenScrub ships a **reproducible redaction benchmark**
+([`benchmark/`](benchmark/)) — you run the same commands and get your
+own numbers instead of trusting ours. It measures what a reader of the
+output actually cares about: is the planted PII still *readable*, and
+can a face recognizer still *identify* the person?
+
+Text PII (v1.0.78, CPU, corpus v2 — 8 scenarios including dark mode,
+sensor noise, 2.5° rotation and 120 kbps re-compression; 64 planted
+values × 5 sampled frames):
+
+    PII recall:           100.0%   (64/64 planted values never readable in the output)
+    Frame leak rate:        0.00%  (0/320 sampled region-frames readable)
+    Benign preservation:  100.0%   (43/43 non-PII samples left readable)
+
+Face re-identification (v1.0.78, default blur, SFace matcher, valid
+same-person controls): **0 of 511 redacted faces re-identified** across
+a handheld party clip at 1080p and native 4K, a public-domain close-up
+interview, and a mosaic-mode run — with post-redaction similarity below the
+*cross-person chance floor*, i.e. a redacted face matches its own
+original no better than a stranger's face does.
+
+Numbers on a fixed corpus mean the corpus no longer finds leaks — not
+that leaks are impossible. The benchmark README lists every defect the
+harness caught (all fixed and regression-pinned), and the rules for
+quoting these numbers honestly. The shipped test suite
 (`pytest test_openscrub.py`) exercises the same end-to-end pipeline on
 synthetic videos — for this tool a regression is not a bug, it's a leak,
 so the suite must stay green on every change.

openscrub-1.0.78/benchmark/README.md

@@ -0,0 +1,159 @@
+# OpenScrub redaction benchmark
+
+There is no industry certification for video redaction — nothing like
+NIST FRVT for face recognition. So instead of quoting a number nobody
+can check, everything here is **reproducible**: you run the same
+commands and get your own numbers.
+
+Two things are measured, because they answer different questions.
+
+| Harness | Question |
+|---|---|
+| `corpus.py` + `score.py` | Is planted PII still **readable** in the output? |
+| `reid.py` | Can a face recognizer still **identify** the person? |
+
+## Why not detection mAP
+
+Detection benchmarks optimise a precision/recall balance. For redaction,
+precision barely matters — over-blur is acceptable by design (fail
+closed) — and per-frame recall understates the problem, because a face
+missed in *one frame out of three hundred* is still a leak. So the
+scorer never asks "did a detection fire?". It crops the rendered output
+and asks whether the sensitive string can still be **read**, and whether
+the face can still be **matched**. A leak counts as a leak even when the
+report claims the region was covered.
+
+## Text PII: corpus + scorer
+
+```
+python benchmark/corpus.py --out benchmark/_corpus
+python benchmark/score.py  --corpus benchmark/_corpus --engine onnx
+```
+
+The corpus is synthetic and deterministic — same seed, same layout, same
+footage for everyone. Every planted value comes from a **reserved or
+documentation range**: 555 phone numbers, `example.com` (RFC 2606),
+`192.0.2.0/24` (RFC 5737), the public `4111…` test card. The benchmark
+for a privacy tool must not itself be a privacy problem.
+
+Eight scenarios (corpus v2), 64 PII + 43 benign samples: a static
+record screen, small text (needs the 2x re-OCR pass), PII on coloured
+highlight bars (the classic OCR disruptor), scrolling notes at a known
+rate, dark mode (light text on dark), heavy sensor noise, a 2.5°
+rotated page (photographed-document tilt), and a brutally re-compressed
+copy (120 kbps — screen-share/messenger quality).
+
+Reported: **PII recall** (samples never readable in any sampled frame),
+**frame leak rate** (region-frames that were readable), and **benign
+preservation** (how much readable non-PII survived — the over-blur cost).
+
+## Faces: re-identification attack
+
+```
+python benchmark/reid.py --video yourclip.mp4
+python benchmark/reid.py --video yourclip.mp4 --mode mosaic --coverage concealed
+```
+
+Finds faces in the original with a deliberately low threshold (the
+oracle should over-find, never grade generously), redacts, then at each
+oracle location asks whether a face is still detectable and whether its
+SFace embedding still matches the identity taken from the original.
+
+**Read the control line before quoting any result.** The harness also
+compares original faces against *other original frames of the same
+footage*. If the recognizer cannot link an unredacted face to another
+unredacted frame of itself, then "the redacted face did not match"
+proves nothing about the redaction — and the tool says so, loudly,
+instead of letting you publish the number.
+
+Supply your own footage or public-domain clips. Do not run this on real
+people's faces and publish the frames.
+
+## Measured: text PII
+
+OpenScrub v1.0.78, CPU only (no GPU), corpus v2 (8 scenarios), 5 frames
+sampled per planted value. Higher is better for recall and benign
+preservation; lower is better for leak rate:
+
+| OCR engine | PII recall | Frame leak rate | Benign preserved |
+|---|---|---|---|
+| ONNX PP-OCRv5 (the default on CPU/Intel/Windows) | 100.0 % | 0.00 % | 100.0 % |
+| Tesseract | 100.0 % | 0.00 % | 97.7 % |
+
+Every planted value — names, SSNs, phone numbers, addresses, cards,
+emails, dates of birth, IP addresses — was unreadable in every sampled
+frame of the rendered output, across all eight scenarios, on both
+engines. The one benign loss (Tesseract) is "Follow-up in 6 weeks"
+garbled under the 120 kbps compression — over-blur, the failure
+direction this tool chooses on purpose.
+
+A 100 % row on a fixed corpus means the corpus no longer finds leaks —
+not that leaks are impossible. The honest reading is the history below.
+
+### What building the benchmark caught (all fixed in 1.0.78)
+
+The first runs of this harness measured the ONNX backend at **40.6 %
+recall with a 57.5 % frame leak rate**. Every point of the gap traced to
+a real, previously-invisible defect; each is now fixed and pinned by a
+regression test:
+
+- **Lost spaces.** PP-OCRv5's space class routinely loses the argmax to
+  blank at word gaps, welding `SSN 123-45-6789` into `SSN123-45-6789` —
+  every multi-token pattern silently stopped matching.
+- **Full-width punctuation.** The Chinese-first vocabulary returned
+  `（555）013-8842` — visually identical to a reader, matched by no regex.
+- **Shredded card numbers.** Compression and noise fragment a spaced
+  PAN into arbitrary digit chunks (`41 11 11 …`, `4111. 1111-1111`);
+  the fixed-format matcher missed them. Fragments now join under a
+  Luhn + brand-prefix gate.
+- **Column-spanning over-blur.** Two-column layouts merged into one OCR
+  line, so an address detection's blur swallowed unrelated text a page
+  width away (benign preservation cost, found by the corpus's benign
+  samples).
+- **Tilted-line grouping.** On the 2.5° rotated page, line grouping by
+  average y-center split a first name from its surname and the name was
+  never detected at all.
+- **Uncovered tails.** Under heavy compression the last scans read only
+  fragments of a card number — real evidence the string was still on
+  screen, but not a full match, so the blur ended ~0.7 s early.
+
+## Measured: face re-identification
+
+OpenScrub v1.0.78, default blur mode, SFace matcher, same-person
+threshold 0.55. "Re-identified" = the redacted face still matched the
+identity embedding taken from the unredacted original. All three rows
+have a valid control (the recognizer reliably re-identifies the same
+faces in the *unredacted* footage, so a 0 % result is meaningful):
+
+| Footage | Faces attacked | Re-identified | Still detected as a face | Control |
+|---|---|---|---|---|
+| Handheld 1080p party clip, five subjects, constant motion ([Pexels 7100826](https://www.pexels.com/video/7100826/), downscaled from 4K) | 125 | **0 (0.0 %)** | 0.0 % | 0.888 |
+| The same party clip at native 4K (3840×2160 — larger faces, more recoverable detail) | 128 | **0 (0.0 %)** | 0.0 % | 0.887 |
+| Public-domain interview, large frontal close-up faces, 1080×1920 ([Wikimedia Commons](https://commons.wikimedia.org/wiki/File:Interview_with_a_Marshall_Center_professor_(999984).webm)) | 133 | **0 (0.0 %)** | 0.7 % | 0.691 |
+| Same party clip, `--mode mosaic` | 125 | **0 (0.0 %)** | 4.8 % | 0.888 |
+
+Mean post-redaction similarity sat **below the cross-person chance
+floor** in all three runs — a redacted face matches its own original no
+better than a random stranger's face does.
+
+This table also records a caught defect: the interview clip's close-up
+faces (200–750 px) were initially re-identified at **8.3 % straight
+through the old blur** — a large face keeps enough low-frequency
+identity when the blur kernel is 1/3 of the region. The kernel is now
+2/3 of the region (pinned by test), which also drove
+"still detected as a face" from 99.2 % to 0 % on the party clip.
+
+Two candidate clips (a street scene with small oblique faces, a
+multi-person interview) produced **invalid controls** — the recognizer
+could not re-identify unredacted faces in them — and the harness
+refused to grade them. That refusal is the feature.
+
+## Rules for quoting these numbers
+
+- Always state **version, engine, corpus version, and hardware**. A
+  number without them is not a claim, it is a vibe.
+- Never say "HIPAA compliant" or "GDPR compliant" — tools are not
+  compliant, processes are.
+- Never say 100 %, "guaranteed", or "fully anonymised".
+- Publish the failures next to the successes. A tool that says where it
+  struggles is the one worth believing.

{openscrub-1.0.76 → openscrub-1.0.78}/openscrub.py

@@ -36,7 +36,7 @@ from dataclasses import dataclass, asdict
 import cv2
 import numpy as np
 
-VERSION = "1.0.76"
+VERSION = "1.0.78"
 
 # ----------------------------------------------------------------------------
 # OCR backends
@@ -481,33 +481,65 @@ def group_lines(words):
     {text, words: [(word, box, conf, char_start, char_end)]}"""
     if not words:
         return []
-    items = sorted(words, key=lambda w: ((w[1][1] + w[1][3]) / 2, w[1][0]))
+    # Sweep words left-to-right and chain each onto the line whose
+    # RIGHTMOST member's y-center is within 0.7x line height. Comparing to
+    # the local neighbour (not the line's average y) makes grouping follow
+    # tilted text: on a 2.5deg-rotated page the two ends of one text row
+    # differ by ~50px — far more than a line height — so an average-y band
+    # seeded at one end rejected words at the other ('Marguerite' was
+    # orphaned from 'Vandersloot' and the name went undetected on the
+    # rotated benchmark), while adjacent words shear by only ~1px per 25px
+    # of gap and always chain.
+    items = sorted(words, key=lambda w: (w[1][0], (w[1][1] + w[1][3]) / 2))
     lines = []
     for w in items:
         yc = (w[1][1] + w[1][3]) / 2
         h = max(w[1][3] - w[1][1], 1)
-        placed = False
+        best = None
+        best_gap = None
         for ln in lines:
-            if abs(ln["yc"] - yc) < 0.7 * max(h, ln["h"]):
-                ln["raw"].append(w)
-                ln["yc"] = (ln["yc"] * (len(ln["raw"]) - 1) + yc) / len(ln["raw"])
-                ln["h"] = max(ln["h"], h)
-                placed = True
-                break
-        if not placed:
+            m = ln["raw"][-1]                       # rightmost so far
+            myc = (m[1][1] + m[1][3]) / 2
+            mh = max(m[1][3] - m[1][1], 1)
+            if abs(yc - myc) < 0.7 * max(h, mh):
+                gap = w[1][0] - m[1][2]
+                if best is None or gap < best_gap:
+                    best, best_gap = ln, gap
+        if best is not None:
+            best["raw"].append(w)
+            best["h"] = max(best["h"], h)
+        else:
             lines.append({"raw": [w], "yc": yc, "h": h})
+    lines.sort(key=lambda ln: sum((m[1][1] + m[1][3]) / 2
+                                  for m in ln["raw"]) / len(ln["raw"]))
     out = []
     for ln in lines:
         ln["raw"].sort(key=lambda w: w[1][0])
-        text = ""
-        entries = []
-        for w, box, conf in ln["raw"]:
-            if text:
-                text += " "
-            start = len(text)
-            text += w
-            entries.append((w, box, conf, start, len(text)))
-        out.append({"text": text, "words": entries})
+        # split at column-sized horizontal gaps: two-column layouts land on
+        # the same y-band, and a merged line makes every line-level match
+        # (address box, spaCy NER context) span BOTH columns — the blur then
+        # swallows unrelated text a page-width away. A gap wider than 2.5x
+        # the line height (same scale as the name-pair adjacency rule) is a
+        # column boundary, not a word gap. PII values never span columns,
+        # so splitting can only tighten boxes, never lose a match.
+        segments = [[]]
+        prev_x2 = None
+        for w in ln["raw"]:
+            if (prev_x2 is not None
+                    and w[1][0] - prev_x2 > 2.5 * max(ln["h"], 10)):
+                segments.append([])
+            segments[-1].append(w)
+            prev_x2 = w[1][2]
+        for seg in segments:
+            text = ""
+            entries = []
+            for w, box, conf in seg:
+                if text:
+                    text += " "
+                start = len(text)
+                text += w
+                entries.append((w, box, conf, start, len(text)))
+            out.append({"text": text, "words": entries})
     return out
 
 
@@ -612,11 +644,32 @@ class NameDetector:
                     if w.strip(".,:;()[]").lower() in self.extra:
                         hits.append((box, w))
 
-            # --- 4. capitalized-pair heuristic (fallback) ---
-            if self.heuristic:
+            # --- 4. capitalized-pair heuristic ---
+            # UNION, not fallback: spaCy's small model is blind to
+            # uncommon names with no sentence context — on a records
+            # screen "Marguerite Vandersloot" returned NO entities at
+            # all (a real benchmarked leak), so NER availability must
+            # never disable this detector (same union principle as the
+            # face models: an extra detector may only ADD coverage).
+            # With NER present the pair test tightens to non-ALLCAPS
+            # tokens so headers ("PATIENT RECORD") don't flood review;
+            # without NER the historical broader behavior stands.
+            if self.heuristic or self.nlp is not None:
+                strict = self.nlp is not None and not self.heuristic
                 for i in range(len(words) - 1):
                     w1, b1 = words[i][0], words[i][1]
                     w2, b2 = words[i + 1][0], words[i + 1][1]
+                    if strict and (w1.strip(".,:;()[]").isupper()
+                                   or w2.strip(".,:;()[]").isupper()):
+                        continue
+                    # a real "First Last" pair sits within a couple of
+                    # character-heights; line reconstruction also glues
+                    # COLUMNS together, and the resulting fake pairs
+                    # ("Vandersloot | Ward", "Road | Priority") blurred
+                    # benign text two columns away (benchmarked)
+                    gap = b2[0] - b1[2]
+                    if gap > 2.5 * max(b1[3] - b1[1], b2[3] - b2[1], 8):
+                        continue
                     pair = False
                     if self._namey(w1) and self._namey(w2):
                         pair = True
@@ -810,6 +863,55 @@ def detect_phi(words, lines, t, offset, namer, mrn_re, custom_res=()):
                     "card", joined, min(g[2] for g in grp))
                 break
 
+    # shredded card: OCR can fragment a spaced PAN into arbitrary digit
+    # chunks ("41 11 11 11 1 1 11 11 11" — a real read of a highlighted
+    # card row under video compression), which the fixed 3/4-group join
+    # above cannot see. Join maximal runs of consecutive same-row digit
+    # tokens; any 13-16 digit concatenation that carries a card-brand
+    # prefix AND passes Luhn is a card. Luhn is the false-positive
+    # gate — an arbitrary digit run fails it 9 times out of 10.
+    digit_run = []
+    for w in words + [("", (0, 0, 0, 0), 0.0)]:      # sentinel flushes
+        tok = re.sub(r"\D", "", w[0])
+        # digits with incidental punctuation still count: motion blur
+        # turned one group into "1111." and the stray period broke every
+        # join (a real scrolling-card leak), and dark/noisy footage welds
+        # neighbouring groups into "1111.1111" (8 digits, 9 chars) — the
+        # old {1,8}/6-digit caps split the run right there and leaked the
+        # card. Any punctuated token up to a full welded PAN (16 digits)
+        # may join; Luhn + brand prefix stay the false-positive gate.
+        digity = (bool(tok) and len(tok) <= 16
+                  and bool(re.fullmatch(r"[\d.,:;\-]{1,19}", w[0])))
+        same_row = (digit_run
+                    and abs((w[1][1] + w[1][3]) - (digit_run[-1][1][1]
+                            + digit_run[-1][1][3])) / 2
+                    < max(8, digit_run[-1][1][3] - digit_run[-1][1][1]))
+        if digity and (not digit_run or same_row):
+            digit_run.append(w)
+            continue
+        if digit_run:
+            for a in range(len(digit_run)):
+                digits = ""
+                for z in range(a, len(digit_run)):
+                    digits += re.sub(r"\D", "", digit_run[z][0])
+                    if len(digits) > 16:
+                        break
+                    if len(digits) >= 13:
+                        grouped = " ".join(digits[k:k + 4]
+                                           for k in range(0, len(digits), 4))
+                        if RE_CARD.search(grouped) and _luhn_ok(digits):
+                            bx = [q[1] for q in digit_run[a:z + 1]]
+                            add((min(b[0] for b in bx),
+                                 min(b[1] for b in bx),
+                                 max(b[2] for b in bx),
+                                 max(b[3] for b in bx)), "card", digits,
+                                min(q[2] for q in digit_run[a:z + 1]))
+                            a = None
+                            break
+                if a is None:
+                    break
+        digit_run = [w] if digity else []
+
     # split-across-words phone: "(501)" "555-0142"
     for i in range(len(words) - 1):
         joined = words[i][0] + " " + words[i + 1][0]
@@ -2510,6 +2612,46 @@ def merge_detections(dets, hold, scans=None, bridge_gap=4.0, fuzz=None,
                 break
         else:
             merged.append(d)
+
+    # Tail extension: a span ends `hold` after its LAST positive detection,
+    # but the text often outlives that detection — under heavy compression
+    # the final scans read only fragments ('4111.' '1111-1111' '1111'),
+    # which fail the full pattern yet are strong evidence the string is
+    # still on screen (the benchmark's compressed card leaked exactly this
+    # tail). Walk the scans PAST each span's last sighting: while a scan's
+    # words in the region fuzzy-match the span text (the same same-word
+    # test contradicted() uses), the content verifiably persisted — extend
+    # coverage through that scan. A scan with no matching read stops the
+    # walk, so a real scene change never accretes stale blur.
+    if scans and fuzz:
+        for m in merged:
+            if getattr(m, "dense", False) or not m.text:
+                continue
+            mx1, my1, mx2, my2 = m.cbox
+            mt = PhiMemory.norm(m.text)
+            for st, _cum, words in scans:
+                if st <= m.last_seen + 0.01:
+                    continue
+                if st - m.last_seen > bridge_gap:
+                    break
+                support = False
+                for txt, (x1, y1, x2, y2), conf in words:
+                    if conf < 0.5:
+                        continue
+                    cxm = (x1 + x2) / 2
+                    cym = (y1 + y2) / 2
+                    if not (mx1 - 6 <= cxm <= mx2 + 6
+                            and my1 - 4 <= cym <= my2 + 4):
+                        continue
+                    n = PhiMemory.norm(txt)
+                    if len(n) >= 3 and (fuzz.ratio(n, mt) >= 70
+                                        or fuzz.partial_ratio(n, mt) >= 85):
+                        support = True
+                        break
+                if not support:
+                    break
+                m.last_seen = st
+                m.t_end = max(m.t_end, st + hold)
     return merged
 
 
@@ -2594,7 +2736,14 @@ def blur_region(frame, x1, y1, x2, y2, mode, shape="rect"):
         filled = cv2.resize(small, (x2 - x1, y2 - y1),
                             interpolation=cv2.INTER_NEAREST)
     else:
-        k = max(31, (((x2 - x1) // 3) | 1))
+        # kernel = 2/3 region width (was 1/3): the re-identification
+        # benchmark showed close-up faces (200-750px) surviving a 1/3-width
+        # blur at SFace similarity 0.57-0.69 — above the 0.55 same-person
+        # bar. At 2/3 width the same faces measure 0.11-0.23, at or below
+        # the cross-person chance floor: a blurred face carries no more
+        # identity than a random stranger's. Small faces were already safe
+        # (the 112px embed crop upsamples them); this closes the close-up.
+        k = max(31, ((2 * (x2 - x1) // 3) | 1))
         filled = _gauss_big(roi, k)
     if shape == "ellipse":
         # elliptical mask hugs a face: no smeared background corners, which
@@ -2673,7 +2822,7 @@ def blur_silhouette(frame, x1, y1, x2, y2, mode, polys, poly_box,
                            interpolation=cv2.INTER_LINEAR)
         filled = cv2.resize(small, (rw, rh), interpolation=cv2.INTER_NEAREST)
     else:
-        k = max(31, ((rw // 3) | 1))
+        k = max(31, ((2 * rw // 3) | 1))
         filled = _gauss_big(roi, k)
     cv2.copyTo(filled, mask, roi)
 
@@ -2981,7 +3130,7 @@ def _blur_yuv10(y, u, v, x1, y1, x2, y2, mode, shape="rect"):
             filled = cv2.resize(small, (px2 - px1, py2 - py1),
                                 interpolation=cv2.INTER_NEAREST)
         else:
-            k = max(kmin, (((px2 - px1) // 3) | 1))
+            k = max(kmin, ((2 * (px2 - px1) // 3) | 1))
             filled = _gauss_big(roi, k)
         if shape == "ellipse":
             rw, rh = px2 - px1, py2 - py1
@@ -3061,7 +3210,7 @@ def _blur_silhouette_yuv10(y, u, v, x1, y1, x2, y2, mode, polys,
             filled = cv2.resize(small, (px2 - px1, py2 - py1),
                                 interpolation=cv2.INTER_NEAREST)
         else:
-            k = max(kmin, (((px2 - px1) // 3) | 1))
+            k = max(kmin, ((2 * (px2 - px1) // 3) | 1))
             filled = _gauss_big(roi, k)
         m = mask
         if m.shape != roi.shape:            # chroma planes are half-res
@@ -4289,21 +4438,83 @@ class QRDetector:
         return _nms_boxes(out, 0.4)
 
 
-def _ctc_greedy_decode(logits, charmap):
+_FULLWIDTH = {ord(c): ord(a) for c, a in zip(
+    "０１２３４５６７８９（）－．，：／＠＃",
+    "0123456789()-.,:/@#")}
+_FULLWIDTH[0x3000] = 0x20          # ideographic space
+
+
+def _ascii_forms(s):
+    """Map full-width/CJK punctuation and digits to their ASCII forms
+    (pure, unit-tested). PP-OCRv5's vocabulary is Chinese-first and it
+    returns '（555）013-8842' for a plain US phone number — no PII regex
+    matches that, so a real phone number sailed through untouched. The
+    glyphs are visually identical to a reader; only the code points
+    differ, so normalizing costs nothing and closes the hole."""
+    return s.translate(_FULLWIDTH)
+
+
+def _ctc_greedy_decode(logits, charmap, space_thresh=0.01, with_pos=False):
     """Greedy CTC decode (pure, unit-tested) for the PP-OCR recognizer.
     Per-timestep argmax, collapse consecutive repeats, drop blank
     (index 0), map indices through charmap. `charmap[0]` is the blank.
-    Returns (text, mean confidence over the kept, non-blank steps)."""
+    Returns (text, mean confidence over the kept, non-blank steps).
+
+    SPACE RECOVERY: PP-OCRv5's space class routinely loses the argmax to
+    blank at real word gaps, so a pure argmax decode welds words
+    together — "SSN 123-45-6789" came back as "SSN123-45-6789" and
+    "4210 Kestrel Hollow Road" as "4210KestrelHollowRoad", which breaks
+    EVERY multi-token PII regex (address/phone/ssn silently stopped
+    matching once this backend became the CPU/Intel/Windows default).
+    The signal is there underneath: measured on real frames, the peak
+    space-class probability inside a word-gap blank run ranges 0.013 to
+    0.165 while inside a word it stays at or below 0.001 — at least a
+    10x separation everywhere it was measured (the first threshold,
+    0.05, sat INSIDE the word-gap range and welded exactly the rows
+    whose gap signal was weakest: small text — "SSN123-45-6789" again).
+    So when a blank run between two characters carries space probability
+    above space_thresh, emit the space the argmax dropped. Languages
+    that do not use spaces are unaffected (space probability stays flat).
+
+    with_pos=True additionally returns the per-character TIMESTEP of
+    each emitted char (recovered spaces get their blank run's start).
+    Timesteps map linearly onto the crop width, which gives true
+    per-word boxes downstream — the proportional character-count split
+    used before drifted off the real word positions and blur landed
+    beside the text it was covering."""
     idx = logits.argmax(1)
     prob = logits.max(1)
-    chars, confs, prev = [], [], -1
+    sp = (len(charmap) - 1) if (charmap and charmap[-1] == " ") else -1
+    chars, confs, pos, prev = [], [], [], -1
+    run = None                      # start index of the current blank run
     for t in range(len(idx)):
         i = int(idx[t])
-        if i != 0 and i != prev and i < len(charmap):
+        if i == 0:
+            if run is None:
+                run = t
+            prev = i
+            continue
+        if i != prev and i < len(charmap):
+            # any nonempty blank run can carry a space: a tight det crop
+            # compresses the word gap to a SINGLE timestep (measured
+            # p=0.02-0.065 there, still ~10x above the within-word
+            # ceiling of ~0.001) — requiring 2+ steps welded exactly
+            # those rows
+            if (sp > 0 and chars and run is not None and t - run >= 1
+                    and charmap[i] != " " and chars[-1] != " "
+                    and float(logits[run:t, sp].max()) > space_thresh):
+                chars.append(" ")
+                pos.append(run)
             chars.append(charmap[i])
             confs.append(float(prob[t]))
+            pos.append(t)
+        run = None
         prev = i
-    return "".join(chars), (float(np.mean(confs)) if confs else 0.0)
+    text = "".join(chars)
+    conf = float(np.mean(confs)) if confs else 0.0
+    if with_pos:
+        return text, conf, pos
+    return text, conf
 
 
 def _decode_db(prob, w, h, nw, nh, thresh=0.3, box_thresh=0.5, pad_frac=0.55):
@@ -4452,12 +4663,14 @@ class OnnxOcrBackend(OcrBackend):
         return _decode_db(prob, w, h, nw, nh, 0.3, 0.5, pad_frac=0.12)
 
     def _read_line(self, crop):
-        """Recognize one text-line crop (BGR) -> (text, mean confidence).
-        PP-OCR rec: resize to H=48 keeping aspect, normalize BGR to
-        [-1,1], greedy CTC decode (collapse repeats, drop blank)."""
+        """Recognize one text-line crop (BGR) -> (text, mean confidence,
+        char timesteps, total timesteps). PP-OCR rec: resize to H=48
+        keeping aspect, normalize BGR to [-1,1], greedy CTC decode
+        (collapse repeats, drop blank). The timesteps map linearly onto
+        the crop width, so callers can place each WORD exactly."""
         h, w = crop.shape[:2]
         if h < 2 or w < 2:
-            return "", 0.0
+            return "", 0.0, [], 1
         rw = max(16, min(2000, int(round(48.0 * w / h))))
         img = cv2.resize(crop, (rw, 48)).astype(np.float32)
         img = img.transpose(2, 0, 1) / 255.0
@@ -4466,8 +4679,9 @@ class OnnxOcrBackend(OcrBackend):
             y = self.rec.run(None, {self._rin:
                                     img[None].astype(np.float32)})[0][0]
         except Exception:
-            return "", 0.0
-        return _ctc_greedy_decode(y, self.charmap)
+            return "", 0.0, [], 1
+        txt, conf, pos = _ctc_greedy_decode(y, self.charmap, with_pos=True)
+        return txt, conf, pos, max(1, len(y))
 
     def read(self, frame):
         out = []
@@ -4477,20 +4691,32 @@ class OnnxOcrBackend(OcrBackend):
             x2 = min(W, int(x2)); y2 = min(H, int(y2))
             if x2 - x1 < 6 or y2 - y1 < 6:
                 continue
-            txt, conf = self._read_line(frame[y1:y2, x1:x2])
-            words = txt.split()
-            if not words:
+            txt, conf, pos, T = self._read_line(frame[y1:y2, x1:x2])
+            txt = _ascii_forms(txt)      # 1:1 mapping — positions stay aligned
+            if not txt.strip():
                 continue
-            # line box -> per-word boxes by proportional width, identical
-            # to PaddleBackend so per-word redaction stays tight.
-            total = sum(len(w) for w in words) + (len(words) - 1)
-            cursor = float(x1)
-            for w in words:
-                frac = (len(w) + 1) / max(total, 1)
-                wx2 = cursor + (float(x2) - float(x1)) * frac
-                out.append((w, (int(cursor), int(y1), int(wx2), int(y2)),
-                            conf))
-                cursor = wx2
+            # line box -> per-word boxes from the CTC TIMESTEPS: the
+            # recognizer says exactly where each character sits across
+            # the crop, so each word gets its true x-range (the old
+            # proportional character-count split drifted off the real
+            # positions and blur landed beside the text). Half a
+            # timestep cell of margin on each side covers glyph edges.
+            span = float(x2) - float(x1)
+            cell = span / T
+            i = 0
+            while i < len(txt):
+                if txt[i] == " ":
+                    i += 1
+                    continue
+                j = i
+                while j < len(txt) and txt[j] != " ":
+                    j += 1
+                w = txt[i:j]
+                wx1 = x1 + pos[i] * cell - 0.8 * cell
+                wx2 = x1 + (pos[j - 1] + 1) * cell + 0.8 * cell
+                out.append((w, (int(max(x1, wx1)), int(y1),
+                                int(min(x2, wx2)), int(y2)), conf))
+                i = j
         return out
 
 

{openscrub-1.0.76 → openscrub-1.0.78}/openscrub_web.py

@@ -2420,9 +2420,17 @@ async function removeCerts(){
 async function updCheck(){
  try{
   const d=await (await fetch("/api/update_check")).json();
-  if(d.available)document.getElementById("upd").innerHTML=
-   '· <a href="#" style="color:#fbbf24" onclick="updRun();return false">v'
-   +d.latest+' available — update</a> ';
+  if(!d.available)return;
+  const el=document.getElementById("upd");
+  if(d.can_update===false){
+   // Docker/Podman: the in-place updater would write into the ephemeral
+   // container and be lost on recreate — inform only, never offer.
+   el.innerHTML='· <span style="color:#fbbf24" title="Pull the new image tag and recreate the container — job data lives in the mounted volume and is preserved.">v'
+    +d.latest+' available — pull the new image to update</span> ';
+  }else{
+   el.innerHTML='· <a href="#" style="color:#fbbf24" onclick="updRun();return false">v'
+    +d.latest+' available — update</a> ';
+  }
  }catch(e){}
 }
 async function updRun(){
@@ -2546,11 +2554,21 @@ UPD = {"running": False, "log": [], "ok": None, "cache": (0.0, None)}
 UPD_LOCK = threading.Lock()
 
 
+def _in_container():
+    """True inside Docker/Podman. The in-place updater writes into the
+    container's ephemeral filesystem — the "update" would vanish on the
+    next recreate while the UI claimed success. Containers update by
+    pulling the new image, so the web UI must inform, never offer."""
+    return (os.environ.get("OPENSCRUB_IN_DOCKER", "") not in ("", "0")
+            or os.path.exists("/.dockerenv")
+            or os.path.exists("/run/.containerenv"))
+
+
 @app.route("/api/update_check")
 def update_check():
     if openscrub_update is None:
         return jsonify({"current": openscrub.VERSION, "latest": None,
-                        "available": False})
+                        "available": False, "can_update": False})
     now = time.time()
     with UPD_LOCK:
         ts, latest = UPD["cache"]
@@ -2564,11 +2582,16 @@ def update_check():
                                                       openscrub.VERSION))
     return jsonify({"current": openscrub.VERSION,
                     "latest": latest["version"] if latest else None,
-                    "available": avail})
+                    "available": avail,
+                    "can_update": not _in_container()})
 
 
 @app.route("/api/update_run", methods=["POST"])
 def update_run():
+    if _in_container():
+        abort(400, "running in a container — update by pulling the new "
+                   "image and recreating the container (job data lives in "
+                   "the mounted volume and is preserved)")
     if openscrub_update is None:
         abort(400, "updater module not available")
     with JOBS_LOCK:

{openscrub-1.0.76 → openscrub-1.0.78}/test_openscrub.py

@@ -2203,7 +2203,12 @@ def test_ctc_greedy_decode():
     missed repeat-collapse turns recognized text into garbage."""
     charmap = ["", "A", "B", "C", " "]        # 0=blank, 4=space
     def step(i, n=5, p=0.9):
-        v = np.full(n, (1.0 - p) / (n - 1))
+        # off-target mass stays at 0.001 — the measured within-word
+        # ceiling of the real model. Spreading (1-p) evenly put 0.025 on
+        # the space class at every step, which no real frame does, and
+        # tripped the space-recovery threshold (a fixture bug, not an
+        # engine bug).
+        v = np.full(n, 0.001)
         v[i] = p
         return v
     # A A blank B  -> "AB": the repeated A collapses, the blank drops
@@ -2280,6 +2285,234 @@ def test_docker_prefetch_matches_engine_pins():
     assert pf.MODELS == expect
 
 
+def test_ocr_space_recovery_and_ascii_forms():
+    """Two PP-OCRv5 quirks that silently broke PII detection on the
+    ONNX OCR backend — the DEFAULT on CPU/Intel/Windows since 1.0.69 —
+    found by the benchmark harness:
+
+    1. The space class routinely loses the argmax to blank at real word
+       gaps, welding "SSN 123-45-6789" into "SSN123-45-6789" and
+       "4210 Kestrel Hollow Road" into one token, so every multi-token
+       regex (address/phone/ssn) stopped matching. The space signal is
+       still present in the blank run and must be recovered.
+    2. Its Chinese-first vocabulary returns FULL-WIDTH punctuation:
+       "（555）013-8842" matches no phone pattern even though it is
+       visually identical to the ASCII form."""
+    # --- full-width normalization
+    assert openscrub._ascii_forms("（555）013-8842") == "(555)013-8842"
+    assert openscrub._ascii_forms("１２３－４５－６７８９") == "123-45-6789"
+    assert openscrub._ascii_forms("already ascii 1-2") == "already ascii 1-2"
+
+    # --- space recovery: charmap[-1] is the space class
+    charmap = [""] + list("AB") + [" "]          # blank, A, B, space
+    SP = len(charmap) - 1
+    n = len(charmap)
+
+    def step(i, space_p=0.0):
+        v = np.full(n, 0.001, np.float32)
+        v[i] = 1.0 - space_p
+        v[SP] = max(v[SP], space_p)
+        return v
+
+    # "A" <blank run carrying space probability> "B"  ->  "A B"
+    hi = np.array([step(1), step(0, 0.4), step(0, 0.4), step(2)])
+    assert openscrub._ctc_greedy_decode(hi, charmap)[0] == "A B"
+    # same shape, but the blank run has NO space probability -> "AB"
+    lo = np.array([step(1), step(0), step(0), step(2)])
+    assert openscrub._ctc_greedy_decode(lo, charmap)[0] == "AB"
+    # a ONE-step gap still carries a space when its space probability is
+    # clear — tight det crops compress real word gaps to single steps
+    # (space_p stays below blank's argmax share, the case this exists for)
+    short = np.array([step(1), step(0, 0.4), step(2)])
+    assert openscrub._ctc_greedy_decode(short, charmap)[0] == "A B"
+    # ...but a one-step gap with a flat space signal is not a word break
+    short_lo = np.array([step(1), step(0, 0.001), step(2)])
+    assert openscrub._ctc_greedy_decode(short_lo, charmap)[0] == "AB"
+    # no leading space, and repeats still collapse
+    lead = np.array([step(0, 0.4), step(0, 0.4), step(1), step(1)])
+    assert openscrub._ctc_greedy_decode(lead, charmap)[0] == "A"
+    # with_pos: timesteps come back aligned with the emitted characters
+    txt, conf, pos = openscrub._ctc_greedy_decode(hi, charmap, with_pos=True)
+    assert txt == "A B" and pos == [0, 1, 3]
+
+
+def test_merge_tail_extends_through_supporting_scans():
+    """A span ends `hold` after its last full detection, but heavily
+    compressed footage keeps showing the string while OCR reads only
+    fragments — the benchmark's compressed card was exposed for its last
+    0.7s exactly this way. Scans whose region words fuzzy-match the span
+    text extend coverage; a scan reading different text stops the walk."""
+    from rapidfuzz import fuzz
+    def mk():
+        return openscrub.Detection(t_start=0.0, t_end=0.0,
+                                   cbox=(100, 100, 300, 120),
+                                   category="card", text="4111111111111111",
+                                   confidence=0.9)
+    frag = lambda t, txt: (t, 0.0, [(txt, (110, 102, 220, 118), 0.9)])
+    scans = [frag(1.0, "4111."), frag(2.0, "1111-1111"),
+             frag(3.0, "TOTALLY DIFFERENT")]
+    m = openscrub.merge_detections([mk()], hold=0.8, scans=scans,
+                                   fuzz=fuzz)[0]
+    assert m.t_end == pytest.approx(2.8), m.t_end   # 2.0 + hold, stopped at 3.0
+    # no supporting scans -> tail stays at plain hold
+    m = openscrub.merge_detections([mk()], hold=0.8,
+                                   scans=[frag(1.0, "UNRELATED")],
+                                   fuzz=fuzz)[0]
+    assert m.t_end == pytest.approx(0.8), m.t_end
+
+
+def test_blur_kernel_is_two_thirds_region():
+    """Re-identification benchmark, close-up faces (200-750px): a
+    1/3-width Gaussian left SFace similarity at 0.57-0.69 — above the
+    0.55 same-person bar, so 11/133 faces re-identified through the
+    blur. 2/3 width measures 0.11-0.23 on the same faces, at or below
+    cross-person chance. Pin the kernel so a 'cosmetic' weakening of the
+    blur can't silently reopen the leak."""
+    seen = []
+    orig = openscrub._gauss_big
+    openscrub._gauss_big = lambda roi, k: (seen.append(k), orig(roi, k))[1]
+    try:
+        frame = np.full((400, 400, 3), 128, np.uint8)
+        openscrub.blur_region(frame, 50, 50, 350, 350, "blur")
+    finally:
+        openscrub._gauss_big = orig
+    assert seen and seen[0] >= (2 * 300 // 3) | 1, seen
+
+
+def test_group_lines_splits_columns():
+    """Two-column layouts share a y-band; a merged line made the address
+    detection's line box (and spaCy's NER context) span BOTH columns, so
+    the blur swallowed unrelated right-column text ('Priority normal' sat
+    on the address row and was over-blurred in 7 of 8 benchmark
+    scenarios). A gap wider than 2.5x line height is a column boundary."""
+    def w(txt, x, y=100):
+        return (txt, (x, y, x + 12 * len(txt), y + 120 - 100 + y), 0.9)
+    words = [("4210", (100, 100, 148, 120), 0.9),
+             ("Kestrel", (160, 100, 244, 120), 0.9),
+             ("Road", (256, 100, 304, 120), 0.9),
+             ("Priority", (820, 100, 916, 120), 0.9),
+             ("normal", (928, 100, 1000, 120), 0.9)]
+    lines = openscrub.group_lines(words)
+    texts = [ln["text"] for ln in lines]
+    assert "4210 Kestrel Road" in texts and "Priority normal" in texts, texts
+    # normal word gaps must NOT split
+    lines = openscrub.group_lines(words[:3])
+    assert [ln["text"] for ln in lines] == ["4210 Kestrel Road"]
+
+
+def test_shredded_card_fragment_join():
+    """OCR can fragment a spaced PAN into arbitrary digit chunks
+    ("41 11 11 11 1 1 11 11 11" — a real read of a highlighted card row
+    under video compression) or attach stray punctuation ("1111." from
+    motion blur). Both defeated the fixed 3/4-group join and the card
+    leaked. Any same-row run of digit tokens whose 13-16 digit
+    concatenation has a card prefix AND passes Luhn must fire."""
+    def w(txt, x):
+        return (txt, (x, 100, x + 12 * max(1, len(txt)), 120), 0.9)
+    shredded = [w(t, 60 + i * 40) for i, t in enumerate(
+        ["41", "11", "11", "11", "1", "1", "11", "11", "11"])]
+    dets = openscrub.detect_phi(shredded, [], 0.0, (0, 0), None, None)
+    assert any(d.category == "card" and d.text == "4111111111111111"
+               for d in dets), [(d.category, d.text) for d in dets]
+    dotted = [w("4111", 60), w("1111", 130), w("1111.", 200), w("1111", 270)]
+    dets = openscrub.detect_phi(dotted, [], 0.0, (0, 0), None, None)
+    assert any(d.category == "card" for d in dets)
+    # dark/noisy footage welds neighbouring groups into one big token
+    # ("4111" ".1111." "1111.1111" — the exact PP-OCRv5 read of the
+    # dark-mode benchmark card row that leaked in 1.0.78 dev): tokens up
+    # to a full welded PAN must still join
+    welded = [w("4111", 60), w(".1111.", 130), w("1111.1111", 210)]
+    dets = openscrub.detect_phi(welded, [], 0.0, (0, 0), None, None)
+    assert any(d.category == "card" and d.text == "4111111111111111"
+               for d in dets), [(d.category, d.text) for d in dets]
+    # and a single fully-welded punctuated PAN fires on its own
+    solo = [w("4111.1111.1111.1111", 60)]
+    dets = openscrub.detect_phi(solo, [], 0.0, (0, 0), None, None)
+    assert any(d.category == "card" for d in dets)
+    # welded junk that fails Luhn stays silent
+    wj = [w("1234", 60), w("5678.9012", 130), w("3456", 240)]
+    dets = openscrub.detect_phi(wj, [], 0.0, (0, 0), None, None)
+    assert not any(d.category == "card" for d in dets)
+    # a random digit run that fails Luhn must NOT fire
+    junk = [w("4111", 60), w("1111", 130), w("1111", 200), w("1112", 270)]
+    dets = openscrub.detect_phi(junk, [], 0.0, (0, 0), None, None)
+    assert not any(d.category == "card" for d in dets)
+    # SHREDDED fragments on different rows never join (the fragment
+    # join is noise-driven, so it stays same-row; the classic 4-group
+    # join may still bridge a wrapped PAN — that one is Luhn-gated)
+    split_rows = ([w(t, 60 + i * 30) for i, t in enumerate(
+                      ["41", "11", "11", "11"])]
+                  + [("1", (60 + i * 30, 300, 80 + i * 30, 320), 0.9)
+                     for i in range(8)])
+    dets = openscrub.detect_phi(split_rows, [], 0.0, (0, 0), None, None)
+    assert not any(d.category == "card" for d in dets)
+
+
+def test_name_pair_heuristic_union_and_adjacency():
+    """The capitalized-pair name heuristic is a UNION with NER, not a
+    fallback: spaCy's small model returned zero entities for a real
+    two-token name on a records screen, and NER availability silently
+    disabled the heuristic that would have caught it. In strict (union)
+    mode ALLCAPS headers stay exempt, and pairs glued together from
+    different COLUMNS by line reconstruction (big x gap) never pair —
+    that fake adjacency blurred benign text two columns away."""
+    class FakeDoc:
+        ents = ()
+
+    class FakeNLP:
+        def __call__(self, text):
+            return FakeDoc()
+
+    nd = openscrub.NameDetector(use_ner=False)
+    nd.nlp = FakeNLP()                       # NER "available", finds nothing
+    nd.heuristic = False                     # auto-mode: heuristic off pre-fix
+
+    def line(pairs):
+        words, text, x = [], "", None
+        entries = []
+        for wtxt, bx in pairs:
+            s = len(text) + (1 if text else 0)
+            text = (text + " " + wtxt) if text else wtxt
+            entries.append((wtxt, bx, 0.9, s, s + len(wtxt)))
+        return [{"text": text, "words": entries}]
+
+    # adjacent Titlecase pair -> caught even though NER found nothing
+    hits = nd.find(line([("Marguerite", (100, 50, 220, 70)),
+                         ("Vandersloot", (228, 50, 360, 70))]))
+    assert {h[1] for h in hits} == {"Marguerite", "Vandersloot"}
+    # ALLCAPS header exempt in strict mode
+    assert nd.find(line([("PATIENT", (100, 50, 200, 70)),
+                         ("RECORD", (208, 50, 300, 70))])) == []
+    # cross-column fake pair (huge x gap) never pairs
+    assert nd.find(line([("Vandersloot", (100, 50, 220, 70)),
+                         ("Ward", (600, 50, 660, 70))])) == []
+
+
+def test_container_update_notice_is_inform_only(monkeypatch):
+    """Inside Docker the in-place updater writes into the ephemeral
+    container filesystem — the "update" would vanish on the next recreate
+    while the UI claimed success (a real CUDA-image user saw the footer
+    offer an update it could not deliver). In a container the API must
+    say can_update=false (footer shows "pull the new image") and
+    update_run must refuse with the pull instruction."""
+    import openscrub_web
+    monkeypatch.setenv("OPENSCRUB_IN_DOCKER", "1")
+    assert openscrub_web._in_container() is True
+    client = openscrub_web.app.test_client()
+    d = client.get("/api/update_check").get_json()
+    assert d["can_update"] is False
+    r = client.post("/api/update_run")
+    assert r.status_code == 400
+    assert b"pulling the new image" in r.data
+    # env cleared -> the env-var gate no longer forces container mode
+    monkeypatch.delenv("OPENSCRUB_IN_DOCKER", raising=False)
+    import os as _os
+    if not (_os.path.exists("/.dockerenv")
+            or _os.path.exists("/run/.containerenv")):
+        assert openscrub_web._in_container() is False
+        assert client.get("/api/update_check").get_json()["can_update"] is True
+
+
 def test_quality_tiers_align_with_registries():
     """Every model id a Detection-quality tier selects must exist in its
     registry with a real download_url — a tier pointing at a missing or