OpenScrub 1.0.78__tar.gz → 1.0.80__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: OpenScrub
3
- Version: 1.0.78
3
+ Version: 1.0.80
4
4
  Summary: Local, GPU-accelerated redaction for video and screen recordings — blur, black-box, or mosaic faces, names, and any regex-matchable PII, with human review.
5
5
  Project-URL: Homepage, https://github.com/austinmabry/OpenScrub
6
6
  Project-URL: Repository, https://github.com/austinmabry/OpenScrub
@@ -783,7 +783,7 @@ own numbers instead of trusting ours. It measures what a reader of the
783
783
  output actually cares about: is the planted PII still *readable*, and
784
784
  can a face recognizer still *identify* the person?
785
785
 
786
- Text PII (v1.0.78, CPU, corpus v2 — 8 scenarios including dark mode,
786
+ Text PII (v1.0.79, CPU, corpus v2 — 8 scenarios including dark mode,
787
787
  sensor noise, 2.5° rotation and 120 kbps re-compression; 64 planted
788
788
  values × 5 sampled frames):
789
789
 
@@ -791,12 +791,15 @@ values × 5 sampled frames):
791
791
  Frame leak rate: 0.00% (0/320 sampled region-frames readable)
792
792
  Benign preservation: 100.0% (43/43 non-PII samples left readable)
793
793
 
794
- Face re-identification (v1.0.78, default blur, SFace matcher, valid
795
- same-person controls): **0 of 511 redacted faces re-identified** across
796
- a handheld party clip at 1080p and native 4K, a public-domain close-up
797
- interview, and a mosaic-mode run — with post-redaction similarity below the
798
- *cross-person chance floor*, i.e. a redacted face matches its own
799
- original no better than a stranger's face does.
794
+ Face re-identification (v1.0.79, default blur, SFace matcher, valid
795
+ same-person controls): **1 of 559 redacted faces re-identified (0.2 %)**
796
+ across a handheld party clip at 1080p and native 4K, a public-domain
797
+ close-up interview, and a mosaic-mode run — with mean post-redaction
798
+ similarity below the *cross-person chance floor*, i.e. a redacted face
799
+ matches its own original no better than a stranger's face does. The one
800
+ residual is published, not hidden: an extreme close-up whose face IS
801
+ covered — the match rides on uniform insignia below the blur (what the
802
+ `person` category exists for).
800
803
 
801
804
  Numbers on a fixed corpus mean the corpus no longer finds leaks — not
802
805
  that leaks are impossible. The benchmark README lists every defect the
@@ -549,7 +549,7 @@ own numbers instead of trusting ours. It measures what a reader of the
549
549
  output actually cares about: is the planted PII still *readable*, and
550
550
  can a face recognizer still *identify* the person?
551
551
 
552
- Text PII (v1.0.78, CPU, corpus v2 — 8 scenarios including dark mode,
552
+ Text PII (v1.0.79, CPU, corpus v2 — 8 scenarios including dark mode,
553
553
  sensor noise, 2.5° rotation and 120 kbps re-compression; 64 planted
554
554
  values × 5 sampled frames):
555
555
 
@@ -557,12 +557,15 @@ values × 5 sampled frames):
557
557
  Frame leak rate: 0.00% (0/320 sampled region-frames readable)
558
558
  Benign preservation: 100.0% (43/43 non-PII samples left readable)
559
559
 
560
- Face re-identification (v1.0.78, default blur, SFace matcher, valid
561
- same-person controls): **0 of 511 redacted faces re-identified** across
562
- a handheld party clip at 1080p and native 4K, a public-domain close-up
563
- interview, and a mosaic-mode run — with post-redaction similarity below the
564
- *cross-person chance floor*, i.e. a redacted face matches its own
565
- original no better than a stranger's face does.
560
+ Face re-identification (v1.0.79, default blur, SFace matcher, valid
561
+ same-person controls): **1 of 559 redacted faces re-identified (0.2 %)**
562
+ across a handheld party clip at 1080p and native 4K, a public-domain
563
+ close-up interview, and a mosaic-mode run — with mean post-redaction
564
+ similarity below the *cross-person chance floor*, i.e. a redacted face
565
+ matches its own original no better than a stranger's face does. The one
566
+ residual is published, not hidden: an extreme close-up whose face IS
567
+ covered — the match rides on uniform insignia below the blur (what the
568
+ `person` category exists for).
566
569
 
567
570
  Numbers on a fixed corpus mean the corpus no longer finds leaks — not
568
571
  that leaks are impossible. The benchmark README lists every defect the
@@ -71,7 +71,7 @@ people's faces and publish the frames.
71
71
 
72
72
  ## Measured: text PII
73
73
 
74
- OpenScrub v1.0.78, CPU only (no GPU), corpus v2 (8 scenarios), 5 frames
74
+ OpenScrub v1.0.79, CPU only (no GPU), corpus v2 (8 scenarios), 5 frames
75
75
  sampled per planted value. Higher is better for recall and benign
76
76
  preservation; lower is better for leak rate:
77
77
 
@@ -83,14 +83,22 @@ preservation; lower is better for leak rate:
83
83
  Every planted value — names, SSNs, phone numbers, addresses, cards,
84
84
  emails, dates of birth, IP addresses — was unreadable in every sampled
85
85
  frame of the rendered output, across all eight scenarios, on both
86
- engines. The one benign loss (Tesseract) is "Follow-up in 6 weeks"
87
- garbled under the 120 kbps compression — over-blur, the failure
88
- direction this tool chooses on purpose.
86
+ engines. The one benign loss (Tesseract) is a genuine over-blur, and
87
+ worth naming precisely: on the 120 kbps copy, compression noise made
88
+ Tesseract read the lowercase "in" of "Follow-up in 6 weeks" as "In", the
89
+ capitalised-pair name heuristic saw "Follow-up In", and the phrase was
90
+ blurred as a name. Wrong, but wrong in the safe direction — the
91
+ failure this tool chooses on purpose.
92
+
93
+ **No zones, no detection windows, no tuning flags.** The scorer runs
94
+ the engine on its defaults (`--engine <x> -o … --report …`) and nothing
95
+ else. Scoping features would raise the numbers by telling the engine
96
+ where the answers are — see "Why the benchmark runs unassisted" below.
89
97
 
90
98
  A 100 % row on a fixed corpus means the corpus no longer finds leaks —
91
99
  not that leaks are impossible. The honest reading is the history below.
92
100
 
93
- ### What building the benchmark caught (all fixed in 1.0.78)
101
+ ### What building the benchmark caught (all fixed in 1.0.78-1.0.79)
94
102
 
95
103
  The first runs of this harness measured the ONNX backend at **40.6 %
96
104
  recall with a 57.5 % frame leak rate**. Every point of the gap traced to
@@ -119,7 +127,7 @@ regression test:
119
127
 
120
128
  ## Measured: face re-identification
121
129
 
122
- OpenScrub v1.0.78, default blur mode, SFace matcher, same-person
130
+ OpenScrub v1.0.79, default blur mode, SFace matcher, same-person
123
131
  threshold 0.55. "Re-identified" = the redacted face still matched the
124
132
  identity embedding taken from the unredacted original. All three rows
125
133
  have a valid control (the recognizer reliably re-identifies the same
@@ -127,33 +135,88 @@ faces in the *unredacted* footage, so a 0 % result is meaningful):
127
135
 
128
136
  | Footage | Faces attacked | Re-identified | Still detected as a face | Control |
129
137
  |---|---|---|---|---|
130
- | Handheld 1080p party clip, five subjects, constant motion ([Pexels 7100826](https://www.pexels.com/video/7100826/), downscaled from 4K) | 125 | **0 (0.0 %)** | 0.0 % | 0.888 |
131
- | The same party clip at native 4K (3840×2160 — larger faces, more recoverable detail) | 128 | **0 (0.0 %)** | 0.0 % | 0.887 |
132
- | Public-domain interview, large frontal close-up faces, 1080×1920 ([Wikimedia Commons](https://commons.wikimedia.org/wiki/File:Interview_with_a_Marshall_Center_professor_(999984).webm)) | 133 | **0 (0.0 %)** | 0.7 % | 0.691 |
133
- | Same party clip, `--mode mosaic` | 125 | **0 (0.0 %)** | 4.8 % | 0.888 |
138
+ | Handheld 1080p party clip, five subjects, constant motion ([Pexels 7100826](https://www.pexels.com/video/7100826/), downscaled from 4K) | 125 | **0 (0.0 %)** | 0.8 % | 0.880 |
139
+ | The same party clip at native 4K (3840×2160 — larger faces, more recoverable detail) | 129 | **0 (0.0 %)** | 2.3 % | 0.895 |
140
+ | Public-domain interview, large frontal close-up faces, 1080×1920 ([Wikimedia Commons](https://commons.wikimedia.org/wiki/File:Interview_with_a_Marshall_Center_professor_(999984).webm)) | 203 | **1 of 180 graded (0.6 %)** ¹ | 0.5 % | 0.695 |
141
+ | Same party clip, `--mode mosaic` | 125 | **0 (0.0 %)** | 4.8 % | 0.880 |
142
+
143
+ ¹ 4 of the 203 attacked crops were excluded as *degenerate* (their
144
+ embeddings match a featureless flat patch at ≥ 0.5 — they carry no
145
+ identity to protect or leak; excluded openly, like unconfirmed
146
+ candidates). The one residual match is an extreme close-up crop where
147
+ the face itself IS covered — the 0.63 similarity rides on the
148
+ subject's uniform insignia and neck below the blur. Face redaction
149
+ covered the face; identifying clothing is what the `person` category
150
+ and `--coverage concealed` exist for. We publish it rather than
151
+ excluding it because the recognizer scored it, and the rule here is
152
+ that the numbers include what the attack found.
134
153
 
135
154
  Mean post-redaction similarity sat **below the cross-person chance
136
- floor** in all three runs — a redacted face matches its own original no
155
+ floor** in every run — a redacted face matches its own original no
137
156
  better than a random stranger's face does.
138
157
 
139
- This table also records a caught defect: the interview clip's close-up
158
+ This table also records caught defects. The interview clip's close-up
140
159
  faces (200–750 px) were initially re-identified at **8.3 % straight
141
160
  through the old blur** — a large face keeps enough low-frequency
142
- identity when the blur kernel is 1/3 of the region. The kernel is now
143
- 2/3 of the region (pinned by test), which also drove
144
- "still detected as a face" from 99.2 % to 0 % on the party clip.
161
+ identity when the blur kernel is 1/3 of the region; the kernel is now
162
+ 2/3 of the region's long side (pinned by test). Frame-border
163
+ half-faces went **entirely undetected** by every detection backend
164
+ until detection ran on a mirror-padded frame, and once detected, the
165
+ narrow slivers re-identified through a width-scaled kernel — both
166
+ fixed, both found by this harness on real footage. The harness itself
167
+ was fixed twice along the way: VFR/HDR inputs are now attacked on the
168
+ pipeline's normalized timeline (misaligned frames once produced
169
+ confident nonsense — the "leaked faces" were a curtain), and
170
+ non-discriminative crops are excluded by the flat-patch floor above.
171
+ All numbers in this table were re-measured on the 1.0.79 build
172
+ after those fixes.
145
173
 
146
174
  Two candidate clips (a street scene with small oblique faces, a
147
175
  multi-person interview) produced **invalid controls** — the recognizer
148
176
  could not re-identify unredacted faces in them — and the harness
149
177
  refused to grade them. That refusal is the feature.
150
178
 
179
+ ## Why the benchmark runs unassisted
180
+
181
+ OpenScrub ships scoping tools that make redaction better in practice:
182
+ **detection windows** (scan only these time ranges) and **zones**
183
+ (look for this category only inside this rectangle), plus never-blur
184
+ **ignore zones**. A benchmark run *with* them would score at least as
185
+ well as the numbers above and would remove false blurs, because a zone
186
+ that excludes a column excludes every false positive in it.
187
+
188
+ They are deliberately not used here, for one reason: on a synthetic
189
+ corpus the person drawing the zones has the answer key. Zoning the
190
+ region where you planted the SSN and then reporting that the SSN was
191
+ covered measures the renderer, not the detector — and the first
192
+ question any skeptical reader asks is "so you told it where to look?".
193
+ The number that means something is the one from the path a new user
194
+ gets by default, on footage the tool knows nothing about.
195
+
196
+ So the rule for this harness: **the engine is never told anything the
197
+ corpus generator knew.** If a future scenario defeats the unassisted
198
+ path, the fix belongs in the detector, not in a hand-drawn rectangle.
199
+
200
+ That is a statement about the *benchmark*, not about the feature.
201
+ Windows and zones are the right tool on real footage — where you, not
202
+ a generator, know the phone number only appears in the lower third, and
203
+ where excluding the rest of the frame genuinely removes false blurs and
204
+ scan time. If a zone-assisted number is ever published, it belongs in
205
+ its own clearly-labelled row ("guided mode"), never merged into the
206
+ unassisted one.
207
+
151
208
  ## Rules for quoting these numbers
152
209
 
153
210
  - Always state **version, engine, corpus version, and hardware**. A
154
211
  number without them is not a claim, it is a vibe.
155
212
  - Never say "HIPAA compliant" or "GDPR compliant" — tools are not
156
213
  compliant, processes are.
157
- - Never say 100 %, "guaranteed", or "fully anonymised".
214
+ - Never say "guaranteed", "fully anonymised", or "undetectable" — those
215
+ describe the tool, and no redaction tool can promise them.
216
+ - A measured 100 % is a **result**, not a promise: quote it only with
217
+ its scope attached ("100 % of the planted values on corpus v2 were
218
+ unreadable in the output"), never bare and never in the future tense
219
+ ("OpenScrub blurs 100 % of PII"). The first is what happened; the
220
+ second is a claim about footage nobody has run yet.
158
221
  - Publish the failures next to the successes. A tool that says where it
159
222
  struggles is the one worth believing.
@@ -36,7 +36,7 @@ from dataclasses import dataclass, asdict
36
36
  import cv2
37
37
  import numpy as np
38
38
 
39
- VERSION = "1.0.78"
39
+ VERSION = "1.0.80"
40
40
 
41
41
  # ----------------------------------------------------------------------------
42
42
  # OCR backends
@@ -1018,13 +1018,22 @@ def probe_camera_motion(path, sample_windows=14, pairs_per_window=4):
1018
1018
  win = None
1019
1019
  positions = ([int(total * i / sample_windows) for i in range(sample_windows)]
1020
1020
  if total > sample_windows * (pairs_per_window + 1) else [0])
1021
+ fps_p = cap.get(cv2.CAP_PROP_FPS) or 30.0
1021
1022
  for pos in positions:
1022
- cap.set(cv2.CAP_PROP_POS_FRAMES, pos)
1023
+ # verified positioning; positions ascend, so _grab_frame's
1024
+ # forward-roll memory turns this into one sequential pass on
1025
+ # files where seeking is slow (single-keyframe exports decode
1026
+ # from frame zero on EVERY raw seek)
1027
+ first = _grab_frame(cap, pos / fps_p, fps_p)
1028
+ if first is None:
1029
+ continue
1023
1030
  prev = None
1024
- for _ in range(pairs_per_window + 1):
1025
- ok, frame = cap.read()
1026
- if not ok:
1027
- break
1031
+ frame = first
1032
+ for i in range(pairs_per_window + 1):
1033
+ if i > 0:
1034
+ ok, frame = cap.read()
1035
+ if not ok:
1036
+ break
1028
1037
  g = cv2.cvtColor(cv2.resize(frame, (320, max(2, int(
1029
1038
  frame.shape[0] * 320 / frame.shape[1])))),
1030
1039
  cv2.COLOR_BGR2GRAY).astype(np.float32)
@@ -1264,18 +1273,60 @@ def smooth_dense_tracks(dets, fps, video, cum=None, win_start=0.0, cb=None):
1264
1273
 
1265
1274
  _fps_g = (cap.get(cv2.CAP_PROP_FPS) or 30.0) if cap is not None else 30.0
1266
1275
 
1276
+ # The onset walks probe BACKWARD from each track's first sample, one
1277
+ # POS_MSEC seek per probe. On a normal file each seek decodes from a
1278
+ # nearby keyframe; a real 4K export with ONE keyframe (common from
1279
+ # screen recorders and render pipelines) decoded the file from frame
1280
+ # ZERO on every probe — ~37 probes x 75 tracks made a 10-second clip
1281
+ # take hours. Every track's onset is known BEFORE the walks start, so
1282
+ # decode the video ONCE, sequentially (the renderer's access pattern,
1283
+ # correct and fast on any file), caching half-scale grays only for
1284
+ # frames inside some track's walk span and evicting behind the
1285
+ # onset-sorted processing order. Memory stays bounded by one span
1286
+ # (~LEAD_MAX seconds of half-scale grays), not the file.
1287
+ gray_cache = {}
1288
+ _seq = {"pos": 0}
1289
+ _span_lo = {} # frame index -> lowest span start needing it
1290
+ if cap is not None and tracks:
1291
+ for samples in tracks.values():
1292
+ t0 = min(d.t_start for d in samples)
1293
+ lo_t = max(0.0, t0 - LEAD_MAX - 0.2, win_start - 0.01)
1294
+ lo = max(0, int(lo_t * _fps_g) - 2)
1295
+ hi = int(round(t0 * _fps_g)) + 2
1296
+ for k in range(lo, hi + 1):
1297
+ _span_lo[k] = min(_span_lo.get(k, lo), lo)
1298
+
1299
+ def _fill_to(k_end):
1300
+ # sequential read, caching grays for needed frames; never seeks
1301
+ while _seq["pos"] <= k_end:
1302
+ ok, fr = cap.read()
1303
+ if not ok:
1304
+ _seq["pos"] = 10 ** 9 # EOF: stop trying
1305
+ return
1306
+ idx = _seq["pos"]
1307
+ _seq["pos"] += 1
1308
+ if idx in _span_lo:
1309
+ g = cv2.resize(cv2.cvtColor(fr, cv2.COLOR_BGR2GRAY), None,
1310
+ fx=SCALE, fy=SCALE)
1311
+ gray_cache[idx] = cv2.GaussianBlur(g, (3, 3), 0)
1312
+
1267
1313
  def _gray(t):
1268
1314
  if cap is None:
1269
1315
  return None
1270
- fr = _grab_frame(cap, t, _fps_g) # verified seek (see there)
1271
- if fr is None:
1272
- return None
1273
- g = cv2.resize(cv2.cvtColor(fr, cv2.COLOR_BGR2GRAY), None,
1274
- fx=SCALE, fy=SCALE)
1275
- # light smoothing before matching: sub-pixel motion at this scale
1276
- # decorrelates fine texture (measured 1.0 -> 0.48 on a half-pixel
1277
- # offset), while a smoothed match stays >0.8 present and ~0 absent
1278
- return cv2.GaussianBlur(g, (3, 3), 0)
1316
+ k = int(round(t * _fps_g))
1317
+ if k not in gray_cache and k >= _seq["pos"] - 1:
1318
+ _fill_to(k)
1319
+ # light smoothing happened at cache time: sub-pixel motion at this
1320
+ # scale decorrelates fine texture (measured 1.0 -> 0.48 on a
1321
+ # half-pixel offset); a smoothed match stays >0.8 present, ~0 absent
1322
+ for kk in (k, k - 1, k + 1): # PTS rounding tolerance
1323
+ if kk in gray_cache:
1324
+ return gray_cache[kk]
1325
+ return None
1326
+
1327
+ def _evict_below(k):
1328
+ for kk in [kk for kk in gray_cache if kk < k]:
1329
+ del gray_cache[kk]
1279
1330
 
1280
1331
  # This is the longest silent stretch of a dense scan (the onset walk-back
1281
1332
  # re-reads the video per track), so report progress: the bar moves via
@@ -1285,7 +1336,14 @@ def smooth_dense_tracks(dets, fps, video, cum=None, win_start=0.0, cb=None):
1285
1336
  _last_log = time.time()
1286
1337
  n_gaps, n_lead, lead_s = 0, 0, 0.0
1287
1338
  added = []
1288
- for n_done, (tid, samples) in enumerate(tracks.items()):
1339
+ # onset order makes the sequential cache a single forward pass: every
1340
+ # later track's span starts at or after this one's, so frames behind
1341
+ # the current span are never needed again
1342
+ _order = sorted(tracks.items(),
1343
+ key=lambda kv: min(d.t_start for d in kv[1]))
1344
+ for n_done, (tid, samples) in enumerate(_order):
1345
+ _evict_below(max(0, int((min(d.t_start for d in samples)
1346
+ - LEAD_MAX - 0.3) * _fps_g)))
1289
1347
  cb.progress("post", n_done, n_total)
1290
1348
  if time.time() - _last_log >= 3.0:
1291
1349
  _last_log = time.time()
@@ -1409,6 +1467,53 @@ def _vittrack_factory(log):
1409
1467
  return make
1410
1468
 
1411
1469
 
1470
+ _grab_state = {} # id(cap) -> {"pos": next-frame idx or None, "slow": bool}
1471
+
1472
+
1473
+ class _RevReader:
1474
+ """Chunked frame provider for BACKWARD iteration. Stepping backward
1475
+ with one seek per frame decodes keyframe-to-target EVERY step — and a
1476
+ real single-keyframe 4K export decodes from frame ZERO every step, so
1477
+ a 10-second backward track cost hundreds of full-file decodes. This
1478
+ reads the file in forward CHUNKS (one verified position + sequential
1479
+ reads), serves frames from the chunk cache, and reloads the previous
1480
+ chunk when iteration walks below it. Memory is bounded by `budget`;
1481
+ chunk loads total at most a handful of file passes on the worst file
1482
+ and exactly one seek per chunk on a healthy one."""
1483
+
1484
+ def __init__(self, cap, fps, budget_bytes=300 * 1024 * 1024):
1485
+ self.cap, self.fps = cap, fps or 30.0
1486
+ self.cache = {}
1487
+ self.budget = budget_bytes
1488
+ self.n = None
1489
+
1490
+ def get(self, t):
1491
+ k = int(round(max(t, 0.0) * self.fps))
1492
+ fr = self.cache.get(k)
1493
+ if fr is not None:
1494
+ return fr
1495
+ lo = max(0, k - (self.n - 1 if self.n else 0))
1496
+ f0 = _grab_frame(self.cap, lo / self.fps, self.fps)
1497
+ if f0 is None:
1498
+ return None
1499
+ if self.n is None:
1500
+ self.n = max(8, int(self.budget // max(1, f0.nbytes)))
1501
+ lo = max(0, k - self.n + 1)
1502
+ if lo != k: # re-position with the real chunk
1503
+ f0 = _grab_frame(self.cap, lo / self.fps, self.fps)
1504
+ if f0 is None:
1505
+ return None
1506
+ self.cache = {lo: f0}
1507
+ idx = lo
1508
+ while idx < k:
1509
+ ok, f = self.cap.read()
1510
+ if not ok:
1511
+ break
1512
+ idx += 1
1513
+ self.cache[idx] = f
1514
+ return self.cache.get(k)
1515
+
1516
+
1412
1517
  def _grab_frame(cap, t, fps):
1413
1518
  """Fetch the frame at time t from an open VideoCapture, ROBUSTLY.
1414
1519
 
@@ -1427,6 +1532,11 @@ def _grab_frame(cap, t, fps):
1427
1532
  file. Returns the frame or None."""
1428
1533
  fps = fps or 30.0
1429
1534
  target = int(round(max(t, 0.0) * fps))
1535
+ st = _grab_state.setdefault(id(cap), {"pos": None, "slow": False})
1536
+
1537
+ def _done(fr):
1538
+ st["pos"] = target + 1 # cap will read target+1 next
1539
+ return fr
1430
1540
 
1431
1541
  def _pts_frame():
1432
1542
  # index of the frame just decoded, from the reported packet PTS;
@@ -1451,16 +1561,39 @@ def _grab_frame(cap, t, fps):
1451
1561
  fr, got = f, got + 1
1452
1562
  return fr
1453
1563
 
1564
+ # FORWARD ROLL, no seek: when the capture's position is known (from a
1565
+ # prior verified grab) and the target is at/ahead of it, reading
1566
+ # forward is always correct and never worse than seeking. On a real
1567
+ # single-keyframe 4K export every seek decoded from frame ZERO — the
1568
+ # demuxer has no closer keyframe to land on — so ascending access
1569
+ # patterns (walk-backs' spans, embedding grabs, probes) went
1570
+ # quadratic. Once a seek MEASURES slow (>2s) the file is marked and
1571
+ # any forward-reachable target rolls instead.
1572
+ if st["pos"] is not None and 0 <= target - st["pos"] + 1 <= (
1573
+ 10 ** 9 if st["slow"] else 300):
1574
+ got = st["pos"] - 1
1575
+ fr = _roll_to(None, got)
1576
+ # the stored position can be STALE (callers may raw-read the same
1577
+ # cap between grabs; id() can be recycled after release) — accept
1578
+ # the roll only when the landed frame's own PTS confirms it, and
1579
+ # fall through to the seek path otherwise
1580
+ if fr is not None and abs(_pts_frame() - target) <= 1:
1581
+ return _done(fr)
1582
+ st["pos"] = None # stale/EOF: state unknown
1583
+
1584
+ _t0 = time.time()
1454
1585
  cap.set(cv2.CAP_PROP_POS_MSEC, max(t, 0.0) * 1000)
1455
1586
  ok, fr = cap.read()
1587
+ if time.time() - _t0 > 2.0:
1588
+ st["slow"] = True
1456
1589
  if ok and fr is not None:
1457
1590
  got = _pts_frame()
1458
1591
  if got < 0 or abs(got - target) <= 1:
1459
- return fr # verified (or unverifiable) hit
1592
+ return _done(fr) # verified (or unverifiable) hit
1460
1593
  if got < target:
1461
1594
  fr = _roll_to(fr, got) # landed early: walk the gap
1462
1595
  if fr is not None:
1463
- return fr
1596
+ return _done(fr)
1464
1597
  # landed past the target (or the walk hit EOF): try from earlier
1465
1598
  for back in (int(fps), int(4 * fps), int(12 * fps), target):
1466
1599
  start = max(0, target - back)
@@ -1476,7 +1609,8 @@ def _grab_frame(cap, t, fps):
1476
1609
  continue # snapped past: seek earlier
1477
1610
  fr = _roll_to(fr, got)
1478
1611
  if fr is not None:
1479
- return fr
1612
+ return _done(fr)
1613
+ st["pos"] = None
1480
1614
  return None
1481
1615
 
1482
1616
 
@@ -1743,6 +1877,12 @@ def _track_person_dense(video, box, t_ref, t0, t1, det, log,
1743
1877
  # robust prime: positions the decoder just past t_ref's
1744
1878
  # frame even when random POS_MSEC seek fails on this file
1745
1879
  _grab_frame(cap, t_ref, fps)
1880
+ rev = None
1881
+ else:
1882
+ # backward: chunked forward reads served in reverse — a
1883
+ # per-step backward seek decodes keyframe-to-target every
1884
+ # step (from frame ZERO on a single-keyframe export)
1885
+ rev = _RevReader(cap, fps)
1746
1886
  while True:
1747
1887
  t2 = t + direction * step
1748
1888
  if t2 < t0 - 1e-6 or t2 > t1 + 1e-6:
@@ -1756,7 +1896,7 @@ def _track_person_dense(video, box, t_ref, t0, t1, det, log,
1756
1896
  break
1757
1897
  fr = f0
1758
1898
  else:
1759
- fr = _frame(t2)
1899
+ fr = rev.get(t2)
1760
1900
  if fr is None:
1761
1901
  break
1762
1902
  # 1. PROPAGATE the pixel anchor (MaskAnyone principle:
@@ -2236,6 +2376,12 @@ def track_manual_region(video, box, t_ref, t0, t1, cb=None,
2236
2376
  # robust prime: positions the decoder just past t_ref's
2237
2377
  # frame even when random POS_MSEC seek fails on this file
2238
2378
  _grab_frame(cap, t_ref, fps)
2379
+ rev = None
2380
+ else:
2381
+ # backward: chunked forward reads served in reverse — a
2382
+ # per-step backward seek decodes keyframe-to-target every
2383
+ # step (from frame ZERO on a single-keyframe export)
2384
+ rev = _RevReader(cap, fps)
2239
2385
  while True:
2240
2386
  t2 = t + direction * step
2241
2387
  if t2 < t0 - 1e-6 or t2 > t1 + 1e-6:
@@ -2249,7 +2395,7 @@ def track_manual_region(video, box, t_ref, t0, t1, cb=None,
2249
2395
  break
2250
2396
  fr = fr0_
2251
2397
  else:
2252
- fr = _frame(t2)
2398
+ fr = rev.get(t2)
2253
2399
  if fr is None:
2254
2400
  break
2255
2401
  ok, bb = trk.update(cv2.resize(fr, None,
@@ -2440,45 +2586,102 @@ def group_persons(dets, video, cb=None):
2440
2586
  rec = _make_sface(sface)
2441
2587
  det = _make_yunet(yunet, (320, 320), 0.5)
2442
2588
  cap = cv2.VideoCapture(video)
2589
+ fpsv = cap.get(cv2.CAP_PROP_FPS) or 30.0
2443
2590
  cb.log(" person grouping: matching %d face track(s) by identity…"
2444
2591
  % len(tracks))
2445
2592
  _last_log = time.time()
2446
- embs = {}
2447
- for n_done, (tid, samples) in enumerate(tracks.items()):
2448
- if time.time() - _last_log >= 3.0:
2449
- _last_log = time.time()
2450
- cb.log(" …track %d/%d embedded"
2451
- % (n_done + 1, len(tracks)))
2593
+ # Collect every needed frame up front and fetch in ASCENDING order
2594
+ # with forward-only decoding. Per-sample seeks decoded a real
2595
+ # single-keyframe 4K export from frame ZERO on every embedding grab
2596
+ # (the second quadratic site that file exposed, after the onset
2597
+ # walk-back): 3 grabs x 75 tracks of full-file decodes. Sorted
2598
+ # forward reads cost at most one pass of the file; on files where
2599
+ # seeking IS cheap the first large jump still seeks, and only a
2600
+ # measured-slow seek (>2 s) switches to roll-forward.
2601
+ reqs = []
2602
+ for tid, samples in tracks.items():
2452
2603
  best = sorted(samples, key=lambda d: -d.confidence)[:3]
2453
- feats = []
2454
2604
  for d in best:
2455
2605
  t = min(max(d.last_seen, d.t_start), d.t_end)
2456
- fr = _grab_frame(cap, t, cap.get(cv2.CAP_PROP_FPS) or 30.0)
2457
- if fr is None:
2458
- continue
2459
- det.setInputSize((fr.shape[1], fr.shape[0]))
2460
- _, rows = det.detect(fr)
2461
- if rows is None:
2462
- continue
2463
- sb = (d.cbox[0] + d.aoff[0], d.cbox[1] + d.aoff[1],
2464
- d.cbox[2] + d.aoff[0], d.cbox[3] + d.aoff[1])
2465
- rbest, riou = None, 0.2
2466
- for r in rows:
2467
- rb = (r[0], r[1], r[0] + r[2], r[1] + r[3])
2468
- iou = _box_iou(rb, sb)
2469
- if iou > riou:
2470
- rbest, riou = r, iou
2471
- if rbest is None:
2472
- continue
2473
- if float(rbest[2]) < 32 or float(rbest[3]) < 32:
2474
- continue # too small to identify: noise embedding
2475
- f = rec.feature(rec.alignCrop(fr, rbest)).flatten()
2476
- n = float(np.linalg.norm(f))
2477
- if n > 0:
2478
- feats.append(f / n)
2479
- if feats:
2480
- e = np.mean(feats, axis=0)
2481
- embs[tid] = e / np.linalg.norm(e)
2606
+ reqs.append((int(round(t * fpsv)), tid, d))
2607
+ reqs.sort(key=lambda r: r[0])
2608
+ _pos = {"i": 0, "slow": False}
2609
+ _memo = {"idx": -1, "fr": None}
2610
+
2611
+ def _fetch(idx):
2612
+ if idx == _memo["idx"]:
2613
+ return _memo["fr"]
2614
+ gap = idx - _pos["i"]
2615
+ if gap < 0:
2616
+ return None # behind (rounding dupe): skip
2617
+ if gap > 300 and not _pos["slow"]:
2618
+ t0 = time.time()
2619
+ fr = _grab_frame(cap, idx / fpsv, fpsv)
2620
+ if time.time() - t0 > 2.0:
2621
+ _pos["slow"] = True # sparse keyframes: stop seeking
2622
+ if fr is not None:
2623
+ _pos["i"] = idx + 1 # _grab_frame just read frame idx
2624
+ _memo.update(idx=idx, fr=fr)
2625
+ return fr
2626
+ fr = None
2627
+ while _pos["i"] <= idx:
2628
+ ok, fr = cap.read()
2629
+ if not ok:
2630
+ return None
2631
+ _pos["i"] += 1
2632
+ _memo.update(idx=idx, fr=fr)
2633
+ return fr
2634
+
2635
+ feats_by = {}
2636
+ for n_done, (idx, tid, d) in enumerate(reqs):
2637
+ if time.time() - _last_log >= 3.0:
2638
+ _last_log = time.time()
2639
+ cb.log(" …%d/%d face samples embedded"
2640
+ % (n_done + 1, len(reqs)))
2641
+ fr = _fetch(idx)
2642
+ if fr is None:
2643
+ continue
2644
+ sb = (d.cbox[0] + d.aoff[0], d.cbox[1] + d.aoff[1],
2645
+ d.cbox[2] + d.aoff[0], d.cbox[3] + d.aoff[1])
2646
+ # re-detect in a CROP around the known box, not the full frame:
2647
+ # only detections overlapping sb are ever used (IoU gate below),
2648
+ # and full-frame YuNet at 4K cost seconds per sample — 225
2649
+ # samples of it was the third hotspot the single-keyframe export
2650
+ # exposed. The crop is the box grown ~1.5x on each side, so a
2651
+ # slightly-shifted re-detection still lands inside.
2652
+ fh, fw = fr.shape[:2]
2653
+ mw = max(48, int(1.5 * (sb[2] - sb[0])))
2654
+ mh = max(48, int(1.5 * (sb[3] - sb[1])))
2655
+ ox1 = max(0, int(sb[0]) - mw)
2656
+ oy1 = max(0, int(sb[1]) - mh)
2657
+ ox2 = min(fw, int(sb[2]) + mw)
2658
+ oy2 = min(fh, int(sb[3]) + mh)
2659
+ crop = fr[oy1:oy2, ox1:ox2]
2660
+ if crop.size == 0:
2661
+ continue
2662
+ det.setInputSize((crop.shape[1], crop.shape[0]))
2663
+ _, rows = det.detect(crop)
2664
+ if rows is None:
2665
+ continue
2666
+ sb_rel = (sb[0] - ox1, sb[1] - oy1, sb[2] - ox1, sb[3] - oy1)
2667
+ rbest, riou = None, 0.2
2668
+ for r in rows:
2669
+ rb = (r[0], r[1], r[0] + r[2], r[1] + r[3])
2670
+ iou = _box_iou(rb, sb_rel)
2671
+ if iou > riou:
2672
+ rbest, riou = r, iou
2673
+ if rbest is None:
2674
+ continue
2675
+ if float(rbest[2]) < 32 or float(rbest[3]) < 32:
2676
+ continue # too small to identify: noise embedding
2677
+ f = rec.feature(rec.alignCrop(crop, rbest)).flatten()
2678
+ n = float(np.linalg.norm(f))
2679
+ if n > 0:
2680
+ feats_by.setdefault(tid, []).append(f / n)
2681
+ embs = {}
2682
+ for tid, feats in feats_by.items():
2683
+ e = np.mean(feats, axis=0)
2684
+ embs[tid] = e / np.linalg.norm(e)
2482
2685
  cap.release()
2483
2686
  # centroid-linkage: big stable tracks seed clusters; each remaining
2484
2687
  # track joins only if it matches the cluster's AVERAGE identity. This
@@ -2736,14 +2939,16 @@ def blur_region(frame, x1, y1, x2, y2, mode, shape="rect"):
2736
2939
  filled = cv2.resize(small, (x2 - x1, y2 - y1),
2737
2940
  interpolation=cv2.INTER_NEAREST)
2738
2941
  else:
2739
- # kernel = 2/3 region width (was 1/3): the re-identification
2740
- # benchmark showed close-up faces (200-750px) surviving a 1/3-width
2741
- # blur at SFace similarity 0.57-0.69 — above the 0.55 same-person
2742
- # bar. At 2/3 width the same faces measure 0.11-0.23, at or below
2743
- # the cross-person chance floor: a blurred face carries no more
2744
- # identity than a random stranger's. Small faces were already safe
2745
- # (the 112px embed crop upsamples them); this closes the close-up.
2746
- k = max(31, ((2 * (x2 - x1) // 3) | 1))
2942
+ # kernel = 2/3 of the region LONG SIDE (was 1/3 width, then 2/3
2943
+ # width): the re-identification benchmark caught both steps. Close-up
2944
+ # faces (200-750px) survived a 1/3-width blur at SFace similarity
2945
+ # 0.57-0.69; 2/3 width dropped them to 0.11-0.23, at or below the
2946
+ # cross-person chance floor. Then frame-border face SLIVERS (a
2947
+ # 32x101 half-face at x=0) re-identified at 0.55-0.71 through the
2948
+ # width-scaled kernel — a narrow-but-tall region got the k=31 floor
2949
+ # against 100+px of content. Scaling by the long side drops the
2950
+ # same slivers to 0.15-0.57. Square faces are unaffected.
2951
+ k = max(31, ((2 * max(x2 - x1, y2 - y1) // 3) | 1))
2747
2952
  filled = _gauss_big(roi, k)
2748
2953
  if shape == "ellipse":
2749
2954
  # elliptical mask hugs a face: no smeared background corners, which
@@ -2822,7 +3027,7 @@ def blur_silhouette(frame, x1, y1, x2, y2, mode, polys, poly_box,
2822
3027
  interpolation=cv2.INTER_LINEAR)
2823
3028
  filled = cv2.resize(small, (rw, rh), interpolation=cv2.INTER_NEAREST)
2824
3029
  else:
2825
- k = max(31, ((2 * rw // 3) | 1))
3030
+ k = max(31, ((2 * max(rw, rh) // 3) | 1))
2826
3031
  filled = _gauss_big(roi, k)
2827
3032
  cv2.copyTo(filled, mask, roi)
2828
3033
 
@@ -3130,7 +3335,7 @@ def _blur_yuv10(y, u, v, x1, y1, x2, y2, mode, shape="rect"):
3130
3335
  filled = cv2.resize(small, (px2 - px1, py2 - py1),
3131
3336
  interpolation=cv2.INTER_NEAREST)
3132
3337
  else:
3133
- k = max(kmin, ((2 * (px2 - px1) // 3) | 1))
3338
+ k = max(kmin, ((2 * max(px2 - px1, py2 - py1) // 3) | 1))
3134
3339
  filled = _gauss_big(roi, k)
3135
3340
  if shape == "ellipse":
3136
3341
  rw, rh = px2 - px1, py2 - py1
@@ -3210,7 +3415,7 @@ def _blur_silhouette_yuv10(y, u, v, x1, y1, x2, y2, mode, polys,
3210
3415
  filled = cv2.resize(small, (px2 - px1, py2 - py1),
3211
3416
  interpolation=cv2.INTER_NEAREST)
3212
3417
  else:
3213
- k = max(kmin, ((2 * (px2 - px1) // 3) | 1))
3418
+ k = max(kmin, ((2 * max(px2 - px1, py2 - py1) // 3) | 1))
3214
3419
  filled = _gauss_big(roi, k)
3215
3420
  m = mask
3216
3421
  if m.shape != roi.shape: # chroma planes are half-res
@@ -5081,30 +5286,52 @@ class FaceDetector:
5081
5286
  dframe = (cv2.resize(frame, (max(1, int(w * s)), max(1, int(h * s))))
5082
5287
  if s < 1.0 else frame)
5083
5288
  dh, dw = dframe.shape[:2]
5289
+ # EDGE ASSIST: mirror-pad before detecting. A face clipped by the
5290
+ # frame border shows the detector half a face, and every backend
5291
+ # scores that below threshold — a real subject half out of the
5292
+ # right edge went completely undetected on benchmark footage while
5293
+ # 20 interior faces were found. Reflection completes the clipped
5294
+ # half with its own mirror image, which detectors score like a
5295
+ # normal (symmetric) face sitting across the border. Boxes are
5296
+ # mapped back below; anything living mostly in the pad is a mirror
5297
+ # phantom of an interior face (which the normal pass already
5298
+ # covers) and is dropped.
5299
+ pad = self._edge_pad(dw, dh)
5300
+ dframe = cv2.copyMakeBorder(dframe, pad, pad, pad, pad,
5301
+ cv2.BORDER_REFLECT_101)
5302
+ pdh, pdw = dframe.shape[:2]
5084
5303
  raw = []
5085
5304
  if self.net is not None or self.ort is not None:
5086
5305
  raw += (self._find_centerface(dframe) if self.arch == "centerface"
5087
5306
  else self._find_scrfd(dframe))
5088
5307
  if self.yunet is not None:
5089
- if self.size != (dw, dh):
5090
- self.yunet.setInputSize((dw, dh))
5091
- self.size = (dw, dh)
5308
+ if self.size != (pdw, pdh):
5309
+ self.yunet.setInputSize((pdw, pdh))
5310
+ self.size = (pdw, pdh)
5092
5311
  _, faces = self.yunet.detect(dframe)
5093
5312
  for f in (faces if faces is not None else []):
5094
5313
  x, y, fw, fh, conf = f[0], f[1], f[2], f[3], float(f[-1])
5095
5314
  raw.append([x, y, x + fw, y + fh, conf])
5096
5315
  if (raw or self.net is not None or self.ort is not None
5097
5316
  or self.yunet is not None):
5098
- out = [(x1 / s, y1 / s, x2 / s, y2 / s, conf)
5099
- for x1, y1, x2, y2, conf in _nms_boxes(raw)]
5317
+ boxes = _nms_boxes(raw)
5100
5318
  else:
5101
- out = []
5319
+ boxes = []
5102
5320
  gray = cv2.cvtColor(dframe, cv2.COLOR_BGR2GRAY)
5103
5321
  for (x, y, fw, fh) in self.haar.detectMultiScale(gray, 1.1, 5,
5104
5322
  minSize=(36, 36)):
5105
5323
  if 0.8 >= self.thresh: # Haar has no score; gate by threshold
5106
- out.append((x / s, y / s, (x + fw) / s,
5107
- (y + fh) / s, 0.8))
5324
+ boxes.append((x, y, x + fw, y + fh, 0.8))
5325
+ out = []
5326
+ for x1, y1, x2, y2, conf in boxes:
5327
+ x1, y1, x2, y2 = x1 - pad, y1 - pad, x2 - pad, y2 - pad
5328
+ bw, bh = x2 - x1, y2 - y1
5329
+ ix = max(0.0, min(x2, dw) - max(x1, 0))
5330
+ iy = max(0.0, min(y2, dh) - max(y1, 0))
5331
+ if bw <= 0 or bh <= 0 or ix * iy < 0.25 * bw * bh:
5332
+ continue # mirror phantom, mostly in the pad
5333
+ out.append((max(0, x1) / s, max(0, y1) / s,
5334
+ min(dw, x2) / s, min(dh, y2) / s, conf))
5108
5335
  expanded = []
5109
5336
  for x1, y1, x2, y2, conf in out:
5110
5337
  ex, ey = (x2 - x1) * self.expand, (y2 - y1) * self.expand
@@ -5112,6 +5339,12 @@ class FaceDetector:
5112
5339
  min(w, x2 + ex), min(h, y2 + ey), conf))
5113
5340
  return expanded
5114
5341
 
5342
+ @staticmethod
5343
+ def _edge_pad(dw, dh):
5344
+ """Mirror-pad width for edge-clipped faces: covers faces up to
5345
+ ~2x the pad in size whose visible sliver is narrower than the pad."""
5346
+ return max(48, int(0.08 * max(dw, dh)))
5347
+
5115
5348
  def _forward(self, blob):
5116
5349
  """Run the optional ONNX face model and return its output arrays as
5117
5350
  a list — through onnxruntime (GPU where available: OpenVINO on
@@ -2361,6 +2361,47 @@ def test_merge_tail_extends_through_supporting_scans():
2361
2361
  assert m.t_end == pytest.approx(0.8), m.t_end
2362
2362
 
2363
2363
 
2364
+ def test_face_detector_edge_assist_geometry():
2365
+ """Detection runs on a mirror-padded frame so a face clipped by the
2366
+ frame border (half out of frame — undetectable to every backend on
2367
+ real benchmark footage) is completed by its own reflection. Boxes
2368
+ map back by subtracting the pad; a detection living mostly IN the
2369
+ pad is a mirror phantom of an interior face and must be dropped."""
2370
+ class CB(openscrub.Callbacks):
2371
+ def log(self, msg):
2372
+ pass
2373
+ fd = openscrub.FaceDetector(CB())
2374
+ frame = np.full((400, 600, 3), 90, np.uint8)
2375
+ pad = fd._edge_pad(600, 400)
2376
+
2377
+ class _Stub:
2378
+ def __init__(self, faces):
2379
+ self.faces = faces
2380
+ def setInputSize(self, sz):
2381
+ pass
2382
+ def detect(self, img):
2383
+ return None, np.array(self.faces, np.float32)
2384
+
2385
+ # a border-straddling detection (half real, half mirror) survives,
2386
+ # clamped to the frame; a detection fully inside the pad is dropped
2387
+ fd.yunet = _Stub([
2388
+ [pad + 570, pad + 100, 60, 80, 0.9], # face half off the right edge
2389
+ [5, pad + 200, 40, 40, 0.9], # phantom fully in the left pad
2390
+ ])
2391
+ fd.net = fd.ort = None
2392
+ fd.size = None
2393
+ boxes = [tuple(round(v) for v in b[:4]) for b in fd.find(frame)]
2394
+ assert len(boxes) == 1, boxes
2395
+ x1, y1, x2, y2 = boxes[0]
2396
+ assert x2 == 600 and x1 >= 560, boxes # clamped at the right border
2397
+ # an interior detection maps back to its exact position
2398
+ fd.yunet = _Stub([[pad + 100, pad + 120, 50, 60, 0.9]])
2399
+ fd.size = None
2400
+ (x1, y1, x2, y2, _c), = fd.find(frame)
2401
+ assert abs(x1 - (100 - 50 * fd.expand)) < 1.5
2402
+ assert abs(y1 - (120 - 60 * fd.expand)) < 1.5
2403
+
2404
+
2364
2405
  def test_blur_kernel_is_two_thirds_region():
2365
2406
  """Re-identification benchmark, close-up faces (200-750px): a
2366
2407
  1/3-width Gaussian left SFace similarity at 0.57-0.69 — above the
@@ -2374,9 +2415,14 @@ def test_blur_kernel_is_two_thirds_region():
2374
2415
  try:
2375
2416
  frame = np.full((400, 400, 3), 128, np.uint8)
2376
2417
  openscrub.blur_region(frame, 50, 50, 350, 350, "blur")
2418
+ # narrow-but-tall frame-border sliver: the kernel must scale with
2419
+ # the LONG side — width-scaled it hit the k=31 floor and border
2420
+ # half-faces re-identified at 0.55-0.71 through the blur
2421
+ openscrub.blur_region(frame, 0, 50, 40, 250, "blur")
2377
2422
  finally:
2378
2423
  openscrub._gauss_big = orig
2379
2424
  assert seen and seen[0] >= (2 * 300 // 3) | 1, seen
2425
+ assert len(seen) == 2 and seen[1] >= (2 * 200 // 3) | 1, seen
2380
2426
 
2381
2427
 
2382
2428
  def test_group_lines_splits_columns():
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes